Re: FreeBSD NFS server benchmarks vs. OpenBSD, NetBSD?
--On Saturday, June 22, 2002 22:23:59 -0700 Alfred Perlstein [EMAIL PROTECTED] wrote: Actually FreeBSD 5.x should have lockd support. I should know, I ported it from BSD/os. Will it be MFCed ? -- Mathieu Arnold To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: FreeBSD NFS server benchmarks vs. OpenBSD, NetBSD?
* Mathieu Arnold [EMAIL PROTECTED] [020623 00:36] wrote: --On Saturday, June 22, 2002 22:23:59 -0700 Alfred Perlstein [EMAIL PROTECTED] wrote: Actually FreeBSD 5.x should have lockd support. I should know, I ported it from BSD/os. Will it be MFCed ? Not by me, it requires more work than I have time for right now. -- -Alfred Perlstein [[EMAIL PROTECTED]] 'Instead of asking why a piece of software is using 1970s technology, start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
m_cat() does not update m_pkthdr.len
Hi, During development of networking code in FreeBSD kernel, we noticed that m_cat(p1, p2) does NOT do some necessary things: p1-m_pkthdr.len += p2-m_pkthdr.len; p2-m_flags = ~M_PKTHDR; Thanks, Yahel. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: inuring FreeBSD to the apache bug without upgrading apache ?
On Sun, 23 Jun 2002, Joshua Lee wrote: On Thu, 20 Jun 2002 19:59:20 -0700 Terry Lambert [EMAIL PROTECTED] wrote: Patrick Thomas wrote: Is it possible to patch/recompile FreeBSD 4.5 in such a way that your system is no longer vulnerable to the chunking attack, even if you are still running a vulnerable apache ? Not FreeBSD, but it's possible to reconfigure Apache. The way you would deal with this would be to tell Apache that it was an HTTP 1.0 server, since chunking is an HTTP 1.1 feature. I've found a better solution! On today's freshports there is something called mod_blowchunks :-) If installed, it will reject chunking and log it. This is an alternative to upgrading Apache. Given the place the problem occurs: I'd be weary of such solutions. Apaceh's myrad of config abilities are second only to sendmail - and it is easy to let a certain case slip through. If nessesary simply do a 'cvs diff' on apache it's cvs (dev.apache.org for anon access; I am willing to work with BSD developers to help if needed) to see the relatively few lines of code which are an issue - and which can be backported easily. Dw To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
Patrick Thomas wrote: I think I'll just decrease my swap size from 2 gigs to 1 gig - is that a reasonable alternative that provides the same benefit and possible solution to this problem ? ...since bsically 0 swap has ever been used on the machine anyway... Not really. The code in machdep.c allocated pmaps for swapped memory based on the size of real memory, rather than based on available swap. The reason it does this is that you can (effectively) add an arbitrary amount of swap later with swapon, without the swap devices at the time being known to the kernel at boot. THis makes it impossible to prereserve the number of pmap pages that will be needed for the actual amount of swap. Matt Dillon made some autosizing changes after I complained about this before. My actual complaint was to implicate the size of real memory available relative to the size of the full address space. The change he made attempts to autosize, and doesn't quite mirror this policy directly. THis code is not available in 4.5. I believe that it was back-ported to 4.6, but you would have to look at the CVS log on machdep.c to be sure about this -- it may only be in -current. The upshot of this is that having a lot of memory reserves pmap entries at 4K per 4M of real OR virtual memory. The result of this is that at 4G of physical RAM, you actually end up allocating more pmap's than 1G of memory can contain, since the total of physical RAM plus swap over 1024 is larger than 1G minus the amount taken by an idle kernel, not including the page mappings. If you have 3G of real RAM (which you do), then you are on the borderline of running out. When you factor in the amount of *potential* swap that machdep.c reserves, plus tuning for maxfiles/sockets/inpcb/tcpcb/mbufs/etc. (if any), PLUS the RAM taken up for things associated with running over 1000 processes (as your system does), then you end up exhausting the amount of VM space available. As I said before, though, the only way to know for sure if this is your real problem is to break to the debugger after the lockup (it's *not* a crash), and check out the wait channels for the processes thar are unable to run. If you want a tweak for 4.5 that has about a 95% proability of masking the problem, then you need to up the KVA space. Unfortunately, it's not really possible to tell you where every byte of memory is going. Also, unfortunately, the pmap's for swappable memory are not themselves swappable (or this would not be a problem). Probably, pmaps for swap and for file backing store for exectuables should be allocated when they are needed, not preallocated (they can be, if you are not out of RAM, or have RAM, but are out of KVA space in which to create mappings) [see growkernel]. Taking out 1G of physical memory from the box might also fix the problem without a kernel tweak, FWIW. However, right now, you need to cause the problem, enter the debugger, and use ps in the debugger to examine the wait channels. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: inuring FreeBSD to the apache bug without upgrading apache ?
Joshua Lee wrote: Terry Lambert [EMAIL PROTECTED] wrote: Patrick Thomas wrote: Is it possible to patch/recompile FreeBSD 4.5 in such a way that your system is no longer vulnerable to the chunking attack, even if you are still running a vulnerable apache ? Not FreeBSD, but it's possible to reconfigure Apache. The way you would deal with this would be to tell Apache that it was an HTTP 1.0 server, since chunking is an HTTP 1.1 feature. I've found a better solution! On today's freshports there is something called mod_blowchunks :-) If installed, it will reject chunking and log it. This is an alternative to upgrading Apache. But if a client uses chunking legitimately, and does so becuase it believes it's talking to an HTTP server, you've just broken that client's ability to POST/PUT. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
ppbus questions
hi! i'm trying to write a driver for an old cd-rom drive that you connect to the parallel port. it is a shuttletech para drive 525. i don't have any driver docs or technical specifications but i believe that it uses some kind of a SCSI to parallel chipset. the linux tree seems to have drivers for shuttletech EPSA-2 chipsets, so i'm currently trying to port their probe routines to FreeBSD to see if my cd-rom is indeed equipped with one of those chipsets. the linux code uses outb() and inb() directly to communicate with the parallel port so i have to port it to use the ppbus interface. here are my questions: 1) do the ppb_[rw][dcs]tr() functions have any side effects, like eg. does ppb_wdtr() drive strobe high or something like that to tell the device that there is data waiting, or is it simply a wrapper around outb() ? (ps. i tried to find the code for these functions but didn't succeed since the macros make use of PPBUS_IO() which is generated and calls some other function.) if the ppb_[rw][dcs]tr() function have side effects, are there any lower level functions which simply wrap around inb() and outb() ? 2) do the ppb_[rw][dcs]tr() functions wait, or do i have to call some function to do that, like DELAY() ? the linux code calls usleep() after sending commands to the device. (i believe it is bad to call DELAY() in a _probe() routine, but perhaps i can do that to start with...) 3) should i use micro sequences instead of doing ppb_*() calls ? i looked briefly at the man page but it seemed pretty complicated and i'm not a very experienced kernel hacker yet. i would be very grateful for any answers or tips you might have! best regards, Martin Faxér To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
ok. I was just looking back at a previous comment you made: Amusingly enough, you might actually have *better* luck with a lot less swap... and thinking that even if removing most of the swap did not _solve/mask_ the problem, at least it would be a step in the same direction as upping KVA (even if it is not as large a step) but if that is not the case... ...then, has anyone written a HOWTO on upping it in 4.5-RELEASE ? You mentioned to look back over your own old posts on the subject - before I jump in and try it, I want to confirm what I believe to understand, I need to set the KVA value in my kernel config _and_ edit those other two files in the kernel source, then just recompile my kernel. Sound like I'm on the right track ? Terry, thanks again for your help and for all the help you regularly give to other people pursuing items such as this on the various FreeBSD lists. --PT On Sun, 23 Jun 2002, Terry Lambert wrote: Patrick Thomas wrote: I think I'll just decrease my swap size from 2 gigs to 1 gig - is that a reasonable alternative that provides the same benefit and possible solution to this problem ? ...since bsically 0 swap has ever been used on the machine anyway... Not really. The code in machdep.c allocated pmaps for swapped memory based on the size of real memory, rather than based on available swap. The reason it does this is that you can (effectively) add an arbitrary amount of swap later with swapon, without the swap devices at the time being known to the kernel at boot. THis makes it impossible to prereserve the number of pmap pages that will be needed for the actual amount of swap. Matt Dillon made some autosizing changes after I complained about this before. My actual complaint was to implicate the size of real memory available relative to the size of the full address space. The change he made attempts to autosize, and doesn't quite mirror this policy directly. THis code is not available in 4.5. I believe that it was back-ported to 4.6, but you would have to look at the CVS log on machdep.c to be sure about this -- it may only be in -current. The upshot of this is that having a lot of memory reserves pmap entries at 4K per 4M of real OR virtual memory. The result of this is that at 4G of physical RAM, you actually end up allocating more pmap's than 1G of memory can contain, since the total of physical RAM plus swap over 1024 is larger than 1G minus the amount taken by an idle kernel, not including the page mappings. If you have 3G of real RAM (which you do), then you are on the borderline of running out. When you factor in the amount of *potential* swap that machdep.c reserves, plus tuning for maxfiles/sockets/inpcb/tcpcb/mbufs/etc. (if any), PLUS the RAM taken up for things associated with running over 1000 processes (as your system does), then you end up exhausting the amount of VM space available. As I said before, though, the only way to know for sure if this is your real problem is to break to the debugger after the lockup (it's *not* a crash), and check out the wait channels for the processes thar are unable to run. If you want a tweak for 4.5 that has about a 95% proability of masking the problem, then you need to up the KVA space. Unfortunately, it's not really possible to tell you where every byte of memory is going. Also, unfortunately, the pmap's for swappable memory are not themselves swappable (or this would not be a problem). Probably, pmaps for swap and for file backing store for exectuables should be allocated when they are needed, not preallocated (they can be, if you are not out of RAM, or have RAM, but are out of KVA space in which to create mappings) [see growkernel]. Taking out 1G of physical memory from the box might also fix the problem without a kernel tweak, FWIW. However, right now, you need to cause the problem, enter the debugger, and use ps in the debugger to examine the wait channels. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: inuring FreeBSD to the apache bug without upgrading apache ?
On Sun, 23 Jun 2002 02:06:20 -0700 Terry Lambert [EMAIL PROTECTED] wrote: Joshua Lee wrote: Terry Lambert [EMAIL PROTECTED] wrote: The way you would deal with this would be to tell Apache that it was an HTTP 1.0 server, since chunking is an HTTP 1.1 feature. I've found a better solution! On today's freshports there is something called mod_blowchunks :-) If installed, it will reject chunking and log it. This is an alternative to upgrading Apache. But if a client uses chunking legitimately, and does so becuase it believes it's talking to an HTTP server, you've just broken that client's ability to POST/PUT. You mean to say it believes it is talking to an HTTP 1.1 server, yes? I guess using HTTP 1.0 is a better solution then. Of course, maybe the *best* solution IMVHO would be to upgrade to the Apache version without this bug. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Cyrus vs. UW IMAP (was: Re: I Volunteer)
On Sat, Jun 22, 2002 at 02:36:44PM +0200, Neil Blakey-Milner wrote: On Sat 2002-06-22 (00:06), Chris Dillon wrote: There is always the option to use SSL, which is my preference, but unfortunately neither SSL nor SASL have widespread IMAP client support yet. Most IMAP clients I know of support SSL. Outlook, Outlook Express, Eudora, Netscape, Evolution, mutt, pine, ... Which IMAP clients don't? MacOS X's Mail.app prior to about 10.1.2 or .3 - about 6 months back. -- You can't do maths without e -- David Walters To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
Patrick Thomas wrote: ok. I was just looking back at a previous comment you made: Amusingly enough, you might actually have *better* luck with a lot less swap... I meant reserve, not physical swap. I can see how it could have been confusing in context; sorry. and thinking that even if removing most of the swap did not _solve/mask_ the problem, at least it would be a step in the same direction as upping KVA (even if it is not as large a step) but if that is not the case... ...then, has anyone written a HOWTO on upping it in 4.5-RELEASE ? You mentioned to look back over your own old posts on the subject - before I jump in and try it, I want to confirm what I believe to understand, I need to set the KVA value in my kernel config _and_ edit those other two files in the kernel source, then just recompile my kernel. Sound like I'm on the right track ? Yes. That's the way to do it for 4.5, specifically. FreeBSD really needs an internals book. But like I said, this changed between 4.5 and 4.6, and everyone who's buying books would be more interested in 5.x, and all the important things change too fast (writing an internals book is an ~2000 hour job, and that basically means that the important stuff can't change for a year, or you have to track it -- which inflates it to an ~3000 hour job). Basically, most of the important internal interfaces need to sit still so that a book can be written, or no book. Even so, the selling life of the book will be limited to the amount of time after publication that things actually sit still. Kirk McKusick is rumored to be writing one; so was Wes Peters. Alfred and I discussed a device driver book that both of us thought needed to be written. Etc.. But no book, yet. I really hesitate to put down an A-B-C set of steps, if I know that not only is it only applicable to a couple of versions, none of them are the current version. 8-(. Terry, thanks again for your help and for all the help you regularly give to other people pursuing items such as this on the various FreeBSD lists. Eh, I'm noisy. 8-). You still need to run the debugger, I think. So far, this is all theory. It fits the facts, but I can think of two other very low probability ways to cause the same symptoms. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: inuring FreeBSD to the apache bug without upgrading apache ?
Joshua Lee wrote: [ ... mod_blowchunks ... ] But if a client uses chunking legitimately, and does so becuase it believes it's talking to an HTTP server, you've just broken that client's ability to POST/PUT. You mean to say it believes it is talking to an HTTP 1.1 server, yes? Yes. I guess using HTTP 1.0 is a better solution then. Of course, maybe the *best* solution IMVHO would be to upgrade to the Apache version without this bug. Yeah; this whole thread is premised on working around the problem without an Apache software change. It's a reasonable premise (IMO) -- if you've got a custom compilation and a lot of modules, that can end up being a lot of software. I build a PHP4+SSL+Apache+IMAP+etc. source tree at one point, and it ended up being ~1.2 million lines of code, all told, that had to be made to work together. If you had just built it, then it would be very hard to update just one component without repeating the whole process. My advice? Use CVS. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
again: FreeBSD NFS server benchmarks vs. OpenBSD, NetBSD?
There have been some useful responses to my original question, but I guess I didn't make it clear enough what the question was, because I got a lot of responses comparing the NFS servers on systems other than FreeBSD, NetBSD, and OpenBSD. I'm only interested in comparing the performance of these three systems (at least for the short run). If you've got a comparison of these three systems, or if someone has these systems running in-house and would be willing to benchmark them, I'd love to see the results! Thanks, -Dan To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
[Fwd: VIRUS IN MAIL FOR YOU (from owner-freebsd-hackers@FreeBSD.ORG) ()]
-- Ted Sikora [EMAIL PROTECTED] http://unixos2.org ---BeginMessage--- V I R U S A L E R T Our virus checker found the virus in an email to you from: [EMAIL PROTECTED] Delivery of the email was stopped! Please contact your system administrator for details. Replies to this notification message will be discarded. For your reference, here are headers from the email: - BEGIN HEADERS - Return-Path: [EMAIL PROTECTED] Received: from hub.freebsd.org (hub.FreeBSD.org [216.136.204.18]) by mx2.freebsd.org (Postfix) with ESMTP id 1214B55802; Sun, 23 Jun 2002 05:32:43 -0700 (PDT) (envelope-from [EMAIL PROTECTED]) Received: by hub.freebsd.org (Postfix, from userid 538) id 7299E37B403; Sun, 23 Jun 2002 05:32:39 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with SMTP id 2702D2E8023; Sun, 23 Jun 2002 05:32:39 -0700 (PDT) Received: by hub.freebsd.org (bulk_mailer v1.12); Sun, 23 Jun 2002 05:32:38 -0700 Delivered-To: [EMAIL PROTECTED] Received: from excalibur.skynet.be (excalibur.skynet.be [195.238.3.90]) by hub.freebsd.org (Postfix) with ESMTP id EC07737B401 for [EMAIL PROTECTED]; Sun, 23 Jun 2002 05:31:13 -0700 (PDT) Received: from relay.compaqnet.be (free19276.powered-by.skynet.be [62.4.203.76]) by excalibur.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.19) with SMTP id g5NCTF815593; Sun, 23 Jun 2002 14:29:15 +0200 (MET DST) (envelope-from [EMAIL PROTECTED]) Date: Sun, 23 Jun 2002 14:29:15 +0200 (MET DST) Message-Id: [EMAIL PROTECTED] From: De Brabandere Jean-Luc [EMAIL PROTECTED] SUBJECT: Die zich bevinden in mijn effectendossier. X-MSMail-Priority: Normal X-Priority: 3 X-Mailer: Microsoft Outlook Express 5.00.2919.6600 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary==_NextPart_000_00C9_01F9F3D5.CDF3D510 Content-Transfer-Encoding: 7bit To: undisclosed-recipients: ; Sender: [EMAIL PROTECTED] List-ID: freebsd-hackers.FreeBSD.ORG List-Archive: http://docs.freebsd.org/mail/ (Web Archive) List-Help: mailto:[EMAIL PROTECTED]?subject=help (List Instructions) List-Subscribe: mailto:[EMAIL PROTECTED]?subject=subscribe%20freebsd-hackers List-Unsubscribe: mailto:[EMAIL PROTECTED]?subject=unsubscribe%20freebsd-hackers X-Loop: FreeBSD.ORG Precedence: bulk -- END HEADERS -- ---End Message---
Locking the passwd subsystem (was Re: bug in pw, -STABLE [patch])
[ Moved to -hackers since this isn't really a STABLE-specific issue, but rather a general thing. Bcc's to -stable for continuity's sake. ] This discussion has gone on, on and off, for months now, so I'm going to recap a bit. For the people in it, we've mostly paged out the details by now, and for the people not in it, this will be necessary catchup. pw(8) can sometimes corrupt the passwd file when it's running. This is due to the fact that, while it munges things, it locks /etc/master.passwd. This fails to really work right between invocations, because as part of its dirty work, pw(8) moves master.passwd away and creates a new one from scratch. So, an external lockfile is needed. There's no real common locking among the other consumers (chpass(1), passwd(1), vipw(8), pwd_mkdb(8), etc) either, so we might as well make a big common locking mechanism out of it. And while we're doing that, it would be nifty to centralize some functions for doing locking like that, too, instead of coding it from scratch in every application that needs it. The patch on this mail accomplishes this part of the objective; once we can all settle on that, we can easily handle locking all the consumers neatly using these functions. And now, for the carrying on. On Sun, Jun 23, 2002 at 09:07:15AM -0400 I heard the voice of Geoffrey C. Speicher, and lo! it spake thus: Actually, now that I think about it, even a daemon wouldn't guarantee better results in every such scenario. At first it would seem that the daemon would get a chance to clean up the pid file long before another process wanted it (during step 4). However, if the timing is just right then p1 can die the moment that its pid would be reused, and chances are that the daemon wouldn't catch it in time. Well, a daemon would have a BETTER chance, though nowhere near a guarantee. The theory being that the daemon would check every so often (5 minutes sounds good to me), whereas with the library, you potentially could not check for days, depending on how often you run pw(8). Even on a heavily used system, 5 minutes wouldn't likely be enough to reuse a PID, unless you had random PID assignment or some such. Now, this is a problem. There's a race condition here. It's a very small window, to be sure, but I'm not quite sure how to close it. After I think you must've figured it out judging by your post that just came through as I was writing that previous paragraph. :) I think so. See comments in the code. (Re: Oliver's statements about the general evil of PID files) The beauty of encapsulating the pid file operations into a library is that the implementation can be changed to create a socket instead of, or (more likely) in addition to, a pid file. In fact, this may be just the thing we need to close that pid gets reused hole. Since Matt already has the functions mostly implemented, I'll defer to him. Well, there's nothing necessarily stopping us from adding more stuff later, of course. Personally, I've set my sights a bit lower; for the moment, I'm more concerned with Let's not corrupt the passwd database than with This must never block unless totally absolutely completely necessary and should always know where its cheese is, which is what the socket-theory is pointing at. One giant code upheaval at a time :) One possibility would be to make procfs(5) show the process start time (rather than the current time) for the ctime of the /proc/pid directory. Then, one could look at the PID file, and if its mtime (not ctime; see in below code where we could use a pre-existing file) is older than the ctime of the /proc/pid directory, we'd know the PID had changed. Getting the actual process start time would require libkvm (and thus setgid kmem), or spawning ps(1) and parsing output, otherwise. ANYWAY: Attached is a patch (relative to src/) for my first run through creating the functions to handle PID-file creation, deletion, and checking-on-creation. It consists of two functions: - pid_begin(), which creates a PID file if one doesn't exist, and checks to see if the listed process is still around (taking over the PID file if it's not) if it does. - pid_end(), which deletes the PID file. Code is reasonably well commented. Manpage included. Shaken, not stirred. Slippery when wet. This has gone through a compile test, but hasn't been live-tested. This will provide the basis then to move forward and lock all the programs that access the passwd database (for writing, of course; no reason to lock readers) in a common way. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ The only reason I'm burning my candle at both ends, is because I haven't figured out how to light the middle yet Index: lib/libutil/Makefile === RCS file: /usr/cvs/src/lib/libutil/Makefile,v retrieving
Re: ppbus questions
On Sun, Jun 23, 2002 at 12:15:01PM +0200 I heard the voice of Martin Faxer, and lo! it spake thus: i'm trying to write a driver for an old cd-rom drive that you connect to the parallel port. it is a shuttletech para drive 525. i don't have any driver docs or technical specifications but i believe that it uses some kind of a SCSI to parallel chipset. Perhaps you've already been over this, but have you tried just using (or altering) the vpo driver? It already provides something resembling a SCSI interface for parallel-port ZIP drives; it would seem the logical place to start. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ The only reason I'm burning my candle at both ends, is because I haven't figured out how to light the middle yet To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Locking the passwd subsystem (was Re: bug in pw, -STABLE [patch])
On Sun, Jun 23, 2002 at 11:17:35AM -0400 I heard the voice of
Geoffrey C. Speicher, and lo! it spake thus:
5-minute window. Still unliklely, but not quite as unlikely as the
non-daemon scenario.
And it all goes out when window when kern.randompid=1, so it's kinda
moot.
Looks good from here. I've attached some minor patches (also relative to
src/) to pid_util.{3,c}. The first corrects some minor typos in the man
page and the second is to make pid_begin() work as advertised when read()
fails (set errno and return -1 rather than return errno).
Yes, I have a brain. No, really I do. I just don't have enough coffee
:)
(I haven't tested any of this yet either.)
I just slapped together a few quick test programs. Here's an updated
patch, incorporating your fixes, as well as a bugfix and an almost-bugfix
my testing showed up (hint: just because you ftruncate() to 0-length,
doesn't mean your offset becomes 0 too, dangit), and eliminated an extra
variable.
--
Matthew Fuller (MF4839) | [EMAIL PROTECTED]
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
The only reason I'm burning my candle at both ends, is because I
haven't figured out how to light the middle yet
Index: lib/libutil/Makefile
===
RCS file: /usr/cvs/src/lib/libutil/Makefile,v
retrieving revision 1.46
diff -u -r1.46 Makefile
--- lib/libutil/Makefile8 May 2002 00:50:07 - 1.46
+++ lib/libutil/Makefile23 Jun 2002 13:31:01 -
@@ -8,7 +8,7 @@
CFLAGS+=-DINET6
SRCS= _secure_path.c auth.c fparseln.c login.c login_auth.c \
login_cap.c login_class.c login_crypt.c login_ok.c login_times.c \
- login_tty.c logout.c logwtmp.c property.c pty.c \
+ login_tty.c logout.c logwtmp.c pid_util.c property.c pty.c \
pw_util.c realhostname.c stub.c \
trimdomain.c uucplock.c
INCS= libutil.h login_cap.h
@@ -16,7 +16,7 @@
MAN+= login.3 login_auth.3 login_tty.3 logout.3 logwtmp.3 pty.3 \
login_cap.3 login_class.3 login_times.3 login_ok.3 \
_secure_path.3 uucplock.3 property.3 auth.3 realhostname.3 \
- realhostname_sa.3 trimdomain.3 fparseln.3
+ realhostname_sa.3 trimdomain.3 fparseln.3 pid_util.3
MAN+= login.conf.5 auth.conf.5
MLINKS+= property.3 properties_read.3 property.3 properties_free.3
MLINKS+= property.3 property_find.3
@@ -39,5 +39,6 @@
MLINKS+=login_auth.3 auth_checknologin.3 login_auth.3 auth_cat.3
MLINKS+=uucplock.3 uu_lock.3 uucplock.3 uu_lock_txfr.3 \
uucplock.3 uu_unlock.3 uucplock.3 uu_lockerr.3
+MLINKS+=pid_util.3 pid_begin.3 pid_end.3
.include bsd.lib.mk
Index: lib/libutil/libutil.h
===
RCS file: /usr/cvs/src/lib/libutil/libutil.h,v
retrieving revision 1.37
diff -u -r1.37 libutil.h
--- lib/libutil/libutil.h 8 May 2002 00:50:07 - 1.37
+++ lib/libutil/libutil.h 23 Jun 2002 12:35:30 -
@@ -82,6 +82,8 @@
struct sockaddr;
intrealhostname_sa(char *host, size_t hsize, struct sockaddr *addr,
int addrlen);
+int pid_begin(const char *pidfile, mode_t mode, int flags);
+int pid_end(const char *pidfile);
#ifdef _STDIO_H_ /* avoid adding new includes */
char *fparseln(FILE *, size_t *, size_t *, const char[3], int);
#endif
@@ -128,5 +130,8 @@
/* pw_scan() */
#define PWSCAN_MASTER 0x01
#define PWSCAN_WARN0x02
+
+/* pid_begin() */
+#define PID_NOBLOCK0x01
#endif /* !_LIBUTIL_H_ */
Index: lib/libutil/pid_util.c
===
--- /dev/null Sun Jun 23 11:44:00 2002
+++ lib/libutil/pid_util.c Sun Jun 23 11:50:59 2002
@@ -0,0 +1,178 @@
+/*-
+ * Copyright (c) 2002 Matthew D. Fuller [EMAIL PROTECTED]
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF
Re: bug in pw, -STABLE [patch]
[ pulled over to -hackers as well ] On Sun, 23 Jun 2002, Matthew D. Fuller wrote: On Sun, Jun 23, 2002 at 09:00:52AM -0700 I heard the voice of Paul Herman, and lo! it spake thus: On Sun, 23 Jun 2002, Geoffrey C. Speicher wrote: How so? I'm not suggesting unlink(2)ing /etc/master.passwd or /etc/spwd.db at all. No, but pw(8) does; making it not do so would require reasonably extensive rewriting. Indeed it does do this. However, that seems to be the crux of the problem. The current pw(8) behavior was (probably) not only a convenient way to update files in /etc, but an attempt to narrow (not eliminate) the window for race conditions. If this is fundamentaly flawed, *this* should be fixed, and not have a larger band aid put over it. I can't imagine it would be too extensive of a rewrite. The temp file code could be kept, and in fileupd.c:fileupdate() instead of rename(/etc/master.passwd.new, /etc/master.passwd), it just copies the contents of the .new file into the original file (that has the O_EXLOCK.) -Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Limiting clients per source IP address (ftpd, inetd, etc.)
On Fri, Jun 21, 2002 at 03:09:25AM +0300, Giorgos Keramidas wrote: I've been thinking for quite some time to add per-client-IP limiting to ftpd, I needed to do this. Then I discovered that ipfw's limit directive lets you limit the number of incoming connections, which proved much more powerful. N -- FreeBSD: The Power to Serve http://www.freebsd.org/ (__) FreeBSD Documentation Projecthttp://www.freebsd.org/docproj/\\\'',) \/ \ ^ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/_) msg35241/pgp0.pgp Description: PGP signature
Re: Limiting clients per source IP address (ftpd, inetd, etc.)
On Sun, Jun 23, 2002 at 06:19:51PM +0100, Nik Clayton wrote: On Fri, Jun 21, 2002 at 03:09:25AM +0300, Giorgos Keramidas wrote: I've been thinking for quite some time to add per-client-IP limiting to ftpd, I needed to do this. Then I discovered that ipfw's limit directive lets you limit the number of incoming connections, which proved much more powerful. the funny thing is that when i implemented it i thought it was completely useless :) As a matter of fact, I still think that, at least for resource management purposes. It may have its good use for protection against denial-of-service attacks though. cheers luigi To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: inuring FreeBSD to the apache bug without upgrading apache ?
Yeah; this whole thread is premised on working around the problem without an Apache software change. It's a reasonable premise (IMO) -- if you've got a custom compilation and a lot of modules, that can end up being a lot of software. I build a PHP4+SSL+Apache+IMAP+etc. source tree at one point, and it ended up being ~1.2 million lines of code, all told, that had to be made to work together. If you had just built it, then it would be very hard to update just one component without repeating the whole process. My advice? Use CVS. Actually, this whole thread is premised on I have a dev system with 16 jailed apaches and it would be a pain to upgrade all 16 of them vs. just making one global kernel/environment change. It sounds like that is probably a pipe dream though.. --PT To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
FreeBSD 2.2.x ISO images.
Where might I find ISO images for the FreeBSD 2.2.x releases? Do such files exist? I have an older system with a non-IDE Creative CD-ROM drive, and FreeBSD 4.x seems to not support that drive any more. I do have several FreeBSD 3.x releases, but they always hang with a kernel panic when booting via the included bootdisks. Would the FreeBSD 2.2.x series of releases work with such a CD-ROM drive? Thanks, Jefferson H. _ MSN Photos is the easiest way to share and print your photos: http://photos.msn.com/support/worldwide.aspx To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: FreeBSD 2.2.x ISO images.
On Sun, Jun 23, 2002 at 06:19:43PM +, Jefferson Harlough wrote: Where might I find ISO images for the FreeBSD 2.2.x releases? Do such files exist? I have 2.1-RELEASE, on up, all on CD. Would those filesystems help? Thanks, Jefferson H. -- Brian 'you Bastard' Reichert[EMAIL PROTECTED] 37 Crystal Ave. #303Daytime number: (603) 434-6842 Derry NH 03038-1713 USA Intel architecture: the left-hand path To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, 23 Jun 2002, Matthew D. Fuller wrote:
On Sun, Jun 23, 2002 at 09:59:10AM -0700 I heard the voice of
Paul Herman, and lo! it spake thus:
I can't imagine it would be too extensive of a rewrite. The temp
file code could be kept, and in fileupd.c:fileupdate() instead of
rename(/etc/master.passwd.new, /etc/master.passwd), it just
copies the contents of the .new file into the original file (that
has the O_EXLOCK.)
Yes, but that would open up the problem that cause the rename(2) to be
used in the first place; i.e., rename() is atomic; read(2)/write(2)'ing a
bunch of data isn't, so a system crash at the wrong time would leave you
with a partial master.passwd.
Fine, then lock them both, and use rename(). Patch attached.
In fact, if you look at fileupdate(), you see that it already gains
an exclusive lock on the temp file, but not the original
/etc/master.passwd (if you will.) I think this is a bug, because
the original is getting modified (at least in name space), so that
should be locked while pw(8) is operating.
What do you think?
-Paul.
Index: fileupd.c
===
RCS file: /u02/ncvs/src/usr.sbin/pw/fileupd.c,v
retrieving revision 1.9
diff -u -r1.9 fileupd.c
--- fileupd.c 26 Oct 1999 04:27:13 - 1.9
+++ fileupd.c 23 Jun 2002 18:30:16 -
@@ -76,7 +76,7 @@
if (pfxlen = 1)
rc = EINVAL;
else {
- intinfd = open(filename, O_RDWR | O_CREAT, fmode);
+ intinfd = open(filename, O_RDWR | O_CREAT | O_EXLOCK | O_NONBLOCK,
+fmode);
if (infd == -1)
rc = errno;
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
jump in and try it, I want to confirm what I believe to understand, I need to set the KVA value in my kernel config _and_ edit those other two files in the kernel source, then just recompile my kernel. Sound like I'm on the right track ? Yes. That's the way to do it for 4.5, specifically. Because I am paranoid, I like to check the state of a measurement before making a change and then after, to see that what I did did indeed induce a change ... I have this irrational fear that sometimes I make changes like this and nothing in fact changed, and I just don't know it :) So, should I just look for the value of: vm.zone_kmem_kvaspace: 179691520 to increase in size even though the physical RAM stays the same at 3gigs, or is there some other measurement I should look at before and after the KVA increase to ensure that it worked (and yes, I know that if it doesn't work I probably will have an inoperable machine, but just out of curiousity...) thanks, PT To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Locking the passwd subsystem (was Re: bug in pw, -STABLE [patch])
And here's a run at Stage 2. This adapts a subset of programs to use the
pid_*() locking supplied. The attached patch updates: pwd.h, libutil
(libutil.h and pw_util.c), chpass (chpass.c), pw (pw.c), pwd_mkdb
(Makefile, pwd_mkdb.8, pwd_mkdb.c), vipw (vipw.c). This does NOT include
passwd(1), since that does its dirty work in PAM, which I definately
don't feel like delving into.
Note that this patch MAY be a little off when applied over the previous
patch; I'm working on them in seperate trees, to try and keep the stages
discrete. But a little manual fiddling should get them reconciled in
short order.
--
Matthew Fuller (MF4839) | [EMAIL PROTECTED]
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
The only reason I'm burning my candle at both ends, is because I
haven't figured out how to light the middle yet
Index: include/pwd.h
===
RCS file: /usr/cvs/src/include/pwd.h,v
retrieving revision 1.12
diff -u -r1.12 pwd.h
--- include/pwd.h 9 Jun 2002 19:39:18 - 1.12
+++ include/pwd.h 23 Jun 2002 18:16:21 -
@@ -66,6 +66,9 @@
#define_PATH_MASTERPASSWD /etc/master.passwd
#define_MASTERPASSWD master.passwd
+#define_PATH_PWDLOCK /var/run/pwd.lock
+#define_MODE_PWDLOCK (S_IRUSR | S_IWUSR)
+
#define_PATH_MP_DB /etc/pwd.db
#define_MP_DB pwd.db
#define_PATH_SMP_DB/etc/spwd.db
Index: lib/libutil/libutil.h
===
RCS file: /usr/cvs/src/lib/libutil/libutil.h,v
retrieving revision 1.37
diff -u -r1.37 libutil.h
--- lib/libutil/libutil.h 8 May 2002 00:50:07 - 1.37
+++ lib/libutil/libutil.h 23 Jun 2002 18:38:18 -
@@ -95,6 +95,8 @@
intpw_init(const char *_dir, const char *_master);
char *pw_make(struct passwd *_pw);
intpw_mkdb(const char *_user);
+intpw_globlock(int);
+intpw_globunlock(void);
intpw_lock(void);
struct passwd *pw_scan(const char *_line, int _flags);
const char *pw_tempname(void);
@@ -124,6 +126,9 @@
#defineFPARSELN_UNESCCOMM 0x04
#defineFPARSELN_UNESCREST 0x08
#defineFPARSELN_UNESCALL 0x0f
+
+/* pw_globlock() */
+#definePW_NOBLOCK PID_NOBLOCK
/* pw_scan() */
#define PWSCAN_MASTER 0x01
Index: lib/libutil/pw_util.c
===
RCS file: /usr/cvs/src/lib/libutil/pw_util.c,v
retrieving revision 1.25
diff -u -r1.25 pw_util.c
--- lib/libutil/pw_util.c 8 May 2002 14:52:32 - 1.25
+++ lib/libutil/pw_util.c 23 Jun 2002 18:46:19 -
@@ -79,6 +79,7 @@
static char passwd_dir[PATH_MAX];
static char tempname[PATH_MAX];
static int initialized;
+static int globlocked = 0;
void
pw_cont(int sig)
@@ -163,6 +164,28 @@
}
/*
+ * Lock the password subsystem globally.
+ */
+int
+pw_globlock(int flags)
+{
+ int retval;
+
+ if( (retval=pid_begin(_PATH_PWDLOCK, _MODE_PWDLOCK, flags)) == 0 );
+ globlocked=1;
+ return(retval);
+}
+
+/*
+ * Unlock the global lock
+ */
+int pw_globunlock(void)
+{
+
+ return(pid_end(_PATH_PWDLOCK));
+}
+
+/*
* Lock the master password file.
*/
int
@@ -258,10 +281,10 @@
case 0:
/* child */
if (user == NULL)
- execl(_PATH_PWD_MKDB, pwd_mkdb, -p,
+ execl(_PATH_PWD_MKDB, pwd_mkdb, -p, -n,
-d, passwd_dir, tempname, NULL);
else
- execl(_PATH_PWD_MKDB, pwd_mkdb, -p,
+ execl(_PATH_PWD_MKDB, pwd_mkdb, -p, -n,
-d, passwd_dir, -u, user, tempname, NULL);
_exit(1);
default:
@@ -353,6 +376,11 @@
}
if (lockfd != -1)
close(lockfd);
+ if (globlocked != 0)
+ {
+ globlocked=0;
+ pid_end(_PATH_PWDLOCK);
+ }
errno = serrno;
}
Index: usr.bin/chpass/chpass.c
===
RCS file: /usr/cvs/src/usr.bin/chpass/chpass.c,v
retrieving revision 1.23
diff -u -r1.23 chpass.c
--- usr.bin/chpass/chpass.c 8 May 2002 00:54:28 - 1.23
+++ usr.bin/chpass/chpass.c 23 Jun 2002 18:42:25 -
@@ -258,6 +258,8 @@
case _PWF_FILES:
if (pw_init(NULL, NULL))
err(1, pw_init());
+ if (pw_globlock(PW_NOBLOCK) 0)
+ err(1, pw_globlock());
if ((pfd = pw_lock()) == -1) {
pw_fini();
err(1, pw_lock());
Index: usr.sbin/pw/pw.c
===
RCS file: /usr/cvs/src/usr.sbin/pw/pw.c,v
retrieving
Call for Submission of 2003 World Wireless Congress (3Gwireless'2003)
[Sorry for multiple copies of this message. Thanks for your support in promotion of education, research, development and business of emerging wireless communications] Dear Fellow Wireless Colleagues: 2002 World Wireless Congress (WWC02) - the official framework of 3Gwireless'2002 has been extremely successful and productive. To continue this great effort and expect much more authoritative in the next year, the 2003 World Wireless Congress (WWC03) will be a very big bang in helping set the standard of performance, innovation and quality of emerging wireless communications focusing on 3G and 4G mobile technologies and business. WWC'03 will continue to be the best global platform of industry-driven, academia-enriched, development-oriented and business-targeted, and demand very high standard of technical leadership as well as professional excellence. The Call-for-Submission (including technical papers, tutorials, industry forums, panels, etc) is available now at the congress website. The submission will be very busy in the coming months, and the space is very limited. Please act ASAP to submit your leadership and innovation as per the guidelines listed in the web. The website of 2003 World Wireless Congress (3Gwireless'2003) is at: http://wirelesscongress.com or http://3gwireless.com The very successful story of 2002 World Wireless Congress (3Gwireless'2002) is at: http://delson.org/wwc02 This congress at news can be found at: http://www.wirelessweek.com/index.asp?layout=storyarticleId=CA220172stt=001 For sponsorship or exhibition opportunities, please contact [EMAIL PROTECTED] For technical leadership issues, please contact the chair at: [EMAIL PROTECTED] Thank you for your time in advance. With best wishes! Office of 2003 World Wireless Congress http://wirelesscongress.com or http://3gwireless.com [This is only one-time educational message. Thanks for your support.] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, Jun 23, 2002 at 11:32:54AM -0700 I heard the voice of
Paul Herman, and lo! it spake thus:
In fact, if you look at fileupdate(), you see that it already gains
an exclusive lock on the temp file, but not the original
/etc/master.passwd (if you will.) I think this is a bug, because
the original is getting modified (at least in name space), so that
should be locked while pw(8) is operating.
I'm not sure. Hm.
I think that MAY prevent some of the corruption cases. On the other
hand, we're really addressing a broader spectrum of issues here. The was
pw(8) manipulates the passwd database is rather different from how
chpass(1) or vipw(8) do it; this unifies it all (well, starts the
process). Also, looking down, it seems that pw(8) will ALWAYS try to do
a line-by-line copy of the temp file back into the main file, which would
be Bad Bad Bad (tm); it does this because using rename() all the time
loses the flock() lock. However, if we use this global locking, we can
dispense with that, and remove some code complexity, to say nothing of a
giant waste of time copying line-by-line.
I'm trying to kill several birds with as small a stone as I can; I've had
my eye on this whole subsystem for a while, and I'd really like to
re-center it all around pw(8). vipw(8) kinda has to be its own beast,
but chpass(1) and adduser(8)/rmuser(8) are prime candidates to be
reworked to use pw(8) for their dirty work.
Let's try this: updated patch for pw(8) including my global locking, the
addition of flock()'ing both old and new files (it's not quite necessary,
since it's all under a global lock, but it never hurts to double-check)
as in your patch, and removing the line-by-line copy and using rename()
all the time. (I suggest tabstop=4 or even =2; that file is *DEEP*)
--
Matthew Fuller (MF4839) | [EMAIL PROTECTED]
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
The only reason I'm burning my candle at both ends, is because I
haven't figured out how to light the middle yet
Index: fileupd.c
===
RCS file: /usr/cvs/src/usr.sbin/pw/fileupd.c,v
retrieving revision 1.9
diff -u -r1.9 fileupd.c
--- fileupd.c 26 Oct 1999 04:27:13 - 1.9
+++ fileupd.c 23 Jun 2002 19:22:16 -
@@ -76,7 +76,8 @@
if (pfxlen = 1)
rc = EINVAL;
else {
- intinfd = open(filename, O_RDWR | O_CREAT, fmode);
+ intinfd = open(filename,
+ O_RDWR | O_CREAT | O_EXLOCK | O_NONBLOCK, fmode);
if (infd == -1)
rc = errno;
@@ -92,7 +93,8 @@
strcpy(file, filename);
strcat(file, .new);
- outfd = open(file, O_RDWR | O_CREAT | O_TRUNC |
O_EXLOCK, fmode);
+ outfd = open(file,
+ O_RDWR | O_CREAT | O_TRUNC | O_EXLOCK |
+O_NONBLOCK, fmode);
if (outfd == -1)
rc = errno;
else {
@@ -169,27 +171,24 @@
rc = errno; /*
Failed to update */
else {
/*
-* Copy data back into
the
+* Move data back into
+the
* original file and
truncate
*/
rewind(infp);
rewind(outfp);
- while (fgets(line,
linesize, outfp) != NULL)
- fputs(line,
infp);
/*
-* If there was a
problem with copying
-* we will just rename
'file.new'
-* to 'file'.
-* This is a gross
hack, but we may have
-* corrupted the
original file
-* Unfortunately, it
will lose the inode
-* and hence the lock.
+* Use rename() to
+move the new file
Bug in wakeup() (stable and current) ?
This doesn't look right at all. It looks like wakeup is not restarting
properly:
s = splhigh();
qp = slpque[LOOKUP(ident)];
restart:
TAILQ_FOREACH(p, qp, p_procq) {
if (p-p_wchan == ident) {
TAILQ_REMOVE(qp, p, p_procq);
p-p_wchan = 0;
if (p-p_stat == SSLEEP) {
...
goto restart;
}
/* XX goto restart should occur HERE XX */
}
}
The goto restart condition should occur one level up, as I show in
the comment.
Could someone take a look at this and tell me if I am blowing smoke?
-Matt
Matthew Dillon
[EMAIL PROTECTED]
Index: kern_synch.c
===
RCS file: /home/ncvs/src/sys/kern/kern_synch.c,v
retrieving revision 1.87.2.4
diff -u -r1.87.2.4 kern_synch.c
--- kern_synch.c14 Nov 2001 17:22:49 - 1.87.2.4
+++ kern_synch.c23 Jun 2002 20:10:37 -
@@ -745,8 +745,8 @@
wakeup((caddr_t)proc0);
}
/* END INLINE EXPANSION */
- goto restart;
}
+ goto restart;
}
}
splx(s);
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
* Matthew Dillon [EMAIL PROTECTED] [020623 13:14] wrote:
This doesn't look right at all. It looks like wakeup is not restarting
properly:
s = splhigh();
qp = slpque[LOOKUP(ident)];
restart:
TAILQ_FOREACH(p, qp, p_procq) {
if (p-p_wchan == ident) {
TAILQ_REMOVE(qp, p, p_procq);
p-p_wchan = 0;
if (p-p_stat == SSLEEP) {
...
goto restart;
}
/* XX goto restart should occur HERE XX */
}
}
The goto restart condition should occur one level up, as I show in
the comment.
Could someone take a look at this and tell me if I am blowing smoke?
I'm pretty sure you only need to 'goto restart' if you call into
maybe_resched() as someone else may have manipulated the queues.
The 'restart' label is only in there for restarting in case one of
the functions called may change the lists, if we restart _every_
time we'll traverse the same procs where p-p_wchan != ident over
and over needlessly.
-Alfred
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, 23 Jun 2002, Matthew D. Fuller wrote: On Sun, Jun 23, 2002 at 11:32:54AM -0700 I heard the voice of Paul Herman, and lo! it spake thus: In fact, if you look at fileupdate(), you see that it already gains an exclusive lock on the temp file, but not the original /etc/master.passwd (if you will.) I think this is a bug, because the original is getting modified (at least in name space), so that should be locked while pw(8) is operating. I think that MAY prevent some of the corruption cases. On the other hand, we're really addressing a broader spectrum of issues here. The was pw(8) manipulates the passwd database is rather different from how chpass(1) or vipw(8) do it; this unifies it all (well, starts the process). Also, looking down, it seems that pw(8) will ALWAYS try to do a line-by-line copy of the temp file back into the main file, which would be Bad Bad Bad (tm); I think you might have your infp confused with your outfp. It's not writing to the original live file, it's just writing the new temp file. That part of the code is OK. As for the giant lock, it would be like closing down the entire airport after someone finds a knife in the snack bar, and in my opinion an this is an unwise and brutal thing to do. Not to mention stale /var/run/ files that might be lying around... I do appreciate your work, but please don't commit this until the real issue is solved, and the dust settles. There still may be some use for it... plenty of time until the next release cycle. Geoff: Does the patch for fileupd.c in the last mail hinder the file corruption you are seeing? (I suggest tabstop=4 or even =2; that file is *DEEP*) Yss! :-) -Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
:I'm pretty sure you only need to 'goto restart' if you call into :maybe_resched() as someone else may have manipulated the queues. : :The 'restart' label is only in there for restarting in case one of :the functions called may change the lists, if we restart _every_ :time we'll traverse the same procs where p-p_wchan != ident over :and over needlessly. : :-Alfred Look at the code carefully. It's *removing* the element from the list, the conditionally restarting rather then removing the element from the list and unconditionally restarting. The only reason it works at all is because sys/queue.h does not clear out the pointers in the node that was just removed. The code is just plain wrong, though, because the queue mechanisms make no such (documented) guarentee. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, Jun 23, 2002 at 01:19:02PM -0700 I heard the voice of Paul Herman, and lo! it spake thus: I think you might have your infp confused with your outfp. It's not writing to the original live file, it's just writing the new temp file. That part of the code is OK. I'm talking about down around line 177 thru (unpatched), where it's copying back to the primary file. As it stands now (I hadn't realized in my prior reading) it's line-by-line'ing it: /* * Copy data back into the * original file and truncate */ rewind(infp); rewind(outfp); while (fgets(line, linesize, outfp) != NULL) fputs(line, infp); /* * If there was a problem with copying * we will just rename 'file.new' * to 'file'. * This is a gross hack, but we may have * corrupted the original file * Unfortunately, it will lose the inode * and hence the lock. */ if (fflush(infp) == EOF || ferror(infp)) rename(file, filename); else ftruncate(infd, ftell(infp)); which is Very Bad (tm) in that it's not very resilient against system crashes. The rename() solution is much safer. As for the giant lock, it would be like closing down the entire airport after someone finds a knife in the snack bar, and in my opinion an this is an unwise and brutal thing to do. No, it's more like closing the snack bar while you clean it, rather than closing the first steam tray while cleaning that, then the second steam tray, then the third booth on the left side, then... That way, when you open the snack bar, you know that the whole thing is clean (or, depending on your skill and work ethic, at least equally dirty ;). In many passwd-manipulations, the file can't be safely modified in-place, so it has to be done out-of-place, then atomically shoved in, which requires a rename() or the ilk; the downside is that you can't flock() the file in question and handle that easily. One also has to consider the impact with other programs, potentially not written by us, that modify things; a wrapper lock on changes to the auth information is really the only way to ensure consistency; otherwise, you can't REALLY be sure that the passwd and group files are in sync, to say nothing of preventing crashes from mangling things. Not to mention stale /var/run/ files that might be lying around... See the code I put in pid_begin() to handle just that case, that being the reason the function was written in the first place (otherwise, I could dispose of a lot of mess, and just use an empty file as a mutex for it). I do appreciate your work, but please don't commit this until the real issue is solved, and the dust settles. There still may be some use for it... plenty of time until the next release cycle. Commit? How would I do that? :P -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ The only reason I'm burning my candle at both ends, is because I haven't figured out how to light the middle yet To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
Terry, Patrick, et al, What is the procedure in 4.5-RELEASE (please say just change KVA_PAGES=260 to KVA_PAGES=512) (snip) For 4.5, you have to hack ldscript.i386 and pmap.h. I've posted on how to do this before (should be in the archives). Actually, in 4.5 you only need to set: options KVA_PAGES=512 and recompile your kernel. It looks like 4.5-RELEASE was the first release version to _not_ require hacking sys/i386/include/pmap.h and sys/conf/ldscript.i386. As you can see by looking at a 4.5-RELEASE pmap.h: #define NKPDE (KVA_PAGES - 2) /* addressable number of page tables/pde's */ #else #define NKPDE (KVA_PAGES - 1) /* addressable number of page tables/pde's */ the offsets that Terry spoke of are already in place. This is in contrast to 4.4-RELEASE: #define NKPDE 254 /* addressable number of page tables/pde's */ #else #define NKPDE 255 /* addressable number of page tables/pde's */ Where everything was hard coded to match the default KVA_PAGES value. Further, looking at ldscript.i386 we see in 4.5-RELEASE: . = kernbase + 0x0010 + SIZEOF_HEADERS; whereas in 4.4-RELEASE and earlier, we saw: . = 0xc010 + SIZEOF_HEADERS; Which means that in 4.4 you had to change 0xc010 to 0x8010 for a 2gig KVA. In 4.5, however, you don't have to change ldscript.i386 at all, because it is now a relative value that takes kernbase into account. - So, if you are running 4.0 - 4.4, you need to edit ldscript.i386 and change 0xc010 to 0x801 (for a 2gig KVA), then you need to edit pmap.h and change the two lines I pasted above from 254 and 255 to 510 and 511, respectively. Finally, you need to set: options KVA_PAGES=512 in your kernel config, then recompile your kernel. But, if you are running 4.5 or 4.6, from the code I pasted above, it looks like all you have to do is set: options KVA_PAGES=512 in your kernel config, then recompile your kernel. - Another explanation of this concept can be found here: http://www.kozubik.com/docs/original_kva_increase.txt I am posting today mainly to get a little more information stored in the archives. In addition, I myself have a question regarding the default settings of 4.5 and 4.6 - by looking at the NKPDE values in the 4.4-RELEASE version of pmap.h, the values of 254 and 255 indicate that they are hard coded for a default of KVA_PAGES=256, however 4.5 and 4.6 have a KVA_PAGES=260 setting in LINT, which I assume is also the default ... why the increase of 4 since 4.4-RELEASE ? - John Kozubik - [EMAIL PROTECTED] - http://www.kozubik.com The pages are all going to be off-by-one from your calculations, for the recursive page mapping, or off-by-two if your kernel is an SMP kernel, for the per CPU page, so remember that, or you will end up with a kernel that simply doesn't boot. The easiest way is to look at the numbers in pmap.h, and figure out how they relate to 0xc000 (remember to OR in 0x0010 after your math, to count the kernel loading at 1M). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
In article
local.mail.freebsd-hackers/[EMAIL PROTECTED] you
write:
:I'm pretty sure you only need to 'goto restart' if you call into
:maybe_resched() as someone else may have manipulated the queues.
:
:The 'restart' label is only in there for restarting in case one of
:the functions called may change the lists, if we restart _every_
:time we'll traverse the same procs where p-p_wchan != ident over
:and over needlessly.
:
:-Alfred
Look at the code carefully. It's *removing* the element from the list,
the conditionally restarting rather then removing the element from the
list and unconditionally restarting. The only reason it works at all
is because sys/queue.h does not clear out the pointers in the node
that was just removed. The code is just plain wrong, though, because
the queue mechanisms make no such (documented) guarentee.
Looks like the original damage happened in r1.21, where the temporary
variable (used to hold the next item on the list) was replaced by a
dereference through the pointer of the item that was just removed.
The code works simply because it relies TAILQ_REMOVE() not changing
the tqe_next pointer. I suppose that this should either be documented,
or the loop changed back to use a temp variable:
for (td = TAILQ_FIRST(qp); td != NULL; td = tdq) {
tdq = TAILQ_NEXT(td, td_slpq);
...
}
--
Jonathan
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
- So, if you are running 4.0 - 4.4, you need to edit ldscript.i386 and change 0xc010 to 0x801 (for a 2gig KVA), then you need to edit pmap.h and change the two lines I pasted above from 254 and 255 to 510 and 511, respectively. Finally, you need to set: options KVA_PAGES=512 An addendum - skip that last step (setting options KVA_PAGES=512 in your kernel config) for versions 4.0-4.4, as it did not yet exist as a config option at that time. Again, for 4.5 and 4.6, adding that line to your kernel config is _all_ you need to do. If you are reading this from the archives, please see my previous post in this thread for specific details. - John Kozubik - [EMAIL PROTECTED] - http://www.kozubik.com To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: FreeBSD 2.2.x ISO images.
On Sun, Jun 23, 2002 at 06:19:43PM +, Jefferson Harlough wrote: Where might I find ISO images for the FreeBSD 2.2.x releases? Do such files exist? I have an older system with a non-IDE Creative CD-ROM drive, and FreeBSD 4.x seems to not support that drive any more. I do have several FreeBSD 3.x releases, but they always hang with a kernel panic when booting via the included bootdisks. Would the FreeBSD 2.2.x series of releases work with such a CD-ROM drive? don't know, but I have all 2.2.8 CDROMs at home. Cyrille. -- Cyrille Lefevre mailto:[EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, 23 Jun 2002, Matthew D. Fuller wrote: I'm talking about down around line 177 thru (unpatched), where it's copying back to the primary file. [...] which is Very Bad (tm) in that it's not very resilient against system crashes. The rename() solution is much safer. Ah, now I see, thanks. No contention there, we are in total agreement that this is a Bad Thing, and that rename() is sufficient. No, it's more like closing the snack bar while you clean it, rather than closing the first steam tray while cleaning that, then the second steam tray, then the third booth on the left side, then... OK, you can change the analogy, but you're still closing the snack bar, and you turn away customers. Most 24 hour snack bars clean each table at a time as each one frees up. Moral of that story: This would mean for pw(8) that I can't update the system passwords and the passwords in all my jails at the same time. There is no reason to enforce that policy. Back to the situation at hand. You make a few points that should be clarified, I'll summarize them in QA format: Q: rename() is good, but you can't flock() the file in question and handle it easily, can you? A: Indeed, you can. flocks() are even preserved across renames. Consider: lockf -k foo sleep 60 mv foo bar lockf -t 1 -k bar true The second lockf will error. Q: What about other programs not written by us that modify things? A: Agreed, this is a problem. However, no system in place will prevent at any time anyone with the proper credentials from doing: echo junk /etc/master.passwd At least if the programs in the base agree on a system (be it flock() or /var/run lock) it will work for the base system. This doesn't speak for or against any one method. Yes, a lock on changes to the auth information is the only way to ensure consistency, which is what this is all about, but HOW to accomplish this issue at hand. Commit? How would I do that? :P Sorry, it wasn't as much directed at you as it was more of a general plea. Kind of like this one: Why use lockfiles, when we can lock files? Let's lock files and keep the snack bar open! :-) -Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
OK, this is the end for me today. I'm fairly sure, somehow, it's time to
get some sleep ;p
On Sun, Jun 23, 2002 at 03:51:46PM -0700 I heard the voice of
Paul Herman, and lo! it spake thus:
Moral of that story: This would mean for pw(8) that I can't update
the system passwords and the passwords in all my jails at the same
time. There is no reason to enforce that policy.
No, it means that (with configuration) you can only update them once at a
time in each jail.
I'm all in favor of adding options, either command line, or pw.conf (or
both) to put the lockfile somewhere else; -V ${dir} already makes pw(8)
read ${dir}/pw.conf instead of the default /etc/pw.conf, so we're 90% of
the way there already. Heck, I wouldn't be too worried about adjusting
the lockfile location based on -V automagically. I'm also not opposed to
adding an option to not bother with this locking; I already added one to
pwd_mkdb(8) (primarily because it's called from libutil's pw_mkdb(3),
which is called by vipw(8)... deadlocks suck). I just haven't done
either NOW; nor have I yet added options to 'block on the lock and keep
trying' (or decided to make blocking the default and have a switch for
non-blocking) - this is experimental implementation stage, after all.
I don't claim this is perfect for every case; I'm just taking aim at the
common case, where it's currently far too easy to accidentally screw
yourself in the foot.
Q: rename() is good, but you can't flock() the file in question
and handle it easily, can you?
A: Indeed, you can. flocks() are even preserved across renames.
Yes, I've seen that.
A: Agreed, this is a problem. However, no system in place will
prevent at any time anyone with the proper credentials from
doing:
I was aiming more at:
User A adds user with pw(8)
* pw(8) gets together stuff, internally adds user and group
* pw(8) builds homedirs, skel files, changes ownership
* pw(8) opens and locks master.passwd (and assorted temp files) and
starts rewriting
- User B adds group
% User B's pw(8) (or other) invocation adds group with GID that User
A's pw(8) had already assigned and chmod'd for user above
* User A's pw(8) either errors out leaving job half done, or adds group
with duplicate GID, or (not coded) backs out everything it's done and
starts over, or (ditto) backs out everything it's done and errors
back to User A, who screams and bitches and dumps User B into the
East River after sundown.
Similar situations can be thought up. My POV on this is that more than 1
file has to be coordinated here, so why not just make 1 file responsible
for the coordination? I don't care THAT much if it's called
/var/run/pwd.lock or /etc/passwd.lock or /kernel (I suggest testing the
latter on a scratch system). It just seems to me the simplest and least
error-prone way of doing it.
Yes, a lock on changes to the auth information is the only
way to ensure consistency, which is what this is all about,
but HOW to accomplish this issue at hand.
Now, mind you; were this a high-traffic subsystem, where one would
normally have 2 or 3 or 4 requests (changes) outstanding, with a
constant stream coming in keeping the queue full, I'd be out there
screaming for fine-grained locking of individual pieces of it. But
really; how many systems are there (that are using stock passwd and
getpw* and pw/pwd_mkdb/etc) that are adding users at even a significant
fraction of the speed the system can process them? A coarse lock is
easier on the programmer (especially being as we have quickie functions
that do the job for them), the admin (easier to see an extra lock file,
then a hung process holding a flock()), and pretty much everything else.
Why use lockfiles, when we can lock files? Let's lock files and
keep the snack bar open! :-)
We are locking a file :)
We're just locking one file for the whole subsystem, so the pieces can be
assured of being in sync.
So, where's a snack bar with no CVS repository or text editor or diff(1);
I need a vacation!
--
Matthew Fuller (MF4839) | [EMAIL PROTECTED]
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
The only reason I'm burning my candle at both ends, is because I
haven't figured out how to light the middle yet
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
* Matthew Dillon [EMAIL PROTECTED] [020623 13:32] wrote: :I'm pretty sure you only need to 'goto restart' if you call into :maybe_resched() as someone else may have manipulated the queues. : :The 'restart' label is only in there for restarting in case one of :the functions called may change the lists, if we restart _every_ :time we'll traverse the same procs where p-p_wchan != ident over :and over needlessly. : :-Alfred Look at the code carefully. It's *removing* the element from the list, the conditionally restarting rather then removing the element from the --^^^ then?? list and unconditionally restarting. The only reason it works at all is because sys/queue.h does not clear out the pointers in the node that was just removed. The code is just plain wrong, though, because the queue mechanisms make no such (documented) guarentee. You're right, but other than await() why would a process find itself on a sleep queue if not in SSLEEP? -- -Alfred Perlstein [[EMAIL PROTECTED]] 'Instead of asking why a piece of software is using 1970s technology, start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
: :You're right, but other than await() why would a process find itself :on a sleep queue if not in SSLEEP? : :-- :-Alfred Perlstein [[EMAIL PROTECTED]] I think we may be ok on -stable, but -current definitely hits the case (at least it does with Julian's KSE branch). Threads have considerably more blocking states in -current then processes do in -stable. Either that or there are races in -current that we don't know about. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: FreeBSD 2.2.x ISO images.
Jefferson Harlough wrote: Where might I find ISO images for the FreeBSD 2.2.x releases? Do such files exist? I have an older system with a non-IDE Creative CD-ROM drive, and FreeBSD 4.x seems to not support that drive any more. I do have several FreeBSD 3.x releases, but they always hang with a kernel panic when booting via the included bootdisks. Would the FreeBSD 2.2.x series of releases work with such a CD-ROM drive? The Silicon Valley BSD User's Group has legacy images of all versions of FreeBSD, available as ISO's. The project is mentioned at: http://www.svbug.com/projects/ Unfortunately, I don't have a link to the project itself. If you need it, you should contact Jesus Monroy or John Sokol, who are the people who actively maintain the archive. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
Paul Herman wrote: Fine, then lock them both, and use rename(). Patch attached. In fact, if you look at fileupdate(), you see that it already gains an exclusive lock on the temp file, but not the original /etc/master.passwd (if you will.) I think this is a bug, because the original is getting modified (at least in name space), so that should be locked while pw(8) is operating. What do you think? Don't use an exclusive open. Use a standard lock file lock mechanism. The problem with your proposed patch is that it breaks the ability to allow authentication against the database while it is undergoing modification, which may be a prolonged period. The correct way to deal with this is to make a hard link against the file, verify that the link count is exactly 2, and if it's not, back off and fail. You are only permitted to write the file if you are the one that made the second link. Since both rename and link are atomic operations in the directory, you are safe. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
Patrick Thomas wrote: Because I am paranoid, I like to check the state of a measurement before making a change and then after, to see that what I did did indeed induce a change ... I have this irrational fear that sometimes I make changes like this and nothing in fact changed, and I just don't know it :) So, should I just look for the value of: vm.zone_kmem_kvaspace: 179691520 to increase in size even though the physical RAM stays the same at 3gigs, or is there some other measurement I should look at before and after the KVA increase to ensure that it worked (and yes, I know that if it doesn't work I probably will have an inoperable machine, but just out of curiousity...) Yes. You will also see the kernel load address during the boot process, which you can interrupt/pause until you are satisfied. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, 23 Jun 2002, Matthew D. Fuller wrote: I don't claim this is perfect for every case; I'm just taking aim at the common case, where it's currently far too easy to accidentally screw yourself in the foot. ...and have the screw (or bullet) delivered as precisely, quickly and efficiently as possible. Clearly, at the root of our disagreement is what we both perceive the problem to be. I don't see problems in the current implementation, aside from bugs that lead to unexpected behavior, i.e. passwd file corruption. You see the problem as a deficiency in the implementation itself, and wish to protect the user from shooting themselves in the foot. Not only do I think that's impossible[*], I choose to fight for my right to shoot myself in the foot as quickly and efficiently as possible, but that's where we'll disagree, and I'll just leave it at that and wish you a good night's sleep. -Paul. [*] Patch to vi refusing to edit filenames containing master.passwd withheld by request. ;-) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
Jonathan Lemon wrote:
Look at the code carefully. It's *removing* the element from the list,
the conditionally restarting rather then removing the element from the
list and unconditionally restarting. The only reason it works at all
is because sys/queue.h does not clear out the pointers in the node
that was just removed. The code is just plain wrong, though, because
the queue mechanisms make no such (documented) guarentee.
Looks like the original damage happened in r1.21, where the temporary
variable (used to hold the next item on the list) was replaced by a
dereference through the pointer of the item that was just removed.
The code works simply because it relies TAILQ_REMOVE() not changing
the tqe_next pointer. I suppose that this should either be documented,
or the loop changed back to use a temp variable:
for (td = TAILQ_FIRST(qp); td != NULL; td = tdq) {
tdq = TAILQ_NEXT(td, td_slpq);
...
}
Too bad the first argument to TAILQ_FOREACH isn't a pointer to a
pointer, instead of just a pointer. I guess the double dereference
is considered too expensive.
It's a bummer that you can't safely use TAILQ_REMOVE inside the
TAILQ_FOREACH.
FWIW, this looks like a general bug in queue.h for all queue and
list types, so there are probably other places it would be an issue.
8-(.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: (jail) problem and a (possible) solution ?
John Kozubik wrote: Terry, Patrick, et al, For 4.5, you have to hack ldscript.i386 and pmap.h. I've posted on how to do this before (should be in the archives). Actually, in 4.5 you only need to set: options KVA_PAGES=512 and recompile your kernel. It looks like 4.5-RELEASE was the first release version to _not_ require hacking sys/i386/include/pmap.h and sys/conf/ldscript.i386. As you can see by looking at a 4.5-RELEASE pmap.h: #define NKPDE (KVA_PAGES - 2) /* addressable number of page tables/pde's */ #else #define NKPDE (KVA_PAGES - 1) /* addressable number of page tables/pde's */ the offsets that Terry spoke of are already in place. This is in contrast to 4.4-RELEASE: #define NKPDE 254 /* addressable number of page tables/pde's */ #else #define NKPDE 255 /* addressable number of page tables/pde's */ Yes; this is 1.65.2.3 This is my bad. It's the system I was using as a reference; it has two kernel source trees; the first one has 1.65, the second is a RELENG_4, which makes it a 1.65.2.3. Where everything was hard coded to match the default KVA_PAGES value. Further, looking at ldscript.i386 we see in 4.5-RELEASE: . = kernbase + 0x0010 + SIZEOF_HEADERS; whereas in 4.4-RELEASE and earlier, we saw: . = 0xc010 + SIZEOF_HEADERS; Which means that in 4.4 you had to change 0xc010 to 0x8010 for a 2gig KVA. In 4.5, however, you don't have to change ldscript.i386 at all, because it is now a relative value that takes kernbase into account. Yes, this is 1.4.2.1. The commit comments for ldscript.i386 are incredibly misleading as to what the merge actually does. The derivation of kernbase itself is also dependent on a third change, which is not documented, either. So, if you are running 4.0 - 4.4, you need to edit ldscript.i386 and change 0xc010 to 0x801 (for a 2gig KVA), then you need to edit pmap.h and change the two lines I pasted above from 254 and 255 to 510 and 511, respectively. Finally, you need to set: options KVA_PAGES=512 in your kernel config, then recompile your kernel. But, if you are running 4.5 or 4.6, from the code I pasted above, it looks like all you have to do is set: options KVA_PAGES=512 in your kernel config, then recompile your kernel. - Another explanation of this concept can be found here: http://www.kozubik.com/docs/original_kva_increase.txt I am posting today mainly to get a little more information stored in the archives. In addition, I myself have a question regarding the default settings of 4.5 and 4.6 - by looking at the NKPDE values in the 4.4-RELEASE version of pmap.h, the values of 254 and 255 indicate that they are hard coded for a default of KVA_PAGES=256, however 4.5 and 4.6 have a KVA_PAGES=260 setting in LINT, which I assume is also the default ... why the increase of 4 since 4.4-RELEASE ? I believe that this would be because of the desire for the number of *usable* pages, since you have to subtract out the ones that are not global to all CPUs. The LINT value is *not* the default. It went in in 1.954 of NOTES (LINT is a generated file). I don't know why Peter did this. It says and a test in the commit, an since it's only comments and the option itself, I guess that means that the value of 260 is the test that the commit message was referencing. So I guess 4.5 is actually OK, but one of my local boxes is not. My main frustration with this has always been that the information in the Handbook has always been insufficient to actually make the change and have it work. I guess I'm glad that it made it into 4.5, even if it surprised me. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
On Sun, 23 Jun 2002, Matthew Dillon wrote: :I'm pretty sure you only need to 'goto restart' if you call into :maybe_resched() as someone else may have manipulated the queues. : :The 'restart' label is only in there for restarting in case one of :the functions called may change the lists, if we restart _every_ :time we'll traverse the same procs where p-p_wchan != ident over :and over needlessly. : :-Alfred Look at the code carefully. It's *removing* the element from the list, the conditionally restarting rather then removing the element from the list and unconditionally restarting. The only reason it works at all is because sys/queue.h does not clear out the pointers in the node that was just removed. The code is just plain wrong, though, because the queue mechanisms make no such (documented) guarentee. u Was this found because of my tailq debugging stuff that sets the forward pointer to -1? -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
On Sun, 23 Jun 2002, Jonathan Lemon wrote:
In article
local.mail.freebsd-hackers/[EMAIL PROTECTED] you
write:
:I'm pretty sure you only need to 'goto restart' if you call into
:maybe_resched() as someone else may have manipulated the queues.
:
:The 'restart' label is only in there for restarting in case one of
:the functions called may change the lists, if we restart _every_
:time we'll traverse the same procs where p-p_wchan != ident over
:and over needlessly.
:
:-Alfred
Look at the code carefully. It's *removing* the element from the list,
the conditionally restarting rather then removing the element from the
list and unconditionally restarting. The only reason it works at all
is because sys/queue.h does not clear out the pointers in the node
that was just removed. The code is just plain wrong, though, because
the queue mechanisms make no such (documented) guarentee.
Looks like the original damage happened in r1.21, where the temporary
variable (used to hold the next item on the list) was replaced by a
dereference through the pointer of the item that was just removed.
The code works simply because it relies TAILQ_REMOVE() not changing
the tqe_next pointer. I suppose that this should either be documented,
or the loop changed back to use a temp variable:
for (td = TAILQ_FIRST(qp); td != NULL; td = tdq) {
tdq = TAILQ_NEXT(td, td_slpq);
...
}
I just added debug code in the TAILQ code that sets the forward pointor
to -1. Since Matt had this it's possible that this is what hit him?
I do this to stop people accessingthings that they shouldn't be counting
on..
--
Jonathan
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
On Sun, Jun 23, 2002 at 06:42:43PM -0700, Julian Elischer wrote:
On Sun, 23 Jun 2002, Jonathan Lemon wrote:
The code works simply because it relies TAILQ_REMOVE() not changing
the tqe_next pointer. I suppose that this should either be documented,
or the loop changed back to use a temp variable:
for (td = TAILQ_FIRST(qp); td != NULL; td = tdq) {
tdq = TAILQ_NEXT(td, td_slpq);
...
}
I just added debug code in the TAILQ code that sets the forward pointor
to -1. Since Matt had this it's possible that this is what hit him?
Most definitely. If you reset tqe_next, the code stops working. Try
the patch below.
--
Jonathan
Index: kern_synch.c
===
RCS file: /ncvs/src/sys/kern/kern_synch.c,v
retrieving revision 1.175
diff -u -r1.175 kern_synch.c
--- kern_synch.c7 Jun 2002 05:39:16 - 1.175
+++ kern_synch.c24 Jun 2002 02:42:40 -
@@ -605,13 +605,14 @@
register void *ident;
{
register struct slpquehead *qp;
- register struct thread *td;
+ register struct thread *td, *tdn;
struct proc *p;
mtx_lock_spin(sched_lock);
qp = slpque[LOOKUP(ident)];
restart:
- TAILQ_FOREACH(td, qp, td_slpq) {
+ for (td = TAILQ_FIRST(qp); td != NULL; td = tdn) {
+ tdn = TAILQ_NEXT(td, td_slpq);
p = td-td_proc;
if (td-td_wchan == ident) {
TAILQ_REMOVE(qp, td, td_slpq);
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
: I just added debug code in the TAILQ code that sets the forward pointor : to -1. Since Matt had this it's possible that this is what hit him? : :Most definitely. If you reset tqe_next, the code stops working. Try :the patch below. :-- :Jonathan : :Index: kern_synch.c :=== :RCS file: /ncvs/src/sys/kern/kern_synch.c,v :retrieving revision 1.175 :diff -u -r1.175 kern_synch.c :--- kern_synch.c 7 Jun 2002 05:39:16 - 1.175 :+++ kern_synch.c 24 Jun 2002 02:42:40 - :@@ -605,13 +605,14 @@ : register void *ident; :.. I already threw a quick fix into Julian's KSE tree. The real fix (Tor's patch) went into -current earlier today. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
: : the conditionally restarting rather then removing the element from the : list and unconditionally restarting. The only reason it works at all : is because sys/queue.h does not clear out the pointers in the node : that was just removed. The code is just plain wrong, though, because : the queue mechanisms make no such (documented) guarentee. : :u Was this found because of my tailq debugging stuff :that sets the forward pointer to -1? : Yah, that's what found it. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Limiting clients per source IP address (ftpd, inetd, etc.)
Hi,
On Thu, 20 Jun 2002 20:25:28 -0700
Terry Lambert [EMAIL PROTECTED] said:
tlambert2 Giorgos Keramidas wrote:
I've been thinking for quite some time to add per-client-IP limiting
to ftpd, and I had almost decided upon something like the following,
where each child of ftpd has two numbers associated with it. The
client IP address, and the PID of the ftpd child that serves it. The
hash at the beginning of the lists serves as a minor assistance in
splitting the 2^32 address space in smaller chunks so that we don't
end up with a singly linked list of a few thousand entries.
tlambert2 Someone just did something similar for inetd (per IP per port).
Yes, it's me. I already rewrote my patch to use open hash as you
mentioned. My patch is in testing on snapshots.jp.FreeBSD.org (thank
you Matusita-san).
You can find my patch from:
http://www.imasy.or.jp/~ume/FreeBSD/inetd-perip-hash-5c.diff (for 5-CURRENT)
http://www.imasy.or.jp/~ume/FreeBSD/inetd-perip-hash-4s.diff (for 4-STABLE)
Sincerely,
--
Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan
[EMAIL PROTECTED] [EMAIL PROTECTED] ume@{,jp.}FreeBSD.org
http://www.imasy.org/~ume/
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
Matthew D. Fuller writes: ... Why use lockfiles, when we can lock files? Let's lock files and keep the snack bar open! :-) We are locking a file :) We're just locking one file for the whole subsystem, so the pieces can be assured of being in sync. Time to implement hierarchic lock. hlock dir lead to fail hlock every file in that dir with ref count == 1 ?? Extremly unefficient -- @BABOLO http://links.ru/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Bug in wakeup() (stable and current) ?
yep
Matt added a similar patch to the KSE branch...
the debug stuff I added to TAILQ
adds (when compiled in) info on the last 2 places to to linkage
operations..
it also initialises the pointers on elements to -1
it's great for helping find places that are screwing up linkages
e.g. in the KSE run queues which are quite complicated :-)
On Sun, 23 Jun 2002, Jonathan Lemon wrote:
On Sun, Jun 23, 2002 at 06:42:43PM -0700, Julian Elischer wrote:
On Sun, 23 Jun 2002, Jonathan Lemon wrote:
The code works simply because it relies TAILQ_REMOVE() not changing
the tqe_next pointer. I suppose that this should either be documented,
or the loop changed back to use a temp variable:
for (td = TAILQ_FIRST(qp); td != NULL; td = tdq) {
tdq = TAILQ_NEXT(td, td_slpq);
...
}
I just added debug code in the TAILQ code that sets the forward pointor
to -1. Since Matt had this it's possible that this is what hit him?
Most definitely. If you reset tqe_next, the code stops working. Try
the patch below.
--
Jonathan
Index: kern_synch.c
===
RCS file: /ncvs/src/sys/kern/kern_synch.c,v
retrieving revision 1.175
diff -u -r1.175 kern_synch.c
--- kern_synch.c 7 Jun 2002 05:39:16 - 1.175
+++ kern_synch.c 24 Jun 2002 02:42:40 -
@@ -605,13 +605,14 @@
register void *ident;
{
register struct slpquehead *qp;
- register struct thread *td;
+ register struct thread *td, *tdn;
struct proc *p;
mtx_lock_spin(sched_lock);
qp = slpque[LOOKUP(ident)];
restart:
- TAILQ_FOREACH(td, qp, td_slpq) {
+ for (td = TAILQ_FIRST(qp); td != NULL; td = tdn) {
+ tdn = TAILQ_NEXT(td, td_slpq);
p = td-td_proc;
if (td-td_wchan == ident) {
TAILQ_REMOVE(qp, td, td_slpq);
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: bug in pw, -STABLE [patch]
On Sun, 23 Jun 2002, Paul Herman wrote: You see the problem as a deficiency in the implementation itself, and wish to protect the user from shooting themselves in the foot. Not only do I think that's impossible[*], I choose to fight for my right to shoot myself in the foot as quickly and efficiently as possible, but that's where we'll disagree, and I'll just leave it at that and wish you a good night's sleep. Not to be snooty, but there will always be plenty of ways for you to shoot yourself in the foot if you want to play that way. For regular users of pw(8) who don't want to get shot, it needs to be fixed when used within its intended bounds. As far as I'm concerned, it comes down to what is most feasible to implement correctly without an unreasonably large effort. In reality, it comes down to how a committer would like to see it work, since the committers are the ones who ultimately have to deal with and are responsible for the change. So we either need to have a compelling solution or get a committer to step in and make up our minds for us. Geoff To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
subscribe
To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: FreeBSD 2.2.x ISO images.
- Original Message - From: Jefferson Harlough [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Sunday, June 23, 2002 12:19 PM Subject: FreeBSD 2.2.x ISO images. Where might I find ISO images for the FreeBSD 2.2.x releases? Do such files exist? I have an older system with a non-IDE Creative CD-ROM drive, and FreeBSD 4.x seems to not support that drive any more. I do have several FreeBSD 3.x releases, but they always hang with a kernel panic when booting via the included bootdisks. Would the FreeBSD 2.2.x series of releases work with such a CD-ROM drive? You CD-ROM is _probably_ usable under 4.x with one of either the mcd, scd, or matcd drivers. These have been removed from the GENERIC kernel, but are still available as options for a custom kernel. See their respective manpages (as well as LINT) for more info. JN To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
