Re: more amd hangs: problem really in syslog?
Mike Smith wrote: On Tue, 13 Jul 1999, Mike Smith wrote: 'siobi' is someone trying to open the serial console, for whatever reason. Without knowing who it was that was stuck there, it's hard to guess what is going on. D'uh, sorry. Long day. It was amd that was hung in the siobi state. No way to clear it without rebooting the box. Dang. Now I need that stack dump from amd that you posted and I deleted. OK, sent under different cover. Specifically, it'd be handy to know why amd felt it was necessary to open the console. Yeah, I'm kind of curious myself. BTW, I was going to work on this some more today, but the boss thought that putting the box into production was more important. The good news is that under real world load my freebsd box had 20-40% free cpu and a load average of 1.5. With load as equal as the switch could make it, the linux box had no free cpu and a load average of 8. :) I also (finally) got the approval to install freebsd on the fourth box (there are already two linux machines up) so A) I'm making progress in the office, and B) I should have a chance to pound on the syslog stuff tomorrow. Happy, Doug To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
Hi everyone, I've been following this discussion almost from the beginning, and I have the feeling that we're not _really_ getting very far. There's good arguments for and against overcommit, depending on your point of view and your requirements. What I do see is a not-so-openly voiced consent that the way resource(sp?) shortages are handled in an overcommitting system (SIGKILL) makes some of us rather unhappy. I therefore suggest those of us who would like to see a change in this area pool their efforts and energies to work on a mechanism that handles resource shortage in a more graceful way. cheerio Michael -- [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: matcd on an SB16
Mike Smith scribbled this message on Jul 15: Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It "works" for some definitions of "work". Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). actually, only certain drives are supported on the interfaces... this is from -current's LINT: # for the Sony CDU31/33A CDROM device scd0at isa? port 0x230 # matcd: Matsushita/Panasonic CD-ROM # for the SoundBlaster 16 multicd - up to 4 devices controller matcd0 at isa? port 0x230 # mcd: Mitsumi CD-ROM device mcd0at isa? port 0x300 irq 10 so, we support all three different interfaces, but the support is quite weak... I'm not even sure if these even work anymore, do people have some hardware they run tests on these interfaces? Presuming the drive is a real Matsushita, and it's one of the few listed in the matcd manpage, you should be able to get it to work by following the manpage carefully. as for getting the code to work, that completely depends upon the code, and if you have the specs for the drive... it'd probably be cheaper to just by a new ide cdrom drive... and a heck of a lot faster... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 "The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought." -- Ralph Waldo Emerson To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: matcd on an SB16
At 01:05 15/07/99 -0700, John-Mark Gurney wrote: Mike Smith scribbled this message on Jul 15: Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It "works" for some definitions of "work". Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). actually, only certain drives are supported on the interfaces... this is from -current's LINT: # for the Sony CDU31/33A CDROM device scd0at isa? port 0x230 # matcd: Matsushita/Panasonic CD-ROM # for the SoundBlaster 16 multicd - up to 4 devices controller matcd0 at isa? port 0x230 # mcd: Mitsumi CD-ROM device mcd0at isa? port 0x300 irq 10 so, we support all three different interfaces, but the support is quite weak... I'm not even sure if these even work anymore, do people have some hardware they run tests on these interfaces? I have an mcd (with a Mitsumi controller, not SB16) on a spammable box, it worked last time I tried (2.2.8R). [But that wasn't the original question.] -- Bob Bishop +44 118 977 4017 [EMAIL PROTECTED]fax +44 118 989 4254 (0800-1800 UK) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: seg fault in mutex_queue_enq
Thomas Gellekum [EMAIL PROTECTED] wrote: Daniel Eischen [EMAIL PROTECTED] writes: There are some bugs in libc_r in stable that have been fixed in -current. I think the one that you've hit is an uninitialized TAILQ_HEAD in a statically declared mutex (in localtime). It's probably about time for a MFC. If someone wants to give me the go-ahead, I can do it... Just Do It (tm). At least you could merge the change from [pthread_private.h] 1.20 Sun Jun 20 8:28:08 1999 UTC by jb Diffs to 1.19 In the words of the author: o The polling mechanism for I/O readiness was changed from select() to poll(). In additon, a wrapped version of poll() is now provided. [...] plus the later formatting fixes. The libc_r version number was bumped in -current because of the addition of poll(). Is this allowed in -stable, or something that waits for a -RELEASE? As an aside: some files still lack $Id$'s, and someone should `s/REGENTS/AUTHOR/' in all the copyright statements. OK. Dan Eischen [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: seg fault in mutex_queue_enq
Daniel Eischen [EMAIL PROTECTED] writes: The libc_r version number was bumped in -current because of the addition of poll(). Is this allowed in -stable, or something that waits for a -RELEASE? Jordan should have to say something about this. AFAIR, bumps are allowed but only by one between releases. We will have to provide libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this anyway by the time 4.x is released). tg To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
NFS problems due to getcwd/realpath
Hi everybody, after wondering for two years why FreeBSD (2.2.x ... 3.2) might lock up when an NFS server is down, I think I have found one reason for that (see kern/12609 - I now know it doesn't belong to kern - sorry). It is the implementation of getcwd (src/lib/libc/gen/getcwd.c). When examining the parent dir of a mounted filesystem, getcwd lstats every directory entry prior to the mountpoint to find out the name of the mountpoint (but it would only need the inodes's device to do a rough check). If one of the prior entries point to another NFS mountpoint and that one is down, getcwd will wait till the mountpoint is up again This of course applies to all routines which use getcwd, e.g. realpath. This is especially funny since mountd calls realpath (from the RPC handler) to check mount points, so when to machines mount dirs from each other, they can lock up, e.g. at boottime (see kern/12609...) I don't fully understand whether the problem is still present in 3.x, since getcwd may call __getcwd to do the job, but as I understand from the sources, __getcwd may fail and then you're back with the problem. Anyhow, how can this be resolved (except for symlinking all mountpoints)? Must getcwd really do an lstat to find out an inodes device?? Is there no other syscall to do that? (I mean: this information must be present somewhere, without going over the net, right?) Unfortunately I don't now such a syscall. In my opinion getcwd should be implemented differently, but maybe some people have a differen opinion on that (And I am not sure how to do that properly). Any suggestions? best regards Jan -- Physikalisches Institut der Universitaet Bonn Nussallee 12 D-53115 Bonn GERMANY To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Replacement for grep(1) (part 2)
Kevin Schoedel wrote: Imagine a reasonably big program, like Netscape or Emacs, of which you usually just use a subset of features. There can easily be many megabytes of code and data in them you never actually use, or you don't _usually_ use (like the people who use emacs like it was vi :). Without overcommit, you need to allocate all that memory for the code, no matter whether you end up using it or not. With overcommit, there is no such problem. Code, static data, and not-yet-written writable data should be backed by the executable file, not by swap space, so unused code and tables should not be a problem. TEXT should be backed by the executable, as long a the program doesn't change it to read/write. That's not the code I was refering to. Not-yet-written blah-blah-blah should be backed by: 1) The executable file if you are overcommitting. 2) RAM/Swap if you are not. If you don't do this, you are overcommitting. Proof: let the system exaust it's memory. Change a single byte in the not-yet-written stuff. Now you need more memory than you have to comply with a regular operation (like changing the value of a global variable), which means you overcommitted. Now comes the people saying "don't overcommit in *this* case, and overcommit in *that* case". Irrelevant. Programs are still getting killed because memory was overcommitted (with the added disadvantage of you not having as much memory as in a full overcommit mode). Stack is more interesting. There might be a place for a global overcommit switch. I think I'd be happier with a scheme in which stack the first page or first few pages are committed (so that reasonable programs will never run into trouble) and remaining stack is over-/un-committed by default, along with means for unusual programs to commit (and/or test commitability of) subsequent pages. Eh? Reasonable programs *never* run into trouble. Trouble only happens when you have unreasonable programs around, or did not configure the system correctly. And if you did not configure the system correctly, why do you think you would be able to correctly estimate the stack needed for the various programs? -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] "Would you like to go out with me?" "I'd love to." "Oh, well, n... err... would you?... ahh... huh... what do I do next?" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit
[EMAIL PROTECTED] wrote: All of the arguments I've seen so far assume that one process is running off and grabbing all the available memory. That may be the most likely scenario, but it's most certainly not the *only* scenario. What if you have a whole bunch of "middle sized" processes running, all using memory efficiently, but in total using 95% of the available VM. A malloc(5*1024*1024) might work, but I need 10 MB instead of 5MB. And my memory footprint is just a little bit bigger than the other guys. Instead of returning NULL to the malloc() request, *zap* I'm dead. How can you possibly call that sensible behaviour? No process is killed at malloc() time. A process is killed when (another) process needs more memory and it is not available. Yes, the machine is under-resourced. I can't help that -- it's not my machine. The machine belongs to a customer who happens to run my IMAP software, who also happens to have ignored our sizing guidelines. In this situation I have no choice but to deal with the low memory condition, and our code does that, if it's given the chance! At least give me the opportunity to deal with the situation gracefully. If it was not for overcommit, you wouldn't be running half of what you are running in that machine in first place. So, overcommit is helping you run much more for the same resources. What if we decided to defer errors from bind just because there weren't any mbufs available, and later killed the process when it tried to do network I/O? People would howl bloody murder! (== this is rhetorical, folks) Out of mbufs does not result in system deadlock, out of memory does. The semantics of malloc() have been defined since almost the dawn of time. From the current manpage: RETURN VALUES The malloc() and calloc() functions return a pointer to the allocated memory if successful; otherwise a NULL pointer is returned. Nowhere does it say that allocated memory might not exist. Nowhere does it say that I have to touch all the allocated pages to make sure they are really there. Nowhere does it say process death at some non-deterministic time in the future might be a side effect of calling malloc(). And nowhere does it say it does not, of course. But that is beside the point. malloc() works as specified. It is the behavior of the system in a low-resource situation that leads to processes being killed. Applications are written assuming that malloc() behaves in the documented manner. It is *not* acceptable to tell applications writers Actually, applications are written assuming that malloc() will not fail, generally speaking. that they have to provide their own management routines on top of malloc() (SEGV catchers and the like) if they want the long standing semantics of malloc() to be preserved. If the current malloc() cannot behave in the documented and expected manner it needs to be renamed, because malloc() it most certainly isn't. It's funny how all these FreeBSD systems manage to gain such a good reputation despite such an obvious flaw, isn't it? :-) -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] "Would you like to go out with me?" "I'd love to." "Oh, well, n... err... would you?... ahh... huh... what do I do next?" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
At 6:29 PM -0700 7/14/99, Matthew Dillon wrote: If 1G isn't enough, spend another $30 and throw 2G of swap online. Or perhaps dedicate an entire $150 disk and throw 6+ GB of swap online. The equivalent setup using a non-overcommit model would require considerably more swap to have the same reliability. Please note that we're talking at cross-purposes here, mainly because I didn't realize this same general topic was being beaten to death in the 'replacement for grep' thread (which I have not been following). Speaking for just me myself and I, I have no problems with the current overcommit model. All I'd like to do is have a way to indicate which processes should not get booted first, if the system does indeed run out of swap and needs to boot some processes. However, other people seem much more worked up about this topic than I am, and thus what I (personally) meant as "just casual questions" seem to be taken as "demands that something be done, RIGHT NOW". I now realize that some people are arguing that malloc should return an error if the system runs out of space, but that's not what I am thinking about. So, I think I'll bow out of this discussion for now, and maybe try to discuss my "casual questions" sometime in a different context... --- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
make fails in 3.2-RELEASE for netboot
Been trying to get nb8390.com compiled under /usr/sys/i386/boot/netboot and every time it does this: (ns2)[6]:/usr/src/sys/i386/boot/netboot# make Warning: Object directory not changed from original /usr/src/sys/i386/boot/netboot cc -O2 -DNFS -DROMSIZE=16384 -DRELOC=0x9 -DPCI -DPCI_VENDOR=0x10ec -DPCI_DEVICE=0x8029 -DPCI_CLASS=0x02,0x00,0x00 -aout -nostdinc -I/usr/src/sys/i386/boot/netboot/../../../../include -I/usr/src/sys/i386/boot/netboot/../../.. -I/usr/src/sys/i386/boot/netboot -DROMSIZE=16384 -static -o makerom /usr/src/sys/i386/boot/netboot/makerom.c ld: scrt0.o: No such file or directory *** Error code 1 Stop. Now being the resourcful kinda guy I am I looked on another one of my box and it worked fine. On that box I have upgraded all the way from 2.7 so I'm assuming that might have something to do with it. The file scrt0.c does not exist so in a make buildworld it's not created on the new box. The source is fresh as it's a new install so I'm curious as to why the file is missing, and why this was released with netboot as stable is it doesn't compile. Second thing I'm noticing is that, I compiled nb8390.com on the older box and attempted to run it on a machine from boot time. tftp worked great and it attempts to load the kernel from my new box (doesn't matter if I use the new boxes kernel or the old boxes kernel for the client it still fails), I get the following error message: Bad kernel not a valid executable! Of course that is a rough message as I'm at work and the box I'm using it on is at home but that's the important part. It's seeing the kernel file however it's not running it. Now justa long shot but I'm guessing nb8390.com is expecting an aout kernel and it's getting an elf kernel. My last question is why did this program get released with 3.x yet not recognize elf kernels if this is the case? Greg +([EMAIL PROTECTED])--+ | Dynamic Networking Solutions InterX Technologies | | Senior Network Administratorbits/keyID 1024/7DF9C285 | | [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] | +[ DC 50 57 59 C3 76 46 E8 EB 75 A8 94 FE 96 9E D3 ]--+ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Major Device Number for High Availability Ethernet Driver
Greetings Hackers, We are in the process of releasing a FreeBSD v 3.2 Ethernet driver that detects link failures and executes failovers, supports Cisco's FEC trunking, and system-to-system trunking. To support these features, some configuration is required...the configuration utility requires a "/dev/zrm" device with a major character device #. The driver (unfortunately a binary only release) also needs to know the major device #. We would greatly appreciate some advice on which device # to use and how to let our driver know which one has been assigned. http://www.freebsd.org/FAQ/FAQ253.html says we can use character device # 32 - ... what if someone else is already using it..don't these have to be unique ? /usr/src/sys/i386/conf/majors.i386 says 200-255 are available for "local use" ... is there a way a configuration utility can determine the next available ? is there a way my driver can tell which # is used for our "/dev/zrm" character device ? ( the driver uses "cdevsw_add( )" to register routines for the device - major dev # is required ) Advice here would be greatly appreciated. Thanks Tim Tim Hayes ZNYX Corporation 187 S Patterson Suite C Santa Barbara CA 93111 [EMAIL PROTECTED] v: 805 683 8841 f: 805 683 1171 p: 800 376 2942 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Advice on deriving accurate time values from the kernel?
Hi, I am in the process of developing a device driver for the purpose of stepper motor control. The timing of each pulse is determined by external timing hardware on an I/O board, which will fire an interrupt after the time requested. Using this method, I am able to generate streams of pulses at approximately 5000Hz on a Pentium II 400MHz system. Everything seems to be working well, but I'd really like to gather some accurate timing data in order to derive some statistics to from the system. Intuition tells me I'll need a clock with a tick rate of at least 2 Hz to derive this. So, is such a thing available in the kernel? I've searched through various mailing list archives and have found reference to the "HZ" option to the kernel, which works to a point. However, it is not ideal as setting HZ to high values generates far too much kernel overhead. Also being considered is additional external timing hardware, but this is something I'd rather avoid for many reasons. What I am after is not a "timer" as such - all I need to do is derive a time value at an initial time, and a subsequent value at a later time. I've used "getmicrouptime", but this appears dependent on the "Hz" option, and as such is of limited use. I've just had some input from a colleauge who has suggested using the Pentium profiling registers, which we are currently investigating... Any advice gratefully received, -- Jennifer Clark http://telepresence.dmem.strath.ac.uk http://www.crmjewellery.co.uk http://www.furniturenet.co.uk To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: seg fault in mutex_queue_enq
On Thu, 15 Jul 1999, Jordan K. Hubbard wrote: :- Jordan should have to say something about this. AFAIR, bumps are :- allowed but only by one between releases. We will have to provide :- libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this :- anyway by the time 4.x is released). :- :-I'd prefer not to bump it... John Birrell and I are already not :-entirely in agreement that the change required a version bump at :-all. It didn't change any interfaces. Which failure is better: An error that you don't have a recent enough version of the library, or one that the routine poll() can't be found at run time? mike To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Another take on /proc statistics (joke of the day)
I thought this amusing.
Take the following program, designed to suck stats out of /proc for the
network devices:
#include stdio.h
#include sys/types.h
#include unistd.h
main()
{
char stuff[4096];
int fd = open("/proc/net/dev", 0);
while(1)
{
int amount = read(fd, stuff, sizeof(stuff));
if (amount 0)
write(1, stuff, amount);
sleep(1);
lseek(fd, (off_t) 0, SEEK_SET);
}
}
Run this on linux, and you'll get the same values for all the stats.
how to make it work right?
#include stdio.h
#include sys/types.h
#include unistd.h
main()
{
char stuff[4096];
while(1)
{
int fd = open("/proc/net/dev", 0);
int amount;
amount = read(fd, stuff, sizeof(stuff));
if (amount 0)
write(1, stuff, amount);
close(fd);
sleep(1);
}
}
What are the implications of this? Well, if you have an rstatd that uses
/proc for statistics, it will have to (for every request) open the status
files, read them, and close them. Net result: very very poor performance
for an rstatd (not even counting the fact that the rstatd has to parse
formatted output back to a binary format ...)
ron
p.s. the rstatd I have for redhat does indeed read stats out of /proc ...
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: tcp windowsize query?
: If I may re-phrase.. How do I determine if the send/recv spaces :are large enough, and if not, how many times I bumped into the :wall? : :Thanks! :John It depends entirely on the type of traffic your machine is handling. A large web server usually uses relatively small (16K or 32K) window sizes in order to avoid wasting memory. It might be handling a thousand active connections. A machine that is running over a long haul fiber or over a very fast link might want to use larger window sizes to keep the pipeline full, but only if it really needs to max out per-connection bandwidth. A machine that is running over slower links can make due with smaller tcp window sizes. Or perhaps if you have a machine running over a very slow link, like a modem link, you may want to significantly reduce the tcp window size for bulk transfers in order to get better interactive performance. e.g. working over a remote connection while at the same time downloading a big tar file via ftp or a browser. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: seg fault in mutex_queue_enq
Jordan should have to say something about this. AFAIR, bumps are allowed but only by one between releases. We will have to provide libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this anyway by the time 4.x is released). I'd prefer not to bump it... John Birrell and I are already not entirely in agreement that the change required a version bump at all. It didn't change any interfaces. I don't care one way or the other. I could leave out the wrapped poll() very easily and avoid the issue all together. This would provide -stable with everything -current has, except of course poll(). I'd prefer to add poll, though... If you don't bump the version in -stable, then you could end up with two versions of libc_r that are not different (assuming -current doesn't make any subsequent changes that warrant a version bump). Just tell me what to do, and I'll do it :-) Dan Eischen [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Advice on deriving accurate time values from the kernel?
:Hi, : :I am in the process of developing a device driver for the purpose of :stepper motor control. The timing of each pulse is determined by :external timing hardware on an I/O board, which will fire an interrupt :after the time requested. Using this method, I am able to generate :streams of pulses at approximately 5000Hz on a Pentium II 400MHz system. : :Everything seems to be working well, but I'd really like to gather some :accurate timing data in order to derive some statistics to from the :system. Intuition tells me I'll need a clock with a tick rate of at :least 2 Hz to derive this. : :So, is such a thing available in the kernel? I've searched through :various mailing list archives and have found reference to the "HZ" :option to the kernel, which works to a point. However, it is not ideal :as setting HZ to high values generates far too much kernel overhead. :Also being considered is additional external timing hardware, but this :is something I'd rather avoid for many reasons. : :What I am after is not a "timer" as such - all I need to do is derive a :time value at an initial time, and a subsequent value at a later time. :I've used "getmicrouptime", but this appears dependent on the "Hz" :option, and as such is of limited use. : :I've just had some input from a colleauge who has suggested using the :Pentium profiling registers, which we are currently investigating... : :Any advice gratefully received, : :-- :Jennifer Clark Hmm. FreeBSD does not guarentee interrupt timing. If the system is busy doing other things your interrupts can be significantly delays (by microseconds, even milliseconds). I would definitely not try to control a stepper motor from an interrupt! What I recommend instead is that you put a microcontroller on the I/O board and have it do all the sensitive stepper motor timing, then write a device driver that does supervisory management of the microcontroller. For example, a small 68HC11F1 or an 8xC51 type of microcontrollor would work well. I prefer the 68HC11F1 myself because it has automatically timed output registers that make it easy to generate perfect waveforms. In regards to your question on accumulating statistics... that's a hard one. An external interrupt pulse is probably the easiest way to do it even though you do not like the idea. It may also be sufficient to call getmicrouptime from the interrupt you are already getting from the I/O board. Another possibility would be to write a user-level process with access to the I/O space (via /dev/mem or /dev/io) to poll in a tight loop and collect statistics that way. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
On Thu, 15 Jul 1999, Daniel C. Sobral wrote: Uh... like any modern unix, Solaris overcommits. On Thu, 15 Jul 1999 08:46:36 -0700 (PDT), "Eduardo E. Horvath" [EMAIL PROTECTED] said: Where do you guys get this misinformation? : Note the `19464k reserved'; that space has been reserved but not yet allocated. Both Dillon and Sobral mistakenly claimed that "Solaris overcommits", this fact seems to be somewhat suggestive. And also, the followings are allocated memory and reserved memory in my environment. (This table also includes Eduardo's example) SunOS allocated reservedtotal total/allocated - - 4.1.4 4268k1248k5516k 1.2924 4.1.2 7732k1492k9224k 1.193 4.1.4 8848k3080k 11928k 1.3481 4.1.4 13532k6772k 20304k 1.5004 5.5.1 15312k5092k 20404k 1.3325 4.1.3 16112k6512k 22624k 1.4042 4.1.2 26356k1620k 27976k 1.0615 4.1.4 26560k3756k 30316k 1.1414 5.526076k 11348k 37424k 1.4352 4.1.4 32984k5556k 38540k 1.1684 5.632448k7072k 39520k 1.2179 4.1.4 38056k3692k 41748k 1.097 4.1.4 49064k7672k 56736k 1.1564 4.1.4 67012k7800k 74812k 1.1164 4.1.4 99348k 16956k 116304k 1.1707 4.1.4 118288k 11780k 130068k 1.0996 5.6 231968k 18880k 250848k 1.0814 5.7 307240k 19464k 326704k 1.0634 (sorted by total amount of used swap) In those examples, non-overcommiting system requires 1.06x ... 1.50x more swap space than overcommiting system. This table also indicates that in proportion as total used swap increase the ratio will decrease. And extra swap space required on non-overcommiting system is approximately several tens mega bytes. i.e. The extra cost of non-overcommiting system is less than ten dollers in my environment. Matt Dillon claimed that non-overcommiting system requires 8x or more swap space than overcommiting system. That's just wrong as above. (There might be cases which requires 8x swap, but it is not typical like Dillon said.) If you don't want non-overcommiting system, because you don't want to pay it's cost. That's OK, but please don't force us to accept your limited view. -- soda To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
:Both Dillon and Sobral mistakenly claimed that "Solaris overcommits", :this fact seems to be somewhat suggestive. : :And also, the followings are allocated memory and reserved memory :in my environment. (This table also includes Eduardo's example) : : SunOS allocated reservedtotal total/allocated : - - : 4.1.4 4268k1248k5516k 1.2924 : 4.1.2 7732k1492k9224k 1.193 : 4.1.4 8848k3080k 11928k 1.3481 : 4.1.4 13532k6772k 20304k 1.5004 : 5.5.1 15312k5092k 20404k 1.3325 : 4.1.3 16112k6512k 22624k 1.4042 : 4.1.2 26356k1620k 27976k 1.0615 : 4.1.4 26560k3756k 30316k 1.1414 : 5.526076k 11348k 37424k 1.4352 : 4.1.4 32984k5556k 38540k 1.1684 : 5.632448k7072k 39520k 1.2179 : 4.1.4 38056k3692k 41748k 1.097 : 4.1.4 49064k7672k 56736k 1.1564 : 4.1.4 67012k7800k 74812k 1.1164 : 4.1.4 99348k 16956k 116304k 1.1707 : 4.1.4 118288k 11780k 130068k 1.0996 : 5.6 231968k 18880k 250848k 1.0814 : 5.7 307240k 19464k 326704k 1.0634 : : (sorted by total amount of used swap) : :In those examples, non-overcommiting system requires 1.06x ... 1.50x :... :soda Umm... how are you getting the reserved numbers? Are you sure that isn't simply cached swap blocks? I.E. when something gets swapped out and then is swapped back in and dirtied, Solaris may be holding the swap block assignment rather then letting it go. FreeBSD-stable does the same thing. FreeBSD-current does not -- it lets it go in order to be able to reallocate it later as part of a contiguous swath for performance reasons. These 'extra' swap blocks are effectively reserved but not actually allocated. They can be reassigned. The numbers above are very similar to what you would see in a redirtying-cache swap block situation on a FreeBSD-stable system. If I add up all the unshared writeable segments on my home box - that is, all segments for which one would potentially have to reserve swap space - I get a total of around 382MB. The machine is currently eating around 100MB of ram and 5MB of swap, or around a 3.5:1 ratio in this case. A non-overcommit model would have to reserve swap space for 382MB - 100MB = 282MB verses the 5MB of swap the machine actually allocates. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
:"pstat -s" on SunOS4, and "swap -s" on SunOS5. From Solaris man page: : ::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space :: (in 1024-byte blocks) :: currently allocated for use as :: backing store. :: :: reservedThe total amount of swap space :: (in 1024-bytes blocks) not :: currentlyallocated,but :: claimed by memory mappings for :: possible future use. :: :: usedThe total amount of swap space :: (in 1024-byte blocks) that is :: either allocated or reserved. :-- :soda Yah, that's what I thought. A solaris expert could tell us for sure but I am pretty sure those are simply cached swap blocks after-the-fact, not actual reservations on potentially swappable space. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space :: (in 1024-byte blocks) :: currently allocated for use as :: backing store. :: :: reservedThe total amount of swap space :: (in 1024-bytes blocks) not :: currentlyallocated,but :: claimed by memory mappings for :: possible future use. :: :: usedThe total amount of swap space :: (in 1024-byte blocks) that is :: either allocated or reserved. :-- :soda It would be really easy to test this. Write a program that malloc's 32MB of space and touches it, then sleeps 10 seconds and forks, with both child and parent sleeping afterwords. ( the parent and the forked child should not touch the memory after the fork occurs ). Do a pstat -s before, after the initial touch, and after the fork. If you do not see the reserved swap space jump by 32MB after the fork, it isn't what you thought it was. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
650 MB MFS?
Are there any design limits to mfs? I want to use cdrecord to write to a dozen or so CD's at once, and fear making lots of coasters if I run them all off a single on-disk file. However, a CD only holds 650 MB, so it seems like I could have the image on mfs and sleep well sans coasters. Would FreeBSD handle an mfs of this size? Thanks! --- David Miller To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 650 MB MFS?
: :Are there any design limits to mfs? I want to use cdrecord to write to a :dozen or so CD's at once, and fear making lots of coasters if I run them :all off a single on-disk file. However, a CD only holds 650 MB, so it :seems like I could have the image on mfs and sleep well sans coasters. : :Would FreeBSD handle an mfs of this size? : :Thanks! : :--- David Miller Well, if you have 650MB of ram available... I suppose. Otherwise MFS is just going to shove the data into swap. The answer is, yes you can create an MFS partition that large. You have to make sure that you have sufficient swap available and that your datasize resource limit is big enough. So, checklist: * You will need 650MB of swap, possibly even more. (unless you have 650MB+ of ram in your system) * from csh, 'unlimit data' then type 'limit' to see what your maximum datasize limit is. You may have to reconfigure your kernel to increase it: options"MAXDSIZ=(1024*1024*1024)" * Look into using the VN device instead of MFS. VN allows you to create a 'disk file' and then turn it into a partition that you can then mount and use normally. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: printing
On Thu, Jul 15, 1999 at 10:44:57AM +0100, Alex Knowles wrote: I hope this is the right place to post, sorry if it's not. I'm really sorry to post what is probably a repeat question, but I've just upgraded to freebsd 3.2-release and I'm having real problems getting the kernel to see my printer ports: here is my kernel device ppc0at isa? port? flags 0x40 net irq 7 controller ppbus0 device lpt0at ppbus? device plip0 at ppbus? device ppi0at ppbus? and here is my dmesg output: ppc0 at 0x3bc irq 7 flags 0x40 on isa ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode ppi0: generic parallel i/o on ppbus 0 plip0: PLIP network interface on ppbus 0 whenever I try to access lpt0 it says that the device is not configured. If I try and use the old configuration of lpt and I try and build the kernel I get a whole load of make errors. Quick guess: Remove your device entries in /dev/ and recreate them with /dev/MAKEDEV. There may well be a different major/minor number for the new device. -- Dom Mitchell -- Palmer Harvey McLane -- Unix Systems Administrator In Mountain View did Larry Wall Sedately launch a quiet plea: That DOS, the ancient system, shall On boxes pleasureless to all Run Perl though lack they C. -- ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. ** To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
On Wed, 14 Jul 1999, John Nemeth wrote: On Jul 15, 2:40am, "Daniel C. Sobral" wrote: } Garance A Drosihn wrote: } At 12:20 AM +0900 7/15/99, Daniel C. Sobral wrote: } In which case the program that consumed all memory will be killed. } The program killed is +NOT+ the one demanding memory, it's the one } with most of it. } } But that isn't always the best process to have killed off... } } Sure it is. :-) Let's see... This statement is absurd. Only a comptetant admin can decide which process can be killed. No arbitrary decision is going to be correct. } It would be nice to have a way to indicate that, a la SIGDANGER. How about assigning something like a class to process, which gives VM a hint which processes should be killed first without much thinking, and which the last (or never)? In other words, let's say class 10 means "totally disposable, kill whenever you want", and class 1 means "never try to kill me". Of course, most processes would get some default value, and superuser could "renice" them to more resistant class. This way both sides of the discussion would be satisfied :-) Andrzej Bialecki // [EMAIL PROTECTED] WebGiro AB, Sweden (http://www.webgiro.com) // --- // -- FreeBSD: The Power to Serve. http://www.freebsd.org // --- Small Embedded FreeBSD: http://www.freebsd.org/~picobsd/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
[Hijacked from freebsd-security]
On Thu, 15 Jul 1999 17:33:29 -0400, Garance A Drosihn wrote:
What I wanted to do was have "estr" routines, where the destination
is specified as the starting point and the ending point of the area
available for the string (as two parameters). The routines would
return the position of the current string-terminator. So you could
do things like:
As I understand it, the goal here is to return to the caller the number
of bytes copied (however you represent it), so that the caller can
easily determine whether or not dst is safe for operations demanding a
null-terminated string.
If that is true, then I think the interface you propose is overly
complex. Looking at the existing functions, their only flaw is that they
return known (and therefore useless) information, "wasting" the return
value. All we need is:
size_t
foocpy(char *dst, const char *src)
size_t
fooncpy(char *dst, const char *src, size_t len)
size_t
foocat(char *s, const char *append)
size_t
fooncat(char *s, const char *append, size_t count)
where the return value is the number of bytes {copied,appended}.
Since the goal is simply to make it easier to do what is already
possible, I think that this approach is better than what you're
suggesting, since the pointer arithmetic required in the caller is
simpler.
And since the prototypes for fooncpy and fooncat above match exactly
those of the proposed strlcpy and strlcat respectively (just had a
look before I "hit the send button"), I'd say that the latter two are
definitely the functions you want.
Ciao,
Sheldon.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: make fails in 3.2-RELEASE for netboot
According to Gregory A. Carter: I'm assuming that might have something to do with it. The file scrt0.c This is the old a.out crt code. The one in 3.0+ is crt1.c, look into /usr/src/lib/csu/i386-elf/. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- [EMAIL PROTECTED] FreeBSD keltia.freenix.fr 4.0-CURRENT #72: Mon Jul 12 08:26:43 CEST 1999 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Fri, Jul 16, 1999 at 12:15:31AM +0200, Sheldon Hearn wrote:
As I understand it, the goal here is to return to the caller the number
of bytes copied (however you represent it), so that the caller can
easily determine whether or not dst is safe for operations demanding a
null-terminated string.
[...]
size_t
fooncat(char *s, const char *append, size_t count)
where the return value is the number of bytes {copied,appended}.
Eeks! This will quickly lead to code like
if (fooncat(string, append, sizeof(string)) != strlen(append))
...
which is rather evil, given that the second strlen(append) would be
completely gratuitous if it weren't for the interface you're
suggesting.
--
This is my .signature which gets appended to the end of my messages.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
:Before program start: :total: 2k bytes allocated + 4792k reserved = 24792k used, 191048k available : :After malloc, before touch: :total: 18756k bytes allocated + 37500k reserved = 56256k used, 159580k available : :After malloc + touch: :total: 52804k bytes allocated + 4852k reserved = 57656k used, 158184k available : :After fork: :total: 52928k bytes allocated + 37644k reserved = 90572k used, 125264k available : :[there has been a little background activity, but the numbers speak for themselves] : : :Daniel Assuming the allocated field is not inclusive of real memory, what we have is swap reservation under solaris for clean pages, and allocation and assignment for dirty pages. The grand total will tell you the total VM potential for malloc'd space but does not appear to tell you how much swap is actually active - i.e. was written to and contains valid data. It would be interesting to see if the stack segment is included in the reservation. Try setting the stack resource limit to 32m and run the same program, except without bothering to malloc() or touch anything. See if the stack segment is included in the reservation field. It would also be interesting to see how solaris deals with MAP_PRIVATE mmap's. If this is correct, then solaris is using a VMSPACE = SWAPSPACE model. FreeBSD uses a VMSPACE = SWAPSPACE + REALMEM model. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
If this is correct, then solaris is using a VMSPACE = SWAPSPACE model. FreeBSD uses a VMSPACE = SWAPSPACE + REALMEM model. AFAIK it has been stated quite explicitly by the Solaris folks that Solaris 2.x uses VMSPACE = SWAPSPACE + REALMEM. This is *different* from SunOS 4.1.x. Steinar Haug, Nethelp consulting, [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Fri, Jul 16, 1999 at 12:15:31AM +0200, Sheldon Hearn wrote:
As I understand it, the goal here is to return to the caller the number
of bytes copied (however you represent it), so that the caller can
easily determine whether or not dst is safe for operations demanding a
null-terminated string.
[...]
size_t
fooncat(char *s, const char *append, size_t count)
where the return value is the number of bytes {copied,appended}.
Eeks! This will quickly lead to code like
if (fooncat(string, append, sizeof(string)) != strlen(append))
...
which is rather evil, given that the second strlen(append) would be
completely gratuitous if it weren't for the interface you're
suggesting.
What's really stupid is that most of the time you're trying to use
these functions to fix code that looks like:
strcpy(buf, str1);
strcat(buf, str2);
strcat(buf, str3);
without overflowing buf. This is dumb! Use asprintf instead:
asprinf(buf, "%s%s%s", str1, str2, str3);
If you can't keep all of the string elements together at once, try:
asprinf(buf, "%s%s", str1, str2);
...
asprintf(buf2, "%s%s", buf, str3);
free(buf);
No, it's not fast, but it _is_ robust.
--
\\ The mind's the standard \\ Mike Smith
\\ of the man. \\ [EMAIL PROTECTED]
\\-- Joseph Merrick \\ [EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Thu, 15 Jul 1999 18:34:42 -0400, Tim Vanderhoek wrote:
if (fooncat(string, append, sizeof(string)) != strlen(append))
...
which is rather evil, given that the second strlen(append) would be
completely gratuitous if it weren't for the interface you're
suggesting.
Tim, you're doing that "I'm right, but too concise to be understood"
thing again. What are you saying here? :-)
If all you're saying is that you want an API that doesn't require a test
against the known length of src (append in your example), then you won't
like strl*. :-)
You'd probably prefer the functions to return the number of bytes which
they did not manage to {copy,append}, yes? Lazy bastard [1]. :-)
While this might be something we add, it shouldn't be called
strl{cpy,cat}. And the original question was whether or not we'd add the
strl{cpy,cat} functions to libc. If we do, I seriously hope I'll be
given the opportunity to submit a replacement manpage, since theirs
sucks.
Ciao,
Sheldon.
[1] It's usually the lazy guy who demands the best API, provided his
demands are tempered by the pedantic guy.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Thu, 15 Jul 1999, Mike Smith wrote: What's really stupid is that most of the time you're trying to use these functions to fix code that looks like: strcpy(buf, str1); strcat(buf, str2); strcat(buf, str3); without overflowing buf. This is dumb! Use asprintf instead: There was a talk on these (strlcpy(3) and strlcat(3)) at USENIX. The logic as to their design was presented and I agree totally with the way that the logic was played out into the functions. They are described in the FreeNIX proceedings on page 175. I feel they make a lot more sense that teh present version sand we should support OpenBSD's application to Posix to make them standard. They lead to faster overall code, (people using present functions often re-count the size of result strings, or check the end of the buffer for a 0) and lead to more readable code. All of these age a "Good Thing" (TM). asprinf(buf, "%s%s%s", str1, str2, str3); Very nice but not always applicable. Particularly when compiling strings under algorythmic control. If you can't keep all of the string elements together at once, try: asprinf(buf, "%s%s", str1, str2); ... asprintf(buf2, "%s%s", buf, str3); free(buf); No, it's not fast, but it _is_ robust. Why not be fast AND robust? :-) I like the new functions and would like to see them supported. julian To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Thu, 15 Jul 1999, Julian Elischer wrote: There was a talk on these (strlcpy(3) and strlcat(3)) at USENIX. The logic as to their design was presented and I agree totally with the way that the logic was played out into the functions. They are described in the FreeNIX proceedings on page 175. I feel they make a lot more sense that teh present version sand we should support OpenBSD's application to Posix to make them standard. Yes, this discussion started out on freebsd-security and when I first wrote about it, I mentioned the paper at USENIX by Todd Miller and Theo de Raadt. It was later mentioned that the paper and accompanying slides are available at: http://www.openbsd.org/papers/strlcpy-paper.ps http://www.openbsd.org/papers/strlcpy-slides.ps I think each function is well thought out and I think they'd be a great addition to FreeBSD. Paul Hart -- Paul Robert Hart8 8 8Verio Web Hosting, Inc. [EMAIL PROTECTED]8 8 8http://www.iserver.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
Here is what I get from one of BEST's mail www proxy machines. ~dillon/br adds the object size's together. 'swap' and 'default' objects refers to unbacked VM objects - and none of the processes running fork shared unbacked objects so we don't have to worry about that. The 'swap' designation means that at least one page in the object has been assigned swap. The default designation means that no pages have been assigned swap. The pages can be dirty or clean. Typical /proc/PID/map output looks like this (taken from one of the sendmail processes). The lines I've marked are the ones being counted as unbacked/swap-backed VM. The rest are vnode-backed and not counted. 0x1000 0x4b000 66 0 r-x COW vnode 0x4b0000x4e0003 3 rwx COW vnode 0x4e0000x87000 5343 rwx COW swap --- 0x870000x373000 738 738 rwx default --- 0x2004b000 0x2005a000 2 0 r-x COW vnode 0x2005a000 0x2005c000 2 0 rwx COW vnode 0x2005c000 0x20065000 6 2 rwx COW swap --- 0x20068000 0x2006d000 3 0 r-x COW vnode 0x2006d000 0x2006e000 1 1 rwx COW vnode 0x2006e000 0x200cc00070 0 r-x COW vnode 0x200cc000 0x200d 4 4 rwx COW vnode 0x200d 0x200e7000 8 6 rwx COW swap --- 0xefbde000 0xefbfe0001414 rwx COW swap --- proxy1:/tmp# cat /proc/*/map | egrep 'swap|default' | ~dillon/br 639168K proxy1:/tmp# pstat -s Device 1K-blocks UsedAvail Capacity Type /dev/sd0b 52428812596 511628 2%Interleaved This machine has 256MB of ram of which around 200MB is in use, we will assume the entire 200MB is used by VM spaces for processes. It is an active machine with around 205 processes at the time of the test. So. 200MB of ram + 12MB of swap = 212MB of actual storage being used out of 639MB of total swap-backable VM. About a factor of 3.2:1. Actual swap utilization is sitting at 2%. If no overcommit were allowed, and assuming a VMSPACE = REALMEM + SWAP model, 200MB of ram would be active and 439MB worth of swap would be either allocated or reserved ( though only 12MB would be actually written, that part doesn't change ). 439MB of swap verses 12MB of swap. In that scenario, the 512MB of swap I assigned to this machine would be dangerously low. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
Ugh. Take the first example in the paper; it rewrites as len = asprintf(path, "%s/.foorc"); ^ , homedir Whoops. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Thu, 15 Jul 1999, Mike Smith wrote: Ugh. Take the first example in the paper; it rewrites as len = asprintf(path, "%s/.foorc"); as opposed to strlcat(path, homedir, sizeof(path)); strlcat(path, "/", sizeof(path)); strlcat(path, ".foord", sizeof(path)); len = strlen(path); Yes, they're a better str*cat/cpy, but they're not the solution that they claim to be. I don't think that anyone has intended them to be anything other than a better replacement for strcpy/strcat than strncpy/strncat (which they certainly are). Sure, you could go around telling people "use snprintf instead" or "use asprintf instead", but is that the issue at hand? Paul Hart -- Paul Robert Hart8 8 8Verio Web Hosting, Inc. [EMAIL PROTECTED]8 8 8http://www.iserver.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
On Fri, Jul 16, 1999 at 12:53:13AM +0200, Sheldon Hearn wrote:
If all you're saying is that you want an API that doesn't require a test
against the known length of src (append in your example), then you won't
like strl*. :-)
Well, if I read your message correctly, the difference between fooncat()
and strlcat() would be that strlcat() returns the total number of
chars in (or would be in) the destination string, whereas fooncat()
returns the total number of chars copied. The former, strlcat(),
avoids the problem I was noting.
Looking at OpenBSD's actual definition of strlcat() which returns the
number of chars that would have been in the final string is
potentially non-useful, but not really too terrible.
[If I'm using strlcat() in the first place, am I _really_ going to
care how many chars would have been copied? Maybe in legacy code,
but in anything newer...]
You'd probably prefer the functions to return the number of bytes which
they did not manage to {copy,append}, yes? Lazy bastard [1]. :-)
Hmm... That's an interesting idea...
strl{cpy,cat}. And the original question was whether or not we'd add the
strl{cpy,cat} functions to libc. If we do, I seriously hope I'll be
Ahh, well, you did hijack this off of the -security list. Adding
strlcpy() and strlcat() is probably a good idea.
given the opportunity to submit a replacement manpage, since theirs
sucks.
Bah. You're in avail now. Just commit ontop of whatever manpage gets
imported. ;-) If your replacement is good, no one will object. :)
--
This is my .signature which gets appended to the end of my messages.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
At 12:15 AM +0200 7/16/99, Sheldon Hearn wrote: [Hijacked from freebsd-security] For those who missed the original article, here's the initial topic (from Paul Hart, but truncated a bit): I was just reviewing the proceedings from the USENIX 1999 Annual Technical Conference where Todd Miller and Theo de Raadt presented a paper on two new functions that OpenBSD has integrated into libc. The new functions, strlcpy(3) and strlcat(3), are intended to provide an easily understood means of safe string copying and concatenation to programmers. I was impressed by the paper and wondered if anyone besides myself would be amenable to including them in FreeBSD's libc. If you are a USENIX member you can access the text of the paper at: http://www.usenix.org/events/usenix99/millert.html (or check:) Paper: http://www.openbsd.org/papers/strlcpy-paper.ps Slides (worth looking at too): http://www.openbsd.org/papers/strlcpy-slides.ps In reply to that, I mentioned some routines that I have been meaning to write, to address what I saw as the problems with using strncat and strncpy (before I had heard of the strl* routines). Sheldon is arguing against the routines I was talking about, not the strl* routines that Paul referred to... On Thu, 15 Jul 1999 17:33:29 -0400, Garance A Drosihn wrote: What I wanted to do was have "estr" routines, where the destination is specified as the starting point and the ending point of the area available for the string (as two parameters). The routines would return the position of the current string-terminator. So you could do things like: As I understand it, the goal here is to return to the caller the number of bytes copied (however you represent it), so that the caller can easily determine whether or not it is safe for operations demanding a null-terminated string. Um, no. that certainly was not my intention with the estr* ideas... It was noticed as a side-effect of what I ended up with, but it wasn't the main objective. And since the prototypes for fooncpy and fooncat above match exactly those of the proposed strlcpy and strlcat respectively (just had a look before I "hit the send button"), I'd say that the latter two are definitely the functions you want. Well, they aren't exactly the functions *I* would want, but that isn't really the point. I do think the strl* routines are a good idea to have. I would like to see them included in "standard C" (or at least FreeBSD), because they are better (IMO) than using strncat and strncpy to avoid buffer overflows. Even looking over my OWN code, I come across times that I've used strncat or strncpy wrong. So, while I still SLIGHTLY prefer my estr* ideas over the strl* ideas, it isn't enough that I would argue against the strl* routines being standard. (and the more platforms that have them, the better). --- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
In that scenario, the 512MB of swap I assigned to this machine would be dangerously low. With 13GB disks available for a couple of hundred bucks, my machines aren't going to run out of swap space any time soon, even if I commit to disk. All I want for Christmas is a knob to disable overcommit. --lyndon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
On Thu, 15 Jul 1999 17:53:52 CST, [EMAIL PROTECTED] wrote: All I want for Christmas is a knob to disable overcommit. And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to disable the policy knows how to do it? Hmmm, I wonder whether that's significant... Ciao, Sheldon. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
but what about
While ( more data items)
{
copy data items onto end of buffer
if full{
write out buffer
clear buffer, copy in rest of last item.
}
}
I'd certainly not want to use xxprintf() for that
This is what stdio does, funnily enough. See fwrite() etc.
--
\\ The mind's the standard \\ Mike Smith
\\ of the man. \\ [EMAIL PROTECTED]
\\-- Joseph Merrick \\ [EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Mike Smith writes: : if (strlen(buf) = sizeof(buf)) : return(error); This can never be true with the strl functions They don't run off the end, so strlen(buf) is always going to be sizeof(buf) since it doesn't include the traling null. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Mike Smith writes: : What's really stupid is that most of the time you're trying to use : these functions to fix code that looks like: : strcpy(buf, str1); : strcat(buf, str2); : strcat(buf, str3); : without overflowing buf. This is dumb! Use asprintf instead: : : asprinf(buf, "%s%s%s", str1, str2, str3); : : If you can't keep all of the string elements together at once, try: : : asprinf(buf, "%s%s", str1, str2); : ... : asprintf(buf2, "%s%s", buf, str3); : free(buf); : : No, it's not fast, but it _is_ robust. That is true for this case, but not always true. I think these APIs have an excellent role to play. Sure, there are other ways to do it, but there are a growing number of systems that have strl* on them (OpenBSD, Linux and Solaris), which is reason enough to improve our portability by using them. The asprintf isn't completely robust becuase you must free() the routine, or face a memory leak. It won't overflow, but it might introduce another bug. The whole point of these APIs was to transition old code to new in a safe manner that isn't prone to potentiall programming errors. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Mike Smith writes: : Ugh. Take the first example in the paper; it rewrites as : : len = asprintf(path, "%s/.foorc"); : : as opposed to : : strlcat(path, homedir, sizeof(path)); : strlcat(path, "/", sizeof(path)); : strlcat(path, ".foord", sizeof(path)); : len = strlen(path); : : Yes, they're a better str*cat/cpy, but they're not the solution that : they claim to be. You've forgotten the free(path) sometime later in your code... That's a can of warms you conveniently ignore... And can be big problems for library routines whose API is defined to return stuff into a static buffer... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Mike Smith writes: : if (strlen(buf) = sizeof(buf)) : return(error); This can never be true with the strl functions They don't run off the end, so strlen(buf) is always going to be sizeof(buf) since it doesn't include the traling null. I actually should have copied the original example from the paper, which was disgustingly more verbose; I mistransposed it for the abovem which should probably have been (strlen(buf) == (sizeof(buf) - 1)) or similar. They recommend using: len = strlcpy(path, homedir, sizeof(path)); if (len = sizeof(path)) return(ENAMETOOLONG) etc. I still think this is the wrong way to deal with the problem. 8) -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message 19990715194203.A54146@mad Tim Vanderhoek writes: : Looking at OpenBSD's actual definition of strlcat() which returns the : number of chars that would have been in the final string is : potentially non-useful, but not really too terrible. No. It is useful. If you look at the return value, you can detect that an overflow would have happened and bail w/o having the overflow actually happen. That is useful (and even documented in the man page by a nice example). : given the opportunity to submit a replacement manpage, since theirs : sucks. : : Bah. You're in avail now. Just commit ontop of whatever manpage gets : imported. ;-) If your replacement is good, no one will object. :) I'm planning on committing their man page. I don't see problems with it, purhaps people could point them out to me so that both our man pages and theirs could be better. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Mike Smith writes: : I still think this is the wrong way to deal with the problem. 8) We mildly disagree here. The strl* functions are the end all, be all of security. They are just designed to make the existing code that uses static buffers easy to make more robust w/o radically altering that code. Of course, strings have always been weak in 'C'. You make them static and they overflow. You malloc them, and often people forget to free them later leading to other problems... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Warner Losh writes: : We mildly disagree here. The strl* functions are the end all, be all : of security. NOTE: This should have read: We mildly disagree here. The strl* functions are NOT the end all, be all of security. which changes its meaning quite a bit... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Mike Smith writes: : I still think this is the wrong way to deal with the problem. 8) We mildly disagree here. The strl* functions are the end all, be all of security. They are just designed to make the existing code that uses static buffers easy to make more robust w/o radically altering that code. Of course, strings have always been weak in 'C'. You make them static and they overflow. You malloc them, and often people forget to free them later leading to other problems... With the addition of a "not" in your first paragraph, I actually think we're in agreement here. I'm just maintaining that in most of the in-tree cases where static buffers are used, a dynamic buffer would have been a better design choice; you might want to disagree there too of course. 8) Regardless, we should definitely adopt these functions for no other reason than portability, no argument there. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
In message [EMAIL PROTECTED] Sheldon Hearn writes: : If you see my point, let me know and I'll send you an alternative : strlcpy.3 . I can see your point. I don't know if I'll like your man pages better or not, but I'd be willing to give them a spin. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: make fails in 3.2-RELEASE for netboot
I found it when I went searching however I still can't get the netboot to compile as it was designed for aout. Any ideas of why it wasn't moved to elf along with the rest of the OS? Or if not how *I* can port it to elf instead? The intention is that loader(8) will provide the same functionality, and the framework is already in place for this. I suggest you use etherboot in the ports collection, at least until the loader support is completed. -- Robert Nordier To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit
Andrew Reilly wrote: On Thu, Jul 15, 1999 at 11:48:41PM +0900, Daniel C. Sobral wrote: Actually, applications are written assuming that malloc() will not fail, generally speaking. Is this really the case? I'm pretty sure I've _never_ ignored the possibility of a NULL return from malloc, and I've been using it for nearly 20 years. I usually print a message and exit, but I never ignore it. I thought that was pretty standard practise. You are always free to inspect how applications deal with malloc(), as far as open source software goes. Anyway, your "usual" behavior is to expect malloc() will not fail. To print a message and exit is to treat it as a fatal error, don't you agree? -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] "Would you like to go out with me?" "I'd love to." "Oh, well, n... err... would you?... ahh... huh... what do I do next?" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
All I want for Christmas is a knob to disable overcommit. And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to disable the policy knows how to do it? Hmmm, I wonder whether that's significant... that's an impressively bold statement to make. by my reconning, at least 4 people who have posted "wanting no overcommit" are more than capable of programming this for NetBSD. .mrg. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Advice on deriving accurate time values from the kernel?
I've done some work on measuring things like interrupt response times and the interval between two interesting events or steps in processing. A cheap way to do this is to use the TSC register in the CPU, though you then need to calibrate the frequency that the CPU really runs at. If you're willing to spend some money, you can get hardware to plug into a PCI slot that can return timestamps in 100ns units, as well as generating interrupt at some time in the future, etc. See http://www.bancomm.com/cbc637PCI.htm for one example of such hardware. I have a device driver for FreeBSD for this board which even allows user processes to get precise timing by mmap()'ing the device registers into user space for easy access. The driver will be contributed to the FreeBSD project "soon." I was pretty close to doing so just prior to the newbus conversion and now need to update the driver for a more recent -current. louie To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
re: HELP!! Slice info disappeared
Hello Niall and Josef, Thanks for your great help. Finally I get all data on the disk back after the struggle on the weekend :) I run the program you sent me, but I got nothing from it. I guess the reason as follows: (1) The program fetches each chunks of 16 blocks from the disk and check if they are superblocks. (2) My disk is sliced into 3 parts. The fist one is for MS-DOS (FAT format), and the rest of them are FFS. It's highly possible that the offset block of the beginning of FFS slice is not a multiple of 16. (3) So I did: # for i in 0 1 2 3 ... 15; do # ./findsb /dev/wd2 $[ (estimated end of block for FAT slice)*512 + $i*512 ] # done After I got the info from the superblocks, getting back data was easy. Again, thanks much for your great help. You saved a lot of my time to reconstruct the data from scratch. Sincerely yours, -- Kazukiyo Ueda From: Josef Karthauser [EMAIL PROTECTED] Subject: re: HELP!! Slice info disappeared Date: Fri, 9 Jul 1999 10:01:16 +0100 Hi Kazukiyo, This is certainly possible. I've enclosed a hack from Niall Smart that should generated enough information to for you to reconstruct it. I'm working on a general solution to this for inclusion FreeBSD as shipped, but it's at home and I'm at work, that said it's Niall's basic code saved my harddisk a few weeks ago :) Joe To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
VMWare plug/quickie tests.
I've been running VM Ware under NT for a few days now, booting FreeBSD and other OS's. In some quick testing: The host machine is NT 4.0, SP5, 384MB RAM, dual 450 PII's. The guest OS is FreeBSD 3.2-RELEASE, configured with the VMWARE 512MB disk, and 32MB RAM allocated Compiling a generic kernel from scratch came out: 156.7u 64.3s 5:04.63 72.5%, ...149pf On a PII 300 with IDE disks, 128MB RAM, it was: (3.2-stable) 205.7u 17.9s 6:56.37 53.7%, ...32pf+0w All in all, not too shabby. So far it hasn't crashed, although I am not running X in the guest session yet, just using it in console mode. It has some weird pauses and stuff when it boots up, and the boot loader spinner creeps along, but once it's up and running, it's fine. I can see the utility of this pretty easily. Just wish it wasn't 400 bucks. I'm going to do some testing between VM's and such, and see how it goes. Next stop, 4.0-current. It doesn't consume any CPU cycles that I can see while idle, and running it at the standard priorities isn't affecting my NT stuff at all... I could grow to like it. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
predictability in an unpredictable world
If there were a mechanism whereby one could opt out of the SIGKILL,
most if not all of the complaints would go away. SIGDANGER would
suffice, but even a rude hack would do in a pinch, such as the one
included below (untested). If you mmap real disk instead of sbrk'ing,
and use this procfs control then the overcommit world won't interfere
with your process (but take care of thrashing stack/text of course).
No more need to fight the holy war!
3.2-STABLE:
===
diff -u vm/vm_pageout.c ../../src3.2/sys/vm/vm_pageout.c
--- vm/vm_pageout.c Thu Jul 15 20:40:56 1999
+++ ../../src3.2/sys/vm/vm_pageout.cThu Mar 18 17:28:39 1999
@@ -1144,7 +1144,7 @@
/*
* if this is a system process, skip it
*/
- if ((p-p_flag
(P_PRECIOUS|P_SYSTEM|P_NOSWAP)) || (p-p_pid == 1) ||
+ if ((p-p_flag (P_SYSTEM|P_NOSWAP)) ||
(p-p_pid == 1) ||
((p-p_pid 48) (vm_swap_size != 0)))
{
continue;
}
===
diff -u sys/proc.h ../../src3.2/sys/sys/proc.h
--- sys/proc.h Thu Jul 15 20:41:50 1999
+++ ../../src3.2/sys/sys/proc.h Fri May 14 01:32:41 1999
@@ -268,8 +268,6 @@
#defineP_NOCLDWAIT 0x40 /* No zombies if child dies
*/
-#defineP_PRECIOUS 0x80 /* Please, please let me live
*/
-
/*
* MOVE TO ucred.h?
===
diff -u miscfs/procfs/procfs_ctl.c
../../src3.2/sys/miscfs/procfs/procfs_ctl.c
--- miscfs/procfs/procfs_ctl.c Thu Jul 15 20:40:59 1999
+++ ../../src3.2/sys/miscfs/procfs/procfs_ctl.c Sat Aug 2 09:32:10
1997
@@ -69,8 +69,6 @@
#define PROCFS_CTL_STEP3
#define PROCFS_CTL_RUN 4
#define PROCFS_CTL_WAIT5
-#define PROCFS_CTL_PRECIOUS6
-#define PROCFS_CTL_NOPRECIOUS 7
static vfs_namemap_t ctlnames[] = {
/* special /proc commands */
@@ -79,8 +77,6 @@
{ "step", PROCFS_CTL_STEP },
{ "run",PROCFS_CTL_RUN },
{ "wait", PROCFS_CTL_WAIT },
- { "precious", PROCFS_CTL_PRECIOUS },
- { "noprecious", PROCFS_CTL_NOPRECIOUS },
{ 0 },
};
@@ -255,14 +251,6 @@
}
}
return (error);
-
- case PROCFS_CTL_PRECIOUS:
- p-p_flag |= P_PRECIOUS;
- return 0;
-
- case PROCFS_CTL_NOPRECIOUS:
- p-p_flag = ~P_PRECIOUS;
- return 0;
default:
panic("procfs_control");
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit
: fail, generally speaking. : :Is this really the case? I'm pretty sure I've _never_ ignored the :possibility of a NULL return from malloc, and I've been using it :for nearly 20 years. I usually print a message and exit, but I :never ignore it. I thought that was pretty standard practise. : :This is just a random comment, orthogonal to the overcommit issue, :but I've seen both you and Matthew say this now, and I was surprised :both times. : :-- :Andrew The are dozens of libc routines which call malloc internally and return allocated storage. strdup(), opendir(), fopen(), setvbuf(), asprintf(), and so forth. Dozens. And while we might check some of these for NULL, we don't check them all, and the ones we do check we tend to conclude a failure other then a memory failure. We would assume that the directory or file does not exist, for example. How many programmers check errno after such a failure? Very few. How many programmers bother to even *clear* errno before making these calls (since some system calls do not set errno if it already non-zero). Virtually nobody. Having malloc() return NULL due to some *unrelated* process running the system out of swap is unnacceptable as it would result in serious instability to a great many programs that were never designed to deal with the case. Rather then crying about the system killing your favorite process, you would be crying about half a dozen processes that are still running no longer being stable. As a sysop, I would reboot a system in such a state instantly. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
: In that scenario, the 512MB of swap I assigned to this machine would be : dangerously low. : :With 13GB disks available for a couple of hundred bucks, my machines aren't :going to run out of swap space any time soon, even if I commit to disk. : :All I want for Christmas is a knob to disable overcommit. : :--lyndon If your machines aren't going to run out of swap, then the overcommit isn't going to hurt you in a million years. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: OpenBSD's strlcpy(3) and strlcat(3)
Any use of str{,n}cat makes me gag. In the past I have used
a composable function that may be of interest. Composable in
the sense that the result can be immediately used as an arg
to another call and it doesn't have the O(N^2) behavior of
strcat. Such a function can be totally safe. Something like:
char* r;
r = scpy(char* dst, char* const dst_end, const char* src)
where the following post-conditions hold:
dst_end = dst
r == MIN(dst + strlen(src)), dst_end)
r[0] == '\0'
That is, the returned ptr points in `dst' _just_ past the
copied data. Note that `dst_end' points to the _last_ char
of `dst'.
Example:
char* string[N];
...
char str[STRSIZE];
char* const strend = str + sizeof str - 1;
char* t = str;
for (int i = 0; i N t strend; i++)
t = scpy(t, strend, string[i]);
or
scpy(scpy(str, strend, "this"), strend, " and that"));
Here is the implementation:
char*
scpy(char* d, char* const e, const char* s)
{
while (*s d e)
*d++ = *s++;
*d = '\0';
return d;
}
This is far too simple to merit a paper or a long name :-)
And I am sure a zillion others have come up with the same idea.
-- bakul
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit
On Thu, 15 Jul 1999, Matthew Dillon wrote: The are dozens of libc routines which call malloc internally and return allocated storage. strdup(), opendir(), fopen(), setvbuf(), asprintf(), and so forth. Dozens. And while we might check some of these for NULL, we don't check them all, and the ones we do check we tend to conclude a failure other then a memory failure. We would assume that the directory or file does not exist, for example. How many programmers check errno after such a failure? Very few. How many programmers bother to even *clear* errno before making these calls (since some system calls do not ^^ We're not supposed to have to clear errno unless we have to explicitly test if it has changed. We're not supposed to clear it before any system call which could possibly fail and set errno. set errno if it already non-zero). Virtually nobody. Erm... WTF?!?! If so, why the HELL are we doing that?!? -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit
: The are dozens of libc routines which call malloc internally and return : allocated storage. strdup(), opendir(), fopen(), setvbuf(), asprintf(), : and so forth. Dozens. And while we might check some of these for NULL, : we don't check them all, and the ones we do check we tend to conclude : a failure other then a memory failure. We would assume that the directory : or file does not exist, for example. How many programmers check errno : after such a failure? Very few. How many programmers bother to even : *clear* errno before making these calls (since some system calls do not : ^^ :We're not supposed to have to clear errno unless we have to explicitly :test if it has changed. We're not supposed to clear it before any system :call which could possibly fail and set errno. : : set errno if it already non-zero). Virtually nobody. : :Erm... WTF?!?! If so, why the HELL are we doing that?!? No, wait, I got that wrong I think. Oh yah, I remember now. Hmm. How odd. I came across a case where read() could return -1 and not set errno properly if errno was already set, but a perusal of the kernel code seems to indicate that this can't happen. Very weird. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
:Technical follow-up: : :Contrary to what I previously said, a number of tests reveal that :Solaris, indeed, does not overcommit. All non-read only segments, :and all malloc()ed memory is reserved upon exec() or fork(), and the :reserved memory is not allowed to exceed the total memory. It makes :extensive use of read only DATA segments, and has a NON_RESERVE :mmap() flag. : :Though the foot firmly planted in my mouth ought to prevent me from :saying anything else, I must say that it does explain a few things :to me... : :-- :Daniel C. Sobral (8-DCS) :[EMAIL PROTECTED] Something is weird here. If the solaris people are using a SWAPSIZE + REALMEM VM model, they have to allow the allocated + reserved space go +REALMEM bytes over available swap space. If not they are using only a SWAPSIZE VM model. Wait - does Solaris normally use swap files or swap partitions? Or is it that weird /tmp filesystem stuff? If it normally uses swap files and allows holes then that explains everything. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
implementing poll() in a device driver (fwd)
Got no response from freebsd-questions. can anyone here help ? Please cc your reply to my email a/c. thanks --Vasudha -- Forwarded message -- Date: Thu, 15 Jul 1999 14:42:34 +0800 (SGT) From: Vasudha Ramnath [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: implementing poll() in a device driver I'm running FreeBSD 3.1-RELEASE. Could someone explain what the poll() function in a device driver should do ? Can it return POLLERR or POLLHUP ? I have a test driver that returns these values from the poll() function. However, the application that called the select() is not getting an error. Instead, the select is returning that the particular file descriptor is, in this case, 'readable' ! Am I missing something ? Any insight appreciated. thank you --Vasudha To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-questions" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit
No, wait, I got that wrong I think. Oh yah, I remember now. Hmm. How odd. I came across a case where read() could return -1 and not set errno properly if errno was already set, but a perusal of the kernel code seems to indicate that this can't happen. Very weird. I thought I saw this somewhere too, but I thought it was more of a case that it was somewhere *inside* read that errno had to be preserved. i.e. errno gets set somewhere at the top of the code, and if it was already set at a certain point, failure was expected, and to pass along the original errno, not the new one. Or perhaps we're sharing a hallucination. :) Well, set/getpriority(2), certainly can return "-1" and not be an error. You would need to clear out errno before that call and check it on return. This is where excpetions would be a great gain. It could also be used to force programmers to check their system calls more closely. Oops, you didn't handle excpetion foo? SIGBADPROGRAMMER. -- David Cross | email: [EMAIL PROTECTED] Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: more amd hangs: problem really in syslog?
On Tue, 13 Jul 1999, Mike Smith wrote: 'siobi' is someone trying to open the serial console, for whatever reason. Without knowing who it was that was stuck there, it's hard to guess what is going on. D'uh, sorry. Long day. It was amd that was hung in the siobi state. No way to clear it without rebooting the box. Dang. Now I need that stack dump from amd that you posted and I deleted. Specifically, it'd be handy to know why amd felt it was necessary to open the console. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
Daniel Eischen eisc...@vigrid.com writes: There are some bugs in libc_r in stable that have been fixed in -current. I think the one that you've hit is an uninitialized TAILQ_HEAD in a statically declared mutex (in localtime). It's probably about time for a MFC. If someone wants to give me the go-ahead, I can do it... Just Do It (tm). At least you could merge the change from [pthread_private.h] 1.20 Sun Jun 20 8:28:08 1999 UTC by jb Diffs to 1.19 In the words of the author: o The polling mechanism for I/O readiness was changed from select() to poll(). In additon, a wrapped version of poll() is now provided. [...] plus the later formatting fixes. As an aside: some files still lack $Id$'s, and someone should `s/REGENTS/AUTHOR/' in all the copyright statements. tg To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: more amd hangs: problem really in syslog?
Mike Smith wrote: On Tue, 13 Jul 1999, Mike Smith wrote: 'siobi' is someone trying to open the serial console, for whatever reason. Without knowing who it was that was stuck there, it's hard to guess what is going on. D'uh, sorry. Long day. It was amd that was hung in the siobi state. No way to clear it without rebooting the box. Dang. Now I need that stack dump from amd that you posted and I deleted. OK, sent under different cover. Specifically, it'd be handy to know why amd felt it was necessary to open the console. Yeah, I'm kind of curious myself. BTW, I was going to work on this some more today, but the boss thought that putting the box into production was more important. The good news is that under real world load my freebsd box had 20-40% free cpu and a load average of 1.5. With load as equal as the switch could make it, the linux box had no free cpu and a load average of 8. :) I also (finally) got the approval to install freebsd on the fourth box (there are already two linux machines up) so A) I'm making progress in the office, and B) I should have a chance to pound on the syslog stuff tomorrow. Happy, Doug To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
Hi everyone, I've been following this discussion almost from the beginning, and I have the feeling that we're not _really_ getting very far. There's good arguments for and against overcommit, depending on your point of view and your requirements. What I do see is a not-so-openly voiced consent that the way resource(sp?) shortages are handled in an overcommitting system (SIGKILL) makes some of us rather unhappy. I therefore suggest those of us who would like to see a change in this area pool their efforts and energies to work on a mechanism that handles resource shortage in a more graceful way. cheerio Michael -- michael.schus...@germany.sun.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: matcd on an SB16
Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It works for some definitions of work. Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). Presuming the drive is a real Matsushita, and it's one of the few listed in the matcd manpage, you should be able to get it to work by following the manpage carefully. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: matcd on an SB16
Mike Smith scribbled this message on Jul 15: Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It works for some definitions of work. Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). actually, only certain drives are supported on the interfaces... this is from -current's LINT: # for the Sony CDU31/33A CDROM device scd0at isa? port 0x230 # matcd: Matsushita/Panasonic CD-ROM # for the SoundBlaster 16 multicd - up to 4 devices controller matcd0 at isa? port 0x230 # mcd: Mitsumi CD-ROM device mcd0at isa? port 0x300 irq 10 so, we support all three different interfaces, but the support is quite weak... I'm not even sure if these even work anymore, do people have some hardware they run tests on these interfaces? Presuming the drive is a real Matsushita, and it's one of the few listed in the matcd manpage, you should be able to get it to work by following the manpage carefully. as for getting the code to work, that completely depends upon the code, and if you have the specs for the drive... it'd probably be cheaper to just by a new ide cdrom drive... and a heck of a lot faster... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought. -- Ralph Waldo Emerson To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: matcd on an SB16
On Thu, 15 Jul 1999, John-Mark Gurney wrote: Mike Smith scribbled this message on Jul 15: Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It works for some definitions of work. Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). actually, only certain drives are supported on the interfaces... this is from -current's LINT: # for the Sony CDU31/33A CDROM device scd0at isa? port 0x230 # matcd: Matsushita/Panasonic CD-ROM # for the SoundBlaster 16 multicd - up to 4 devices controller matcd0 at isa? port 0x230 # mcd: Mitsumi CD-ROM device mcd0at isa? port 0x300 irq 10 so, we support all three different interfaces, but the support is quite weak... I'm not even sure if these even work anymore, do people havesome hardware they run tests on these interfaces? I have a 486 with a matcd drive. Currently it runs 2.2.8R. The CD works, though occasionaly I get errors on it. I intend to upgrade it to 3.2R or -STABLE as soon as I get its monitor fixed (died a few weeks ago :-( )... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought. -- Ralph Waldo Emerson Nadav To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: matcd on an SB16
At 01:05 15/07/99 -0700, John-Mark Gurney wrote: Mike Smith scribbled this message on Jul 15: Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It works for some definitions of work. Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). actually, only certain drives are supported on the interfaces... this is from -current's LINT: # for the Sony CDU31/33A CDROM device scd0at isa? port 0x230 # matcd: Matsushita/Panasonic CD-ROM # for the SoundBlaster 16 multicd - up to 4 devices controller matcd0 at isa? port 0x230 # mcd: Mitsumi CD-ROM device mcd0at isa? port 0x300 irq 10 so, we support all three different interfaces, but the support is quite weak... I'm not even sure if these even work anymore, do people have some hardware they run tests on these interfaces? I have an mcd (with a Mitsumi controller, not SB16) on a spammable box, it worked last time I tried (2.2.8R). [But that wasn't the original question.] -- Bob Bishop +44 118 977 4017 r...@gid.co.uk fax +44 118 989 4254 (0800-1800 UK) To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
I'll be in Dublin July 19th and July 28th
Hi, Is-it any FreeBSD related events near or anybody who want to drink a Beer and help me to visit the Town ? S. -- FranceNet Security Administrator sebastien.gio...@francenet.fr French FreeBSD Documentation Projectgio...@freebsd.org Tout FreeBSD en Francais http://www.FreeBSD-fr.org To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: more amd hangs: problem really in syslog?
On Wed, Jul 14, 1999 at 10:56:05PM -0700, Mike Smith wrote: On Tue, 13 Jul 1999, Mike Smith wrote: 'siobi' is someone trying to open the serial console, for whatever reason. Without knowing who it was that was stuck there, it's hard to guess what is going on. D'uh, sorry. Long day. It was amd that was hung in the siobi state. No way to clear it without rebooting the box. Dang. Now I need that stack dump from amd that you posted and I deleted. Specifically, it'd be handy to know why amd felt it was necessary to open the console. http://www.egroups.com/group/freebsd-hackers/40590.html Greg -- Gregory S. Sutter My reality check just bounced. mailto:gsut...@pobox.com http://www.pobox.com/~gsutter/ PGP DSS public key 0x40AE3052 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
printing
I hope this is the right place to post, sorry if it's not. I'm really sorry to post what is probably a repeat question, but I've just upgraded to freebsd 3.2-release and I'm having real problems getting the kernel to see my printer ports: here is my kernel device ppc0at isa? port? flags 0x40 net irq 7 controller ppbus0 device lpt0at ppbus? device plip0 at ppbus? device ppi0at ppbus? and here is my dmesg output: ppc0 at 0x3bc irq 7 flags 0x40 on isa ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode ppi0: generic parallel i/o on ppbus 0 plip0: PLIP network interface on ppbus 0 whenever I try to access lpt0 it says that the device is not configured. If I try and use the old configuration of lpt and I try and build the kernel I get a whole load of make errors. what am I doing wrong!? please help thanks Alex P.S. I'm not actually subscribed to the mailing list, so can you reply to me. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: a BSD identd
And pidentd will still be supported. Eventually, I'd like to have those huge majority who do not use DES tokens with pidentd move to the inetd identd (when committed)... How about a standalone identd with DES `tokens' and any other nice to haves that it doesn't make sense to implement in a built-in identd? Regards, Niall To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
Thomas Gellekum t...@ihf.rwth-aachen.de wrote: Daniel Eischen eisc...@vigrid.com writes: There are some bugs in libc_r in stable that have been fixed in -current. I think the one that you've hit is an uninitialized TAILQ_HEAD in a statically declared mutex (in localtime). It's probably about time for a MFC. If someone wants to give me the go-ahead, I can do it... Just Do It (tm). At least you could merge the change from [pthread_private.h] 1.20 Sun Jun 20 8:28:08 1999 UTC by jb Diffs to 1.19 In the words of the author: o The polling mechanism for I/O readiness was changed from select() to poll(). In additon, a wrapped version of poll() is now provided. [...] plus the later formatting fixes. The libc_r version number was bumped in -current because of the addition of poll(). Is this allowed in -stable, or something that waits for a -RELEASE? As an aside: some files still lack $Id$'s, and someone should `s/REGENTS/AUTHOR/' in all the copyright statements. OK. Dan Eischen eisc...@vigrid.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
Daniel Eischen eisc...@vigrid.com writes: The libc_r version number was bumped in -current because of the addition of poll(). Is this allowed in -stable, or something that waits for a -RELEASE? Jordan should have to say something about this. AFAIR, bumps are allowed but only by one between releases. We will have to provide libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this anyway by the time 4.x is released). tg To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
Thomas Gellekum t...@ihf.rwth-aachen.de writes: libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this /usr/src/lib/compat/compat3x Sorry. tg To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
NFS problems due to getcwd/realpath
Hi everybody, after wondering for two years why FreeBSD (2.2.x ... 3.2) might lock up when an NFS server is down, I think I have found one reason for that (see kern/12609 - I now know it doesn't belong to kern - sorry). It is the implementation of getcwd (src/lib/libc/gen/getcwd.c). When examining the parent dir of a mounted filesystem, getcwd lstats every directory entry prior to the mountpoint to find out the name of the mountpoint (but it would only need the inodes's device to do a rough check). If one of the prior entries point to another NFS mountpoint and that one is down, getcwd will wait till the mountpoint is up again This of course applies to all routines which use getcwd, e.g. realpath. This is especially funny since mountd calls realpath (from the RPC handler) to check mount points, so when to machines mount dirs from each other, they can lock up, e.g. at boottime (see kern/12609...) I don't fully understand whether the problem is still present in 3.x, since getcwd may call __getcwd to do the job, but as I understand from the sources, __getcwd may fail and then you're back with the problem. Anyhow, how can this be resolved (except for symlinking all mountpoints)? Must getcwd really do an lstat to find out an inodes device?? Is there no other syscall to do that? (I mean: this information must be present somewhere, without going over the net, right?) Unfortunately I don't now such a syscall. In my opinion getcwd should be implemented differently, but maybe some people have a differen opinion on that (And I am not sure how to do that properly). Any suggestions? best regards Jan -- Physikalisches Institut der Universitaet Bonn Nussallee 12 D-53115 Bonn GERMANY To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: rtfm rewritten in C (updated)
hi, there! On Sun, 11 Jul 1999, Chris Costello wrote: I've implemented a few new features. First off, strdup() isn't abused anymore so the program should run much smoother and smaller, as well as more quickly. Secondly, I have (thanks in part to Alfred Perlstein) added both case-insensitive searches (-i) and local FAQ search (can be overridden with -o, online, flag). Too many others to enumerate. As usual, it's at http://www.calldei.com/~chris/rtfm.tar.gz what about to use shell patterns (fnmatch) instead of substring search? :) /fjoe To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit
Danny Thomas wrote: Killing the biggest is simple to implement and usually right. ... but some people don't want that policy, at least on some of their systems. Does FreeBSD offer alternatives? Is so, they've been conspicuously absent from discussion, which might have taken things into a more productive vein. What do other over-committing systems offer? Absent, eh? FreeBSD offers soft and hard limits on resources. -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org Would you like to go out with me? I'd love to. Oh, well, n... err... would you?... ahh... huh... what do I do next? To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit
lyn...@orthanc.ab.ca wrote: All of the arguments I've seen so far assume that one process is running off and grabbing all the available memory. That may be the most likely scenario, but it's most certainly not the *only* scenario. What if you have a whole bunch of middle sized processes running, all using memory efficiently, but in total using 95% of the available VM. A malloc(5*1024*1024) might work, but I need 10 MB instead of 5MB. And my memory footprint is just a little bit bigger than the other guys. Instead of returning NULL to the malloc() request, *zap* I'm dead. How can you possibly call that sensible behaviour? No process is killed at malloc() time. A process is killed when (another) process needs more memory and it is not available. Yes, the machine is under-resourced. I can't help that -- it's not my machine. The machine belongs to a customer who happens to run my IMAP software, who also happens to have ignored our sizing guidelines. In this situation I have no choice but to deal with the low memory condition, and our code does that, if it's given the chance! At least give me the opportunity to deal with the situation gracefully. If it was not for overcommit, you wouldn't be running half of what you are running in that machine in first place. So, overcommit is helping you run much more for the same resources. What if we decided to defer errors from bind just because there weren't any mbufs available, and later killed the process when it tried to do network I/O? People would howl bloody murder! (== this is rhetorical, folks) Out of mbufs does not result in system deadlock, out of memory does. The semantics of malloc() have been defined since almost the dawn of time. From the current manpage: RETURN VALUES The malloc() and calloc() functions return a pointer to the allocated memory if successful; otherwise a NULL pointer is returned. Nowhere does it say that allocated memory might not exist. Nowhere does it say that I have to touch all the allocated pages to make sure they are really there. Nowhere does it say process death at some non-deterministic time in the future might be a side effect of calling malloc(). And nowhere does it say it does not, of course. But that is beside the point. malloc() works as specified. It is the behavior of the system in a low-resource situation that leads to processes being killed. Applications are written assuming that malloc() behaves in the documented manner. It is *not* acceptable to tell applications writers Actually, applications are written assuming that malloc() will not fail, generally speaking. that they have to provide their own management routines on top of malloc() (SEGV catchers and the like) if they want the long standing semantics of malloc() to be preserved. If the current malloc() cannot behave in the documented and expected manner it needs to be renamed, because malloc() it most certainly isn't. It's funny how all these FreeBSD systems manage to gain such a good reputation despite such an obvious flaw, isn't it? :-) -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org Would you like to go out with me? I'd love to. Oh, well, n... err... would you?... ahh... huh... what do I do next? To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
At 6:29 PM -0700 7/14/99, Matthew Dillon wrote: If 1G isn't enough, spend another $30 and throw 2G of swap online. Or perhaps dedicate an entire $150 disk and throw 6+ GB of swap online. The equivalent setup using a non-overcommit model would require considerably more swap to have the same reliability. Please note that we're talking at cross-purposes here, mainly because I didn't realize this same general topic was being beaten to death in the 'replacement for grep' thread (which I have not been following). Speaking for just me myself and I, I have no problems with the current overcommit model. All I'd like to do is have a way to indicate which processes should not get booted first, if the system does indeed run out of swap and needs to boot some processes. However, other people seem much more worked up about this topic than I am, and thus what I (personally) meant as just casual questions seem to be taken as demands that something be done, RIGHT NOW. I now realize that some people are arguing that malloc should return an error if the system runs out of space, but that's not what I am thinking about. So, I think I'll bow out of this discussion for now, and maybe try to discuss my casual questions sometime in a different context... --- Garance Alistair Drosehn = g...@eclipse.acs.rpi.edu Senior Systems Programmer or dro...@rpi.edu Rensselaer Polytechnic Institute To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
make fails in 3.2-RELEASE for netboot
Been trying to get nb8390.com compiled under /usr/sys/i386/boot/netboot and every time it does this: (ns2)[6]:/usr/src/sys/i386/boot/netboot# make Warning: Object directory not changed from original /usr/src/sys/i386/boot/netboot cc -O2 -DNFS -DROMSIZE=16384 -DRELOC=0x9 -DPCI -DPCI_VENDOR=0x10ec -DPCI_DEVICE=0x8029 -DPCI_CLASS=0x02,0x00,0x00 -aout -nostdinc -I/usr/src/sys/i386/boot/netboot/../../../../include -I/usr/src/sys/i386/boot/netboot/../../.. -I/usr/src/sys/i386/boot/netboot -DROMSIZE=16384 -static -o makerom /usr/src/sys/i386/boot/netboot/makerom.c ld: scrt0.o: No such file or directory *** Error code 1 Stop. Now being the resourcful kinda guy I am I looked on another one of my box and it worked fine. On that box I have upgraded all the way from 2.7 so I'm assuming that might have something to do with it. The file scrt0.c does not exist so in a make buildworld it's not created on the new box. The source is fresh as it's a new install so I'm curious as to why the file is missing, and why this was released with netboot as stable is it doesn't compile. Second thing I'm noticing is that, I compiled nb8390.com on the older box and attempted to run it on a machine from boot time. tftp worked great and it attempts to load the kernel from my new box (doesn't matter if I use the new boxes kernel or the old boxes kernel for the client it still fails), I get the following error message: Bad kernel not a valid executable! Of course that is a rough message as I'm at work and the box I'm using it on is at home but that's the important part. It's seeing the kernel file however it's not running it. Now justa long shot but I'm guessing nb8390.com is expecting an aout kernel and it's getting an elf kernel. My last question is why did this program get released with 3.x yet not recognize elf kernels if this is the case? Greg +(o...@dynmc.net)--+ | Dynamic Networking Solutions InterX Technologies | | Senior Network Administratorbits/keyID 1024/7DF9C285 | | o...@interx.net o...@itstudio.net o...@undernet.org o...@webpop3.com | +[ DC 50 57 59 C3 76 46 E8 EB 75 A8 94 FE 96 9E D3 ]--+ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: NFS problems due to getcwd/realpath
In message pine.bsf.4.10.9907151329040.9501-100...@merlin.th.physik.uni-bonn.d
e, Jan Conrad writes:
after wondering for two years why FreeBSD (2.2.x ... 3.2) might lock up
when an NFS server is down, I think I have found one reason for that (see
kern/12609 - I now know it doesn't belong to kern - sorry).
It is the implementation of getcwd (src/lib/libc/gen/getcwd.c). When
examining the parent dir of a mounted filesystem, getcwd lstats every
directory entry prior to the mountpoint to find out the name of the
mountpoint (but it would only need the inodes's device to do a rough
check).
This should no longer be an issue with FreeBSD 3.x, as the system normally
uses the new _getcwd syscall. The old code is still in getcwd.c, but is
only used if the syscall isn't present (e.g. if running a 3.x executable
on a 2.2 system).
We use the following patch on all our 2.2-stable machines, which works
around the problem. This was submitted as PR bin/6658, but it wasn't
committed, as a backport of 3.x's _getcwd (which never occurred) was
considered to be a more appropriate change.
Ian
--- getcwd.c.orig Tue Jun 30 15:38:44 1998
+++ getcwd.cTue Jun 30 15:39:08 1998
@@ -36,6 +36,7 @@
#endif /* LIBC_SCCS and not lint */
#include sys/param.h
+#include sys/mount.h
#include sys/stat.h
#include errno.h
@@ -169,7 +170,28 @@
if (dp-d_fileno == ino)
break;
}
- } else
+ } else {
+ struct statfs sfs;
+ char *dirname;
+
+ /*
+* Try to get the directory name by using statfs on
+* the mount point.
+*/
+ if (!statfs(up[3] ? up + 3 : ., sfs)
+ (dirname = rindex(sfs.f_mntonname, '/')))
+ while((dp = readdir(dir))) {
+ if (ISDOT(dp))
+ continue;
+ bcopy(dp-d_name, bup, dp-d_namlen+1);
+ if (!strcmp(dirname + 1, dp-d_name)
+ !lstat(up, s)
+ s.st_dev == dev
+ s.st_ino == ino)
+ goto found;
+ }
+ rewinddir(dir);
+
for (;;) {
if (!(dp = readdir(dir)))
goto notfound;
@@ -187,7 +209,9 @@
if (s.st_dev == dev s.st_ino == ino)
break;
}
+ }
+found:
/*
* Check for length of the current name, preceding slash,
* leading slash.
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: tcp windowsize query?
Hi, Thanks for the reply(s)... If I understand you correctly, then: %route -n get netapp01 route to: 192.168.21.52 destination: 192.168.21.52 interface: fxp1 flags: UP,HOST,DONE,LLINFO,WASCLONED recvpipe sendpipe ssthresh rtt,msecrttvar hopcount mtu expire 0 0 0 0 0 0 150010 says that I have a specific interface route to get to the fileserver... The recv/send values being 0 means that we are using the default memory/window sizes. I'm not sure what the 3rd 4th columns are... (possibly what I'm looking for below)... From SunOS, I can get the following values: tcp_cwnd_max == 262144-- same as recv/send sizes, or windowsize ?? tcp_xmit_hiwat == 8192 tcp_recv_hiwat == 8192 udp_xmit_hiwat == 8192 udp_recv_hiwat == 8192 I'm now trying to determine how to get the hi lo water marks from FreeBSD... net.inet.tcp.rfc1323: 0 net.inet.tcp.rfc1644: 0 net.inet.tcp.mssdflt: 512 net.inet.tcp.rttdflt: 3 net.inet.tcp.keepidle: 14400 net.inet.tcp.keepintvl: 150 net.inet.tcp.sendspace: 16384-- SunOS uses 260k... net.inet.tcp.recvspace: 16384-- are these small? net.inet.tcp.keepinit: 150 net.inet.tcp.log_in_vain: 0 net.inet.tcp.delayed_ack: 1 net.inet.tcp.path_mtu_discovery: 1 net.inet.tcp.pcbcount: 155 net.inet.tcp.always_keepalive: 1 net.inet.udp.checksum: 1 net.inet.udp.maxdgram: 9216 net.inet.udp.recvspace: 41600 net.inet.udp.log_in_vain: 0 If I may re-phrase.. How do I determine if the send/recv spaces are large enough, and if not, how many times I bumped into the wall? Thanks! John : net.inet.tcp.recvspace: 16384 :... :send/recv space might be what I'm looking for... : :They're the default send/receive window sizes, yes. You can tweak them :with socket ioctls on a per-socket basis. : :delayed ack sounds interesting : :Turning that off disables TCP slow-start. It's a huge performance :booster for things like SMB service, where you have lots of short-lived :TCP connections on a local net. Note that you can also tweak TCP send/receive window sizes on a per-route basis. The recvpipe and sendpipe parameters in route table entries can be changed. This will override the sysctl defaults. However, it may be a little complex for some people to grasp. The route table is a radix tree. In order to set the send/receive pipes for particular ip addresses you may have to create a route to that IP address in order to effect it rather then a more global route. For example, if I am getting the route to some host that runs through my default gateway, I get my default route entry. If I were to change this route entry I would be changing it for default, not just for idiom.com: route -n get idiom.com # route -n get idiom.com route to: 209.157.64.1 destination: default mask: default - this is not a host route! gateway: 209.157.86.1 - this is not a host route! interface: de0 flags: UP,GATEWAY,DONE,STATIC,PRCLONING recvpipe sendpipe ssthresh rtt,msecrttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 On the other hand, a route to another host on the same ethernet is usually specific: # route -n get lander route to: 209.157.86.6 destination: 209.157.86.6 interface: de0 flags: UP,HOST,DONE,LLINFO,WASCLONED recvpipe sendpipe ssthresh rtt,msecrttvar hopcount mtu expire 0 0 10411044 172 219 0 1500 1131 To change the pipes associated with a route, I would do the following. But in this example if I were to try to change the pipe size to idiom.com, I would actually wind up changing the pipe size for my default route. route -n change idiom.com -sendpipe BYTES -recvpipe BYTES route -n change 209.157.86.6 -sendpipe BYTES -recvpipe BYTES If I really want to change the pipe size just to idiom.com, I would have to first create a specific route to idiom.com, then change that. route add idiom.com default route -n change idiom.com -sendpipe BYTES -recvpipe BYTES -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message -- To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: matcd on an SB16
Is the matcd driver known to work on FreeBSD 3.2 ? If not, does anyone have any estimate of the amount of effort that'd be required to fix it? It works for some definitions of work. Firstly, there are three different CDROM interfaces that can be hung off an SB16; one is the Matsushita drive, there's also a Mutsumi interface (I don't think we support it) and a Sony interface (also, I believe, unsupported). Presuming the drive is a real Matsushita, and it's one of the few listed in the matcd manpage, you should be able to get it to work by following the manpage carefully. FYI this no longer works in -current. FreeBSD 4.0-CURRENT #2: Sun Jul 11 23:42:24 PDT 1999 j...@io.yi.org:/usr/src/sys/compile/JAKE I have a Matsushita 2x drive and a sb16; the drive stopped being probed around when newbus went in. No probe message at all. It shows up in visual kernel config, but not after. I figured it was just rot due to the newbus switch, but it hasn't been fixed. I also imagine I'm the only person in the world running -current with one of these :) probably the driver could just be ripped out; these are ancient drives, and consume horendous amounts of CPU. also, something's gone amiss in the reorganization of GENERIC and/or LINT GENERIC: device matcd0 at isa? port 0x230 LINT: controller matcd0 at isa? port 0x230 The man page says it should be controller. Thanks -- we are but packets in the internet of life To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Major Device Number for High Availability Ethernet Driver
Greetings Hackers, We are in the process of releasing a FreeBSD v 3.2 Ethernet driver that detects link failures and executes failovers, supports Cisco's FEC trunking, and system-to-system trunking. To support these features, some configuration is required...the configuration utility requires a /dev/zrm device with a major character device #. The driver (unfortunately a binary only release) also needs to know the major device #. We would greatly appreciate some advice on which device # to use and how to let our driver know which one has been assigned. http://www.freebsd.org/FAQ/FAQ253.html says we can use character device # 32 - ... what if someone else is already using it..don't these have to be unique ? /usr/src/sys/i386/conf/majors.i386 says 200-255 are available for local use ... is there a way a configuration utility can determine the next available ? is there a way my driver can tell which # is used for our /dev/zrm character device ? ( the driver uses cdevsw_add( ) to register routines for the device - major dev # is required ) Advice here would be greatly appreciated. Thanks Tim Tim Hayes ZNYX Corporation 187 S Patterson Suite C Santa Barbara CA 93111 tim_ha...@znyx.com v: 805 683 8841 f: 805 683 1171 p: 800 376 2942 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
Jordan should have to say something about this. AFAIR, bumps are allowed but only by one between releases. We will have to provide libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this anyway by the time 4.x is released). I'd prefer not to bump it... John Birrell and I are already not entirely in agreement that the change required a version bump at all. It didn't change any interfaces. - Jordan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
On Thu, 15 Jul 1999, Jordan K. Hubbard wrote: :- Jordan should have to say something about this. AFAIR, bumps are :- allowed but only by one between releases. We will have to provide :- libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this :- anyway by the time 4.x is released). :- :-I'd prefer not to bump it... John Birrell and I are already not :-entirely in agreement that the change required a version bump at :-all. It didn't change any interfaces. Which failure is better: An error that you don't have a recent enough version of the library, or one that the routine poll() can't be found at run time? mike To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Another take on /proc statistics (joke of the day)
I thought this amusing.
Take the following program, designed to suck stats out of /proc for the
network devices:
#include stdio.h
#include sys/types.h
#include unistd.h
main()
{
char stuff[4096];
int fd = open(/proc/net/dev, 0);
while(1)
{
int amount = read(fd, stuff, sizeof(stuff));
if (amount 0)
write(1, stuff, amount);
sleep(1);
lseek(fd, (off_t) 0, SEEK_SET);
}
}
Run this on linux, and you'll get the same values for all the stats.
how to make it work right?
#include stdio.h
#include sys/types.h
#include unistd.h
main()
{
char stuff[4096];
while(1)
{
int fd = open(/proc/net/dev, 0);
int amount;
amount = read(fd, stuff, sizeof(stuff));
if (amount 0)
write(1, stuff, amount);
close(fd);
sleep(1);
}
}
What are the implications of this? Well, if you have an rstatd that uses
/proc for statistics, it will have to (for every request) open the status
files, read them, and close them. Net result: very very poor performance
for an rstatd (not even counting the fact that the rstatd has to parse
formatted output back to a binary format ...)
ron
p.s. the rstatd I have for redhat does indeed read stats out of /proc ...
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: tcp windowsize query?
: If I may re-phrase.. How do I determine if the send/recv spaces :are large enough, and if not, how many times I bumped into the :wall? : :Thanks! :John It depends entirely on the type of traffic your machine is handling. A large web server usually uses relatively small (16K or 32K) window sizes in order to avoid wasting memory. It might be handling a thousand active connections. A machine that is running over a long haul fiber or over a very fast link might want to use larger window sizes to keep the pipeline full, but only if it really needs to max out per-connection bandwidth. A machine that is running over slower links can make due with smaller tcp window sizes. Or perhaps if you have a machine running over a very slow link, like a modem link, you may want to significantly reduce the tcp window size for bulk transfers in order to get better interactive performance. e.g. working over a remote connection while at the same time downloading a big tar file via ftp or a browser. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: seg fault in mutex_queue_enq
Jordan should have to say something about this. AFAIR, bumps are allowed but only by one between releases. We will have to provide libc_r.so.3 in /usr/lib/compat/compat3x, though (we'll have to do this anyway by the time 4.x is released). I'd prefer not to bump it... John Birrell and I are already not entirely in agreement that the change required a version bump at all. It didn't change any interfaces. I don't care one way or the other. I could leave out the wrapped poll() very easily and avoid the issue all together. This would provide -stable with everything -current has, except of course poll(). I'd prefer to add poll, though... If you don't bump the version in -stable, then you could end up with two versions of libc_r that are not different (assuming -current doesn't make any subsequent changes that warrant a version bump). Just tell me what to do, and I'll do it :-) Dan Eischen eisc...@vigrid.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Advice on deriving accurate time values from the kernel?
:Hi, : :I am in the process of developing a device driver for the purpose of :stepper motor control. The timing of each pulse is determined by :external timing hardware on an I/O board, which will fire an interrupt :after the time requested. Using this method, I am able to generate :streams of pulses at approximately 5000Hz on a Pentium II 400MHz system. : :Everything seems to be working well, but I'd really like to gather some :accurate timing data in order to derive some statistics to from the :system. Intuition tells me I'll need a clock with a tick rate of at :least 2 Hz to derive this. : :So, is such a thing available in the kernel? I've searched through :various mailing list archives and have found reference to the HZ :option to the kernel, which works to a point. However, it is not ideal :as setting HZ to high values generates far too much kernel overhead. :Also being considered is additional external timing hardware, but this :is something I'd rather avoid for many reasons. : :What I am after is not a timer as such - all I need to do is derive a :time value at an initial time, and a subsequent value at a later time. :I've used getmicrouptime, but this appears dependent on the Hz :option, and as such is of limited use. : :I've just had some input from a colleauge who has suggested using the :Pentium profiling registers, which we are currently investigating... : :Any advice gratefully received, : :-- :Jennifer Clark Hmm. FreeBSD does not guarentee interrupt timing. If the system is busy doing other things your interrupts can be significantly delays (by microseconds, even milliseconds). I would definitely not try to control a stepper motor from an interrupt! What I recommend instead is that you put a microcontroller on the I/O board and have it do all the sensitive stepper motor timing, then write a device driver that does supervisory management of the microcontroller. For example, a small 68HC11F1 or an 8xC51 type of microcontrollor would work well. I prefer the 68HC11F1 myself because it has automatically timed output registers that make it easy to generate perfect waveforms. In regards to your question on accumulating statistics... that's a hard one. An external interrupt pulse is probably the easiest way to do it even though you do not like the idea. It may also be sufficient to call getmicrouptime from the interrupt you are already getting from the I/O board. Another possibility would be to write a user-level process with access to the I/O space (via /dev/mem or /dev/io) to poll in a tight loop and collect statistics that way. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
On Thu, 15 Jul 1999, Daniel C. Sobral wrote: Uh... like any modern unix, Solaris overcommits. On Thu, 15 Jul 1999 08:46:36 -0700 (PDT), Eduardo E. Horvath e...@one-o.com said: Where do you guys get this misinformation? : Note the `19464k reserved'; that space has been reserved but not yet allocated. Both Dillon and Sobral mistakenly claimed that Solaris overcommits, this fact seems to be somewhat suggestive. And also, the followings are allocated memory and reserved memory in my environment. (This table also includes Eduardo's example) SunOS allocated reservedtotal total/allocated - - 4.1.4 4268k1248k5516k 1.2924 4.1.2 7732k1492k9224k 1.193 4.1.4 8848k3080k 11928k 1.3481 4.1.4 13532k6772k 20304k 1.5004 5.5.1 15312k5092k 20404k 1.3325 4.1.3 16112k6512k 22624k 1.4042 4.1.2 26356k1620k 27976k 1.0615 4.1.4 26560k3756k 30316k 1.1414 5.526076k 11348k 37424k 1.4352 4.1.4 32984k5556k 38540k 1.1684 5.632448k7072k 39520k 1.2179 4.1.4 38056k3692k 41748k 1.097 4.1.4 49064k7672k 56736k 1.1564 4.1.4 67012k7800k 74812k 1.1164 4.1.4 99348k 16956k 116304k 1.1707 4.1.4 118288k 11780k 130068k 1.0996 5.6 231968k 18880k 250848k 1.0814 5.7 307240k 19464k 326704k 1.0634 (sorted by total amount of used swap) In those examples, non-overcommiting system requires 1.06x ... 1.50x more swap space than overcommiting system. This table also indicates that in proportion as total used swap increase the ratio will decrease. And extra swap space required on non-overcommiting system is approximately several tens mega bytes. i.e. The extra cost of non-overcommiting system is less than ten dollers in my environment. Matt Dillon claimed that non-overcommiting system requires 8x or more swap space than overcommiting system. That's just wrong as above. (There might be cases which requires 8x swap, but it is not typical like Dillon said.) If you don't want non-overcommiting system, because you don't want to pay it's cost. That's OK, but please don't force us to accept your limited view. -- soda To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
:Both Dillon and Sobral mistakenly claimed that Solaris overcommits, :this fact seems to be somewhat suggestive. : :And also, the followings are allocated memory and reserved memory :in my environment. (This table also includes Eduardo's example) : : SunOS allocated reservedtotal total/allocated : - - : 4.1.4 4268k1248k5516k 1.2924 : 4.1.2 7732k1492k9224k 1.193 : 4.1.4 8848k3080k 11928k 1.3481 : 4.1.4 13532k6772k 20304k 1.5004 : 5.5.1 15312k5092k 20404k 1.3325 : 4.1.3 16112k6512k 22624k 1.4042 : 4.1.2 26356k1620k 27976k 1.0615 : 4.1.4 26560k3756k 30316k 1.1414 : 5.526076k 11348k 37424k 1.4352 : 4.1.4 32984k5556k 38540k 1.1684 : 5.632448k7072k 39520k 1.2179 : 4.1.4 38056k3692k 41748k 1.097 : 4.1.4 49064k7672k 56736k 1.1564 : 4.1.4 67012k7800k 74812k 1.1164 : 4.1.4 99348k 16956k 116304k 1.1707 : 4.1.4 118288k 11780k 130068k 1.0996 : 5.6 231968k 18880k 250848k 1.0814 : 5.7 307240k 19464k 326704k 1.0634 : : (sorted by total amount of used swap) : :In those examples, non-overcommiting system requires 1.06x ... 1.50x :... :soda Umm... how are you getting the reserved numbers? Are you sure that isn't simply cached swap blocks? I.E. when something gets swapped out and then is swapped back in and dirtied, Solaris may be holding the swap block assignment rather then letting it go. FreeBSD-stable does the same thing. FreeBSD-current does not -- it lets it go in order to be able to reallocate it later as part of a contiguous swath for performance reasons. These 'extra' swap blocks are effectively reserved but not actually allocated. They can be reassigned. The numbers above are very similar to what you would see in a redirtying-cache swap block situation on a FreeBSD-stable system. If I add up all the unshared writeable segments on my home box - that is, all segments for which one would potentially have to reserve swap space - I get a total of around 382MB. The machine is currently eating around 100MB of ram and 5MB of swap, or around a 3.5:1 ratio in this case. A non-overcommit model would have to reserve swap space for 382MB - 100MB = 282MB verses the 5MB of swap the machine actually allocates. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
