Re: No MAXUID ?
I've come up empty-handed hunting for a constant that defines the maximum UID supported by the system. I'm working on our passwd and pwd_mkdb stuff and want to get rid of the artificial limitation of 65535 (USHRT_MAX) imposed in (at least) pwd_mkdb. Have I missed a useful define, or should I add one? If I should add one, does it go in pwd.h, types.h or syslimits.h? It probably belongs in param.h, and you can probably safely calculate it as (uid_t)0 - 1; -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: reserved/local ioctl values?
I'l looking at defining about a dozen ioctl calls for a local device driver. When looking at the _IO, _IO, _IOW, _IOR, and _IOWR macros, I'm interested if there are any "reserved" or "local" values for the first parameter? Not really, as such. In short, I'd hate to use a seemly unused value, just to suddenly be in conflict with a major set of ioctls (particularly terminal, as this is the type of driver it will be). Also, would it be wise to reuse an existing value? If you want to add private ioctls to a given subsystem, either pick an arbitrarily high function number, or another letter that's not commonly associated with that subsystem. Eg. I would probably pick 'z' if I were going to add a local ioctl to the tty subsystem. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: BSDI Pthread + gdb (Re: Free BSDI CD!)
On Thu, 29 Jul 1999, Martin Cracauer wrote: In [EMAIL PROTECTED], Rayson Ho wrote: Hi, I am not sure whether this is known to this list or not, but here is the URL for ordering: http://www.bsdi.com/products/evalcd/ The thing that gets my attention is the gdb with threads support. Surely they need to provide source for the GPL gdb. Anyone knows what kind of Pthread library/kernel support they have? I have some patches to get our gdb to understand the uthread version of pthreads. It still needs a bit of work and I have been (extremely) busy with unrelated projects recently. I may be able to finish it after SIGGRAPH. -- Doug Rabson Mail: [EMAIL PROTECTED] Nonlinear Systems Ltd. Phone: +44 181 442 9037 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Changing Interface IP address
Mike Smith wrote: I would like to change the IP address (netmask, gateway etc.) of an Interface (eg. fxp0) from within my C source code. 1. Is this possible to do without the SIOC ioctl call? (i am already in the kernel). Make the call from within the kernel. But I was under the impression that I can't do ioctl calls in the kernel? Which call should I then make from within the kernel? Thanks for the Help To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Changing Interface IP address
Mike Smith wrote: I would like to change the IP address (netmask, gateway etc.) of an Interface (eg. fxp0) from within my C source code. 1. Is this possible to do without the SIOC ioctl call? (i am already in the kernel). Make the call from within the kernel. But I was under the impression that I can't do ioctl calls in the kernel? Which call should I then make from within the kernel? The current BSD kernel architecture does not well handle ABI calls from within kernel space, no. You will probably have to duplicate the code that handles the SIOC[AD]IFADDR in netinet/in.c in your module. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FTPd authentication with PAM/MySQL
hi, there! On Mon, 26 Jul 1999, Steven Fletcher wrote: I've seen the PAM modules/libraries/etc for MySQL and noticed that the FTPD Makefile has a Kerberos PAM option, and was wondering if anyone knows of a way to get FTPd talking to MySQL... or if it would work at all? ftpd PAM patches (and other PAM stuff) are available at http://iclub.nsu.ru/~fjoe /fjoe To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: freebsd-hackers-digest V4 #565
Sorry I know i should not mail to the mailinglist. But I can not unsubscribe Please help me To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
James Howard [EMAIL PROTECTED] writes: DES tells me he has a new version (0.10) which mmap()s. It supposedly cuts the run time down significantly, I do not have the numbers in front of me. Unfortunetly he has not posted this version yet so I cannot download it and run it myself. It's in the usual place (ftp://ftp.ofug.org/pub/grep/). He also says that if mmap fails, he drops back to stdio. This should only happen in the NFS case, the 2G case, etc. Any case in which a) the file is too large to mmap, b) the file is not a regular file, or c) mmap() fails (e.g. NFS). DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
John-Mark Gurney [EMAIL PROTECTED] writes: it was VERY simple to do... and attached is the patch... this uses the option REG_STARTEND to do what the copy was trying to do... all of the code to use REG_STARTEND was already there, it just needed to be enabled.. Funnily, I experience a near-doubling of running time with similar patches. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: No MAXUID ?
Sheldon Hearn [EMAIL PROTECTED] writes: Another question I should have asked in my original mail is this: are there magical reasons why we should want pwd_mkdb to bleat for every encountered UID greater that 65535 ? How many times do I have to go through this? There is no "artificial limitation in pwd_mkdb". pwd_mkdb warns against UIDs larger than 65535 because legacy software that uses unsigned short instead of uid_t will break with large UIDs. There were even a few such cases in our tree that I fixed less than a year ago IIRC. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: No MAXUID ?
On 30 Jul 1999 15:38:30 +0200, Dag-Erling Smorgrav wrote: How many times do I have to go through this? Until I stuff a comment in the source that explains this. :-) There is no "artificial limitation in pwd_mkdb". pwd_mkdb warns against UIDs larger than 65535 because legacy software that uses unsigned short instead of uid_t will break with large UIDs. Would you be happy with changing things so that only one warning is generated? Something like "9 max_uid 65535: others may exist"? The current behaviour is quite annoying with large passwd files. :-) Ciao, Sheldon. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: No MAXUID ?
Sheldon Hearn [EMAIL PROTECTED] writes: Would you be happy with changing things so that only one warning is generated? Something like "9 max_uid 65535: others may exist"? The current behaviour is quite annoying with large passwd files. :-) Sure, and maybe modify the warning to say something like "legacy software may not support UIDs larger than 65535" DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
John-Mark Gurney wrote: ok, I just made a patch to eliminate the copy that was happening in procfile, and it sped up a grep of a 5meg termcap from about 2.9sec down to .6 seconds... this includes time spent profiling the program.. GNU grep w/o profiling only takes .15sec so we ARE getting closer to GNU grep... Rather impressive. But... did you run these tests more than once, to account for vm caching? it was VERY simple to do... and attached is the patch... this uses the option REG_STARTEND to do what the copy was trying to do... all of the code to use REG_STARTEND was already there, it just needed to be enabled.. Just for the record... :-) This eliminates one of the "added complexities" I pointed out. -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] "Is it true that you're a millionaire's son who never worked a day in your life?" "Yeah, I guess so." "Lemme tell you, son, you ain't missed a thing." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
grep-0.11
ftp://ftp.ofug.org/pub/grep/grep-0.11.tar.gz More (gprof-assisted) speedups. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
"Daniel C. Sobral" [EMAIL PROTECTED] writes: Dag-Erling Smorgrav wrote: To be precise, I experience a 30% decrease in system time and a 100% increase in user time when I use RE_STARTEND and eliminate the malloc() / memcpy() calls in procfile(). Could you please test my patch that removes malloc() but bot memcpy()? Here it is again, though against an old version: Yeah. You can do even better by declaring ln static and never free()ing it. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
"Daniel C. Sobral" [EMAIL PROTECTED] writes: Could you please test my patch that removes malloc() but bot memcpy()? Here it is again, though against an old version: Bingo. REG_STARTEND is significantly more expensive than memcpy(). DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Documenting writev(2) ENOBUFS error
Ville-Pertti Keinonen wrote: [EMAIL PROTECTED] (Wes Peters) writes: [ENOBUFS] Insufficient system buffer space exists to complete the op- eration. Do you know what kind of circumstances that error *really* occurs under? If it happened with files, that would be a bug and should be fixed. The call is supposed to block to wait for writes to be possible. This applies to stream sockets in most cases, as well. Based on a quick look at the code, out-of-band TCP data seems to be the only case where ENOBUFS might be returned for streams, and that obviously doesn't apply to write/writev. The only way writev can pick up an ENOBUFS error is in the call to (*fp-f_ops-fo_write)(fp, auio, fp-f_cred, 0) on line 447 (in -CURRENT). ENOBUFS can happen on any network object if the system is completely out of mbufs, right? So it could return ENOBUFS for any connected socket specified as the fd in a write or writev operation. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
:Your new "behavior" flag isn't known by vm_map_simply_entry, so :map simplification could drop the behavior setting on the floor. :I would prefer that the behavior is folded into eflags. : :Overall, I agree with the direction. Behavior is correctly a map :attribute rather than an object attribute. : : Alan : :P.S. The MADV_FREE's by malloc/free were disabled by default in -CURRENT :and -STABLE prior to the release of 3.2. They were a performance :pessimization, slowing down "make world" by ~2%. No problem, I think the behavior can easily be folded into eflags, which deals with the map simplification code as well. I'll work up a new patch. - I'm not sure I see how MADV_FREE could slow performance down unless, perhaps, it is the overhead of the system call itself. e.g. if malloc is calling it on a page-by-page basis and not implementing any hysteresis. 2% isn't a big deal. MADV_FREE theoretically makes a big impact on paging performance in a heavily loaded paging system. If your tests were run on a system that wasn't paging, you weren't testing the right thing. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
bootloader....
I'm looking at booting(embedded devices) and I've been looking at lilo boot loader code and booteasy bootloader code... does anyone know of any documentation that anyone out there has done on this topic? -- more specifically without bios calls/support? I've seen the booteasy code at: ftp://ftp.freebsd.org/pub/FreeBSD/tools/srcs/bteasy/ is there a newer version? this code is supposed to be compiled with TASM/Borland C right? is there source that can be compiled with gnu tools? I'll take any and all suggestions :) Thanks, -roy To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
On Fri, Jul 30, 1999 at 09:51:35AM -0700, Matthew Dillon wrote: I'm not sure I see how MADV_FREE could slow performance down unless, perhaps, it is the overhead of the system call itself. e.g. if malloc is calling it on a page-by-page basis and not implementing any hysteresis. It's the system call overhead. Adding (more) hysteresis would reduce the overhead by some factor, but we'd still be making unnecessary MADV_FREE calls. Calling MADV_FREE accomplishes nothing unless the system is actually short of memory. The right way to address this problem is likely to add a mechanism to the VM system that sends a signal to the process when MADV_FREE would actually be beneficial. 2% isn't a big deal. MADV_FREE theoretically makes a big impact on paging performance in a heavily loaded paging system. If your tests were run on a system that wasn't paging, you weren't testing the right thing. 99% of our user base, whose machines aren't thrashing during a "make world" or other normal operation, shouldn't pay a 2% penalty to produce a theoretical improvement for the 1% that are. At best, that's "optimizing" the infrequent case at the expense of the frequent, not a good trade-off. In any case, the man page for malloc/free explains how to change the default, if you're a part of the "oppressed" 1%. Alan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
:In any case, the man page for malloc/free explains how to change :the default, if you're a part of the "oppressed" 1%. : :Alan Not with a gig of memory :-). My only concern is for the performance curve to look good and not be too jagged. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
No elf(5) man page (docs/7914)
Hi folks, We have an a.out(5), but no elf(5) (as pointed out in docs/7914). Does anyone feel up to writing one? N -- [intentional self-reference] can be easily accommodated using a blessed, non-self-referential dummy head-node whose own object destructor severs the links. -- Tom Christiansen in [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Is the _Device Driver Writers Guide_ still apropos?
-hackers, Is the FreeBSD Device Driver Writers Guide at http://www.freebsd.org/tutorials/ddwg/ddwg.html still correct? I know there have been changes to this area of the tree over the past 6 months or so, but I don't know how much of that document is still appropriate for what we have now. Thanks, N -- [intentional self-reference] can be easily accommodated using a blessed, non-self-referential dummy head-node whose own object destructor severs the links. -- Tom Christiansen in [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
So, back on the topic of enabling bpf in GENERIC...
We got off onto a big tangent about switches and vlans and stuff and I learned a number of interesting things, don't get me wrong, but we still haven't established any consensus on the trade-offs of enabling bpf. This wasn't meant to be a hypothetical discussion, I'm truly trying to measure the trade-off between enabling bpf and (by some fraction) opening things up to easier attack by sniffers in a root-compromise situation vs not having DHCP work properly at all after installation. This is a clear security vs functionality issue and I need to get a good feel for which "cause" is ascendent here in knowing which way to jump on the matter. Can we now hear the closing arguments from the pro and con folks? Thanks, - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: DOC volunteer WAS:RE: userfs help needed.
On Thu, Jul 29, 1999 at 10:45:51AM -0500, Alton, Matthew wrote: I ran screaming into the woods last year from trying to grok VOP_foo. The prospect of a rewrite fills me with warmth and fuzziness. I hereby volunteer to maintain the VOP(9) man pages and to flesh out my notes into a big, beefy FS-implementer's Guide to the new VOP_foo. All the coders have to do is answer my (at least marginally clueful) questions along the way and keep posted as to what's going on in new VM-land. I won't even pester anyone with design suggestions... very much. Solid offer, kids. Everybody wins. Give it some thought. Sounds good. Yell for any assistance you need from the Doc. Proj. In particular, the FS Implementer's Guide sounds good. What (markup) language were you planning on using? N PS: Reply-to: should probably be set to doc, but I've left that at your discretion. -- [intentional self-reference] can be easily accommodated using a blessed, non-self-referential dummy head-node whose own object destructor severs the links. -- Tom Christiansen in [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: No MAXUID ?
On Fri, Jul 30, 1999 at 09:13:52AM -0700, a little birdie told me that Mike Smith remarked I think that the administrator should be forced to override the warning manually to indicate that they are aware of the issues they are getting themselves in for, or at the very least that there should be some very serious warnings placed in the relevant manpages (mount_nfs, passwd(5), vipw, etc) covering these issues.a How about a bit of a compromise on it? Make pwd_mkdb just spew a warning like --- WARNING: UID(s) over X detected, may cause problems. --- by default, with a flag (-v?) to list the specific ones causing problems. Warning in manpages are, of course, always a Good Thing. -- Matthew Fuller (MF4839) |[EMAIL PROTECTED] Unix Systems Administrator |[EMAIL PROTECTED] Specializing in FreeBSD |http://www.over-yonder.net/ FutureSouth Communications |ISPHelp ISP Consulting "The only reason I'm burning my candle at both ends, is because I haven't figured out how to light the middle yet" To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
Here is a pro vote for enabling BPF in GENERIC: It will let us use a dhcp client in the install programs, this is of tremendous use to many people as DHCP starts to become much more popular. I cannot net install a machine at home since that is on a DHCP cable modem service. Also, if root is compromised on a system, even if you don't have bpf installed you would be a fool to believe that they are not sniffing packets/passwords. At the very least Mr. Pragmatic(sp?) has shown the world the power and flexability of KLDs... I am sure someone could write a KLD to impliment the functionality of a packet sniffer. Also an attacker, once obtaining root, could certainly trojan ftpd/sshd/telnetd/login/whatever. I think disabling bpf for "security reasons" is a false sense of security. -- David Cross | email: [EMAIL PROTECTED] Systems Administrator/Research Programmer | Web: http://www.cs.rpi.edu/~crossd Rensselaer Polytechnic Institute, | Ph: 518.276.2860 Department of Computer Science| Fax: 518.276.4033 I speak only for myself. | WinNT:Linux::Linux:FreeBSD To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
Time for this oppressed person to go ln -s H /etc/malloc.conf... Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
If root is compromised, that's the only way bpf can be gotten to by default. When root's compromised, if no bpf is available, the mem devices can still be created (if not there) and network queues can be listened to. And can't IFF_PROMISC be turned on too? There's no good reason to not have bpf in at least the boot disk kernel. Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
It will let us use a dhcp client in the install programs, this is of tremendous use to many people as DHCP starts to become much more popular. I cannot net install a machine at home since that is on a DHCP cable modem service. Well, just to clarify this, if you're installing a 4.0 snapshot then you can indeed now do so; it will just be a bit savage in that the installation-time lease will be snapshotted into your /etc/rc.conf in order that the box can reboot for the first time with network connectivity. After that, it's a very good idea to build a new kernel with bpf support enabled and change the ifconfig_foo line in your /etc/rc.conf to say "DHCP" instead of the static lease information it will have in it. Then you're set from then on out. My only desire is that this process be made entirely automatic by enabling you to skip that kernel rebuild part. :) - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
There's no good reason to not have bpf in at least the boot disk kernel. It already is. That's not the question under discussion here - we're talking about how to make things work in the post-installation boot scenario. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
On Fri, 30 Jul 1999, Jordan K. Hubbard wrote: There's no good reason to not have bpf in at least the boot disk kernel. It already is. That's not the question under discussion here - we're talking about how to make things work in the post-installation boot scenario. When did that happen? :) In that case, my argument changes to: "There's no good reason not to have bpf in the GENERIC kernel." - Jordan Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
: But even if you turn off the bpf device, you still have /dev/mem and : /dev/kmem to worry about. For that matter, the intruder can still write : raw devices. Also, there is another kernel feature called kldload(8). BTW, I wrote this section because a hacker actually installed the bpf device via the module loader during one of the root compromises at BEST, a year or two ago. He had gotten it from a hackers cookbook of exploits which he convieniently left on-disk long enough for our daily backups to catch it :-). -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
On Fri, 30 Jul 1999, Matthew Dillon wrote: : But even if you turn off the bpf device, you still have /dev/mem and : /dev/kmem to worry about. For that matter, the intruder can still write : raw devices. Also, there is another kernel feature called kldload(8). BTW, I wrote this section because a hacker actually installed the bpf device via the module loader during one of the root compromises at BEST, a year or two ago. He had gotten it from a hackers cookbook of exploits which he convieniently left on-disk long enough for our daily backups to catch it :-). Want to post the ocde for it? It would be interesting to see how that was done! -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
: But even if you turn off the bpf device, you still have /dev/mem and : /dev/kmem to worry about. For that matter, the intruder can still write : raw devices. Also, there is another kernel feature called kldload(8). BTW, I wrote this section because a hacker actually installed the bpf device via the module loader during one of the root compromises at BEST, a year or two ago. He had gotten it from a hackers cookbook of exploits which he convieniently left on-disk long enough for our daily backups to catch it :-). This doesn't actually help the attacker much, since at that point in time the network drivers wouldn't have been calling the bpf tap points, so it might well have been loaded, but it wouldn't have been _doing_ anything useful. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
This is a clear security vs functionality issue and I need to get a good feel for which "cause" is ascendent here in knowing which way to jump on the matter. Can we now hear the closing arguments from the pro and con folks? I thought we decided that the networking gurus we're going to make it possible to send out broadcast packets on an unconfigured interface so that DHCP would work, so that bpf wasn't required. I thought that was the over-riding reason for adding bpf to the kernel. Otherwise, BPF is mostly (completely) un-necessary for general purpose computing machines. Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
: BTW, I wrote this section because a hacker actually installed the bpf : device via the module loader during one of the root compromises at BEST, : a year or two ago. He had gotten it from a hackers cookbook of exploits : which he convieniently left on-disk long enough for our daily backups to : catch it :-). : :This doesn't actually help the attacker much, since at that point in :time the network drivers wouldn't have been calling the bpf tap points, :so it might well have been loaded, but it wouldn't have been _doing_ :anything useful. Whatever it was, it was recording packets. This was a year or so ago, I don't have the code handy. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
docs for 3Com 3C515-TX NIC?
Anyone have detailed docs for 3Com's 3C515 NIC? I found Guy Helmer's driver, http://www.freebsd.org/~ghelmer/3c515/ but it didn't recognize the card I have. I was going to spend some time this weekend to see if I could figure out what was up. Just thought it might be easier if I had a little light reading material that described how the chip on it was supposed to work. :) Thanks. -steve To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
On Fri, 30 Jul 1999, Nate Williams wrote: This is a clear security vs functionality issue and I need to get a good feel for which "cause" is ascendent here in knowing which way to jump on the matter. Can we now hear the closing arguments from the pro and con folks? I thought we decided that the networking gurus we're going to make it possible to send out broadcast packets on an unconfigured interface so that DHCP would work, so that bpf wasn't required. This is by far the preferred solution. I might have the details archived from the dhcp mailing list, but IIRC it boils down to dealing properly with an address of 255.255.255.255. Whoever it was that mentioned it recently on this list clearly had the right idea. Doug -- On account of being a democracy and run by the people, we are the only nation in the world that has to keep a government four years, no matter what it does. -- Will Rogers To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
I thought we decided that the networking gurus we're going to make it possible to send out broadcast packets on an unconfigured interface so that DHCP would work, so that bpf wasn't required. I believe we decided that this would be the preferred method, yes. I don't think, however, that we decided that this was likely to show up in any reasonable amount of time or that anyone had decided to do the actual work. :-) - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: docs for 3Com 3C515-TX NIC?
On Fri, 30 Jul 1999, Steve Price wrote: Anyone have detailed docs for 3Com's 3C515 NIC? I found Guy Helmer's driver, http://www.freebsd.org/~ghelmer/3c515/ but it didn't recognize the card I have. I was going to spend some time this weekend to see if I could figure out what was up. Just thought it might be easier if I had a little light reading material that described how the chip on it was supposed to work. :) Have you tried booting with the "-v" flag to see the driver's probe messages? Did you configure the card using the configuration "pnp" commands before the kernel starts booting? For example, config pnp 1 0 irq0 9 port0 0x360 enable os did the trick for me on a 3.2 system. When I did an install of FreeBSD 3.2 on a machine with a 3C515, I was pleasantly surprised when sysinstall captured this config command and wrote it to the kernel.conf file for future boots. Anyway, the card is mostly the same as the Vortex (if_vx.c driver) but with some ports changed. Sorry, I don't happen to have tech docs for the card. Good luck, Guy Guy Helmer, Ph.D. Candidate, Iowa State University Dept. of Computer Science Research Assistant, Ames Laboratory --- [EMAIL PROTECTED] Research Assistant, Dept. of Computer Science --- [EMAIL PROTECTED] http://www.cs.iastate.edu/~ghelmer To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: DOC volunteer WAS:RE: userfs help needed.
I prefer to work in flat ASCII. Perhaps the doc project can HTMLize the final product. Currently, I'm hatching a plan to free up an AMD386/40 running FreeBSD 2.1.5 for use as a 4.x sandbox. The little thing has been faithfully and unerringly natd-ing my whole Solaris/NetBSD/Linux/ UNIXWare LAN onto the big cloud for more than 2 years. It really is emotionally difficult to pull the little box out of production. It has a personality and everything. Anyway, Mr. Dillon, once I have a development box to smack around, I intend to start with your suggestion of implementing a filesystem of my own concoction by returning an error for all VOP calls and issuing a kernel printf. How visible will the new VOP code be to me at this level? The Penguins are rewriting the bejesus out of their VFS system to the point where all the existing FS code must be redone to conform. Please debifurcate: 1) Any attempt from-scratch FS development should definitely wait for the new VFS code. Start now and you'll only end up rewiting in the Fall. 2) Hack away. All changes will be completely transparent to the FS coder. Your code, as well as everything in 2.x and 3.x will drag and drop right into the new model and build like the very wind. Thanks -Original Message- From: Nik Clayton [mailto:[EMAIL PROTECTED]] Sent: Friday, July 30, 1999 05:09 To: Alton, Matthew Cc: 'Matthew Dillon'; David E. Cross; [EMAIL PROTECTED] Subject: Re: DOC volunteer WAS:RE: userfs help needed. On Thu, Jul 29, 1999 at 10:45:51AM -0500, Alton, Matthew wrote: I ran screaming into the woods last year from trying to grok VOP_foo. The prospect of a rewrite fills me with warmth and fuzziness. I hereby volunteer to maintain the VOP(9) man pages and to flesh out my notes into a big, beefy FS-implementer's Guide to the new VOP_foo. All the coders have to do is answer my (at least marginally clueful) questions along the way and keep posted as to what's going on in new VM-land. I won't even pester anyone with design suggestions... very much. Solid offer, kids. Everybody wins. Give it some thought. Sounds good. Yell for any assistance you need from the Doc. Proj. In particular, the FS Implementer's Guide sounds good. What (markup) language were you planning on using? N PS: Reply-to: should probably be set to doc, but I've left that at your discretion. -- [intentional self-reference] can be easily accommodated using a blessed, non-self-referential dummy head-node whose own object destructor severs the links. -- Tom Christiansen in [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: No elf(5) man page (docs/7914)
* Nik Clayton ([EMAIL PROTECTED]) [990730 23:37]: Hi folks, We have an a.out(5), but no elf(5) (as pointed out in docs/7914). Does anyone feel up to writing one? Saw it before, noticed it, placed on my to-do list. If no-one objects I'll submit a manpage per a.out(5) style tomorrow for review untill it's ready for inclusion. -- Jeroen Ruigrok van der Werven asmodai(at)wxs.nl The BSD Programmer's Documentation Project http://home.wxs.nl/~asmodai Network/Security SpecialistBSD: Technical excellence at its best Cum angelis et pueris, fideles inveniamur. Quis est iste Rex gloriae...? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Is the _Device Driver Writers Guide_ still apropos?
* Nik Clayton ([EMAIL PROTECTED]) [990730 23:37]: -hackers, Is the FreeBSD Device Driver Writers Guide at http://www.freebsd.org/tutorials/ddwg/ddwg.html still correct? I know there have been changes to this area of the tree over the past 6 months or so, but I don't know how much of that document is still appropriate for what we have now. As far as I have been able to learn and glance from Bill Paul and some other device driver guru's too much has changed in order for it to be correct under CURRENT. A rewrite is on my to-do list for the PDP, however, should people feel like they want to rewrite it, be my guest and cc: me on any submissions so I can include it in the PDP. Else I will start this ASAHP. -- Jeroen Ruigrok van der Werven asmodai(at)wxs.nl The BSD Programmer's Documentation Project http://home.wxs.nl/~asmodai Network/Security SpecialistBSD: Technical excellence at its best Cum angelis et pueris, fideles inveniamur. Quis est iste Rex gloriae...? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mentioning RFC numbers in /etc/services
On Wed, 28 Jul 1999, Matthew Dillon wrote: : Sheldon Hearn [EMAIL PROTECTED] writes: : I plan to mention in the comments for each service in /etc/services, the : latest RFC describing the service. : : Good idea. : : H... I'm not sure what this gets us. Wouldn't it be better to :place this kind of information in the man page that you suggest below? As :often as /etc/services gets read, do we really want to bloat it with :non-functional information? :... :Doug I kinda like the idea of putting the RFC numbers in comments in /etc/services. It makes the comments more meaningful. I still haven't heard anyone answer the two key (IMO) questions. Why is it better to have this in the file than in a man page, and how do you justify the additional cost to parse the file for every single system call that uses it? Please note that I _do_ think that the information is valuable. My only concern is that putting it IN the file has more costs than benefits. It would be nice if we DBM'd /etc/services. Well that would definitely solve the problem of the "cost" of comments that I mention above, and it could also be handy in the sense that you could build an error-checker into the dbm'ing process. However this raises additional questions, like how to deal with the situation where the file is updated but the db is not. Thinking in mergemaster terms, I already have to special case master.passwd for this reason, and probably should special case login.conf too (just made myself a note). Doug -- On account of being a democracy and run by the people, we are the only nation in the world that has to keep a government four years, no matter what it does. -- Will Rogers To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mentioning RFC numbers in /etc/services
On Thu, 29 Jul 1999, Ben Rosengart wrote: On Thu, 29 Jul 1999, Josef Karthauser wrote: Ok - but it's a bit misleading having both values in /etc/services.. Shouldn't be: http 80/tcpwww www-http #World Wide Web HTTP http 80/udpwww www-http #World Wide Web HTTP Should be: http 80/tcpwww www-http #World Wide Web HTTP http 80/udp #[not used] Don't you think? At least that way you don't have to read all of the rfcs to construct a firewall ;). And the output of netstat (trafshow, etc.) would be considerably easier to read. The -n option to trafshow disables number-name translation for both addresses and ports, although that might be more than what is wanted. I do know what you mean though. On some of the machines I administer I have some custom entries for /etc/services that make more sense than the defaults, especially for the ports 1023. Doug -- On account of being a democracy and run by the people, we are the only nation in the world that has to keep a government four years, no matter what it does. -- Will Rogers To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mentioning RFC numbers in /etc/services
On Fri, 30 Jul 1999, Doug wrote: The -n option to trafshow disables number-name translation for both addresses and ports, although that might be more than what is wanted. I do know what you mean though. On some of the machines I administer I have some custom entries for /etc/services that make more sense than the defaults, especially for the ports 1023. Yeah, it's not that I don't want to see the service names, it's just that I want to see the accurate ones and not the inaccurate ones. As you implied, that often means the ones below port 1024 and not the others. -- Ben UNIX Systems Engineer, Skunk Group StarMedia Network, Inc. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
On Fri, Jul 30, 1999 at 10:56:55PM +0900, Daniel C. Sobral wrote: I said that I did not care whether the thing is inside or outside the regexp library. Yes, although I think at this point it's obvious we're coming at this discussion from fairly different perspectives. By the time you brought-up complexity originally, I had more or less decided that I did not want to see the new grep imported without significant speed improvements and was concerned with how to improve grep. Your interest is in debating that point (fortunately arguing for the side I agree with :). 4) grep -e 123 456 world.build [I assume "grep -e 123 -e 124 world.build"] One can clearly see that GNU grep has a much better complexity in the cases of longer patterns or multiple patterns with common prefix. Alright, someone else already mentioned to me in email that I totally ignored what differences involved multiple patterns. Combining multiple patterns is a big win if those two patterns have a common prefix (I hadn't considered the case of similar patterns before, actually). Combining multiple patterns when they're dissimilar doesn't appear to help much (which is the only case I had considered -- my mistake, and also the reason I ignored what you said about multiple patterns). I'm surprised by the way GNU grep is able to handle longer patterns, and I probably wouldn't have noticed it unless I'd taken some time to examine the GNU source. Congratulations, you win. :) The rest of your lengthy message mostly goes on to repeat the fact that GNU grep is able to merge multiple patterns with a common prefix (and postfix?) to good effect. It also shows that the new grep spends a lot of time in an activity not related to the search itself, since it does multiple patterns by Well, duh. This is really why my reaction to "complexity analysis" is (still) what it is. Complexity analysis is almost only useful for comparing two different algorithms and the fact that the new grep spends a lot of time doing things other than pattern searching is quite obvious after a casual perusal of the source. Complexity analysis does not (directly) help improving an algorithm. With the possible exception of the idea of merging common prefixes, most of this is not useful (at this stage) to improving grep. If I was going to propose replacing the existing GNU grep, I would (and always would have) done considerable more speed trials than the simple one in my last message. It would seem that GNU grep is superior in the case of partial matches without a full match too, but the standard deviation for the That is almost certainly something inside the regex library, which I have repeatedly said I'm not interested in even looking at. If our regex library is too slow, then we need to look into the newer one the Henry Spencer is rumoured to be sitting on. -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
On Fri, Jul 30, 1999 at 03:27:20PM +0200, Dag-Erling Smorgrav wrote: it was VERY simple to do... and attached is the patch... this uses the option REG_STARTEND to do what the copy was trying to do... all of the code to use REG_STARTEND was already there, it just needed to be enabled.. Funnily, I experience a near-doubling of running time with similar patches. Strange... His patches made grep on my system much faster than the original 0.10 and almost as fast as GNU grep. b$ /usr/bin/time ./grep-10 -e printer longfile /dev/null 1.16 real 0.97 user 0.19 sys b$ /usr/bin/time ./grep-10-jmg -e printer longfile /dev/null 0.48 real 0.43 user 0.04 sys b$ /usr/bin/time grep -e printer longfile /dev/null 0.28 real 0.09 user 0.18 sys This is one of the original Celerons, FWIW. Once-in-a-while that gives me performance numbers somewhat different from any other Intel. -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: docs for 3Com 3C515-TX NIC?
On Fri, 30 Jul 1999, Guy Helmer wrote: # Have you tried booting with the "-v" flag to see the driver's probe # messages? Did you configure the card using the configuration "pnp" # commands before the kernel starts booting? For example, I did the former but not the latter. I'll stick it in a machine here at home (right after dinner) and give it a try. # config pnp 1 0 irq0 9 port0 0x360 enable os # # did the trick for me on a 3.2 system. When I did an install of FreeBSD # 3.2 on a machine with a 3C515, I was pleasantly surprised when sysinstall # captured this config command and wrote it to the kernel.conf file for # future boots. # # Anyway, the card is mostly the same as the Vortex (if_vx.c driver) but # with some ports changed. Sorry, I don't happen to have tech docs for the # card. Thanks for the hints. If I have to change anything to get it to work I'll send you some diffs. -steve To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
In message [EMAIL PROTECTED] "Jordan K. Hubbard" writes: : It already is. That's not the question under discussion here - we're : talking about how to make things work in the post-installation boot : scenario. I'm in favor of having it in the kernel by default. With one proviso. Any place where we talk about locking down a FreeBSD machine, we'd need to make it explicit that bpf should be turned off when you wish to make it hard for intruders to get packets off your wire in a root compromize situation. I wonder if /dev/bpf should be disabled when secure level is 1 or 2... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
In message [EMAIL PROTECTED] "Brian F. Feldman" writes: : And how about having : if (securelevel 3) : return (EPERM); : in bpf_open()? There are no security levels 3. I'd be happy with 0. This is consistant with the meaning of "raw devices". Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
On Fri, 30 Jul 1999, Warner Losh wrote: In message [EMAIL PROTECTED] "Brian F. Feldman" writes: : And how about having : if (securelevel 3) : return (EPERM); : in bpf_open()? There are no security levels 3. I'd be happy with 0. This is consistant with the meaning of "raw devices". What about the one-way sysctls that have been suggested? -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
In message [EMAIL PROTECTED] Alfred Perlstein writes: : What about the one-way sysctls that have been suggested? They need to be implemente dfirst. A quick securelevel 0 in bpf_open would allow many people's objections to bpf in the kernel by default. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
There are no security levels 3. I'd be happy with 0. This is consistant with the meaning of "raw devices". Would you be willing to make this change? - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
In message [EMAIL PROTECTED] "Jordan K. Hubbard" writes: : There are no security levels 3. I'd be happy with 0. This is : consistant with the meaning of "raw devices". : : Would you be willing to make this change? Yes. I will make this change tomorrow unless there is significant objections that cannot be resolved in the mean time. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
readdirplus is very cool, any other nfs client suggestions?
Spent quite a bit of time today playing around with the newly
repaired readdirplus option for nfs clients in -current. My thanks to Matt
Dillon and Bill Paul. For those that don't remember, I'm trying to use
amd/nfs client stuff on some freebsd web servers that read their data from
our sun (and now sun + netapp) web farm. I have a script that used to lock
up amd and/or nfs and/or the whole machine pretty regularly. I've run it
about 100 times today in various conditions with no ill effects. About
this I am quite pleased. :)
Since I'm new to nfs, and wasn't aware of the readdirplus option,
it dawns on me that there might be other cool things I'm not using that
also might be a benefit. I'm using amd for now, although this might be
replaced with plain old hard mounts at some time in the near future. The
following options I've cobbled together from advice on the lists, my
reading of the man pages and other documentation, and a lot of
experimentation. Any comments or suggestions for improvements would be
welcome.
Thanks,
Doug
amd.conf:
[ global ]
map_type = file
search_path =/etc
auto_dir = /usr/amd/realmounts
normalize_hostnames = yes
plock = yes
selectors_on_default = yes
[ /usr/amd/Interfaces ]
map_name = amd.Interfaces
amd.Interfaces:
/defaults type:=nfs;opts:=rw,vers=3,readdirplus,intr,proto=udp
* rhost:=IP${key};rfs:=/Space/${key}
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: readdirplus is very cool, any other nfs client suggestions?
:
: amd.Interfaces:
:
: /defaults type:=nfs;opts:=rw,vers=3,readdirplus,intr,proto=udp
:
: * rhost:=IP${key};rfs:=/Space/${key}
:
:Not to rain on your parade, but Dag-Erling informed me that there
:was problems using NFSv3 over the loopback interface that can cause
:lockups.
:
:DES: can you elaborate? you think it may cause problems with amd
:since it's like an NFS buffer isn't it and would work over the
:loopback...
:
:-Alfred
If there are easily reproducable problems with the kernel nfs client
or server, please report them to me and we can almost certainly get
them fixed.Problems related to amd are harder - it's really up to
the author to fix those if it turns out to be a bug in amd rather then
the kernel. So far, though, most of the recent bugs have been on
the kernel side.
-Matt
Matthew Dillon
[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: So, back on the topic of enabling bpf in GENERIC...
Warner Losh wrote: In message [EMAIL PROTECTED] "Brian F. Feldman" writes: : And how about having : if (securelevel 3) : return (EPERM); : in bpf_open()? There are no security levels 3. I'd be happy with 0. This is consistant with the meaning of "raw devices". Disabling bpf it will break rarpd (and also rbootd but it is less important). I think such a thing should be mentioned in documentation. -SB To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: readdirplus is very cool, any other nfs client suggestions?
On Fri, 30 Jul 1999, Matthew Dillon wrote: The only other major feature is the nq leasing stuff for cache coherency, but it is highly experimental and you probably shouldn't use it. Plus very few other OS's support it. heh... I'm just happy when the stuff works that's supposed to work. I really don't want to push my luck. There is a lot of tweaking you can do with 'normal' nfs options, such as tuning the read write block size, adjusting cache timeouts for various parameters, and so forth. man mount_nfs and look at the various -o options available. Yeah, I've been reading the various man pages and things for quite a while now and it's starting to synch in. Do you have any suggestions for additional reading? The O'Reilly book seems a bit out of date, but I'll try it if folks say it's good. Basically my problem is that there are so MANY options, and my knowledged about potential side effects is very limited. Thanks for your response, Doug -- On account of being a democracy and run by the people, we are the only nation in the world that has to keep a government four years, no matter what it does. -- Will Rogers To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: replacing grep(1)
On Fri, Jul 30, 1999 at 03:27:20PM +0200, Dag-Erling Smorgrav wrote: Funnily, I experience a near-doubling of running time with similar patches. Incidentally, it seems that it's not possible to assume that our regex library is even anywhere in the same league as the GNU regex library. b$ time ./grep -E '(vt100)|(printer)' longfile /dev/null real0m21.284s user0m22.034s sys 0m0.083s Now, with a profiled executable with optimization turned off it takes about 25 seconds. Regardless, it appears to spend 98% of its time in regexec(), which is good, since that's where it should be spending time. [I had been intending to combine multiple patterns, ultimately combining in a '\n' to avoid the memchr() in mmopen]. b$ time grep '(vt100)|(printer)' longfile /dev/null real0m0.267s user0m0.109s sys 0m0.157s 98% * 20 = ~19... Without an improved regex library, any mildly complicated pattern will bring the new grep to its knees. This could be the dfa helping GNU grep more than having a better regexp library... Probably both. I wonder how well the devel/pcre port would do POSIX regular expressions. -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Unsubscribe
Unsubscribe To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Unsubscribe
Unsubscribe To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Hey kernel hackers, this is worth a read.
http://features.linuxtoday.com/stories/8191.html A story on upcoming plans for the Linux 2.4 kernel. Since they're going after a lot of the same performance goals we are, it's worth a read. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Hey kernel hackers, this is worth a read.
On Fri, 30 Jul 1999, Jordan K. Hubbard wrote: http://features.linuxtoday.com/stories/8191.html A story on upcoming plans for the Linux 2.4 kernel. Since they're going after a lot of the same performance goals we are, it's worth a read. It seems to me that a lot of the features mentioned are already in -CURRENT and some in -STABLE. Hmm. I rather liked this paragraph: Users who connect to Windows shares via SMB will be pleased that there is no longer a compile time option for enabling bug workarounds for Win9x. Instead, Linux 2.4 will be able to detect the type of machine it is connected to and react accordingly. This will make Linux a much better option overall in homogenous networks. :^) - alex You wear guilt, like shackles on your feet, Like a halo in reverse - Depeche Mode To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Unsubscribe
On Sat, 31 Jul 1999, kylee wrote: Unsubscribe No. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Fw: thunks
- jir ji jimaria [EMAIL PROTECTED] irc#tokyo15 icq jir 3941247- http://www.enjoynight.com/cgi-bin/friends/ji/familychat.cgi VAIO PCG-C1 FreeBSD be late sorry Thunks kindness to Mr Dirk GOUDERS From: Dirk GOUDERS [EMAIL PROTECTED] Subject: Re: [FreeBSD-net-jp 1746] [FYI] Adaptec AIC-6915 "Starfire" ethernet controller driver and plus question compaq presario dec et Date: Thu, 22 Jul 1999 09:55:05 +0200 o.k., Bill, I'll try to translate it for you: To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Changing Interface IP address
I would like to change the IP address (netmask, gateway etc.) of an Interface (eg. fxp0) from within my C source code. 1. Is this possible to do without the SIOC ioctl call? (i am already in the kernel). Make the call from within the kernel. 2. Which structure and which member should I write the new information to (ifaddr?)? Call the ioctl, which will do whatever needs to be done. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: No MAXUID ?
I've come up empty-handed hunting for a constant that defines the maximum UID supported by the system. I'm working on our passwd and pwd_mkdb stuff and want to get rid of the artificial limitation of 65535 (USHRT_MAX) imposed in (at least) pwd_mkdb. Have I missed a useful define, or should I add one? If I should add one, does it go in pwd.h, types.h or syslimits.h? It probably belongs in param.h, and you can probably safely calculate it as (uid_t)0 - 1; -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: reserved/local ioctl values?
I'l looking at defining about a dozen ioctl calls for a local device driver. When looking at the _IO, _IO, _IOW, _IOR, and _IOWR macros, I'm interested if there are any reserved or local values for the first parameter? Not really, as such. In short, I'd hate to use a seemly unused value, just to suddenly be in conflict with a major set of ioctls (particularly terminal, as this is the type of driver it will be). Also, would it be wise to reuse an existing value? If you want to add private ioctls to a given subsystem, either pick an arbitrarily high function number, or another letter that's not commonly associated with that subsystem. Eg. I would probably pick 'z' if I were going to add a local ioctl to the tty subsystem. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: BSDI Pthread + gdb (Re: Free BSDI CD!)
On Thu, 29 Jul 1999, Martin Cracauer wrote: In owner-freebsd-hackersatfreebsd.org--19990727011825.11692.rocketm...@web805.mail.yahoo.com, Rayson Ho wrote: Hi, I am not sure whether this is known to this list or not, but here is the URL for ordering: http://www.bsdi.com/products/evalcd/ The thing that gets my attention is the gdb with threads support. Surely they need to provide source for the GPL gdb. Anyone knows what kind of Pthread library/kernel support they have? I have some patches to get our gdb to understand the uthread version of pthreads. It still needs a bit of work and I have been (extremely) busy with unrelated projects recently. I may be able to finish it after SIGGRAPH. -- Doug Rabson Mail: d...@nlsystems.com Nonlinear Systems Ltd. Phone: +44 181 442 9037 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Changing Interface IP address
Mike Smith wrote: I would like to change the IP address (netmask, gateway etc.) of an Interface (eg. fxp0) from within my C source code. 1. Is this possible to do without the SIOC ioctl call? (i am already in the kernel). Make the call from within the kernel. But I was under the impression that I can't do ioctl calls in the kernel? Which call should I then make from within the kernel? Thanks for the Help To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Changing Interface IP address
Mike Smith wrote: I would like to change the IP address (netmask, gateway etc.) of an Interface (eg. fxp0) from within my C source code. 1. Is this possible to do without the SIOC ioctl call? (i am already in the kernel). Make the call from within the kernel. But I was under the impression that I can't do ioctl calls in the kernel? Which call should I then make from within the kernel? The current BSD kernel architecture does not well handle ABI calls from within kernel space, no. You will probably have to duplicate the code that handles the SIOC[AD]IFADDR in netinet/in.c in your module. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: FTPd authentication with PAM/MySQL
hi, there! On Mon, 26 Jul 1999, Steven Fletcher wrote: I've seen the PAM modules/libraries/etc for MySQL and noticed that the FTPD Makefile has a Kerberos PAM option, and was wondering if anyone knows of a way to get FTPd talking to MySQL... or if it would work at all? ftpd PAM patches (and other PAM stuff) are available at http://iclub.nsu.ru/~fjoe /fjoe To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
RE: freebsd-hackers-digest V4 #565
Sorry I know i should not mail to the mailinglist. But I can not unsubscribe Please help me To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Documenting writev(2) ENOBUFS error
w...@softweyr.com (Wes Peters) writes: [ENOBUFS] Insufficient system buffer space exists to complete the op- eration. Do you know what kind of circumstances that error *really* occurs under? If it happened with files, that would be a bug and should be fixed. The call is supposed to block to wait for writes to be possible. This applies to stream sockets in most cases, as well. Based on a quick look at the code, out-of-band TCP data seems to be the only case where ENOBUFS might be returned for streams, and that obviously doesn't apply to write/writev. As I mentioned to Nik in private mail, for datagram sockets, the description in send(2) more or less applies. Programs should generally use send rather than write for such objects, anyhow. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: No MAXUID ?
On Fri, 30 Jul 1999 00:53:56 MST, Mike Smith wrote: It probably belongs in param.h, and you can probably safely calculate it as (uid_t)0 - 1; Excellent. Another question I should have asked in my original mail is this: are there magical reasons why we should want pwd_mkdb to bleat for every encountered UID greater that 65535 ? Can you think of anything other than hysterical raisins why I shouldn't bump that artificial limit to the new MAX_UID when that arrives? Thanks, Sheldon. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
HOWTO: FreeBSD with integrated IPv6
Hi! As I promised earlier, here is some information how to integrate KAME and PAO into the FreeBSD 3.2-RELEASE source tree. The current status is: - FreeBSD+KAME - no conflicts during merge - make world is working - systems are up and running without any problems :-) - FreeBSD+PAO - no conflicts during merge - hasn't been tested (because I need IPv6) - FreeBSD+KAME+PAO - one conflict during merge (if_ed.c) but it's trivial to fix - hasn't been tested (I didn't have time to test this) - make release hasn't been tested (could some test this in FREEBSD+KAME and FREEBSD+KAME+PAO and report if it's not working) - disk usage is not optimized at all (current requirement with all the working files is about 1 GB !!) - we have been using this internally for awhile now and it seems to work quite well (although it's slow :-) The document, configuration files and scripts can be found at http://www.hut.fi/~kuparine/papers/kame/kame.html Comments, improvements, patches etc. are welcome. I'll be on vacation so I won't be reading my Ericsson mails until September 6th (but you can send email to my HUT-address). Have fun and welcome to the wonderful world of IPv6! Martti PS. Should we setup a cvsup server somewhere (not in the USA) which would have these branches? KAME people, what do you say of having such a service in one of your servers? --- Martti Kuparinen martti.kupari...@ericsson.com http://www.hut.fi/~kuparine/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: (KAME-snap 936) HOWTO: FreeBSD with integrated IPv6
PS. Should we setup a cvsup server somewhere (not in the USA) which would have these branches? KAME people, what do you say of having such a service in one of your servers? guys at imasy.or.jp has been doing it for FreeBSD228+KAME+PAO for a while, so please feel free to start (if you feel like so, of course). itojun To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Tim Vanderhoek vand...@ecf.utoronto.ca writes: I do. Still far too slow. I'll work on this tomorrow, since that seems the only way to convince people that mmap is not such a big win. :-( mmap() gives a fourfold speed increase. I call that a big win. I have a few other ideas which will make 0.11 even faster. DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
James Howard howar...@wam.umd.edu writes: DES tells me he has a new version (0.10) which mmap()s. It supposedly cuts the run time down significantly, I do not have the numbers in front of me. Unfortunetly he has not posted this version yet so I cannot download it and run it myself. It's in the usual place (ftp://ftp.ofug.org/pub/grep/). He also says that if mmap fails, he drops back to stdio. This should only happen in the NFS case, the 2G case, etc. Any case in which a) the file is too large to mmap, b) the file is not a regular file, or c) mmap() fails (e.g. NFS). DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
John-Mark Gurney gurne...@efn.org writes: it was VERY simple to do... and attached is the patch... this uses the option REG_STARTEND to do what the copy was trying to do... all of the code to use REG_STARTEND was already there, it just needed to be enabled.. Funnily, I experience a near-doubling of running time with similar patches. DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Dag-Erling Smorgrav d...@flood.ping.uio.no writes: John-Mark Gurney gurne...@efn.org writes: it was VERY simple to do... and attached is the patch... this uses the option REG_STARTEND to do what the copy was trying to do... all of the code to use REG_STARTEND was already there, it just needed to be enabled.. Funnily, I experience a near-doubling of running time with similar patches. To be precise, I experience a 30% decrease in system time and a 100% increase in user time when I use RE_STARTEND and eliminate the malloc() / memcpy() calls in procfile(). DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: No MAXUID ?
Sheldon Hearn sheld...@uunet.co.za writes: Another question I should have asked in my original mail is this: are there magical reasons why we should want pwd_mkdb to bleat for every encountered UID greater that 65535 ? How many times do I have to go through this? There is no artificial limitation in pwd_mkdb. pwd_mkdb warns against UIDs larger than 65535 because legacy software that uses unsigned short instead of uid_t will break with large UIDs. There were even a few such cases in our tree that I fixed less than a year ago IIRC. DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: No MAXUID ?
On 30 Jul 1999 15:38:30 +0200, Dag-Erling Smorgrav wrote: How many times do I have to go through this? Until I stuff a comment in the source that explains this. :-) There is no artificial limitation in pwd_mkdb. pwd_mkdb warns against UIDs larger than 65535 because legacy software that uses unsigned short instead of uid_t will break with large UIDs. Would you be happy with changing things so that only one warning is generated? Something like 9 max_uid 65535: others may exist? The current behaviour is quite annoying with large passwd files. :-) Ciao, Sheldon. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: No MAXUID ?
Sheldon Hearn sheld...@uunet.co.za writes: Would you be happy with changing things so that only one warning is generated? Something like 9 max_uid 65535: others may exist? The current behaviour is quite annoying with large passwd files. :-) Sure, and maybe modify the warning to say something like legacy software may not support UIDs larger than 65535 DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
John-Mark Gurney wrote: ok, I just made a patch to eliminate the copy that was happening in procfile, and it sped up a grep of a 5meg termcap from about 2.9sec down to .6 seconds... this includes time spent profiling the program.. GNU grep w/o profiling only takes .15sec so we ARE getting closer to GNU grep... Rather impressive. But... did you run these tests more than once, to account for vm caching? it was VERY simple to do... and attached is the patch... this uses the option REG_STARTEND to do what the copy was trying to do... all of the code to use REG_STARTEND was already there, it just needed to be enabled.. Just for the record... :-) This eliminates one of the added complexities I pointed out. -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org Is it true that you're a millionaire's son who never worked a day in your life? Yeah, I guess so. Lemme tell you, son, you ain't missed a thing. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Tim Vanderhoek wrote: I'm sorry. I've read your message and have decided that you're wrong. Not that you did bother to counter the points I made. You only comment on the one thing I said was probably insignificant. Are you taking your clues from me? :-) Outside of the regexp library, algorithmic complexity is not a factor here. It would take a beanbag to write anything other than an O(N) algorithm. I said that I did not care whether the thing is inside or outside the regexp library. And a N*search+N*copy, as opposed to N*search, *is* relevant. And that N*copy is outside regexp. And, just for the reference, GNU Grep uses a dfa to identify likely matches before letting gnuregexp work. The proposed grep is slow, very slow, and I've sent a long message to James outlining how to make it much faster, but algorithmic complexity is not an issue. So you say without having checked. The test you suggested doesn't show anything about that algorithmic complexity, though. Yeah? Try to back that with the results of the tests I suggested. No, it's not even worth my time. Now look. You've gotten me so upset I actually went and did a simple test. The test showed I'm right and you're wrong. Catting X number of copies of /etc/termcap into longfile causes the time grep uses to pass longfile searching for all occurrences of printer causes it to use an extra 0.03 seconds for every repetition of /etc/termcap in longfile. Gee, linear complexity wrt to file length. Who could've guessed!? That does not *begin* to cover the cases I outlined. What'ya bet GNU grep also exhibits linear complexity? :) Admit it, you jumped in with some bullshit about complexity when had you taken the time to look into what James meant when he said it now spends 50% of its time in procline() you would have kept quiet, realizing that he was talking about a constant factor in the complexity analysis, an subject where comments such as it now spends 50% of its time in procline() are relevent. Ok, here is the _DATA_ backing my bullshit. First table: searching for non-existent patterns Tests: 1) grep -e 123 world.build 2) grep -e 123456 world.build 3) grep -e 123 124 world.build 4) grep -e 123 456 world.build These were made with GNU grep, the version 0.9 of the new grep, and that version with the patch I sent previously (this later was non-intended -- only after completing the test I realized the executable was the one with my patches). Each test was repeated five times after both the executable and the target file were cached. I show here the averages of the line real for time. The user and sys values were actually more interesting, but with much greater deviation. :-) GNU grepNew grepPatched grep 1) 0.09945s0.4460s 0.3870s 2) 0.07225s0.4424s 0.3894s 3) 0.12200s0.6352s 0.5814s 4) 0.18240s0.6364s 0.5796s One can clearly see that GNU grep has a much better complexity in the cases of longer patterns or multiple patterns with common prefix. It also shows that the new grep spends a lot of time in an activity not related to the search itself, since it does multiple patterns by calling regexec() multiple times, but 2:1 is not the proportion you see up there. Also, the patch I introduced to eliminate N*(malloc()+free()), N being the number of lines searched, significantly reduces that overhead (overhead as in, *beyond* the time spent in regexec()). Second table: searching for existing patterns Tests: 1) grep -e net world.build /dev/null 2) grep -e netipx world.build /dev/null 3) grep -e netinet world.build /dev/null 4) grep -e netinet -e netipx world.build /dev/null GNU grepNew grep 1) 0.10750s0.57060s 2) 0.07575s0.46375s 3) 0.07416s0.46700s 4) 0.09950s0.67440s Though these tests involve more factors because each has a different number of matches, it again shows very clearly that the new grep has increased complexity in the case of multiple patterns. See there, cases 1 and 4. The latter has *less* matches than the former. Third table: non-existing pattern on different sized files Tests: 1) grep 123 world.build 2) grep 123 world.build.2 (two times world.build) 3) grep 123 world.build.3 (three times world.build) 4) grep 123 world.build.4 (four times world.build) GNU grepNew grep 1) 0.09600s0.44750s 2) 0.16425s0.89075s 3) 0.24760s1.30850s 4) 0.31833s1.75900s Linear, it would seem... but, alas, this is to be expected. Grep searches inside lines, and the above does not increase the size of a line, only the number of them. Still, it's a relief that the new grep does not have a worse performance in this most simple test. Fourth table: non-existing patterns on files with different line sizes. Tests: 1) grep abc line10 2) grep abc line20 3) grep 124 line10 4)
grep-0.11
ftp://ftp.ofug.org/pub/grep/grep-0.11.tar.gz More (gprof-assisted) speedups. DES -- Dag-Erling Smorgrav - d...@yes.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Dag-Erling Smorgrav wrote:
To be precise, I experience a 30% decrease in system time and a 100%
increase in user time when I use RE_STARTEND and eliminate the
malloc() / memcpy() calls in procfile().
Could you please test my patch that removes malloc() but bot
memcpy()? Here it is again, though against an old version:
--- util.c.orig Thu Jul 29 19:14:17 1999
+++ util.c Thu Jul 29 20:49:16 1999
@@ -107,6 +107,8 @@
ln.file = fn;
ln.line_no = 0;
+ ln.bufsize = 81; /* Magical constants, yeah! */
+ ln.dat = grep_malloc(81);
linesqueued = 0;
if (Bflag 0)
@@ -115,11 +117,14 @@
ln.off = grep_tell();
if ((tmp = grep_getln(ln.len)) == NULL)
break;
- ln.dat = grep_malloc(ln.len + 1);
+ if (ln.bufsize ln.len + 1)
+ ln.dat = grep_realloc(ln.dat, ln.len + 1);
memcpy(ln.dat, tmp, ln.len);
- ln.dat[ln.len] = 0;
if (ln.len 0 ln.dat[ln.len - 1] == '\n')
ln.dat[--ln.len] = 0;
+ else
+ ln.dat[ln.len] = 0;
+
ln.line_no++;
z = tail;
@@ -127,9 +132,9 @@
enqueue(ln);
linesqueued++;
}
- free(ln.dat);
c += t;
}
+ free(ln.dat);
if (Bflag 0)
clearqueue();
grep_close();
--- grep.h.orig Thu Jul 29 20:47:52 1999
+++ grep.h Thu Jul 29 20:48:34 1999
@@ -35,6 +35,7 @@
typedef struct {
size_t len;
+ size_t bufsize;
int line_no;
int off;
char*file;
--
Daniel C. Sobral(8-DCS)
d...@newsguy.com
d...@freebsd.org
Is it true that you're a millionaire's son who never worked a day
in your life?
Yeah, I guess so.
Lemme tell you, son, you ain't missed a thing.
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Daniel C. Sobral d...@newsguy.com writes: Dag-Erling Smorgrav wrote: To be precise, I experience a 30% decrease in system time and a 100% increase in user time when I use RE_STARTEND and eliminate the malloc() / memcpy() calls in procfile(). Could you please test my patch that removes malloc() but bot memcpy()? Here it is again, though against an old version: Yeah. You can do even better by declaring ln static and never free()ing it. DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Daniel C. Sobral d...@newsguy.com writes: Could you please test my patch that removes malloc() but bot memcpy()? Here it is again, though against an old version: Bingo. REG_STARTEND is significantly more expensive than memcpy(). DES -- Dag-Erling Smorgrav - d...@flood.ping.uio.no To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Documenting writev(2) ENOBUFS error
Ville-Pertti Keinonen wrote: w...@softweyr.com (Wes Peters) writes: [ENOBUFS] Insufficient system buffer space exists to complete the op- eration. Do you know what kind of circumstances that error *really* occurs under? If it happened with files, that would be a bug and should be fixed. The call is supposed to block to wait for writes to be possible. This applies to stream sockets in most cases, as well. Based on a quick look at the code, out-of-band TCP data seems to be the only case where ENOBUFS might be returned for streams, and that obviously doesn't apply to write/writev. The only way writev can pick up an ENOBUFS error is in the call to (*fp-f_ops-fo_write)(fp, auio, fp-f_cred, 0) on line 447 (in -CURRENT). ENOBUFS can happen on any network object if the system is completely out of mbufs, right? So it could return ENOBUFS for any connected socket specified as the fd in a write or writev operation. -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: No MAXUID ?
On Fri, 30 Jul 1999 00:53:56 MST, Mike Smith wrote: It probably belongs in param.h, and you can probably safely calculate it as (uid_t)0 - 1; Excellent. Another question I should have asked in my original mail is this: are there magical reasons why we should want pwd_mkdb to bleat for every encountered UID greater that 65535 ? v2 NFS doesn't support UIDs 65535, and UIDs around that number are magic to it as well. There are serious security issues here (files will appear to be owned by the wrong user). Can you think of anything other than hysterical raisins why I shouldn't bump that artificial limit to the new MAX_UID when that arrives? I think that the administrator should be forced to override the warning manually to indicate that they are aware of the issues they are getting themselves in for, or at the very least that there should be some very serious warnings placed in the relevant manpages (mount_nfs, passwd(5), vipw, etc) covering these issues.a -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Documenting writev(2) ENOBUFS error
:w...@softweyr.com (Wes Peters) writes: : : [ENOBUFS] Insufficient system buffer space exists to complete the op- :eration. : :Do you know what kind of circumstances that error *really* occurs :under? : :If it happened with files, that would be a bug and should be fixed. :The call is supposed to block to wait for writes to be possible. This I am almost certain that this error can only occur when writing to sockets, and only then of the network mbuf pool is completely exhausted. UDP is probably the most vulernable. -Matt To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
:Your new behavior flag isn't known by vm_map_simply_entry, so :map simplification could drop the behavior setting on the floor. :I would prefer that the behavior is folded into eflags. : :Overall, I agree with the direction. Behavior is correctly a map :attribute rather than an object attribute. : : Alan : :P.S. The MADV_FREE's by malloc/free were disabled by default in -CURRENT :and -STABLE prior to the release of 3.2. They were a performance :pessimization, slowing down make world by ~2%. No problem, I think the behavior can easily be folded into eflags, which deals with the map simplification code as well. I'll work up a new patch. - I'm not sure I see how MADV_FREE could slow performance down unless, perhaps, it is the overhead of the system call itself. e.g. if malloc is calling it on a page-by-page basis and not implementing any hysteresis. 2% isn't a big deal. MADV_FREE theoretically makes a big impact on paging performance in a heavily loaded paging system. If your tests were run on a system that wasn't paging, you weren't testing the right thing. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: replacing grep(1)
Daniel C. Sobral scribbled this message on Jul 30: Dag-Erling Smorgrav wrote: To be precise, I experience a 30% decrease in system time and a 100% increase in user time when I use RE_STARTEND and eliminate the malloc() / memcpy() calls in procfile(). Could you please test my patch that removes malloc() but bot memcpy()? Here it is again, though against an old version: wierd, I was running your patch, and at first I would get from .69 up to 1.03 seconds run time, but I can't seem to generate that problem right now... w/ your patches I'm getting around .67 to .7 seconds for: time ./grep THIS /tmp/ports/freegrep/work/grep-0.10/termcap.long /dev/null 0.68 real 0.63 user 0.03 sys 0.67 real 0.65 user 0.01 sys 0.67 real 0.63 user 0.03 sys 0.67 real 0.63 user 0.03 sys 0.67 real 0.66 user 0.00 sys 0.67 real 0.64 user 0.02 sys summary of gprof output: [3] 50.10.020.21 108213 procline [3] [4] 46.70.020.19 108213 regexec [4] [7] 28.50.130.00 108214 mmfgetln [7] [10] 4.80.000.022393 grep_realloc [10] with my patch and the exact same command, I get .58 to .59 seconds... 0.58 real 0.54 user 0.03 sys 0.58 real 0.53 user 0.04 sys 0.58 real 0.55 user 0.02 sys 0.58 real 0.57 user 0.00 sys 0.59 real 0.55 user 0.02 sys 0.58 real 0.55 user 0.02 sys summary of gprof output: [3] 57.10.040.19 108213 procline [3] [4] 48.00.020.17 108213 regexec [4] [7] 34.10.130.00 108214 mmfgetln [7] [10] 2.00.010.00 1 _munmap [10] (I include _munmap because realloc/malloc/free are in the 0.0% on my patch) and grep 0.10 w/o patches: 2.82 real 1.63 user 1.12 sys 2.79 real 1.53 user 1.20 sys 2.80 real 1.65 user 1.08 sys 2.84 real 1.67 user 1.10 sys 2.82 real 1.67 user 1.08 sys 2.91 real 1.66 user 1.14 sys summary of gprof output: [5] 55.11.120.00 74985 _madvise [5] [7] 13.30.040.23 108213 regexec [7] [9] 8.40.000.17 108217 grep_malloc [9] [13] 6.50.130.00 108214 mmfgetln [13] all of the programs were compiled w/ the exact same options... that is I added -g -pg to the CFLAGS in the Makefile to generate profiling info.. I'm not sure about you, but on my k6/200, the STARTEND is more efficient than the memcpy/realloc, and to tell you the truth, I can't see why it'd be more effecient to copy possible multiple kilobytes of data than to just use indexes instead of modifing a ptr... right now, I'm trying to think of a way to eliminate the fgetln searching for end of line... of course this would eliminate some of the simplicity of design, but we can get a BIG speed increase if we simply don't scan for the new line unless we NEED to... and if we do, why not use regexec to search for us? -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought. -- Ralph Waldo Emerson To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
bootloader....
I'm looking at booting(embedded devices) and I've been looking at lilo boot loader code and booteasy bootloader code... does anyone know of any documentation that anyone out there has done on this topic? -- more specifically without bios calls/support? I've seen the booteasy code at: ftp://ftp.freebsd.org/pub/FreeBSD/tools/srcs/bteasy/ is there a newer version? this code is supposed to be compiled with TASM/Borland C right? is there source that can be compiled with gnu tools? I'll take any and all suggestions :) Thanks, -roy To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
On Fri, Jul 30, 1999 at 09:51:35AM -0700, Matthew Dillon wrote: I'm not sure I see how MADV_FREE could slow performance down unless, perhaps, it is the overhead of the system call itself. e.g. if malloc is calling it on a page-by-page basis and not implementing any hysteresis. It's the system call overhead. Adding (more) hysteresis would reduce the overhead by some factor, but we'd still be making unnecessary MADV_FREE calls. Calling MADV_FREE accomplishes nothing unless the system is actually short of memory. The right way to address this problem is likely to add a mechanism to the VM system that sends a signal to the process when MADV_FREE would actually be beneficial. 2% isn't a big deal. MADV_FREE theoretically makes a big impact on paging performance in a heavily loaded paging system. If your tests were run on a system that wasn't paging, you weren't testing the right thing. 99% of our user base, whose machines aren't thrashing during a make world or other normal operation, shouldn't pay a 2% penalty to produce a theoretical improvement for the 1% that are. At best, that's optimizing the infrequent case at the expense of the frequent, not a good trade-off. In any case, the man page for malloc/free explains how to change the default, if you're a part of the oppressed 1%. Alan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: patch for behavior changes and madvise MADV_DONTNEED
:In any case, the man page for malloc/free explains how to change :the default, if you're a part of the oppressed 1%. : :Alan Not with a gig of memory :-). My only concern is for the performance curve to look good and not be too jagged. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
