Re: Broadcom BCM5701 Chipset problems
Jamie Heckford wrote: I read in the mailling lists mentions of hardware checksum problems with the chipset.. did anyone know what the final outcome with this problem was and if any changes had been MFC'd? The problems were only when coupled with VLAN tagging. I'm not sure if the 5701 was succeptible to the problem, or not. The patches disabled hardware checksumming. To fix the problem on the card would require changes to the card firmware, at least (hardware assisted checksum, subtracting out the VLAN encapsulation incrementally; assumes that hardware can run a partial checksum, and it's not tied to the buffer shift registers directly). If you aren't using VLAN tagging, you shouldn't care. No, that is absolutely not correct. The checksum problems happend in many situations, depending on the chipset and other factors. The problem that resulted in the commit to disable the receive hardware checksum was caused by small packets with certain byte patterns, NOT VLAN ENCAPSULATION. -DG David Greenman-Lawrence Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Broadcom BCM5701 Chipset problems
My local copy of the -STABLE source tree leaves BGE_CSUM_FEATURES set on in the driver; is there a change that needs to be MFC'ed to turn these suckers off? ... There's no similar comment in the if_bge.c ... See rev 1.5 and 1.3.2.5 of if_bge.c. -DG David Greenman-Lawrence Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Broadcom BCM5701 Chipset problems
:Can you guys elaborate on the problem? Was it on incoming checksums, :outgoing checksums, or both? It was on incoming checksums I believe. Was the result a rejected packet that didn't get transferred, or transferred packets with bad checksums? If the latter, then it's workaroundable in software, which might be worth doing... if only rechecking packets with bad checksums. I fear the former makes more sense, though. 8-(. The chip would calculate the wrong checksum and basically say the packet was bad when it was good (and by inference, good when it was bad). I was not able to figure out what they got wrong in the algorithm, but if that were known, then it is conceivable that the problem could be fixed in software. -DG David Greenman-Lawrence Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Broadcom BCM5701 Chipset problems
David Greenman-Lawrence wrote: Was the result a rejected packet that didn't get transferred, or transferred packets with bad checksums? If the latter, then it's workaroundable in software, which might be worth doing... if only rechecking packets with bad checksums. I fear the former makes more sense, though. 8-(. The chip would calculate the wrong checksum and basically say the packet was bad when it was good (and by inference, good when it was bad). I was not able to figure out what they got wrong in the algorithm, but if that were known, then it is conceivable that the problem could be fixed in software. Was the bad packet DMA'ed in anyway, or just dropped at the card? The card doesn't drop the packet if the IP/TCP checksum is wrong. In my tests, I did a software checksum on the supposedly bad packet, and found it to be good every time. So it DMA's correctly, the checksum is just calculated incorrectly by the hardware. -DG David Greenman-Lawrence Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Broadcom BCM5701 Chipset problems
* David Greenman-Lawrence [EMAIL PROTECTED] [020513 13:10] wrote: The card doesn't drop the packet if the IP/TCP checksum is wrong. In my tests, I did a software checksum on the supposedly bad packet, and found it to be good every time. So it DMA's correctly, the checksum is just calculated incorrectly by the hardware. Probably pretty obvious, but adding a flag if bad hwsum, then try softsum probably wouldn't be too hard. Not obvious at all. How do you know if a packet is good or bad without always doing the software checksum? -DG David Greenman-Lawrence Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Broadcom BCM5701 Chipset problems
Actually, if you could find one of these, it would probably give enough information that we could figure out what kind of error it was giving. If it's just the incremental checksum 0x (-0) problem from one's complement vs. two's complement, then you could be sure that it was only bad packets being marked good. It's not. The problem is much more complicated then that. It appears that some portions are either not added in or are added incorrectly. -DG David Greenman-Lawrence Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Fatal trap 12: page fault while in kernel mode
Fatal trap 12: page fault while in kernel mode fault virtual address = 0x70 #12 0xc014f61d in panic () #13 0xc025c02f in trap_fatal () #14 0xc025bcdd in trap_pfault () #15 0xc025b883 in trap () #16 0xc0152220 in tsleep () #17 0xc016abfe in m_clalloc_wait () #18 0xc01c8b14 in nfs_realign () #19 0xc01c9653 in nfsrv_rcv () #20 0xc01701d0 in sowakeup () #21 0xc01abd7c in udp_input () #22 0xc01a1bfb in ip_input () #23 0xc01a1c5b in ipintr () This is basically telling you that there is a bug in the NFS code that is incorrectly trying to do a wait type of allocation in an interrupt context, which is not valid. You can't sleep when there is no process context. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Swapping performance
Another point that I should bring up in regards to swap performance: systems that require good swap performance will almost always have more then one physical disk to swap to. FreeBSD is very good at paging to swap with a single disk, but it kicks ass paging to swap with multiple disks because it implements a very good interleaving algorithm. I think it's also useful to point out that the performance of this test is likely going to vary greatly as it is run. This is because it is heavily affected by the order of the pageouts in terms of their physical swap block allocations. In other words, what you're really testing is how sequentially the blocks are paged out vs. how randomly at the swap block level. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Zero copy usually means zero unnecessary copies; but what someone thinks of as necessary is really based on their bias towards an existing implementation. zero copy these days has come to mean no copies that involve the CPU. In my experiance, raw memory bandwidth to DMA packets to/from main memory is not the bottleneck on modern hardware. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
I'll agree with your experience. At this point, the limiting factor is PCI bandwith, at least for general purpose hardware. I haven't found PCI bandwidth to be a problem, either, at least when using gigabit ethernet NICs on 64bit and/or 66MHz PCI. When one writes an efficient HTTP server that takes a tiny amount of memory per process and uses sendfile() to crank out the bits, the bottleneck becomes the CPU for doing context switches, packet header creation, and TCP protocol processing. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Actually, I was talking about the Super Micro 2x64 bit PCI with two Tigon III cards, with TCP processing to completion at interrupt, the problem in doing fast forwarding of flows becomes the PCI bus bandwith, whose top end is 64x66 = Um, I thought we were talking about HTTP servers, not IP routers. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Patch to remove MFREE() macro entirely
Oh what a tangled web we weave. This should be really easy for people to take a quick look at to see if I made any mistakes. I'm basically untangling the (small) mess that people made of the code while trying to use the MFREE() macro over the last N years. If nobody sees any problems it will go into -current next week some time and then be MFC'd to stable. Looks good to me. I'm definately very much in favor of killing MFREE(). -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: bge + hardware checksum hangs
It looks like the TCP recieve checksum issues weren't the only ones we had to contend with. I've got a couple of new iXsystems 2650's with 3Com 3C996-T's in them and while running cvsup I get long hangs usually resulting in a lost connection. When the machines recover I see watchdog timeout messages in /var/log/messages. The current system configuration is a bit weird in that I've got the nic hooked up to a 10/100 HUB so I'm currently running 100 half-duplex. Acting on the theory that HW checksuming had already failed in some situations, I modified the BGE_CSUM_FEATURES define to 0 and so far things seem to be working. I'm in the middle of a ports cvsup and I completed a cvsup over the 4.5 branch and tagging without a hitch. This seems to imply that at least TCP checksuming is broken across the board. The really odd thing is that I haven't had any real problems with local connections, only cvsups and possiably one hang due to a whole lot of console output over ssh. I've been able to do 10 minute long netperf runs in both TCP_STREAM and TCP_RR modes to local hosts without any hangs. Does anyone have any ideas other them the current disabling of hardware checksuming? That's probably fine for now, but it's really going to suck on the core NFS server for this cluster once we're up and running. I think the brokeness is chipset revision dependant. We're using SysKonnect cards here at Download Technologies extensively and have only seen the input checksum bug (which we worked around prior to deployment of the servers) - no hangs and this is with typically 30-50Mbps sustained per server out to the Internet over a two month period. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: FreeBSD performing worse than Linux?
Yes, you are correct. There is no real reason for ssh to set TCP_NODELAY on FreeBSD and, in fact, I believe it didn't used to. We should just turn it off. Nagle + delayed acks don't really get along all that well, and often result in character echoes lagging .25-.5 seconds behind the keystrokes (due to round trip + 200ms). This really makes editing files and other interactive jobs rather painful. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Does anyone know if the Broadcom BCM5700 has problems with HW csum?
David Greenman writes: David Greenman wrote: In any case, disabling it is what ClickArray ended up doing, as well, for the Tigon II, until the firmware could be fixed. We're talking about the Tigon III (bge driver for Broadcom BCM5700/BCM5701). Crap. Thanks for the info. Have you manually calculated the checksum on a bad packet to see how it's off? Yes. It's typically off by 0x1051, but varies depending on the TCP/IP header contents. Hmm.. Since you've already got the code for calculating the checksum in the driver written, why not use it? Eg, why not pass the csum up set CSUM_DATA_VALID iff the csum ends up being 0? Are you worried that the firmware will yield false posatives too? Yes, I think it can just as easily return false positives. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Does anyone know if the Broadcom BCM5700 has problems with HW csum?
Brooks Davis wrote: There was a commit to current a few hours ago disabling hardware checksums on recieve due to corruption problems. It will be MFC'd in three days though it's a two line fix so you could apply it your self: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/bge/if_bge.c.diff?r1=1.4r2=1.5 I believe you will find that the problem is related to the firmware handling of VLAN tagging, and that the problem only exists if VLAN tagging is enabled. You would believe wrongly, then, because the problem that I was seeing did not involve VLAN tags. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Does anyone know if the Broadcom BCM5700 has problems with HW csum?
I am playing with a driver for the Broadcom 5700/5701. It recognizes the 5700 in my 3Com cards OK, but seems to screw up the TCP checksum. Switching off hardware checksum capability fixes it. Does anyone know the details of which stepping this stuff worked on? I haven't nailed down the problem that I've seen with them to a specific chipset, but I can confirm that they incorrectly calculate the checksum on input packets in some cases. It seems to be related to both packet size and certain TCP options (or lack of them). I've only seen the problem occur with very small (0-4 byte payload) packets. In any case, after discussing this problem with Bill Paul, I disabled input checksum in the -current driver and intend to merge that to -stable in a few days. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Does anyone know if the Broadcom BCM5700 has problems with HW csum?
* David Greenman [EMAIL PROTECTED] [011215 03:12] wrote: Brooks Davis wrote: There was a commit to current a few hours ago disabling hardware checksums on recieve due to corruption problems. It will be MFC'd in three days though it's a two line fix so you could apply it your self: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/bge/if_bge.c.diff?r1=1.4r2=1.5 I believe you will find that the problem is related to the firmware handling of VLAN tagging, and that the problem only exists if VLAN tagging is enabled. You would believe wrongly, then, because the problem that I was seeing did not involve VLAN tags. You're probably incorrect, it doesn't matter if vlan tags are active or not, it's most likely wheather or not the firmware is being asked to handle them at all. I would think it would get the checksum wrong most of the time if that were the case. It seems to only have problems with small packets, but the behavior is pretty strange, so who knows. Do you have some specific knowledge about Broadcom and brokeness related to VLAN tag support when not using VLANs? -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Does anyone know if the Broadcom BCM5700 has problems with HW csum?
David Greenman wrote: In any case, disabling it is what ClickArray ended up doing, as well, for the Tigon II, until the firmware could be fixed. We're talking about the Tigon III (bge driver for Broadcom BCM5700/BCM5701). Crap. Thanks for the info. Have you manually calculated the checksum on a bad packet to see how it's off? Yes. It's typically off by 0x1051, but varies depending on the TCP/IP header contents. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Does anyone know if the Broadcom BCM5700 has problems with HW csum?
David Greenman wrote: Alfred Perlstein wrote: You're probably incorrect, it doesn't matter if vlan tags are active or not, it's most likely wheather or not the firmware is being asked to handle them at all. I would think it would get the checksum wrong most of the time if that were the case. It seems to only have problems with small packets, but the behavior is pretty strange, so who knows. Do you have some specific knowledge about Broadcom and brokeness related to VLAN tag support when not using VLANs? If it's very small payload, it's probably a byte-order-in-buffer issue (several Eagle manufactured cards had similar problems, and so did the NE1000, when it came to DMA transfers, back when 16 bit transfers were new 8^). The packet itself is fine, it's just the checksum that the hardware calculates is wrong. For VLANs, yes, there are specific problems known with the Broadcom cards when the firmware support for VLANs is enabled. The first card known to work with checksum offload enable and VLAN support enabled (whether it's used or not) is the Tigon III. I don't know if Bill Paul fixed the firmware for the Tigon II in this case (he has been known to hack Tigon II firmware), but it could have been fixed by now. In any case, disabling it is what ClickArray ended up doing, as well, for the Tigon II, until the firmware could be fixed. We're talking about the Tigon III (bge driver for Broadcom BCM5700/BCM5701). -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Found NFS data corruption bug... (was Re: NFS: How to make FreeBSD fall on its face in one easy step )
Very cool. Good job! -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: fxp driver - receive interrupt bundling
On http://fly.cc.fer.hr/~zec/index.html you can find a 4.4-RELEASE fxp driver source, with patches that incorporate receive interrupt bundling microcode, borrowed from the Intel's Linux e100 driver. Bundling interrupts for a couple of received Ethernet frames can significantly lower interrupt processing overhead, so if you have a really busy server or router or whatever this code can make a noticeable difference. On an 1200 MHz Athlon machine, the microcode saves around 10% of CPU utilization, with incoming traffic of 20k pps on a single interface. The code is tested on 82558 rev B0 hardware, I'd be glad to know how it works on other versions of Intel's fxp cards. Pls. send your comments, suggestions etc. to [EMAIL PROTECTED] Have fun! Nifty! -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine
Well, this has turned into a rather sticky little problem. I've spent all day going through the vnode/name-cache reclaim code, looking both at Seigo's cache_purgeleafdirs() and my own patch. Can you forward me your patch? I'd like to try it out on some machines in the TSI lab. The bigger problem is exactly as DG has stated... it isn't the namei cache that is our enemy, it's the VM Page cache preventing vnodes from being recycled. For the moment I believe that cache_purgeleafdirs() or my patch solves the problem well enough that we can run with it for a while. The real solution, I believe, is to give us the ability to take cached VM Pages associated with a file and rename them to cached VM Pages associated with the filesystem device - we can do this only for page-aligned blocks of course, not fragments (which we would simply throw away)... but it would allow us to reclaim vnodes independant of the VM Page cache without losing the cached pages. I think this is doable but it will require a considerable amount of work. It isn't something I can do in a day. I also believe that this can dovetail quite nicely into the I/O model that we have slowly been moving towards over the last year (Poul's work). Inevitably we will have to manage device-based I/O on a page-by-page basis and being able to do it via a VM Object seems to fit the bill in my opinion. Hmmm. This would seem to be a step back to the days when caching was done relative to the device as opposed to the file-relative scheme we have now. One of the problems with the old scheme as I recall is that some filesystems like NFS don't have a 'device' and thus no physical block numbers to associate the cached pages with. There is also some cost in moving the pages between the file object and the device object. For these reasons, I would prefer that we keep the existing model, but just make sure that we can handle the degenerate case of one page per file object. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: More on the cache_purgeleafdirs() routine
Well, wait a sec... all I do is zap the namei cache for the vnode. The check to see if the vnode's object still has resident pages is still in there so I don't quite understand how I turned things around. In my tests it appears to cache vnodes as long as there are resident pages associated with them. Sounds like a very good first step. I would like to point out that the problem may still occur on large memory systems with a few hundred thousand tiny files (that consume just one page of memory). There really needs to be a hard limit as well - something low enough so that the FFS node KVM malloc limit isn't reached, but still large enough to not significantly pessimize the use of otherwise free physical memory for file caching. Considering that a 4GB machine has about 1 million pages and that the malloc limit hits at about 250,000 vnodes, this is an impossible goal to acheive in that case without increasing the malloc limit by at least 4X. Of course this many 1 page files is extremely rare, however, and I don't think we should optimize for it. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: mmap MAP_INHERIT question.
From: Matt Dillon [EMAIL PROTECTED] Subject: Re: mmap MAP_INHERIT question. Date: Thu, Aug 23, 2001 at 10:38:31AM -0700 MAP_INHERIT is broken and always has been. -Matt Is then a send-pr to remove the MAP_INHERIT description from mmap(2) manpage the best we can do? Now it says the following: MAP_INHERIT Permit regions to be inherited across execve(2) system calls. Perhaps this should be changed to something along the lines of the following? MAP_INHERIT This is supposed to permit regions to be inherited across execve(2) system calls, but is currently broken. Support for the flag and reference to it in the manpage should just be removed. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Recommendation for minor KVM adjustments for the release
There are two things I would like to commit for the release: - I would like to cap the SWAPMETA zone reservation to 70MB, which allows us to manage a maximum of 29GB worth of swapped out data. This is plenty and saves us 94MB of KVM which is roughly equivalent to 30,000 nmbclusters/mbufs. It's seems really hard to justify even that much SWAPMETA. A more reasonable amount would be more like 20MB. - I would like to cap the size of the buffer cache at 200MB, giving us another 70MB or so of KVM which is equivalent to another 30,000 or so nmbclusters. That also seems like overkill for the vast majority of systems. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: exec() doesn't update access time
On Wed, Jul 25, 2001 at 02:25:19PM -0700, David Greenman wrote: Guessing, I think the correct fix is probably to set the IN_ACCESS flag in ufs_open() [and similarly with other filesystems where this makes sense] if the filesystem is not mounted with the noatime flag. However, I'm not sure of the symantics of the access time in the relavent standards, and I seem to recall Bruce saying that it was incorrect to indicate an access on just an open(), but I may be mistaken. Wouldn't setting the access time on open mess up the last read time for people's mail boxes when mail was delivered? Probably. Now that you mention it, it seems kind of bogus for a file write to not be considered an access. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: exec() doesn't update access time
I noticed that exec(2) does not update the last access time of a file... is this intentional? Not exactly intentional (I never had that as a goal when I wrote execve()), but it's a side-effect of exec not doing a 'read' on the file in the traditional sense. This has been discussed several times over the past many years and the end result is that 1) Noone really seems to care very much, and 2) There are performance reducing implications if the atime update is forced. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: exec() doesn't update access time
Hmm... would it be as easy as VOP_GETATTR(); . . . VOP_SETATTR(); within the exec() code? Certainly this would be an 'easy' fix (and I can work up diffs for review), but is it the 'correct' fix? No, it's not the correct fix. You shouldn't need to do the GETATTR first, and doing a SETATTR will cause a synchronous update of the atime, which is not what you want. This also doesn't fix that standard case of open/mmap() not updating the access time, which is the real problem, not execve. Guessing, I think the correct fix is probably to set the IN_ACCESS flag in ufs_open() [and similarly with other filesystems where this makes sense] if the filesystem is not mounted with the noatime flag. However, I'm not sure of the symantics of the access time in the relavent standards, and I seem to recall Bruce saying that it was incorrect to indicate an access on just an open(), but I may be mistaken. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: exec() doesn't update access time
Hmm... would it be as easy as
VOP_GETATTR();
.
.
.
VOP_SETATTR();
within the exec() code?
Certainly this would be an 'easy' fix (and I can work up diffs for review),
but is it the 'correct' fix?
No, it's not the correct fix. You shouldn't need to do the GETATTR first,
and doing a SETATTR will cause a synchronous update of the atime, which is
not what you want. This also doesn't fix that standard case of open/mmap() not
updating the access time, which is the real problem, not execve.
Guessing, I think the correct fix is probably to set the IN_ACCESS flag in
ufs_open() [and similarly with other filesystems where this makes sense] if
the filesystem is not mounted with the noatime flag. However, I'm not sure
of the symantics of the access time in the relavent standards, and I seem
to recall Bruce saying that it was incorrect to indicate an access on just
an open(), but I may be mistaken.
Here is a patch that I just wrote that should implement the above. Please
test and report results (good or bad). :-)
-DG
David Greenman
Co-founder, The FreeBSD Project - http://www.freebsd.org
President, TeraSolutions, Inc. - http://www.terasolutions.com
Pave the road of life with opportunities.
Index: ufs_vnops.c
===
RCS file: /home/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v
retrieving revision 1.131.2.3
diff -c -r1.131.2.3 ufs_vnops.c
*** ufs_vnops.c 2001/02/26 04:23:21 1.131.2.3
--- ufs_vnops.c 2001/07/25 23:52:38
***
*** 249,255
/*
* Open called.
*
! * Nothing to do.
*/
/* ARGSUSED */
int
--- 249,255
/*
* Open called.
*
! * Update last accessed time.
*/
/* ARGSUSED */
int
***
*** 261,273
struct proc *a_p;
} */ *ap;
{
/*
* Files marked append-only must be opened for appending.
*/
! if ((VTOI(ap-a_vp)-i_flags APPEND)
(ap-a_mode (FWRITE | O_APPEND)) == FWRITE)
return (EPERM);
return (0);
}
--- 261,280
struct proc *a_p;
} */ *ap;
{
+ struct inode *ip;
+ ip = VTOI(ap-a_vp);
/*
* Files marked append-only must be opened for appending.
*/
! if ((ip-i_flags APPEND)
(ap-a_mode (FWRITE | O_APPEND)) == FWRITE)
return (EPERM);
+ /*
+* Update file access time.
+*/
+ if ((ap-a_vp-v_mount-mnt_flag MNT_NOATIME) == 0)
+ ip-i_flag |= IN_ACCESS;
return (0);
}
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: Intel ISP1100 or similar 1U experience with 4.3 stable
In a message dated 07/12/2001 6:44:38 PM Eastern Daylight Time, [EMAIL PROTECTED] writes: The cooling in the 2U solution is better than the 1U... Which is pretty much all I was recommending. Im not sure why everyone is getting so hot under the collar. You agree with me, yet you are arguing anyway. If you have the space, 2U is better. Why is that statement so irritating to you. Its a fact. you agree with it. So what is the problem? The problem is that you said that 1U solutions are inherently unreliable, which is not true. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Intel ISP1100 or similar 1U experience with 4.3 stable
In a message dated 07/10/2001 11:54:26 PM Eastern Daylight Time, [EMAIL PROTECTED] writes Its generally a bad idea to house a multi-processor system in a 1U enclosure, as there isnt enough cooling space and 3/4 fans are simply not powerful enough. Unless space is ridiculously scarce, you can get much better cooling and reliability with a 2U unit. In fact there are other considerations as well, namely the power supply, which is typically not all that beefy in 1U systems. For both of these reasons and others, our current 1U offering, although using an MP motherboard, can only be ordered with one CPU. We've recently improved the cooling in our 1U server with very powerful 40mm fans that will cool an MP system more than adequately, but the power supply issue still remains. We should have a solution for that as well in a few weeks. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Intel ISP1100 or similar 1U experience with 4.3 stable
On Wed, Jul 11, 2001 at 02:24:11PM -0400, [EMAIL PROTECTED] wrote: Its generally a bad idea to house a multi-processor system in a 1U enclosure, as there isnt enough cooling space and 3/4 fans are simply not powerful enough. Unless space is ridiculously scarce, you can get much better cooling and reliability with a 2U unit. I have not had any problems (that were related to environment) with any of my multi-processor 1U machines. There are companies[1] who know how to build them. We've never had any cooling problems with our 1U servers either and we provide fan and other enclosure monitoring under FreeBSD as well, so any temperature issues would get attention before they become a real problem. We have both center of the cabinet fans as well as multiple high output exhaust fans and can sustain multiple fan failures before things start to warm up inside. I'm not sure how this thread got moved to -hackers...it started out on freebsd-isp and really does not belong on this list. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Re. The Foundation [was Re: FreeBSD Mall now BSDCentral]
The Foundation has yet to approach Wind River about the Trademark, so I cannot speculate on their disposition. What are your plans to change this situation? The Foundation isn't planning to do anything about the trademark or in regards to any of its other proposed activities until sufficient donations have arrived. I believe that the financial statement released with our announcement makes it clear why this must be the case. How much do you think it will cost to transfer the trademark? -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ftpd....
Mike Smith wrote: Unless I'm mistaken, the current ftpd doesn't offer the functionality you're asking for either, so there's no valid reason to block the import of the lukem ftpd on these grounds. And, I'm quite certain that Luke would happily consider patches, if you were to put them forward. Neither one of them hold a candle to the load CDROM.COM can handle. How about we import dg-ftpd instead? I'm sure we'd all like to be able to support 1TB a day of data transferred... dg-ftpd isn't freeware. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: max kernel memory
:Hi, : :I'm trying to give the kernel (4.0-RELEASE) 2Gb of memory to work with. I :can afford to have 4Gb of physical memory on one of my servers, and hence :the experiments. : :Is it safe to play around with KERNBASE, and get away without breaking :code ? Is there any other advisable method if this one is risky ? : :-ASR Yes, you can change KERNBASE. I'm not entirely sure but I believe if you have an old set of bootblocks you may have to reinstall them to get a set that is kernbase-agnostic. e.g. the disklabel -B command on the appropriate slice. DG changed KERNBASE a while back to reserve a gigabyte of VM for the kernel. This should be sufficient on a 4G machine but it depends where your resources are going. If your server's resources are user-process centric then you don't need to change KERNBASE. If your server's resources are network-mbuf centric then you may have to give the kernel more KVM (e.g. like 2GB instead of 1GB... 0x8000 instead of 0xC000). But be careful. The more KVM reserved for the kernel, the less VM is available for user processes to allocate and mmap. I'm sure people who run 4G machines can give you better information, I've never run anything larger then 2G myself, though expect down the line I'll begin needing 4G machines to support larger databases. Don't forget to also change NKPDE as well when relocating the start of kernel VM. For kernbase = 0x8000 on a non-SMP machine, NKPDE needs to be 511. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: panics with 4GB on an IBM xSeries 330
] We have a 4GB IBM xSeries 330 (1GHz PIII) and I can't get 4.3-RELEASE to ] boot on it. I did set NKPT to 64 as suggested by DG about a week ago on ] this list (this is also the reason I take this to -hackers rather than ] -questions). Still, I get ] panic: swap_pager_swap_init: swap_zone=NULL ] when booting (both the modified kernel and GENERIC behave the same). An ] identical machine with 1GB works like a champ. Anything else other than ] NKPT I should set? Personally, I think NKPT is a red herring. I am running several 4G machines with the default of 32, and have not had problems (any problems will be automatically fixed for you by grow_kernel()). NKPT is the initial number of kernel page table pages. It has to be large enough so that all of the initial/bootstrap VM system data structures can be allocated. pmap_growkernel() doesn't work during this time since the VM system has not be initialized yet. The data structures can be quite large and the default of NKPT isn't large enough and a panic will result in the bootstrap prior to the device initialization. This was the problem in the first 4G problem report a few days ago, but is apparantly not the problem in the most recently reported problem. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: 4GB
Hello all, I've been experiencing some major problems running FreeBSD on 4GB of memory, and I was curious if anyone has experienced this before. Bascially, the problem is that FreeBSD just won't boot. Shortly after the kernel gains control of the system, it panics... Fatal trap 12: page fault while in kernel mode mp_lock = 000b; cpuid = 0; lapic.id = fault virtual address = 0xbff11000 fault code = supervisor write, page not present instruction pointer = 0x8:0xc027e48c stack pointer = 0x10:0xc03c4f30 frame pointer = 0x10:0xc03c4f3c 95 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = Idle interrupt mask = net tty bio cam - SMP: XXX kernel: type 12 trap, code=0 Stopped at 0xc027e48c: movl%edx,0xbfc0(,%eax,4) The IP is pointing to the pmap_map() loop. So it's clearly having problems mapping out the VM. I have yet to do any rigorous debugging, but I thought I would run this by the mailing list before I do. The system is running on a Tyan Tiger LE board with dual PIIIs (860Mhz) and 4 1GB (133Mhz) SiliconTech chips. I have tested Linux on this same configuration, and while it boots normally, it will eventually panic with a similar error once you start doing some real work. Any thoughts? The kernel is probably running out of the initially allocated kernel page table pages. Try upping it with 'options NKPT=64' in your kernel config file. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Extremely large (70TB) File system/server planning
While talking to a friend about what his company is planning to do, I found out that he is planning a 70TB filesystem/servers/cluster/db. (Yes, seventy t-e-r-a-b-y-t-e...) We could do this using about 44 of the not-yet-announced TSR-3100 fibre channel RAID storage systems. These are 1.8TB (1.62TB usable) capacity units in a 3U cabinet. It would take around 200A @ 120VAC (about 18KW) to power all of them and should fit in about 5 rack cabinets. Total cost would be about $3 million. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Realtek card support
PS. for the record: I also still have an SMC EtherEZ 10Mb UTP and a 3Com 3c503 for those who want to work on drivers for them. Both of those should work with the ed driver. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: if_fxp driver info
I don't know what list you are looking at, but the download list that I was looking at did not include SCO, Unixware or any other Unix variant except Linux. This is the list. NDIS2, NDIS3, NDIS4 and NDIS5 drivers Novell Netware* Client 3.11, 3.12 Novell Netware Server 4.1x, 5 SunSoft Solaris* SCO Unix 3, 5 SCO UnixWare* 2, 7, OpenDesktop*, OpenServer* These are "licensed" drivers. The linux driver is free. How do you know that the above drivers are developed by Intel? The above could easily be OS vendor supplied. It's anybody's guess without the source. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: if_fxp driver info
I'll look into the Linux driver, however, and see if it has anything useful in it. Historically the Linux Pro/100+ driver has totally sucked and was chalk-full of magic numbers being anded and ored. That's "chock full", and you're confusing the Becker driver (bad) with the Intel-supplied driver (slightly less bad). The intel driver seems to cover all the bases and has some nice glue routines for determining the part and features available. I havent tested it under load, but I wonder if intel would consider supporting it if someone ported it over to freebsd? they have drivers for just about every other major OS except BSD. it would be nice if the driver was updated BEFORE cards and MBs that dont work started showing up on the loading dock. Every time I get a shipment we have to hold our breath until we try one out. "drivers for every major OS"? They have drivers for Windows, Window/NT, and Linux. Of those Linux is the closest to FreeBSD, but that's like saying that a penguin is similar to a human because they are both mammals. After looking at the driver source, it's my opinion that porting it to FreeBSD would be a major undertaking and it would likely be easier to just rewrite it from scratch. In total it is more than 14,000 lines of code - much of which is highly specific to the Linux internals. The current FreeBSD driver, by comparison, is about 2600 lines of code and a whole lot easier to read and maintain. I do believe that there are some useful tidbits to be gotten out of the Intel/Linux driver, but that's about it. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: if_fxp driver info
In message [EMAIL PROTECTED], David Greenman writes: "drivers for every major OS"? They have drivers for Windows, Window/NT, and Linux. Of those Linux is the closest to FreeBSD, but that's like saying that a penguin is similar to a human because they are both mammals. Pinguins are birds... Oh, yeah. Well, even more different, then. :-) -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: if_fxp driver info
David Greenman wrote: supporting it if someone ported it over to freebsd? they have drivers for just about every other major OS except BSD. it would be nice if the driver was updated BEFORE cards and MBs that dont work started showing up on the loading dock. Every time I get a shipment we have to hold our breath until we try one out. "drivers for every major OS"? They have drivers for Windows, Window/NT, and Linux. Of those Linux is the closest to FreeBSD, but that's like saying Count in SCO as well - that's at least 3 somewhat different drivers, for OpenServer, UnixWare2 (DLPI), UnixWare7 (MDI), plus perversive customized OEM versions for Compaq and ICL. Plus probably they have drivers for Netware and Solaris too. I don't know what list you are looking at, but the download list that I was looking at did not include SCO, Unixware or any other Unix variant except Linux. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: if_fxp driver info
I guess they changed their policy on the part. I've tested the linux driver with the new part on the supermicro board and it works, so the driver is reasonably up to date. The source-available Intel driver does actually look pretty good. I don't know why David has failed to track it wrt. PHY numbers. I've dumped the last board I had with an unrecognised PHY, so I don't have any incentive to poke him about it anymore. Primarily for two reasons: 1) I didn't know that Intel had released Linux driver source, and 2) I don't have any boards that don't work correctly. I'll look into the Linux driver, however, and see if it has anything useful in it. Historically the Linux Pro/100+ driver has totally sucked and was chalk-full of magic numbers being anded and ored. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: FreeBSD vs Linux, Solaris, and NT
Your stupidity is also is emphasized by the fact that no major manufacturer has supported drivers for freebsd. Intel wont even help by providing docs. Bravo. What a WIN for the freebsd community. You've done a tremendous job marketing your concept. So that's why Intel provides free bound copies of their IA-64 books and PDF's of the other chip manuals, PXE manuals, etc. I guess all those data sheets I've seen Bill Paul pore over while working on his ethernet drivers were just figments of my imagination as well. I think he's refering to the 82559 manual. It is available from Intel to developers, but only with an NDA. For various reasons, I can't sign an NDA for that information without putting myself in legal jeopardy. That has always been true, but I was able to obtain the [now older] 82557 manual without an NDA due to a screwup at Intel - which allowed me to write the original fxp driver. Unfortunately, a few things have changed since then, especially in the SEEPROM area and the only method I have of fixing those problems these days is by reverse-engineering. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: yet another unsupported PHY in fxp driver
Say what? I lived under the impression that MAC addresses were unique. Here I have three cards that have identical MAC adresses. Finally, we finish off with: bpf: fxp0.0 attached fxp1: warning: unsupported PHY, type = 0, addr = 0 bpf: fxp1.0 attached fxp2: warning: unsupported PHY, type = 0, addr = 0 bpf: fxp2.0 attached fxp3: warning: unsupported PHY, type = 0, addr = 0 bpf: fxp3.0 attached fxp3: link media DOWN 10Mb / half-duplex fxp2: link media DOWN 10Mb / half-duplex fxp0: link media DOWN 10Mb / half-duplex fxp1: link media DOWN 10Mb / half-duplex The dmesg is from FreeBSD 2.2.8 (boot -v) by the way, but booting 4.1 yields the same results. Now (finally) my question: how do we get this particular PHY supported? This seems to have remained unanswered back in October. All of the above is caused by the SEEPROM not being read properly. Since it doesn't work with 4.1, this probably indicates that you're using an on-motherboard NIC (Supermicro?). I'm running out the door to the airport, however, and won't be able to get a fix to you until next week. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: sendfile for raw disk (was: zero copy TCP)
Both, but I may do either way, depending on which way is easier. If we can directly DMA from a disk drive to a NIC, that will be great. If the current implementation requires preloaded buffer, that works. So, where can I look for the patch? Please see sendfile(2). It seems that sendfile(2) can read only a regular file. I wiould like to modify it to send a raw disk block. Before going for it, I would like to know what was the reason for not doing this, and what may be the issus for doing this? The main issue is that sendfile() is very much VM-system centric in the way that it implements zero-copy sends. In order for raw devices to work, you would need to have a raw device vm_object to hold the pages. The problem with this is that it creates cache coherency problems with any cached file data. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: yet another unsupported PHY in fxp driver
I'm sure it's just like all the other 82559 parts, with perhaps some new features that we won't take advantage of (due to not having the documentation). If there is something useful indicated in the 'unsupported PHY" message that you mentioned (a type, for example), then it could easily be added. The 82559 has an integrated 82555 PHY, so I really doubt there is actually a new PHY to deal with. nope, type 0, addr 0. does this indicate (perhaps) another size change? It indicates that something is wrong with the SEEPROM. Is it a SuperMicro motherboard? If so, they changed the layout in the SEEPROM. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: yet another unsupported PHY in fxp driver
It seems that the NetBSD folks have eliminated this ongoing pain in the butt by using the mii interface for the intel cards. Is freebsd also moving in this direction? Every few months there seems to be a problem, and its difficult to fix it without docs The primary problems that have resulted in the "unsupported PHY" messages in the past year or so have been related to either the format or size of the SEEPROM changing. Using generic MII code doesn't fix the problem; the fxp's would still not work due to the MAC address being wrong, among other things, which is also read from the SEEPROM. This said, I think it is generally the right approach to use a generic MII PHY software interface and at some point the driver will likely be updated for that. It is low priority, however, since it doesn't solve any problems. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Moving FreeBSD towards glibc (or: FreeBSD and Hurd/Mach)
[please Cc: to me, since I'm not subscribed to this list. Thanks] are there plans to replace FreeBSD's libc with GNU glibc in the near or medium future? Linux moved also from it's own libc5 to glibc (=libc6) some time ago and it may be useful to do the same in FreeBSD too. No, that is not going to happen. The reason I'm asking this is to investigate further, wether the Hurd can be dropped into a FreeBSD system as an alternative to the current FreeBSD kernel. Using a non-FreeBSD kernel with GNU libc would pretty much make it not FreeBSD anymore. FreeBSD is what it is; if you don't like most of it, then may I suggest that you use one of the alternatives that better suits your needs. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf re-write(s), v 0.1
I'm getting the unfortunate impression that evolution is being frowned upon here. Are their other people that frown the proposal out there to this extent? (i.e. "don't change it if it works") I'd like to hear some important voices on this issue so that I can decide whether to just drop this entire thing and forget about it. (in other words, what do committers and/or core have to say about this?) Aside from this, I've gotten several other "pro" opinions on this; some people have even sent suggestions. So I know that I am not the only one (not by far, in fact) to see an opportunity to benefit from this. Either way, I know *I* will be using this code in time to come, so I suppose the question is: Would you consider committing this code or should I stop posting any changes I make in the future altogether? What I'm doing is challenging your assertions that spending CPU cycles to save memory in the networking code is the right thing to do. I'm further saying that I have direct experiance in this area since I'm one of the primary people in FreeBSD's history that have spent major amounts of effort in improving its performance, especially in the networking area. We (actually John Dyson and I) made a conscience decision to waste memory in trade for performance and if we (FreeBSD developers in general) decide to go in the opposite direction, then it sure ought to be well thought out and have solid reasoning behind it. In our discussions so far, I haven't yet seen any real numbers to back up the claims. What is needed is: 1) Some numbers that show that the memory wastage is significant - and I'm talking about multiple megabytes at least. If its not 'significant' by that definition (and in my experiance it isn't), than I'd like to hear why you think much smaller numbers are significant. 2) I'd like to see some more numbers that show that the additional CPU wastage is very minimal (say less than 1% of the total amount of time spent doing the allocs/frees). I'm not trying to 'frown upon evolution', unless the particular form of evolution is to make the software worse than it was. I *can* be convinced that your proposed changes are a good thing and I'm asking you to step up to the plate and prove it. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf re-write(s), v 0.1
On Thu, 29 Jun 2000, David Greenman wrote: We used to do this in FreeBSD, but found that it was a bad idea for performance reasons. Freeing and reallocating memory from the high-level VM system is quite expensive and the trend in NICs these days is towards needing the code to be even faster, not slower. Further, if the 'peak' is reached often, then you're probably not really gaining much by freeing the memory back to the common pool. What was previously done at some point was use the kernel malloc() to allocate mbufs. As you know, this is a general purpose allocator that has to first determine what algorithm to use and then store the object correctly according to its size. This allocator is faster than that one. This allocator knows that it only has to deal with mbufs and knows that all of these mbufs are of the same size. Yes, malloc is slow for other reasons, but it is especially slow when VM pages are freed back to the general pool. Of course it is possible to introduce hysteresis in the algorithm such that it doesn't free the pages as often, but this (and all the tunables that you proposed) has the negative effect of making the allocator more complex. We've tried very hard not to do this in the current mbuf allocator, making it nearly as efficient as you can get. I guess I just don't see the problem on any of the servers that I manage (ftp.cdrom.com and ftp.freesoftware.com, for example). There are peaks in usage, but they tend to reach the peaks often enough that freeing the pages for short term memory gain is just a waste of CPU cycles. Memory is so cheap these days that throwing memory at the problem seems to be a very reasonable solution, especially when the system clearly needs it during the peaks. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf re-write(s), v 0.1
In an attempt to eliminate or significantly reduce the hogging of physical memory by unused mbufs, I have begun re-writing some of the mbuf subsystem. I've re-written the allocator and designed an actual free routine, and have also considerably re-written the MGET, MGETHDR, and MFREE macros. I still have some work to do with this, notably optimisation, but I have not been able to do any profiling whatsoever as profiling, I repeat, seems presently broken on -CURRENT. This is particularily useful for machines which see "peak" mbuf usage periods, where many mbufs are allocated, only to be freed a little while later, but which will unfortunately remain on the free list, holding on to physical memory (for a graphical example, see the THIRD graph at http://www.technokratis.com/stats/mbuf.html). We used to do this in FreeBSD, but found that it was a bad idea for performance reasons. Freeing and reallocating memory from the high-level VM system is quite expensive and the trend in NICs these days is towards needing the code to be even faster, not slower. Further, if the 'peak' is reached often, then you're probably not really gaining much by freeing the memory back to the common pool. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: freebsd bios.
So, I repeat: easily done, not acceptable to freebsd core. As has been mentioned by several people already, 'freebsd core' hasn't discussed this as a group and hasn't made any declaration of acceptabilty. That said, I'll say (as a core member, but representing only myself) that I think the idea is pretty cool and I'd very much like it to succeed if at all possible. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Conflict between Intel 82558/9 and VIA MVP4?
to reduce interrupt overhead. It shouldn't peak higher than about 250 or so, however, if things are working correctly. It sounds to me as though there is a serious problem with the DMA operating properly. I'm wondering if the Apollo chipset doesn't support some PCI operation that the Pro/100 wants, causing major problems. Unfortunately I don't believe that there isn't anything that can be done in the driver to work around this. What you need is someone with a PCI bus analyzer to look into the behavior on the bus more closely. You may wish to look for any BIOS settings that might affect the DMA - things like write buffering, burst size, etc., and tweak with those to see if you can affect the behavior. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: sendfile should use getfp.
May I commit this? I'm going to need getfp to be non-static for some stuff I have in the queue, I figured sendfile might as well use it. Fine by me. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: The Design and Implementation of the 4.4bsd Operating System
On Sun, 7 May 2000, Adrian Filipi-Martin wrote: On Sun, 7 May 2000, Chuck Robey wrote: On Sun, 7 May 2000, Alkis Evlogimenos wrote: Can you also recommend any other books describing the internals of the FreeBSD OS which are a closer match than the above? Nope. It's still the best ref. Ask me again in 9 months, maybe there'll be a different answer, because another one is in the works (I think David Greenman is one of the authors of a new one) but reading that book will help a whole lot, it's very definitely not a waste of time. Correct me if I'm wrong, but I recall hearing that Kirk McKusick is working on a new edition called "The Design and Implemntation of the FreeBSD OS". Maybe I heard this at FreeBSDCon. Maybe it was all the beer during Kirk's talk combined with wishful thinking. ;-) Reread what I said (which talks about the book that's upcoming but NOT available yet). I couldn't remember if it was Kirk or not, but I was sure David was one of the authors, so I named him. It's basically Kirk, me, and Sam Lefler. It won't be ready until Q1 2001. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel NIC
Maybe I got lost in the hub-bub, but I am still having trouble getting my onboard NIC to work, still get unsupported phy errors. When you say its fixed are you refering to the patch the DG sent out the other day, or your patch. I applied DG's patch and the problem still exists. And I cant compile my kernel with your patch. Your problem is unrelated to the problems that other people were having. I'll work with you privately to narrow it down. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel NIC
The only thing remaining is that the card is reported as Intel EtherExpress Pro 10/100B (which used to have i82557), but it's a card with i82559. This can be detected by reading the revision code of integrated PHY (registers 2 and 3) through MDI in CSR space (pg. 87 in i82559 datasheet or pg. 67 for i82559ER). The value is different for i82558 and i82559[ER]. It is 0x02A8:0150 for i82558 and 0x02A8:0154 for both 82559 and i82559ER. Do you plan to change/improve the card/chipset reporting mechanism? I wasn't planning to in the short term, but I'd be happy to review some patches for it! :-) -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel Networking - i82559
What is the status of support for onboard Intel networking? Intel
EtherExpress PRO/100 PCI Cards work fine with the fxp0 driver, but I am
have having alot of problems with the onboard intel networking. For
example, SuperMicro PIIIDM3 motherboards, while setting up a firewall
server, I get "Unsupported PHY type" errors from the onboard
networking.. Whats the deal, Linux supports this onboard networking with
their eepro100.c driver, why can't freebsd support this?
Some minor changes are needed. I'll make them as soon as I can find the
time. Meanwhile, this change might get it working:
Index: if_fxp.c
===
RCS file: /home/ncvs/src/sys/pci/if_fxp.c,v
retrieving revision 1.77
diff -c -r1.77 if_fxp.c
*** if_fxp.c1999/09/30 19:03:12 1.77
--- if_fxp.c2000/03/27 08:30:21
***
*** 839,845
/*
* Shift in address.
*/
! for (x = 6; x 0; x--) {
if ((i + offset) (1 (x - 1))) {
reg = FXP_EEPROM_EECS | FXP_EEPROM_EEDI;
} else {
--- 839,845
/*
* Shift in address.
*/
! for (x = 8; x 0; x--) {
if ((i + offset) (1 (x - 1))) {
reg = FXP_EEPROM_EECS | FXP_EEPROM_EEDI;
} else {
This will probably break support for those that were working, however.
Let me know if it works.
-DG
David Greenman
Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org
Creator of high-performance Internet servers - http://www.terasolutions.com
Pave the road of life with opportunities.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel NIC
I've asked about this several times and was beaten silly by Wes Peters and Co. . It seems that they dont like to be bothered by the thousands of people suffering from a driver that is not up to date (linux and netbsd were fixed months ago)..they are way too busy working on 5.0 or something else that about 1% of their user base is using to be bothered by routine driver maintenance. While the card "works" you wont be able to set media-related features..so you really do need the fix. I've successfully fixed our version and its working nicely.I've forwarded the info to DG so hopefully it will be patched soon. While I appreciate the assistence that Dennis has provided me in the past few days with regard to this problem, I really don't think these barbs are justified. There have been some recent Intel boards that haven't worked with FreeBSD, but until just recently I wasn't able to find one and the person who orginally reported the problem just returned the card for one that did work. In fact I wasn't able to determine for several weeks that it even was a software issue. As for "thousands of people people suffering from a driver that is not up-to-date", I think you are *way* overstating the breadth of the problem. I'll also point out that hundreds of thousands of people find the fxp driver a lifesaver and one of the best functioning network drivers in the source tree. Your comments definately piss me off and definately provide for major negative motivation as far as my interest in solving this problem is concerned. Bludgeoning tactics like yours might sometimes work in business, but they sure as hell don't work in free software development. As said in previous mail, I'll implement a solution as soon as I can find some time. I'm extremely busy these days and if I can somehow squeeze in some quality development time inbetween multiple trips across te country, then I will. Otherwise people will either have to use the patch I mailed out or just wait. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel NIC
If you dont have time then perhaps someone else should do it. THATS the point. PLEASE, someone else fix the driver!!! Please, please, please!!! There, did that help? Probably not. I don't know what point you're trying to make. The driver isn't fixed because noone to this point has had both the knowledge and time to fix it, not because I've been some sort of impediment. I have yet to see a real fix for the problem. A real fix requires detecting the type of SEEPROM (6 vs. 8bit address) and doing the right thing. If NetBSD has it working, then great - I suggest that someone look at what they do and form an appropriate fix. If that doesn't happen before I get some time to work on it, then I will when I do. Its an important driver. People get upset when they can't run full-duplex mode. It makes linux MUCH more attractive when you can run your $39. card at twice the speed. Do you really think that whether some small number of Intel cards work correctly in full-duplex with FreeBSD or not will have *any* measurable effect on the attractiveness of FreeBSD vs. Linux? If that prevented someone from installing FreeBSD, then I might agree, but let's be realistic. Whether you are angry at me or not is no reason to punish the communiity. How is me not doing free work punishing the community? The 'community' should be happy that the driver (and FreeBSD!) exist at all. If you think that you are just doing this for me because I'm the only one with the nerve to complain then you dont understand your obligation to the user base. I do have an obligation, yes. I also have many other obligations. Like anyone with 5 times more to do than can be done in a day, I have to prioritize and do the really important things first and if there is any time left over to do "fun" stuff (like work on free software), then I do. Life hasn't been fun for me for quite awhile. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel NIC
At the moment I have a spare machine
and the NIC available for testing.
Please let me know if you are interested
in my offer to test your patches to 4.0 driver?
Thanks, Milon. I've attached patches which I believe will fix the problem
as seen with the Compaq cards/motherboards, SuperMicro motherboards, and
certain newer Pro/100+ cards. Please test and let me know if they work for
you. I've tested this with (Compaq and Pro/100+) cards that some FreeBSD users
shipped to me a few months ago when the problem was first noticed, and the
driver now works fine with those. The algorithm for sizing the SEEPROM was
taken from the NetBSD version of the driver. Thanks for your patience.
-DG
David Greenman
Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org
Creator of high-performance Internet servers - http://www.terasolutions.com
Pave the road of life with opportunities.
Index: if_fxp.c
===
RCS file: /home/ncvs/src/sys/pci/if_fxp.c,v
retrieving revision 1.77
diff -c -r1.77 if_fxp.c
*** if_fxp.c1999/09/30 19:03:12 1.77
--- if_fxp.c2000/03/28 02:58:41
***
*** 232,237
--- 232,238
static int fxp_add_rfabuf __P((struct fxp_softc *, struct mbuf *));
static int fxp_mdi_read __P((struct fxp_softc *, int, int));
static void fxp_mdi_write __P((struct fxp_softc *, int, int, int));
+ static void fxp_autosize_eeprom __P((struct fxp_softc *));
static void fxp_read_eeprom __P((struct fxp_softc *, u_int16_t *,
int, int));
static int fxp_attach_common __P((struct fxp_softc *, u_int8_t *));
***
*** 748,753
--- 749,759
}
/*
+* Find out how large of an SEEPROM we have.
+*/
+ fxp_autosize_eeprom(sc);
+
+ /*
* Get info about the primary PHY
*/
fxp_read_eeprom(sc, (u_int16_t *)data, 6, 1);
***
*** 802,807
--- 808,883
}
/*
+ * From NetBSD:
+ *
+ * Figure out EEPROM size.
+ *
+ * 559's can have either 64-word or 256-word EEPROMs, the 558
+ * datasheet only talks about 64-word EEPROMs, and the 557 datasheet
+ * talks about the existance of 16 to 256 word EEPROMs.
+ *
+ * The only known sizes are 64 and 256, where the 256 version is used
+ * by CardBus cards to store CIS information.
+ *
+ * The address is shifted in msb-to-lsb, and after the last
+ * address-bit the EEPROM is supposed to output a `dummy zero' bit,
+ * after which follows the actual data. We try to detect this zero, by
+ * probing the data-out bit in the EEPROM control register just after
+ * having shifted in a bit. If the bit is zero, we assume we've
+ * shifted enough address bits. The data-out should be tri-state,
+ * before this, which should translate to a logical one.
+ *
+ * Other ways to do this would be to try to read a register with known
+ * contents with a varying number of address bits, but no such
+ * register seem to be available. The high bits of register 10 are 01
+ * on the 558 and 559, but apparently not on the 557.
+ *
+ * The Linux driver computes a checksum on the EEPROM data, but the
+ * value of this checksum is not very well documented.
+ */
+ static void
+ fxp_autosize_eeprom(sc)
+ struct fxp_softc *sc;
+ {
+ u_int16_t reg;
+ int x;
+
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL, FXP_EEPROM_EECS);
+ /*
+* Shift in read opcode.
+*/
+ for (x = 3; x 0; x--) {
+ if (FXP_EEPROM_OPC_READ (1 (x - 1))) {
+ reg = FXP_EEPROM_EECS | FXP_EEPROM_EEDI;
+ } else {
+ reg = FXP_EEPROM_EECS;
+ }
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL, reg);
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL,
+ reg | FXP_EEPROM_EESK);
+ DELAY(1);
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL, reg);
+ DELAY(1);
+ }
+ /*
+* Shift in address.
+* Wait for the dummy zero following a correct address shift.
+*/
+ for (x = 1; x = 8; x++) {
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL, FXP_EEPROM_EECS);
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL,
+ FXP_EEPROM_EECS | FXP_EEPROM_EESK);
+ DELAY(1);
+ if ((CSR_READ_2(sc, FXP_CSR_EEPROMCONTROL) FXP_EEPROM_EEDO) == 0)
+ break;
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL, FXP_EEPROM_EECS);
+ DELAY(1);
+ }
+ CSR_WRITE_2(sc, FXP_CSR_EEPROMCONTROL, 0);
+ DELAY(1);
+ sc-eeprom_size = x;
+ }
+ /*
* Read from the serial EEPROM. Basically, you manually shift in
* the read opcode (one bit at a time) and then shift in the address,
* and then you shift out the data (all of this one bit at a time
Re: if_fxp driver
I hope your happy, but do you know the answer to my question? Has the driver been updated recently? Not to fix the problem that you are reporting. The solution might be as simple as adding another PHY identifier to the list of supported ones. I need to find some time to sit down with one of the not-working cards and fiddle with it. I was going to do that this past weekend, but then got sick with a virus. It's on my list. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: limiting buffer cache
Does anybody here knows how to limit the size of my FS buffer cache ? I have already looked over LINT config file, but it seems not to have any option! I have ever send this message to questions, but nobody there seems to care/know. I would really appreciate your help. There really isn't a 'buffer cache' in FreeBSD. About 5 years ago FreeBSD was changed so that buffers weren't used to cache filesystem data - instead they are used as a mechanism to map cached pages from the VM system into the kernel address space. All file caching now occurs in the VM system and is completely dynamic in size, and varies with other activity in the system. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: limiting buffer cache
So, that is it! When my users try to copy very big directories the system simply prevent anyone else from spawwing any other process, return the followind: swapper: out of memory! Are you telling me that there is no way to prevent my users from eating all my memory ? Uh, I'm not sure what is generating that error, but it has nothing to do with file caching. Sounds like you don't have enough swap space configured. What version of FreeBSD are you using? -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Is FreeBSD dead ?
I fail to see how you can read anything bad into this announcement. If you're really concerned, you have just as much right to the code as any one else, feel free to take the 4.0 code base and create your own system. BSDidier has a nice ring to it. Personally, I've been running FreeBSD since 1.0, and I'll be sticking with it for quite some time to come. Ever read Animal Farm? Remember that BSD/OS started out as "cheap with source" and grew into "just another OS company". Good ideas can turn bad very quickly. The is all just FUD. From the FreeBSD side of things, BSDI is going to opensource a bunch of their software that we can then integrate into FreeBSD. We're still very much in control of the FreeBSD development effort and what fundamentally comprises FreeBSD. In fact nothing really changes as far as the FreeBSD Project is concerned - it's the same core team, the same developers, and the same BSD-license source code. It's just as "free" as ever, and nothing is going to change that. BSDI may decide not to make all of their BSD/OS open-sourced, but that's their decision and that will in no way deminish what we have in FreeBSD today. As others have said, this is a win for everyone and will result in FreeBSD being a much better open-source OS in the future. I really hope that people will go and read the various press releases and look at this in an objective and rational frame of mind. If you do, then there is only one conclusion that you can get from the facts: This is a great thing for FreeBSD and our future couldn't be brighter. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Is FreeBSD dead? Well, not in theory...
instead of NT. Instead of Linux. The existing BSD market is too small. They have failed to convince the world that BSD is the answer. Outside of the US. linux is totally dominant. I'm not sure where you get your market demographics, but at least in Japan, FreeBSD is on par with Linux in popularity. If we had even half of the success in the US that we've enjoyed in Japan, then we'd be a lot further along in mindshare. But all of this speaks to the past and doesn't consider what may be ahead. It's true that BSD in general has sucked at marketing. One of the primary goals with the new company is to change that. In time we'll know if this was just wishful thinking. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel fxp network chip, unknown PHY 17 type 2
Hmm. That reminds me: I've also got a box with an onboard 8255X that isn't recognized. The relevant parts of "boot -v" output are: found- vendor=0x8086, dev=0x1209, revid=0x09 class=02-00-00, hdrtype=0x00, mfdev=0 subordinatebus=0secondarybus=0 intpin=a, irq=11 map[10]: type 1, range 32, base ffaff000, size 12 map[14]: type 1, range 32, base ef00, size 6 map[18]: type 1, range 32, base ffac, size 17 [...] pci0: unknown card (vendor=0x8086, dev=0x1209) at 14.0 irq 11 ^ Don't know what that is, but's not a part that is supported by the fxp driver. It would help if you could find out the part number (8255X isn't sufficient since it isn't really just one series - some of the parts are similar, and others are completely different). -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Is FreeBSD dead ?
Could somebody clear this up for me? If FreeBSD is still going to go along doing what it does, then what happens if I write a device driver for WhizzoNewProduct(TM), that the commercial side is developing as an "added value feature"? Say, for example, I beat them to the punch. As pointed to Simple answer: BSD, Inc. loses. What BSD, Inc. tries to do in the value-add arena is entirely their problem and if FreeBSD developers develop something that conflicts with BSD, Inc.'s value-add, then tough - BSD, Inc. will have to go and find another value-add. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel fxp network chip, unknown PHY 17 type 2
Hi, My new i840 chipset motherboard has onboard networking. It ises the Intel chipset and the FXP driver detects it. However, it reports PHY Unknown PHY 17 type 2. on the 4.0-current 2307 snapshot. Is there a quick fix for this? I'd really like to get networking going with this board. Can you look on the motherboard and find out what type of chip it uses? It should be one of: 82557, 82558, or 82559. Let me know. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Onboard Intel fxp network chip, unknown PHY 17 type 2
here is: GD82559 L942SH98 There's what might be the PHY next to it (assuming it's not internal). MVT203011 CW 722490-001 I don't recognise the logo on this, unless it's a new/changed International Rectifier logo (looks like two diodes pointing at each other inside a circle and square). The 82559 has an integrated PHY. Looks like someone has changed the identifier again. What type of motherboard is this on (sorry if I missed this in a previous message). -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Multiple instances of the same character device
The question is very simple. Is it possible to open the same character pseudo device, for example /dev/foo0, simultaneously from other programs, and to work with this instances independently? I'm asked as the developer of a driver with such requirements, so please don't complain about such technique. I read some discursion about this in the mailing list http://www.FreeBSD.org/cgi/getmsg.cgi?fetch=854845+857264+/usr/local/www/db/text/1997/freebsd-hackers/19970727.freebsd-hackers . But it's old one (dated by begin of '97). The simple answer is yes. The only issue has to do with closing the device. If you don't care about the device driver close routine being called except on the last close, then everything should work just fine. I assume this question is related to you work on VMware for FreeBSD? -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: new Intel 100Mbps card
Wes Peters wrote: Wes Peters wrote: This might be the new 82559ER; I'm downloading the datasheet now. Have a peek at: http://developer.intel.com/design/network/datashts/index.htm Nope, that one is apparently device ID 0x1209. Too bad they don't have a PCI device ID cross-reference on the web site. Bleh. if_fxpreg.h presently has: #define FXP_VENDORID_INTEL 0x8086 #define FXP_DEVICEID_i82557 0x1229 /* 82557 - 82559 "classic" */ #define FXP_DEVICEID_i82559 0x1030 /* New 82559 device id.. */ Oops, I forgot that you had already added this. This should be merged into 3.x for the 3.4 release (with Jordan's permission of course). Are you going to take care of that, Peter, or would you like me to? -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: new Intel 100Mbps card
#define FXP_VENDORID_INTEL 0x8086 #define FXP_DEVICEID_i825570x1229 +#define FXP_DEVICEID_i825580x1030 This wouldn't be correct. The 82558 has been used for years on Pro/100+ boards and they ID as 0x1229. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: new Intel 100Mbps card
#define FXP_VENDORID_INTEL 0x8086 #define FXP_DEVICEID_i825570x1229 +#define FXP_DEVICEID_i825580x1030 This wouldn't be correct. The 82558 has been used for years on Pro/100+ boards and they ID as 0x1229. Sorry, I forgot to say about the above. Since Wes Peters suggested that it might be a 82558, it put the above name. Please correct it to whatever the name should be. From your other email it sounds like it has an 82559. Intel has been shipping that for more than a year as well on boards that ID as 0x1229, so apparantly the chip being used doesn't correlate with the ID number. In this case I'd recommend changing the above defines to "FXP_DEVICE_ID_1" and "FXP_DEVICE_ID_2" respectively. If you are confident that your 0x1230 Pro/100 is working correctly, including stuff related to manual selection of the speed and duplex, then I'll take care of making the changes to the driver. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: fxp related kernel panic
Let me guess...your system has an Intel N440BX motherboard, right? If so, then it's a known problem with no solution yet. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. I have a 3.3-stable machine that I use as a news router (running diablo). The fxp0 interface averages 10-15 Mbps bandwidth continously. About once a week the machine crashes reboots. We enabled the debugger this time and captured the following debug output: Fatal trap 12: page fault while in kernel mode fault virtual address = 0x382e4641 fault code = supervisor write, page not present instruction pointer = 0x8:0xc01a372e stack pointer = 0x10:0xc02523b0 frame pointer = 0x10:0xc02523c0 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = Idle interrupt mask = net kernel: type 12 trap, code=0 Stopped at fxp_add_rfabuf+0x1de: movw%ax,0x4(%esi) db %uname -a FreeBSD feeder.via.net 3.3-STABLE FreeBSD 3.3-STABLE #7: Mon Oct 18 17:14:40 PDT 1999 [EMAIL PROTECTED]:/usr/src/sys/compile/DIABLO i386 %dmesg Copyright (c) 1992-1999 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.3-STABLE #7: Mon Oct 18 17:14:40 PDT 1999 [EMAIL PROTECTED]:/usr/src/sys/compile/DIABLO Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 498752616 Hz CPU: Pentium III (498.75-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x672 Stepping = 2 Features=0x387fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CM OV,PAT,PSE36,b18,MMX,FXSR,b25 real memory = 536870912 (524288K bytes) avail memory = 519340032 (507168K bytes) Preloaded elf kernel "kernel" at 0xc02ca000. Pentium Pro MTRR support enabled Probing for devices on PCI bus 0: chip0: Intel 82443BX host to PCI bridge (AGP disabled) rev 0x03 on pci0.0.0 ncr0: ncr 53c875 fast20 wide scsi rev 0x37 int a irq 11 on pci0.13.0 ncr1: ncr 53c875 fast20 wide scsi rev 0x37 int b irq 10 on pci0.13.1 fxp0: Intel EtherExpress Pro 10/100B Ethernet rev 0x05 int a irq 5 on pci0.15. 0 fxp0: Ethernet address 00:a0:c9:fc:45:7f chip1: Intel 82371AB PCI to ISA bridge rev 0x02 on pci0.18.0 ide_pci0: Intel PIIX4 Bus-master IDE controller rev 0x01 on pci0.18.1 chip2: Intel 82371AB Power management controller rev 0x02 on pci0.18.3 vga0: Cirrus Logic model 00bc VGA-compatible display device rev 0x23 on pci0.2 0.0 Probing for PnP devices: Probing for devices on the ISA bus: sc0 on isa sc0: VGA color 16 virtual consoles, flags=0x0 atkbdc0 at 0x60-0x6f on motherboard atkbd0 irq 1 on isa sio0: configured irq 4 not in bitmap of probed irqs 0 sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa sio0: type 8250 sio1: configured irq 3 not in bitmap of probed irqs 0 sio1 not found at 0x2f8 fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa wdc0 at 0x1f0-0x1f7 irq 14 flags 0xb0ffb0ff on isa wdc0: unit 0 (wd0): WDC WD273BA, LBA, DMA, 32-bit, multi-block-16 wd0: 26105MB (53464320 sectors), 3328 cyls, 255 heads, 63 S/T, 512 B/S wdc0: unit 1 (wd1): WDC WD273BA, LBA, DMA, 32-bit, multi-block-16 wd1: 26105MB (53464320 sectors), 3328 cyls, 255 heads, 63 S/T, 512 B/S wdc1 not found at 0x170 vga0 at 0x3b0-0x3df maddr 0xa msize 131072 on isa npx0 on motherboard npx0: INT 16 interface ccd0-3: Concatenated disk drivers Waiting 15 seconds for SCSI devices to settle changing root device to da0s1a da0 at ncr0 bus 0 target 0 lun 0 da0: QUANTUM VIKING II 4.5WLS 4110 Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz, offset 16, 16bit), Tagged Queueing Enabled da0: 4350MB (8910423 512 byte sectors: 255H 63S/T 554C) WARNING: / was not properly dismounted Any ideas ? Thanks, Joe -- Joe McGuckin ViaNet Communications 994 San Antonio Road Palo Alto, CA 94303 Phone: 650-969-2203 Cell: 650-207-0372 Fax: 650-969-2124 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: fxp related kernel panic
Hi David -- What if I install a *real* EtherExpress Pro-100B (or whatever it's known as today) in the PCI slot, and use it instead of the on-board (N440BX motherboard) fxp0 interface? Judging that you probably know the nature of the problem, do you think this might circumvent it? I think it is caused by the NCR/Symbios controller. It might be a side effect of the NCR just using up a lot of PCI bandwidth, with the real bug being in the fxp driver (although I've looked and haven't found one). So I don't think putting in a real Pro/100 will have any effect on the problem. Of course I don't really know what is causing it, so just about anything is possible. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: SMP + fxp0 wierdness
David Greenman wrote: So if this problem is NOT related to specific hardware, how can we get the driver fixed? Talk to the maintainer (David). We've offered him cores and kernels before. Alternatively, you'll need to experiment with your setup to determine what characterises the failures and help David out with more data. Hotmail has troubleshooted the problem down to the NCR controller. It appears that the problem only occurs when using one of those. If they plug in an Adaptec 2940 and use it instead of the onboard NCR then the problems disappear. Well that's not good, since I have almost convinced my boss to replace the crappy IDE drives on our shiny new Intel N440BX mb's with scsi drives since the controller is built in. :-/ Does this look like a soluble problem, or is it just going to be a case of "don't do that?" Anything I can do to help mail me and let me know. Intel generally makes good stuff. On the other hand, I'm not too happy with the NCR/Symbios support in FreeBSD...the conversion to CAM wasn't all that great and the driver really needs a rewrite. I wouldn't personally put a machine with an NCR/Symbios into production - I have just too much negative history with it. I don't understand why some machines are having this problem with the Intel Pro/100B/100+ and (most) others never do. All indications right now is that it is a DMA corruption problem of some kind, but I don't have any clue what might be causing it. I don't think it is a software bug, but it's conceivable that the problem could be worked around with software if I knew what was causing it. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: SMP + fxp0 wierdness
So if this problem is NOT related to specific hardware, how can we get the driver fixed? Talk to the maintainer (David). We've offered him cores and kernels before. Alternatively, you'll need to experiment with your setup to determine what characterises the failures and help David out with more data. Hotmail has troubleshooted the problem down to the NCR controller. It appears that the problem only occurs when using one of those. If they plug in an Adaptec 2940 and use it instead of the onboard NCR then the problems disappear. We did this (you remember that we went through all this a while back, right?), but we've also been seing reports from people that aren't using NCR controllers. I haven't seen a report yet from someone not using an NCR/Symbios controller. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: SMP + fxp0 wierdness
So if this problem is NOT related to specific hardware, how can we get the driver fixed? Talk to the maintainer (David). We've offered him cores and kernels before. Alternatively, you'll need to experiment with your setup to determine what characterises the failures and help David out with more data. Hotmail has troubleshooted the problem down to the NCR controller. It appears that the problem only occurs when using one of those. If they plug in an Adaptec 2940 and use it instead of the onboard NCR then the problems disappear. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: SMP + fxp0 wierdness
We're running 3.3-REL on dual processor PII-450's, with a N440BX motherboard, using the onboard EtherExpress Pro (fxp) NIC and 512MB RAM. These machines are running custom software that excercises the disk, CPU and network quite heavily. The SMP machines seem to have both "fxp0: device timeout" problems, and spontaneous reboots. We were uable to get a working savecore until now, and have traced the reboots back to the fxp driver as well. Here are the debug outputs, and any custom changes to our kernel config. Could this be a problem with SMP + fxp combination? Any other thoughts or ideas? We've serached, and read, and searched all the FAQ's for both of these problems, and have pretty well come up empty. Suggestions for the next course of action? Thanks in advance for anyones help. There is some kind of hardware problem with the Intel N440BX motherboard that is causing memory corruption during the DMA. This is the third nearly identical report I've gotten about it. It does not appear to be a FreeBSD bug and so far only occurs when using the N440BX. You might try messing with the BIOS options and see if changing any of the DMA related settings will make the problem go away...I'd be very interested in the results. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: CFD: bogomips CPU performance metric
Description of BogoMIPS deleted Would anyone scream and projectile-vomit if I added this to identcpu.c? I might. :-) Why exactly, except to keep up with the Linux kidz, do we need this? I recognize that this is solely a cosmetic change, but one of the things I hold over the heads of the Linux folks I deal with is the fact that FreeBSD is a professional quality operating system which doesn't need useless blinking lights like BogoMIPS. Chalk me up as one of the people who considers "Linux works like that" as a negative. I'm with you on this, Keith. I'd rather that we kept the professional FreeBSD look and feel. If we look too much like Linux, then people will just use Linux. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: CFD: bogomips CPU performance metric
Description of BogoMIPS deleted Would anyone scream and projectile-vomit if I added this to identcpu.c? I might. :-) Why exactly, except to keep up with the Linux kidz, do we need this? I recognize that this is solely a cosmetic change, but one of the things I hold over the heads of the Linux folks I deal with is the fact that FreeBSD is a professional quality operating system which doesn't need useless blinking lights like BogoMIPS. Chalk me up as one of the people who considers Linux works like that as a negative. I'm with you on this, Keith. I'd rather that we kept the professional FreeBSD look and feel. If we look too much like Linux, then people will just use Linux. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
The structure appears to be backwards compatible. The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It seems reasonable to me, although there may be issues with finding a bit in the minor number - I think they've pretty much all been taken. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
The structure appears to be backwards compatible. The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It seems reasonable to me, although there may be issues with finding a bit in the minor number - I think they've pretty much all been taken. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: network performance vs. linux on small transfers
I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: network performance vs. linux on small transfers
I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mmap bug
This report seems to be severely lacking in details. First, I don't
understand why it is called "mmap" since it doesn't do an mmap and the
"addr" that is being frobbed with isn't even initialized. Second, I
get a core dump when I run it on a -stable machine:
[speedy:tmp9] mmap
unlink files? NO
mmaping 10485760 byte region on file 0
Segmentation fault (core dumped)
...which is exactly what I'd expect when dealing with a bogus pointer.
You didn't specify what version of FreeBSD you saw a hang and didn't provide
any details of the hang. Please provide more information so that we can
help you. Thanks.
-DG
David Greenman
Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org
Creator of high-performance Internet servers - http://www.terasolutions.com
This small program, running as 'mmap', not 'mmap -u', can hang my machine.
Is this a known bug in FreeBSD's kernel, or it is my fantasy ?
Thank you for answer.
#include stdio.h
#include stdlib.h
#include sys/types.h
#include sys/mman.h
#include unistd.h
#include fcntl.h
#include errno.h
main(int argc, char *argv[])
{
int fd;
int i;
int len=1024*1024*10; /*ie 10Mbytes*/
caddr_t addr;
char ttt[80];
int bunlink = 0;
if ( argc 1 strcmp(argv[1], "-u") == 0 ) {
bunlink = 1;
}
printf("unlink files? %s\n", bunlink ? "YES" : "NO");
for (i=0;;i++)
{
sprintf (ttt,"%d",i);
printf("mmaping %ld byte region on file %s\n", len, ttt);
fd=open(ttt,O_CREAT|O_RDWR,0666);
if (fd0)
{
printf("mmap error %ld",errno);
exit(1);
}
memset(addr,'x',len);
if ( munmap(addr, len) != 0 ) {
fprintf(stderr, "munmap failed\n");
exit(EXIT_FAILURE);
}
close(fd);
if ( bunlink ) unlink(ttt);
}
}
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: mmap bug
Looks like Oleg made a mistake in posting the code. I saw an earlier version of this in freebsd-questions and followed up with him. I've appended the version I think he meant to include. He's reporting this behavior with 3.2R. Runs fine with 'mmap -u', appears to hang the machine on the second iteration (file "1") with 'mmap'. Runs fine on Solaris 2.6 and Digital Unix 4.0D -- with the exception of filling the disk without "-u" :^). He's trying to ask if this is a problem with the code in question or 3.2R's mmap. That's better. It appears to be a classic resource related deadlock that is caused by the VFS code needing pages in order to page things out (and thus free up pages), but is unable to since no memory is available. Matt Dillon was working on deadlocks like this in -current awhile back and it would be interesting to know if the hang occurs there as well. I don't have a -current machine at the moment so I can't test it myself. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mmap bug
This report seems to be severely lacking in details. First, I don't
understand why it is called mmap since it doesn't do an mmap and the
addr that is being frobbed with isn't even initialized. Second, I
get a core dump when I run it on a -stable machine:
[speedy:tmp9] mmap
unlink files? NO
mmaping 10485760 byte region on file 0
Segmentation fault (core dumped)
...which is exactly what I'd expect when dealing with a bogus pointer.
You didn't specify what version of FreeBSD you saw a hang and didn't provide
any details of the hang. Please provide more information so that we can
help you. Thanks.
-DG
David Greenman
Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org
Creator of high-performance Internet servers - http://www.terasolutions.com
This small program, running as 'mmap', not 'mmap -u', can hang my machine.
Is this a known bug in FreeBSD's kernel, or it is my fantasy ?
Thank you for answer.
#include stdio.h
#include stdlib.h
#include sys/types.h
#include sys/mman.h
#include unistd.h
#include fcntl.h
#include errno.h
main(int argc, char *argv[])
{
int fd;
int i;
int len=1024*1024*10; /*ie 10Mbytes*/
caddr_t addr;
char ttt[80];
int bunlink = 0;
if ( argc 1 strcmp(argv[1], -u) == 0 ) {
bunlink = 1;
}
printf(unlink files? %s\n, bunlink ? YES : NO);
for (i=0;;i++)
{
sprintf (ttt,%d,i);
printf(mmaping %ld byte region on file %s\n, len, ttt);
fd=open(ttt,O_CREAT|O_RDWR,0666);
if (fd0)
{
printf(mmap error %ld,errno);
exit(1);
}
memset(addr,'x',len);
if ( munmap(addr, len) != 0 ) {
fprintf(stderr, munmap failed\n);
exit(EXIT_FAILURE);
}
close(fd);
if ( bunlink ) unlink(ttt);
}
}
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: mmap bug
Looks like Oleg made a mistake in posting the code. I saw an earlier version of this in freebsd-questions and followed up with him. I've appended the version I think he meant to include. He's reporting this behavior with 3.2R. Runs fine with 'mmap -u', appears to hang the machine on the second iteration (file 1) with 'mmap'. Runs fine on Solaris 2.6 and Digital Unix 4.0D -- with the exception of filling the disk without -u :^). He's trying to ask if this is a problem with the code in question or 3.2R's mmap. That's better. It appears to be a classic resource related deadlock that is caused by the VFS code needing pages in order to page things out (and thus free up pages), but is unable to since no memory is available. Matt Dillon was working on deadlocks like this in -current awhile back and it would be interesting to know if the hang occurs there as well. I don't have a -current machine at the moment so I can't test it myself. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Benchmarking server app on FreeBSD
What do you have the listen queue limit set for? What is the kern.somaxconn sysctl variable set to? -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com I'm benchmarking the performance of a server application on various platforms. To do so, I've developed a program on FreeBSD 3.x to generate heavy loads. This program can, for example, generate 200 simultaneous connections to the server and process them all appropriately, and I have it running on a bunch of machines to simulate high load on the server. However, I'm running into an unexpected problem on a server running FreeBSD 3.2-RELEASE. If a single client opens 200 simultaneous connections to the FreeBSD server, all but 30 to 40 of those get an immediate Connection refused. If a whole row of clients open 200 simultaneous connections each, they still get only 30 to 40 each, but the server is now accepting upwards of 200 connections all at once. This seems to indicate that it's not a server load question, but rather that the server will not accept more than ~40 connections at once from the same client. I've tried the same test using the FreeBSD clients against a Solaris server, and the Solaris server accepts all 200 connections from each client machine without refusing any connections, so I'm sure it's not on the client end. A fellow admin suggested that my load simulation program quite possibly looks like a SYN attack and FreeBSD might be rejecting it for that reason. Since Solaris doesn't have the same SYN attack protection, that would account for the difference. If this is the case, how would I disable the SYN attack protection for these tests? Or is it something else limiting FreeBSD? For reference, the FreeBSD server has a custom kernel compiles with MAXUSERS set to 512, and other performance enhancements I've gleaned off these lists. Thanks, Ken Bolingbroke hac...@bolingbroke.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: tcp windowsize query?
delayed ack sounds interesting Turning that off disables TCP slow-start. It's a huge performance booster for things like SMB service, where you have lots of short-lived TCP connections on a local net. Uh, that's not what it does. Slow start is a behavior where the sender opens the window slowly - starting with one segment for the window and adding one more segment for each ack that comes back successfully. What the above option does is disable delayed acks on the receiver, thus reducing the round-trip time and increasing the speed of transaction style (small) TCP sends. Actually the real purpose of it is to eliminate the internal overhead that is normally imposed by the delayed ack timers, which can become substantial on large systems like wcarchive. That it has other beneficial side effects is almost accidental. :-) -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: tcp windowsize query?
delayed ack sounds interesting Turning that off disables TCP slow-start. It's a huge performance booster for things like SMB service, where you have lots of short-lived TCP connections on a local net. Uh, that's not what it does. Slow start is a behavior where the sender opens the window slowly - starting with one segment for the window and adding one more segment for each ack that comes back successfully. What the above option does is disable delayed acks on the receiver, thus reducing the round-trip time and increasing the speed of transaction style (small) TCP sends. Actually the real purpose of it is to eliminate the internal overhead that is normally imposed by the delayed ack timers, which can become substantial on large systems like wcarchive. That it has other beneficial side effects is almost accidental. :-) -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Replacement for grep(1) (part 2)
: :Well, all I can say is: : : I'm sure glad you don't have any influence over the code : base I run. : :-- Jason R. Thorpe [EMAIL PROTECTED] I'm sure the feeling is mutual. More to the point, I really seriously doubt that any of the core developers would consider this idea either because it's been rejected in the past and, so far, nobody has offered anything that hasn't been heard before. You are welcome to ask them, of course, but that is the feeling I get. There are much easier ways to accomplish the level of control required. I'm not fundamentally opposed to a no-overcommit knob, but I think implementing it properly is more difficult than people think. There are things that do implied swap allocation (automatic stack allocation and fork() are two examples) that make this a difficult problem to solve. I wouldn't personally want to run a system with such a knob turned on, however, and I tend to agree with Matt that there are other better ways to solve the embedded system case. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
The point is, the OS should have provided *some* mechanism to insure that the long-running process wasn't affected. It didn't. That's a clear failure of the OS to provide a reasonable environment for this type of computing. Whether this should be solved by switching to a no-overcommit policy, fiddling with the overcommit policy in some way, or whatever, is a different issue. But you have not yet proposed any mechanism that would have prevented this problem while still permitting me to get work done. I've long felt that the best solution to problems like this is a per-user swap space quota. This gives admins a knob to manage the allocation of swap space while still allowing overcommit. The downside is that it doesn't provide a graceful way for a program to recover from it's overconsumption sins. I'd argue, however, that buggy software or incorrectly tuned systems should get what they deserve. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Replacement for grep(1) (part 2)
: :Well, all I can say is: : : I'm sure glad you don't have any influence over the code : base I run. : :-- Jason R. Thorpe thor...@nas.nasa.gov I'm sure the feeling is mutual. More to the point, I really seriously doubt that any of the core developers would consider this idea either because it's been rejected in the past and, so far, nobody has offered anything that hasn't been heard before. You are welcome to ask them, of course, but that is the feeling I get. There are much easier ways to accomplish the level of control required. I'm not fundamentally opposed to a no-overcommit knob, but I think implementing it properly is more difficult than people think. There are things that do implied swap allocation (automatic stack allocation and fork() are two examples) that make this a difficult problem to solve. I wouldn't personally want to run a system with such a knob turned on, however, and I tend to agree with Matt that there are other better ways to solve the embedded system case. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))
The point is, the OS should have provided *some* mechanism to insure that the long-running process wasn't affected. It didn't. That's a clear failure of the OS to provide a reasonable environment for this type of computing. Whether this should be solved by switching to a no-overcommit policy, fiddling with the overcommit policy in some way, or whatever, is a different issue. But you have not yet proposed any mechanism that would have prevented this problem while still permitting me to get work done. I've long felt that the best solution to problems like this is a per-user swap space quota. This gives admins a knob to manage the allocation of swap space while still allowing overcommit. The downside is that it doesn't provide a graceful way for a program to recover from it's overconsumption sins. I'd argue, however, that buggy software or incorrectly tuned systems should get what they deserve. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
