Re: Current, XEON and MP performance
Of all the gin joints in all the towns in all the world, Achim Patzner had to walk into mine and say: > > Can you show us the actual results from your testing (an hopefully your > > testing methods as well) that led you to this conclusion? Details matter. > > First test was running an apache and fcgi noticing that it took considerably > longer to respond than the old server. Next test was a number of homegrown > programs under time(1). > > > Are these programs I/O bound, CPU bound, or a little of both? > > All of it; one is page-view accounting for the current web server, others > are meteorological simulations. > > > FreeBSD's SMP support still depends largely on the big giant lock approach > > which means that while you can indeed get processes running on multiple > > CPUs at the same time, you end up using only one CPU once you enter the > > kernel. > > Which would account for needing more wall-clock-time. But heaps of CPU time? > > > And you have to enter the kernel in order to perform any disk, network > > or even console I/O. If your programs suck large datasets into memory, > > do lots of number crunching on them, then spit the results back out to > > a disk file, then they should benefit from more CPUs. However if they > > read and write data a lot while running, you're going to be limited by > > the big giant lock. > > Hm. Both are MP systems. Both were - at the time of test - virtually idle > besides running the tests. I could even retry in single user mode. I'm taking about your test programs themselves, not the rest of the system. It doesn't matter if the system is otherwise idle: if *your* programs perform a lot of I/O, then the big giant lock will hurt you. > > There may also be scalability issues (i.e. does FreeBSD perform better > > as you add more CPUs or does it spend so much time trying to stay out > > of its own way that it actually performs worse) however I don't know > > enough to say if you could be running into such problems as the only > > SMP machines I have access to have only 2 CPUs. > > As far as I understood HP Germany, they'd be glad to lend you one of them > for a few months. You'd just have to ask. I'm not an SMP expert, though perhaps those who are SMP experts might want to take advantage of this. > > > [Worse: The LH4 > > > behaves like a spoilt brat when it comes to hardware, disliking the Int= > el > > > EtherExpress that came with it (generating bus mastering problems after > > > bringing it up), > >=20 > > Which model Intel EtherExpress? What chipset? > > I'll have to put it into another machine - it's sitting on a shelf right > now. But it was working reliably AND fast on another ASUS P2B-DS machine. > > > What bus mastering problems exactly? > > Time outs, followed by a panic. This is still a bit vague. It would be nice to know exactly what the timeout error messages said, as well as what the panic message said. > > > having interrupt routing problems with two DEC TULIP based ethernet > > > cards sharing the same IRQ > >=20 > > Which tulip cards? > > CNet CN100 with an 21443 21143, not 21443. > > What driver? > > if_de Try if_dc instead. > > What kind of problems? > > Constant time outs under higher load than a few packets a second; no more > than 18 KB/second on a switch, no matter whether full duplex or not. This is probably due to a bug in the if_de driver. I finally got a proper handle on it the other day. The 21143 is different from the 21140 in a couple of ways, the most important being that it has built-in NWAY autonegotiation support. If you want to make a 10/100 autonegotiating NIC with a 21140 chip, you need to use an external MII-compliant tranceiver chip with NWAY support. You can design board with the 21143 like this as well, however you can *also* do it using the 21143's own built-in NWAY support. The DEC DE500-BA card works like this. The problem is that in adding the built-in autoneg support, DEC/Intel did a really dumb thing. In the CSR6 register, there's a bit that controls full duplex/half duplex operation: turn it on, the chip runs in full duplex mode, turn it off and it runs in half duplex. On the 21140, this is the *only* function that this bit has. But in the 21143, this bit has two functions: if the internal autoneg is shut off, then it behaves like it does on the 21140, but if autoneg is turned *on*, then it controls whether or not the chip advertises 10Mbps half duplex when performing autonegotiation. Why is this dumb? For the following reasons: - There are plenty of unused bits in other registers that DEC/Intel could have used instead of overloading the meaning of the full duplex enable bit in CSR6. - The chip defaults to autoneg turned on after a reset. It also defaults to half duplex. So if you want to manually enable full duplex, you *must* first turn off the autoneg (by clearing CSR14). The de driver doesn't know about this, and the result is that full duplex mode just won't work. If your boa
Re: Current, XEON and MP performance
Of all the gin joints in all the towns in all the world, Achim Patzner had to walk into mine and say: > I don't know where to ask first (or what to look at) so I'd like some > creative guessing by some people closer to the sources... > > Running the same programs on nearly identically configured -CURRENT kernels > on a > HP NetServer LH4 > (four 550 MHz PIII Xeon with 512MB Cache, 512MB cache? I think you mean KB. > supposed to be an INTEL 450NX-based chipset) > with one GB RAM > and a home-grown ASUS P2-BDS based system > (two 450 MHz PIII) > with 512 MB RAM > I find that the > programs (running on the same input data) on the "smaller" machine tend to > take only a third of the CPU time they need on the LH4. Can you show us the actual results from your testing (an hopefully your testing methods as well) that led you to this conclusion? Details matter. Are these programs I/O bound, CPU bound, or a little of both? FreeBSD's SMP support still depends largely on the big giant lock approach which means that while you can indeed get processes running on multiple CPUs at the same time, you end up using only one CPU once you enter the kernel. And you have to enter the kernel in order to perform any disk, network or even console I/O. If your programs suck large datasets into memory, do lots of number crunching on them, then spit the results back out to a disk file, then they should benefit from more CPUs. However if they read and write data a lot while running, you're going to be limited by the big giant lock. There may also be scalability issues (i.e. does FreeBSD perform better as you add more CPUs or does it spend so much time trying to stay out of its own way that it actually performs worse) however I don't know enough to say if you could be running into such problems as the only SMP machines I have access to have only 2 CPUs. > [Worse: The LH4 > behaves like a spoilt brat when it comes to hardware, disliking the Intel > EtherExpress that came with it (generating bus mastering problems after > bringing it up), Which model Intel EtherExpress? What chipset? What bus mastering problems exactly? > having interrupt routing problems with two DEC TULIP based > ethernet cards sharing the same IRQ Which tulip cards? What driver? What kind of problems? I find it unusual that two PCI devices would wind up with the same IRQ with the APIC enabled since it's supposed to give you a lot more IRQs than in UP mode. > and being picky just which 3C906B-TX it > gets plugged in. There is no such card as a 3c906B. There's a 3c905B, and there's a 3c905C. Unfortunately, 3Com did go through several different ASIC revisions with the 3c905B series, some of which work better than others, but again, I see no details here. > It's a bitch and I'd like shooting it. Oh yes - HP has been > very helpful, telling me that I was at least 10 years behind wanting to run > a BSD and that only WinNT, HP-Sux and Linux were supported on this hardware.] If somebody at HP actually told you that HP-UX runs on anything besides the PA-RISC architecture (and, in the distant past, the m68k architecture), they were either a) jerking your chain, b) working at HP in an parallel dimension, c) misinformed, or d) not terribly bright. (I'm sure HP wouldn't mind having HP-UX/x86, but they certainly don't offer it as a product now.) > Back to the topic: Are there any reasons for these observations? If someone > liked taking a closer look at it I could provide them with access to the > machine (and its console). I ran out of clues... Hard to tell really without more info. We don't know what your test programs do, so it's impossible to predict what their behavior should or shouldn't be. -Bill -- = -Bill Paul(212) 854-6020 | System Manager, Master of Unix-Fu Work: [EMAIL PROTECTED] | Center for Telecommunications Research Home: [EMAIL PROTECTED] | Columbia University, New York City = "It is not I who am crazy; it is I who am mad!" - Ren Hoek, "Space Madness" = To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Current, XEON and MP performance
>I don't know where to ask first (or what to look at) so I'd like some >creative guessing by some people closer to the sources... > >Running the same programs on nearly identically configured -CURRENT kernels >on a HP NetServer LH4 (four 550 MHz PIII Xeon with 512MB Cache, supposed to >be an INTEL 450NX-based chipset) with one GB RAM and a home-grown ASUS >P2-BDS based system (two 450 MHz PIII) with 512 MB RAM I find that the >programs (running on the same input data) on the "smaller" machine tend to >take only a third of the CPU time they need on the LH4. [Worse: The LH4 >behaves like a spoilt brat when it comes to hardware, disliking the Intel >EtherExpress that came with it (generating bus mastering problems after >bringing it up), having interrupt routing problems with two DEC TULIP based >ethernet cards sharing the same IRQ and being picky just which 3C906B-TX it >gets plugged in. It's a bitch and I'd like shooting it. Oh yes - HP has been >very helpful, telling me that I was at least 10 years behind wanting to run >a BSD and that only WinNT, HP-Sux and Linux were supported on this hardware.] > >Back to the topic: Are there any reasons for these observations? If someone >liked taking a closer look at it I could provide them with access to the >machine (and its console). I ran out of clues... What about wall-clock time? -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Current, XEON and MP performance
I don't know where to ask first (or what to look at) so I'd like some creative guessing by some people closer to the sources... Running the same programs on nearly identically configured -CURRENT kernels on a HP NetServer LH4 (four 550 MHz PIII Xeon with 512MB Cache, supposed to be an INTEL 450NX-based chipset) with one GB RAM and a home-grown ASUS P2-BDS based system (two 450 MHz PIII) with 512 MB RAM I find that the programs (running on the same input data) on the "smaller" machine tend to take only a third of the CPU time they need on the LH4. [Worse: The LH4 behaves like a spoilt brat when it comes to hardware, disliking the Intel EtherExpress that came with it (generating bus mastering problems after bringing it up), having interrupt routing problems with two DEC TULIP based ethernet cards sharing the same IRQ and being picky just which 3C906B-TX it gets plugged in. It's a bitch and I'd like shooting it. Oh yes - HP has been very helpful, telling me that I was at least 10 years behind wanting to run a BSD and that only WinNT, HP-Sux and Linux were supported on this hardware.] Back to the topic: Are there any reasons for these observations? If someone liked taking a closer look at it I could provide them with access to the machine (and its console). I ran out of clues... Achim To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
