Re: ixg(4) performances
On 2014/09/04 11:24, Masanobu SAITOH wrote: On 2014/09/04 0:40, Emmanuel Dreyfus wrote: On Wed, Sep 03, 2014 at 04:11:29PM +0200, Bert Kiers wrote: NetBSD 6.1 says: vendor 0x8086 product 0x1528 (ethernet network, revision 0x01) at pci1 dev 0 function 0 not configured In src/sys/dev/pci/ixgbe/ we know about producct Id 0x1529 and 0x152A but not 0x1528. But this can probably be easily borrowed from FreeBSD: http://svnweb.freebsd.org/base/head/sys/dev/ixgbe/ They call it IXGBE_DEV_ID_X540T, You can try to add in ixgbe_type.h: #define IXGBE_DEV_ID_82599_X540T 0x1528 Then in ixgbe.c add a IXGBE_DEV_ID_82599_X540T line in ixgbe_vendor_info_array[] In ixgbe_82599.c you need a case IXGBE_DEV_ID_82599_X540T next to case IXGBE_DEV_ID_82599_BACKPLANE_FCOE if media is indeed backplane. Otherwise add it at the appropriate place in the switch statement. And finally you need to add a case IXGBE_DEV_ID_82599_X540T next to case IXGBE_DEV_ID_82599_BACKPLANE_FCOE in ixgbe_api.c Our ixg(4) driver doesn't support X520. At least there is no s/X520/X540/ file ixgbe/ixgbe_x540.c of FreeBSD. BusFreeBSD NetBSD 82597PCI-Xixgbdge 82598PCIeixgbeixg 82599(X520)PCIeixgbeixg X540PCIeixgbe(ixg)(not yet) -- --- SAITOH Masanobu (msai...@execsw.org msai...@netbsd.org)
Re: ixg(4) performances
On 2014/09/04 0:40, Emmanuel Dreyfus wrote: On Wed, Sep 03, 2014 at 04:11:29PM +0200, Bert Kiers wrote: NetBSD 6.1 says: vendor 0x8086 product 0x1528 (ethernet network, revision 0x01) at pci1 dev 0 function 0 not configured In src/sys/dev/pci/ixgbe/ we know about producct Id 0x1529 and 0x152A but not 0x1528. But this can probably be easily borrowed from FreeBSD: http://svnweb.freebsd.org/base/head/sys/dev/ixgbe/ They call it IXGBE_DEV_ID_X540T, You can try to add in ixgbe_type.h: #define IXGBE_DEV_ID_82599_X540T 0x1528 Then in ixgbe.c add a IXGBE_DEV_ID_82599_X540T line in ixgbe_vendor_info_array[] In ixgbe_82599.c you need a case IXGBE_DEV_ID_82599_X540T next to case IXGBE_DEV_ID_82599_BACKPLANE_FCOE if media is indeed backplane. Otherwise add it at the appropriate place in the switch statement. And finally you need to add a case IXGBE_DEV_ID_82599_X540T next to case IXGBE_DEV_ID_82599_BACKPLANE_FCOE in ixgbe_api.c Our ixg(4) driver doesn't support X520. At least there is no file ixgbe/ixgbe_x540.c of FreeBSD. Bus FreeBSD NetBSD 82597 PCI-X ixgbdge 82598 PCIeixgbe ixg 82599(X520)PCIeixgbe ixg X540 PCIeixgbe (ixg)(not yet) -- --- SAITOH Masanobu (msai...@execsw.org msai...@netbsd.org)
RE: FW: ixg(4) performances
>From Emmanuel Dreyfus
> You are right;
> # pcictl /dev/pci5 read -d 0 -f 1 0xa8
> 00092810
> # pcictl /dev/pci5 write -d 0 -f 1 0xa8 0x00094810
> # pcictl /dev/pci5 read -d 0 -f 1 0xa8
> 4810
That's reassuring. The dump confirms that we're looking at the right
registers, thank you.
As I read the spec, 0x4810 in the Device Control Register means:
Max_Read_Request_Size: 100b -> 4096 bytes
Enable_No_Snoop: 1
Max_Payload_Size: 000b --> 128 bytes
Enable_Relaxed_Ordering: 1
All other options turned off.
I think you should try:
Max_Read_Request_Size: 100b -> 4096 bytes
Enable_No_Snoop: 1
Max_Payload_Size: 100b --> 4096 bytes
Enable_Relaxed_Ordering: 1
This would give 0x4890 as the value, not 0x4810.
It's odd that the BIOS set the max_payload_size to 000b. It's possible that
this indicates that the root complex has some limitations. Or it could be a
buggy or excessively conservative BIOS. ("It's safer to program Add-In
boards conservatively -- fewer support calls due to dead systems." Or
something like that.)
So you may have to experiment. This would explain that you saw 2.5 GB/sec
before, and 2.7 GB/sec after -- you increased the max *read* size, but not
the max *write* size. Increasing from 2048 to 4096 would improve read
throughput but not enormously. Depends, of course, on your benchmark.
--Terry
RE: FW: ixg(4) performances
> -Original Message- > From: Hisashi T Fujinaka [mailto:ht...@twofifty.com] > Sent: Sunday, August 31, 2014 12:39 > To: Terry Moore > Cc: tech-kern@netbsd.org > Subject: RE: FW: ixg(4) performances > > I may be wrong in the transactions/transfers. However, I think you're > reading the page incorrectly. The signalling rate is the physical speed of > the link. On top of that is the 8/10 encoding (the Ethernet controller > we're talking about is only Gen 2), the framing, etc, and the spec > discusses the data rate in GT/s. Gb/s means nothing. Hi, I see what the dispute is. The PCIe 3.0 spec *nowhere* uses "transfers" in the sense of UNIT INTERVALs (it uses UNIT INTERVAL). The word "transfer" is not used in that way in the spec. Transfer is used, mostly in the sense of a larger message. It's not in the glossary, etc. However, you are absolutely right, many places in the industry (Intel, Wikipedia, etc.) refer to 2.5GT/s to mean 2.5G Unit Intervals/Second; and it's common enough that it's the de-facto standard terminology. I only deal with the spec, and don't pay that much attention to the ancillary material. We still disagree about something: GT/s does mean *something* in terms of the raw throughput of the link. It tells the absolute upper limit of the channel (ignoring protocol overhead). If the upper limit is too low, you can stop worrying about fine points. If you're only trying to push 10% (for example) and you're not getting it, you look for protocol problems. At 50%, you say "hmm, we could be hitting the channel capacity". I think we were violently agreeing, because the technical content of what I was writing was (modulo possible typos) identical to 2.5GT/s. 2.5GT/s is (after 8/10 encoding) a max raw data rate of 250e6 bytes/sec, and when you go through things and account for overhead, it's very possible that 8 lanes (max 2e9 bytes/sec) of gen1 won't be fast enough for 10G Eth. Best regards, --Terry
PUFFS race on parent mtime update
Hi When running regression tests on FUSE filesystems, I hit a i race condition: when I create a node in a directory, the parent directory mtime/ctime seems to be updated asynchronously, and some tests that check it report an error because it has not been updated at test time. Here is a PoC: stat -f '%m' . ; mkfifo x; stat -f '%m' . ; sleep 1; stat -f '%m' . This gives the result: 1409757799 1409757799 1409757818 Which means that when mkfifo returns, the parent directory mtime is not yet updated. If I wait one second I get it alrgiht. I have trouble to understand where parent directory ctime/mtime is updated; Anyone can tell me how this should work? -- Emmanuel Dreyfus m...@netbsd.org
Re: ixg(4) performances
On Wed, Sep 03, 2014 at 04:11:29PM +0200, Bert Kiers wrote: > NetBSD 6.1 says: > vendor 0x8086 product 0x1528 (ethernet network, revision 0x01) at pci1 dev 0 > function 0 not configured In src/sys/dev/pci/ixgbe/ we know about producct Id 0x1529 and 0x152A but not 0x1528. But this can probably be easily borrowed from FreeBSD: http://svnweb.freebsd.org/base/head/sys/dev/ixgbe/ They call it IXGBE_DEV_ID_X540T, You can try to add in ixgbe_type.h: #define IXGBE_DEV_ID_82599_X540T 0x1528 Then in ixgbe.c add a IXGBE_DEV_ID_82599_X540T line in ixgbe_vendor_info_array[] In ixgbe_82599.c you need a case IXGBE_DEV_ID_82599_X540T next to case IXGBE_DEV_ID_82599_BACKPLANE_FCOE if media is indeed backplane. Otherwise add it at the appropriate place in the switch statement. And finally you need to add a case IXGBE_DEV_ID_82599_X540T next to case IXGBE_DEV_ID_82599_BACKPLANE_FCOE in ixgbe_api.c -- Emmanuel Dreyfus m...@netbsd.org
Re: ixg(4) performances
On Wed, Sep 03, 2014 at 04:11:29PM +0200, Bert Kiers wrote: > NetBSD 6.1 says: > > vendor 0x8086 product 0x1528 (ethernet network, revision 0x01) at pci1 dev 0 > function 0 not configured > > Complete messages: http://netbsd.itsx.net/hw/x9drw.dmesg > > NetBSD current from today also does not configure them. Btw, FreeBSD 9.3-RELEASE says: ix0: port 0x8020-0x803f mem 0xde20-0xde3f,0xde404000-0xde407fff irq 26 at device 0.0 on pci1 ix0: Using MSIX interrupts with 9 vectors ix0: Ethernet address: 00:25:90:f9:49:20 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 -- B*E*R*T
Re: ixg(4) performances
On Sat, Aug 30, 2014 at 10:24:52AM +0100, Justin Cormack wrote: > On Sat, Aug 30, 2014 at 8:22 AM, Thor Lancelot Simon wrote: > > On Fri, Aug 29, 2014 at 12:22:31PM -0400, Terry Moore wrote: > >> > >> Is the ixg in an expansion slot or integrated onto the main board? > > > > If you know where to get a mainboard with an integrated ixg, I wouldn't > > mind hearing about it. > > They are starting to appear, eg > http://www.supermicro.co.uk/products/motherboard/Xeon/C600/X9SRH-7TF.cfm We have some SuperMicro X9DRW with 10 GbE Intel NICs on board. NetBSD current does not configure them. NetBSD 6.1 says: vendor 0x8086 product 0x1528 (ethernet network, revision 0x01) at pci1 dev 0 function 0 not configured Complete messages: http://netbsd.itsx.net/hw/x9drw.dmesg NetBSD current from today also does not configure them. (I boot from an USB-stick with current kernel and 6.1 userland. It wants to mount sda0 but there is only dk0, dk1. So I end up with read/only disk. I have to sort that out before I can save kernels messages.) Grtnx, -- B*E*R*T
