Re: [E1000-devel] ixgbe and SRIOV failure in driver?

[Rose, Gregory V]

-Original Message-
From: netdev-ow...@vger.kernel.org [mailto:netdev-ow...@vger.kernel.org]
On Behalf Of Fischer, Anna
Sent: Friday, May 21, 2010 9:33 AM
To: e1000-devel@lists.sourceforge.net; net...@vger.kernel.org
Subject: ixgbe and SRIOV failure in driver?

I am running a system with 3 Intel NICs. Two of them are 82598 devices,
and one is a SRIOV capable 82599.

All devices use the ixgbe driver. What happens (I believe) now is that
when the driver loads at first, it sees the 82598 first (because of its
position in the PCI tree) and then it says Device not IOV capable -
switching off IOV.

So then it switches into non-IOV mode, and I can never enable SRIOV on
my 82599, because the driver does not enable it any more for further
devices.

So to get around this issue, I tried to use pciback.hide to hide the
82598 devices from the OS. That way I was hoping that the driver would
switch on SRIOV on my 82599. However, then I got a kernel panic on boot
(see below).

I am running Xen 4 and the Dom0 kernel is a 2.6.31 kernel.

The ixgbe driver included with the 2.6.31 kernel does not support SR-IOV.
Where did you get the driver that does?  Please run ethtool -i ethx and
post the results.

- Greg Rose
Intel Corp.
Lan Access Division


--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

[Rose, Gregory V]

-Original Message-
From: Gerd v. Egidy [mailto:li...@egidy.de]
Sent: Friday, May 21, 2010 5:24 AM
To: e1000-devel@lists.sourceforge.net
Cc: Rose, Gregory V
Subject: Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

Hi Greg,

thanks for your quick reply.

 But the host can't send packets to the guests (or these packets are
 never
 received on the guests). When using tcpdump, I can see these packets
 leaving
 the device on the host, but they never arrive on the guest.

 So you're saying that the guests don't even receive packets from the
host
 with broadcast as the destination address?

that is correct.

I just tried it again to verify:

eth1 on the host is connected to an otherwise unused switch.
The host is at 192.168.100.1, the vm at 192.168.100.2. The vm is not
connected
to any other network and has only the vf as nic (eth0).

[r...@host]# arping -I eth1 192.168.100.2

- no response

during this I ran on the vm:

[r...@vm]# tcpdump -i eth0 -n

- no packets

When looking at the packet counter on the vm, the RX-counter is still 0.
So no
broadcasts from the host can reach the vm.

When doing it the other way round:

[r...@vm]# arping -I eth0 192.168.100.1

I still get no response. But that is because the host can't answer. The
host
sees the packets and replies:

[r...@host]# tcpdump -i eth1 -n
14:05:48.473303 ARP, Request who-has 192.168.100.1 (Broadcast) tell
192.168.100.2, length 28
14:05:48.473329 ARP, Reply 192.168.100.1 is-at 00:1b:21:60:xx:xx, length
28
14:05:49.473478 ARP, Request who-has 192.168.100.1 (Broadcast) tell
192.168.100.2, length 28
14:05:49.473492 ARP, Reply 192.168.100.1 is-at 00:1b:21:60:xx:xx, length
28
[...]

This evening I will upgrade the vm to 2.6.34 too. But aside from that I
don't
know what else to try.

Any ideas what I could do to further trace this down?

Pretty strange but one thing I can think of to do is make sure that the source
MAC address in the ARP packets the host is receiving is the same one you 
assigned
via the ip link set... commands.

Other than that I'm in the process of setting something up to reproduce the 
problem.

- Greg

--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

[Rose, Gregory V]

-Original Message-
From: Rose, Gregory V
Sent: Friday, May 21, 2010 12:29 PM
To: 'Gerd v. Egidy'; e1000-devel@lists.sourceforge.net
Subject: RE: [E1000-devel] guest with igbvf on 82576 can't talk to host

-Original Message-
From: Gerd v. Egidy [mailto:li...@egidy.de]
Sent: Friday, May 21, 2010 5:24 AM
To: e1000-devel@lists.sourceforge.net
Cc: Rose, Gregory V
Subject: Re: [E1000-devel] guest with igbvf on 82576 can't talk to host


[snip]

This evening I will upgrade the vm to 2.6.34 too. But aside from that I
don't
know what else to try.

Any ideas what I could do to further trace this down?

Pretty strange but one thing I can think of to do is make sure that the
source
MAC address in the ARP packets the host is receiving is the same one you
assigned
via the ip link set... commands.

Other than that I'm in the process of setting something up to reproduce
the problem.

Good news of a sort.  I can reproduce the problem.  Which means I can debug it 
and hopefully fix it.

I'll get back to you when I have some results.

- Greg


--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

[Rose, Gregory V]

-Original Message-
From: Gerd v. Egidy [mailto:li...@egidy.de]
Sent: Friday, May 21, 2010 1:08 PM
To: e1000-devel@lists.sourceforge.net
Cc: Rose, Gregory V
Subject: Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

  But the host can't send packets to the guests (or these packets are
  never
  received on the guests). When using tcpdump, I can see these
packets
  leaving
  the device on the host, but they never arrive on the guest.

 This evening I will upgrade the vm to 2.6.34 too. But aside from that
I
 don't know what else to try.

host and vm are on 2.6.34 now. But it didn't change anything, the
problem
still persists.

Yes, I reproduced the problem on 2.6.34.  I'm debugging it now.

Will let you know of results ASAP.

- Greg


--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] ixgbe: macvlan on PF/VF when SRIOV is enabled

[Rose, Gregory V]

-Original Message-
From: Shirley Ma [mailto:mashi...@us.ibm.com]
Sent: Monday, May 24, 2010 10:09 AM
To: Rose, Gregory V
Cc: Kirsher, Jeffrey T; da...@davemloft.net; k...@vger.kernel.org;
net...@vger.kernel.org; e1000-devel@lists.sourceforge.net
Subject: RE: ixgbe: macvlan on PF/VF when SRIOV is enabled

Hello Greg,

Thanks for your prompt response.

On Sat, 2010-05-22 at 10:53 -0700, Rose, Gregory V wrote:
 As of 2.6.34 the ixgbe driver does not support multiple queues for
 macvlan.
 Support for multiple queues for macvlan will come in a subsequent
 release.

When it might happen?

Support for multiple queues in the PF driver is in the planning stage right 
now.  Hopefully we'll get it in sooner rather than later but I can't give any 
solid dates for it.

 I will double check my test to see whether macvlan
was multiple queue or not.

Ah, so you just want to set multiple macvlan filters for the PF driver but 
aren't concerned about directing the traffic to different queues?  We haven't 
tested that in SR-IOV modes of operation but we can have a look at it.

Then submitting my experimental patch for
review.

We look forward to it and will be happy to provide feedback.


 The VF driver does not support macvlan.  Future releases may but there
 are no immediate plans to support it.

When it might be support in future. For performance reason, we are
interested in macvlan + VF for multiples VMs.

There is a resource contention issue in this case.  There are 128 MAC filters 
available.  When VFs are allocated each will use a MAC filter entry.  In the 
case where the maximum number of VFs are allocated (63 in this case) there 
aren't a whole lot of MAC filters left over to spread across that many VFs.  
Our view has been that it wouldn't be that much added value to support macvlan 
in the VFs but if there is a good case for it we'll consider it.

One thing you can do is allocate VFs and then load the VF driver in your host 
domain and then assign each of them a macvlan filter.  You'd get a similar 
effect.


One more question here: Does VF support promiscuous mode? I don't see
the flag in ixgbevf driver.

No, there is no per-VF support for promiscuous mode in the HW as such, however 
you could set the unicast hash table array to all ones and then set the VFs you 
want to be in promiscuous mode to accept untagged packets via the VM filtering 
and offload register at offset 0F000h.  However, re-provisioning the UTA for 
this purpose would preclude using it for the designed purpose.

I suggest that you pick up a copy of the developer's manual for the 82599 at 
the Intel developer's site.

http://developer.intel.com/products/ethernet/index.htm?iid=nc+ethernet#s1=alls2=82576EBs3=all

- Greg

--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] ixgbe: macvlan on PF/VF when SRIOV is enabled

[Rose, Gregory V]

-Original Message-
From: Shirley Ma [mailto:mashi...@us.ibm.com]
Sent: Tuesday, May 25, 2010 9:51 AM
To: Rose, Gregory V
Cc: Kirsher, Jeffrey T; da...@davemloft.net; k...@vger.kernel.org;
net...@vger.kernel.org; e1000-devel@lists.sourceforge.net
Subject: RE: ixgbe: macvlan on PF/VF when SRIOV is enabled

On Tue, 2010-05-25 at 08:57 -0700, Rose, Gregory V wrote:
 Just use ifconfig and vconfig utilities to set the MAC and VLAN for
 each VF.  There shouldn't be any need for secondary addresses because
 they're not like physical devices where each VF has a pre-programmed
 HW MAC address.  The initial MAC address of each VF is generated on
 the fly during the PF driver initialization. You can change it as you
 see fit and then put the VF on a VLAN using vconfig.  After you do
 that you have a macvlan filter for that VF.

I run macvlan test not vlan. macvlan is used to give a second MAC
address to a network adapter and see it as a new device at the higher
levels. The command is used as follow:

ip link add link eth4 address 54:52:00:35:e3:20 macvlan2 type macvlan

It will create an interface name macvlan2 with above MAC address. In
kernel, netdev eth4 maintains this secondary address.

Right, so what I'm saying is that if you load the VF driver it will create and 
eth interface for each VF you've created.  You can assign a MAC and VLAN to 
that eth interface and will get essentially the same behavior.

- Greg

--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH net-next] ixgbe: make macvlan on PF working when SRIOV is enabled

[Rose, Gregory V]

-Original Message-
From: Shirley Ma [mailto:mashi...@us.ibm.com]
Sent: Tuesday, May 25, 2010 10:16 AM
To: Kirsher, Jeffrey T
Cc: Rose, Gregory V; da...@davemloft.net; k...@vger.kernel.org;
net...@vger.kernel.org; e1000-devel@lists.sourceforge.net
Subject: Re: [PATCH net-next] ixgbe: make macvlan on PF working when
SRIOV is enabled

To produce this problem:

1. modprobe ixgbe max_vfs=2
   eth4 is PF, eth5 is VF
2. ip link set eth4 up
3. ip link add link eth4 address 54:52:00:35:e3:20 macvlan2 type macvlan
4. ip addr add 192.168.7.74/24 dev macvlan2
5. ping macvlan2 from remote host, works
6. ip link set eth5 up
7. ping macvlan2 from remote host failed.

Based on my understanding, the problem is:
1. PF set_rar use rar index is 0, and vmdq index is adapter-num_vfs,
2. when macvlan2 is created, rar index is based rar_used_count, which
would be 1.
3. later when VF is up, the rar index is vf+1, and vmdq index is vf, so
VF0 will overwrite macvlan2 rar entry.

The fix here:
1. make sure PF uses vmdq index = adapter-num_vfs during
initialization, reset.
2. reserve rar index for all VFs from 1 to num_vfs + 1.


Please let me know whether my understanding is correct or not.

Yes, that appears to be correct.

We'll test your patch but I think you're on the right track.

- Greg


--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

[Rose, Gregory V]

-Original Message-
From: Gerd v. Egidy [mailto:li...@egidy.de]
Sent: Thursday, May 27, 2010 3:13 PM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] guest with igbvf on 82576 can't talk to host

Hi Greg,

Very good. Hope you find something soon.

Alright we've found something and it seems to be a pretty solid lead.  What I'd 
like for you to do if you can find the time is repeat the tests you've been 
running but put up a window that watches the packet stats per queue.  Something 
like this:

watch -n 2 'ethtool -S ethx | grep packets'

Then watch the tx queue packet stats.  We're thinking that when the problem 
occurs that the VF is not able to communicate with the PF you will see packets 
from the PF to the VF going out on tx queue 1.  When it works you'll see 
packets going out on tx queue 0.

If you could confirm that for us it would help out a great deal.

Thanks for your help and your patience.

- Greg



--

___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH] igbvf: avoid name clash between PF and VF

[Rose, Gregory V]

-Original Message-
From: Stefan Assmann [mailto:sassm...@redhat.com]
Sent: Friday, July 09, 2010 2:32 AM
To: Arnd Bergmann
Cc: netdev; e1000-devel@lists.sourceforge.net; Duyck, Alexander H; Rose,
Gregory V; Kirsher, Jeffrey T; Andy Gospodarek
Subject: Re: [PATCH] igbvf: avoid name clash between PF and VF

On 08.07.2010 15:41, Arnd Bergmann wrote:
 On Wednesday 30 June 2010, Stefan Assmann wrote:
 diff --git a/drivers/net/igbvf/netdev.c b/drivers/net/igbvf/netdev.c
 index 5e2b2a8..2fb665b 100644
 --- a/drivers/net/igbvf/netdev.c
 +++ b/drivers/net/igbvf/netdev.c
 @@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev
*pdev,
 netif_carrier_off(netdev);
 netif_stop_queue(netdev);

 -   strcpy(netdev-name, eth%d);
 +   strcpy(netdev-name, veth%d);
 err = register_netdev(netdev);
 if (err)
 goto err_hw_init;

 Note that 'veth' is the name used for a virtual ethernet pair by
 drivers/net/veth.c. If a variant of your patch gets applied, it would
 probably be useful to use a different naming scheme to avoid confusion
 with the veth driver.

Good point!
Greg suggested vfeth, that should be more descriptive and unique.

  Stefan
---
diff --git a/drivers/net/igbvf/netdev.c b/drivers/net/igbvf/netdev.c
index 5e2b2a8..4d02af8 100644
--- a/drivers/net/igbvf/netdev.c
+++ b/drivers/net/igbvf/netdev.c
@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev
*pdev,
   netif_carrier_off(netdev);
   netif_stop_queue(netdev);

-  strcpy(netdev-name, eth%d);
+  strcpy(netdev-name, vfeth%d);
   err = register_netdev(netdev);
   if (err)
   goto err_hw_init;

Acked-by: Greg Rose gregory.v.r...@intel.com


--
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH 26/49] drivers/net/ixgbevf: Use vzalloc

[Rose, Gregory V]

 -Original Message-
 From: Joe Perches [mailto:j...@perches.com]
 Sent: Thursday, November 04, 2010 8:08 PM
 To: Jiri Kosina
 Cc: Kirsher, Jeffrey T; Brandeburg, Jesse; Allan, Bruce W; Wyborny,
 Carolyn; Skidmore, Donald C; Rose, Gregory V; Waskiewicz Jr, Peter P;
 Duyck, Alexander H; Ronciak, John; e1000-devel@lists.sourceforge.net;
 net...@vger.kernel.org; linux-ker...@vger.kernel.org
 Subject: [PATCH 26/49] drivers/net/ixgbevf: Use vzalloc
 
 Signed-off-by: Joe Perches j...@perches.com
 ---
  drivers/net/ixgbevf/ixgbevf_main.c |6 ++
  1 files changed, 2 insertions(+), 4 deletions(-)
 
 diff --git a/drivers/net/ixgbevf/ixgbevf_main.c
 b/drivers/net/ixgbevf/ixgbevf_main.c
 index dc03c96..6aeaf54 100644
 --- a/drivers/net/ixgbevf/ixgbevf_main.c
 +++ b/drivers/net/ixgbevf/ixgbevf_main.c
 @@ -2488,10 +2488,9 @@ int ixgbevf_setup_tx_resources(struct
 ixgbevf_adapter *adapter,
   int size;
 
   size = sizeof(struct ixgbevf_tx_buffer) * tx_ring-count;
 - tx_ring-tx_buffer_info = vmalloc(size);
 + tx_ring-tx_buffer_info = vzalloc(size);
   if (!tx_ring-tx_buffer_info)
   goto err;
 - memset(tx_ring-tx_buffer_info, 0, size);
 
   /* round up to nearest 4K */
   tx_ring-size = tx_ring-count * sizeof(union ixgbe_adv_tx_desc);
 @@ -2555,14 +2554,13 @@ int ixgbevf_setup_rx_resources(struct
 ixgbevf_adapter *adapter,
   int size;
 
   size = sizeof(struct ixgbevf_rx_buffer) * rx_ring-count;
 - rx_ring-rx_buffer_info = vmalloc(size);
 + rx_ring-rx_buffer_info = vzalloc(size);
   if (!rx_ring-rx_buffer_info) {
   hw_dbg(adapter-hw,
  Unable to vmalloc buffer memory for 
  the receive descriptor ring\n);
   goto alloc_failed;
   }
 - memset(rx_ring-rx_buffer_info, 0, size);
 
   /* Round up to nearest 4K */
   rx_ring-size = rx_ring-count * sizeof(union ixgbe_adv_rx_desc);
 --
 1.7.3.1.g432b3.dirty

Acked By: Greg Rose gregory.v.r...@intel.com


--
The Next 800 Companies to Lead America's Growth: New Video Whitepaper
David G. Thomson, author of the best-selling book Blueprint to a 
Billion shares his insights and actions to help propel your 
business during the next growth cycle. Listen Now!
http://p.sf.net/sfu/SAP-dev2dev
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] 82599EB - PCI Passthrough gives Error -5

[Rose, Gregory V]

 -Original Message-
 From: Robert Dunkley [mailto:rob...@saq.co.uk]
 Sent: Thursday, March 24, 2011 1:22 AM
 To: E1000-devel@lists.sourceforge.net
 Cc: Linux NICS
 Subject: [E1000-devel] 82599EB - PCI Passthrough gives Error -5
 
 I'm trying to setup some X520-DA2s with one copper cable and one Intel LR
 SFP in each for standard PF PCI device pass through. The same systems are
 working OK with virtual function pass through of 82576 ports so I think
 the Xen  configuration is fine. The 82599EB works correctly in Dom0,
 appears in pci assignable devices when hidden and is set as permissive in
 pciback. All messages in Dom0 seem encouraging but the paravirt Domu just
 gives error -5 when using the 3.2.10 driver or the Redhat supplied
 drivers. Kernel is RHEL 2.6.18-247.

Error -5 is EIO.  Without more information it's hard for me to determine what 
your exact problem is but Intel does not officially support pass through (aka 
Direct Assignment) of PF devices to guests.  Vendors who support that feature 
do so at their own risk.  We would suggest using the SR-IOV feature to pass a 
virtual function through to a guest and that is our only supported 
configuration.

Regards,

- Greg
Greg Rose
LAN Access Division
Intel Corp.



--
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software 
be a part of the solution? Download the Intel(R) Manageability Checker 
today! http://p.sf.net/sfu/intel-dev2devmar
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] 82599EB - PCI Passthrough gives Error -5

[Rose, Gregory V]

 -Original Message-
 From: Robert Dunkley [mailto:rob...@saq.co.uk]
 Sent: Friday, March 25, 2011 1:11 AM
 To: Rose, Gregory V; E1000-devel@lists.sourceforge.net
 Cc: Linux NICS
 Subject: RE: 82599EB - PCI Passthrough gives Error -5
 
 Hi Greg,
 
 
 I believe I have to pass through the PF because I need to support an MTU
 of 1600 within the VM which from what I've read is impossible with
 SR-IOV, is that correct? With Debian Squeeze PF pass through seems to
 work (Detects card and SFPs), I assume per port PF pass through is known
 to work but not officially supported?

You are correct, we don't' support jumbo frames or frames larger than the 
standard Ethernet MTU size on the 82599EB in SR-IOV mode.  Also, as you state 
per port PF pass through is known to work on a large number of platforms and OS 
configurations but we do not officially support it.  You might be able to get 
help from your OS vendor if they claim to support it.

- Greg


--
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software 
be a part of the solution? Download the Intel(R) Manageability Checker 
today! http://p.sf.net/sfu/intel-dev2devmar
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] I have a Problem in vf rate limit

[Rose, Gregory V]

One other thing to note is that Tx rate limiting is a cap, not a guarantee that 
the bandwidth allocation can be provisioned.  Other issues, especially the 
resource allocation to your VMs, but also other matters such as the kernel used 
in the VMs, the hypervisor, etc. can impact total bandwidth capacity.

From looking at the results you've shown none of the VMs are exceeding their 
rate limit.  It's impossible for us to tell from your email why you're not 
getting line rate but as I mentioned above there could be many reasons for that.

- Greg

 -Original Message-
 From: Brandeburg, Jesse [mailto:jesse.brandeb...@intel.com]
 Sent: Monday, April 11, 2011 9:04 AM
 To: 박대영; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] I have a Problem in vf rate limit
 
 How are you measuring bandwidth?
 
 -Original Message-
 From: 박대영 [mailto:likebulle...@gmail.com]
 Sent: Monday, April 11, 2011 4:49 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] I have a Problem in vf rate limit
 
 hello I'm using a Ixgbe 3.3.8.
 
 there are new feature vf rate limit.
 
 when I use this feature with 3 VM in xen. and rate limit configuration VM
 1:
 2Gb/s VM 2: 3Gb/s VM 3: 5Gb/s
 
 but result is VM 1: 1962Mb/sec VM2: 1910Mb/sec VM3: 3165Mb/sec
 
 SUM of bandwidth is only 7Gb/sec.
 
 what is problem.
 --
 
 Xperia(TM) PLAY
 It's a major breakthrough. An authentic gaming
 smartphone on the nation's most reliable network.
 And it wants your games.
 http://p.sf.net/sfu/verizon-sfdev
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174; Ethernet, visit
 http://communities.intel.com/community/wired
--
Forrester Wave Report - Recovery time is now measured in hours and minutes
not days. Key insights are discussed in the 2010 Forrester Wave Report as
part of an in-depth evaluation of disaster recovery service providers.
Forrester found the best-in-class provider in terms of services and vision.
Read this report now!  http://p.sf.net/sfu/ibm-webcastpromo
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] VM can not get ip and with error igbvf 0000:00:04.0: Unable to allocate interrupt, Error: -1 failed

[Rose, Gregory V]

 -Original Message-
 From: guan qin [mailto:qinguan0...@gmail.com]
 Sent: Saturday, April 23, 2011 8:46 PM
 To: e1000-de...@lists.sf.net
 Subject: [E1000-devel] VM can not get ip and with error igbvf
 :00:04.0: Unable to allocate interrupt, Error: -1 failed
 
 Hi，
 
 I want to use SRIOV under debian 6.0.1a ，and I had installed the drive
 igb
 2.4.13 and igbvf 1.0.7。
 I boot the VM by virsh tool :virsh create sriov.xml ,just create a
 ubuntu10.04 VM.
 when I enter the VM ,I can't access to the network.
 If I use sudo ifconfig eth0 192.168.1.181, I always  got the message
 SIOCSIFFLAGS:Operation not permitted.

It looks to me like you don't have suitable account permissions to perform 
network configuration.

- Greg

 according the message Unable to allocate interrupt, Error: -1  from
 dmesg in VM, I see that the message coming from a function:*
 igbvf_request_irq*, failed *initialize interrupts *. But I didn't  know
 how to solve it ?
 (http://tomoyo.sourceforge.jp/cgi-
 bin/lxr/source/drivers/net/igbvf/netdev.c:
 *line 1074*)
 
 besides, I  assigned a VF to the VM which HWaddr aa:2a:91:5a:eb:c8, but in
 the VM the HWaddr randomly change into onother mac such  as
 b2:c1:6e:ad:67:db
 It also puzzled me many days.
 can you help me ?
 
 thanks,
 qinguan
 
 *some dmesg info  in the VM:*
 JBD:barrier-based sync failed on vda1-8 - disabling barriers igbvf
 :00:04.0:Unable to allocate interrupt,Error:-1
 
 *some dmesg info in the host:*
 pci-stub :02:10.1 enabling device( - 0002) assign device 0:2:10.1
 *
 /var/log/libvirt/qemu/testnew_srion.log:*
 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 HOME=/ QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 512 -
 smp 1,sockets=1,cores=1,threads=1 -name testnew_sriov -uuid ea040095-01e4-
 61cb-bb79-10b92712eb4f -nodefaults -chardev
 socket,id=monitor,path=/var/lib/libvirt/qemu/testnew_sriov.monitor,server,
 nowait
 -mon chardev=monitor,mode=readline -rtc base=utc -boot c -drive
 file=/home/qinguan/exp/ubuntu_1.img,if=none,id=drive-virtio-
 disk0,boot=on,format=raw
 -device
 virtio-blk-pci,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0
 -usb -device usb-mouse,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device
 pci-assign,host=02:10.1,id=hostdev0,bus=pci.0,addr=0x4 -device
 virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
 
 *sriov.xml:*
 
 domain type='kvm'
 nametestnew_sriov/name
 descriptionjust test using sriov!/description
 os
 typehvm/type
 boot dev='hd'/
 /os
 memory524288/memory
 currentMemory524288/currentMemory
 on_poweroffdestroy/on_poweroff
 on_rebootrestart/on_reboot
 on_crashrestart/on_crash
 
 featurespae/acpi/apic//features
 clock offset='utc'/
 devices
 input type='mouse' bus='usb'/
 disk type='file' device='disk'
 source file='/home/qinguan/exp/ubuntu_1.img'/
 target dev='hda' bus='virtio'/
 /disk
 emulator/usr/bin/kvm/emulator
 graphics type='vnc' port='-1'/
 hostdev mode='subsystem' type='pci' managed='yes'
 source
 address bus='0x2' slot='0x10' function='0x1'/
 /source
 boot order='1'/
 /hostdev
 /devices
 /domain
 
 
 *more system information please look into the attachment,thanks.*

--
Fulfilling the Lean Software Promise
Lean software platforms are now widely adopted and the benefits have been 
demonstrated beyond question. Learn why your peers are replacing JEE 
containers with lightweight application servers - and what you can gain 
from the move. http://p.sf.net/sfu/vmware-sfemails
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH] ixgbevf: remove unnecessary ampersands

[Rose, Gregory V]

 -Original Message-
 From: Stephen Hemminger [mailto:shemmin...@vyatta.com]
 Sent: Wednesday, June 08, 2011 11:11 AM
 To: e1000-de...@lists.sf.net
 Subject: [E1000-devel] [PATCH] ixgbevf: remove unnecessary ampersands
 
 Use standard format for net_device_ops (without )
 
 Signed-off-by: Stephen Hemminger shemmin...@vyatta.com

Acked-by: Greg Rose gregory.v.r...@intel.com



--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [patch net-next-2.6 18/47 V2] igbvf: do vlan cleanup

[Rose, Gregory V]

 -Original Message-
 From: Jiri Pirko [mailto:jpi...@redhat.com]
 Sent: Thursday, July 21, 2011 6:23 AM
 To: net...@vger.kernel.org
 Cc: da...@davemloft.net; shemmin...@linux-foundation.org;
 eric.duma...@gmail.com; gree...@candelatech.com; mir...@gmail.com;
 Kirsher, Jeffrey T; Brandeburg, Jesse; Waskiewicz Jr, Peter P; Allan,
 Bruce W; Wyborny, Carolyn; Skidmore, Donald C; Rose, Gregory V; Duyck,
 Alexander H; Ronciak, John; e1000-devel@lists.sourceforge.net;
 je...@nicira.com
 Subject: [patch net-next-2.6 18/47 V2] igbvf: do vlan cleanup
 
 - unify vlan and nonvlan rx path
 - kill adapter-vlgrp and igbvf_vlan_rx_register
 
 Signed-off-by: Jiri Pirko jpi...@redhat.com
 ---
  drivers/net/igbvf/igbvf.h  |4 +-
  drivers/net/igbvf/netdev.c |   55 +++
 
  2 files changed, 26 insertions(+), 33 deletions(-)
 
 diff --git a/drivers/net/igbvf/igbvf.h b/drivers/net/igbvf/igbvf.h
 index d5dad5d..fd4a7b7 100644
 --- a/drivers/net/igbvf/igbvf.h
 +++ b/drivers/net/igbvf/igbvf.h
 @@ -34,7 +34,7 @@
  #include linux/timer.h
  #include linux/io.h
  #include linux/netdevice.h
 -
 +#include linux/if_vlan.h
 
  #include vf.h
 
 @@ -173,7 +173,7 @@ struct igbvf_adapter {
 
   const struct igbvf_info *ei;
 
 - struct vlan_group *vlgrp;
 + unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
   u32 bd_number;
   u32 rx_buffer_len;
   u32 polling_interval;
 diff --git a/drivers/net/igbvf/netdev.c b/drivers/net/igbvf/netdev.c
 index 64b47bf..d924b09 100644
 --- a/drivers/net/igbvf/netdev.c
 +++ b/drivers/net/igbvf/netdev.c
 @@ -100,12 +100,12 @@ static void igbvf_receive_skb(struct igbvf_adapter
 *adapter,
struct sk_buff *skb,
u32 status, u16 vlan)
  {
 - if (adapter-vlgrp  (status  E1000_RXD_STAT_VP))
 - vlan_hwaccel_receive_skb(skb, adapter-vlgrp,
 -  le16_to_cpu(vlan) 
 -  E1000_RXD_SPC_VLAN_MASK);
 - else
 - netif_receive_skb(skb);
 + if (status  E1000_RXD_STAT_VP) {
 + u16 vid = le16_to_cpu(vlan)  E1000_RXD_SPC_VLAN_MASK;
 +
 + __vlan_hwaccel_put_tag(skb, vid);
 + }
 + netif_receive_skb(skb);
  }
 
  static inline void igbvf_rx_checksum_adv(struct igbvf_adapter *adapter,
 @@ -1167,22 +1167,29 @@ static int igbvf_poll(struct napi_struct *napi,
 int budget)
   */
  static void igbvf_set_rlpml(struct igbvf_adapter *adapter)
  {
 - int max_frame_size = adapter-max_frame_size;
 + int max_frame_size;
   struct e1000_hw *hw = adapter-hw;
 
 - if (adapter-vlgrp)
 - max_frame_size += VLAN_TAG_SIZE;
 -
 + max_frame_size = adapter-max_frame_size + VLAN_TAG_SIZE;
   e1000_rlpml_set_vf(hw, max_frame_size);
  }
 
 -static void igbvf_vlan_rx_add_vid(struct net_device *netdev, u16 vid)
 +static bool __igbvf_vlan_rx_add_vid(struct igbvf_adapter *adapter, u16
 vid)
  {
 - struct igbvf_adapter *adapter = netdev_priv(netdev);
   struct e1000_hw *hw = adapter-hw;
 
   if (hw-mac.ops.set_vfta(hw, vid, true))
   dev_err(adapter-pdev-dev, Failed to add vlan id %d\n,
 vid);
 + return false;
 + return true;
 +}

I'm pretty sure you intended to put a curly brace after the if statement here.

Other than that it seems fine.

- Greg



--
5 Ways to Improve  Secure Unified Communications
Unified Communications promises greater efficiencies for business. UC can 
improve internal communications as well as offer faster, more efficient ways
to interact with customers and streamline customer service. Learn more!
http://www.accelacomm.com/jaw/sfnl/114/51426253/
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Unicast hash for IXGBEVF driver

[Rose, Gregory V]

The ROPE bits should not be set for any of the VFs and only on the PF when it 
is in promiscuous mode.

The PFUTA bits are imperfect MAC address hash filters.  By setting them all to 
ones and then turning on the ROPE bit for each of the VFs you're essentially 
putting the VF into a sort of fake promiscuous mode.

This is not recommended.  Older drivers that did this had a bug in them.

- Greg

 -Original Message-
 From: tar...@gmail.com [mailto:tar...@gmail.com] On Behalf Of Jeff Kirsher
 Sent: Saturday, August 27, 2011 12:35 AM
 To: J.Hwan Kim; Rose, Gregory V
 Cc: netdev; e1000-devel@lists.sourceforge.net
 Subject: Re: Unicast hash for IXGBEVF driver
 
 On Sat, Aug 27, 2011 at 00:23, J.Hwan Kim frog1...@gmail.com wrote:
  Hi, everyone
 
  How can I distribute the packets according to destination MAC address
  into multi-virtual fucntion queue?
  Now, my setting is that all bit of PFUTA are '1' and ROPE bit is 1,
  so all mac packet is duplicated to all VF queue.
  I cannot understand the meaning of bits of PFUTA and the relation
  with mac address.
  I want to distribute the received packets to RX queues respectively,
  not duplicated.
 
 
  Thanks in advance.
 
  Best Regards,
  J.Hwan Kim
 
 
 Adding Greg Rose, since he is the ixgbevf driver maintainer.  He
 should be able to answer your questions early next week, unless he is
 checking his email over the weekend.
 
 --
 Cheers,
 Jeff
--
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free Love Thy Logs t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sdnews
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [ixgbevf]: patch to fix incorrect PF NACK assertion...

[Rose, Gregory V]

 -Original Message-
 From: Venugopal Busireddy [mailto:v...@nextio.com]
 Sent: Tuesday, September 20, 2011 9:37 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] [ixgbevf]: patch to fix incorrect PF NACK
 assertion...
 
 diff -r 194e296c8d2e ixgbevf_main.c
 --- a/ixgbevf_main.cThu Sep 15 10:46:23 2011 -0500
 +++ b/ixgbevf_main.cThu Sep 15 10:47:08 2011 -0500
 @@ -997,15 +997,18 @@
 if (!hw-mbx.ops.check_for_ack(hw, 0))
 got_ack = true;
 
 -   hw-mbx.ops.read(hw, msg, 1, 0);
 -
 -   if ((msg  IXGBE_MBVFICR_VFREQ_MASK) == IXGBE_PF_CONTROL_MSG)
 -   mod_timer(adapter-watchdog_timer,
 - round_jiffies(jiffies + 1));
 -
 -   if (msg  IXGBE_VT_MSGTYPE_NACK)
 -   DPRINTK(DRV, ERR, Last Request of type %2.2x to PF
 Nacked\n,
 -   msg  0xFF);
 +   if (!hw-mbx.ops.check_for_msg(hw, 0)) {
 +   hw-mbx.ops.read(hw, msg, 1, 0);
 +
 +   if ((msg  IXGBE_MBVFICR_VFREQ_MASK) ==
 IXGBE_PF_CONTROL_MSG)
 +   mod_timer(adapter-watchdog_timer,
 + round_jiffies(jiffies + 1));
 +
 +   if (msg  IXGBE_VT_MSGTYPE_NACK)
 +   DPRINTK(DRV, ERR,
 +   Last Request of type %2.2x to PF
 Nacked\n,
 +   msg  0xFF);
 +   }
 
 /*
  * checking for the ack clears the PFACK bit.  Place

Thanks for submitting the patch.  I'm curious what steps you were taking to 
cause a problem with this code.  In our own testing we haven't been seeing an 
issue with this.  If you could provide some information on the steps to 
reproduce the problem I'd appreciate it.

Thanks,

- Greg


--
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] igb/i350 VF/PF mailbox locking race leading to VF link staying down

[Rose, Gregory V]

James,

I'm sorry to hear you're having problems with this and I really appreciate the 
extensive and detailed investigation you've done on the matter.

I'll do some further investigation on my side and we will review and test your 
patch.  Given that no one here at Intel has seen the problem it might be that 
the best we can do is incorporate the patch and then make sure that it doesn't 
break anything else.  If that is the case and the patch fixes your issue then I 
can't see any reason for us to not accept the patch.

Thanks,

- Greg

 -Original Message-
 From: James Bulpin [mailto:james.bul...@eu.citrix.com]
 Sent: Wednesday, November 02, 2011 11:31 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] igb/i350 VF/PF mailbox locking race leading to VF
 link staying down
 
 Hello,
 
 Summary:
  1. An observation/theory of a race condition in VF/PF mailbox protocol
 leading to VF showing link down
  2. Belief that the race is exploited due to the VF not waiting on PF
 replies for some messages
  3. Patch for the above
 
 I've been trying to get i350 SR-IOV VFs working in my Xen environment
 (XenServer 6.0 [Xen 4.1], igb 3.2.10, HVM CentOS 5.x guest with igbvf
 1.1.5, Dell T3500). We've had a lot of experience and success with SR-IOV
 with both the 82599 and 82576 ET2; this work was to extend our support to
 include the i350. The problem I observed was that around 9 times out of 10
 the VF link was showing as down, both after the initial module load and
 after a ifconfig up. On the occasions the link did come up the VF worked
 perfectly. Adding a few printks showed the link was being declared down
 due to the VF/PF mailbox timeout having been hit (setting mbx-timeout to
 zero). Further instrumentation suggests that the timeout occurred when the
 VF was waiting for an ACK from the PF that never came, due to the original
 VF to PF message not being received by the PF. Based on limited logging
 the following sequence is what I believe happened:
 
  1. VF sends message (e.g. 0x00010003) to PF (get VFU lock; write buffer;
 release VFU lock/signal REQ), VF waits for ACK
  2. PF igb_msg_task handler checks the VFICR and finds REQ set
  3. PF igb_msg_task handler calls igb_rcv_msg_from_vf which reads message
 and ACKs (get PFU lock; read buffer; release PFU lock/signal ACK)
  4. PF deals with message, e.g. calling igb_set_vf_multicasts
  5. VF receives ACK
  6. VF moves on to send next message (e.g. 0x0006) to PF (get VFU
 lock; write buffer; release VFU lock/signal REQ), VF waits for ACK
  7. PF igb_rcv_msg_from_vf sends reply message (orig msg |
 E1000_VT_MSGTYPE_ACK|E1000_VT_MSGTYPE_CTS) from PF to VF (get PFU lock;
 clear REQ/ACK bits in VFICR ***clearing the REQ flag just set by VF***;
 write buffer ***overwriting VF's pending message***; release PFU
 lock/signal STS)
  8. PF igb_msg_task handler runs but REQ flag is zero so message not
 handled
  9. VF times out waiting for ACK to the second message
 
 From inspecting the code in both drivers my understanding is that the
 VFU/PFU locking mechanism is only being used to protect the message buffer
 while it is being read or written, it is not protecting against the buffer
 being re-used by either driver before the existing message has been
 handled (the lock is released when setting REQ/STS, not on receiving the
 ACK as the protocol description in the 82576 datasheet suggests). Adding a
 5000usec sleep after each message send from the VF makes the original
 link-down failure go away giving some confidence to the race condition
 theory.
 
 I believe that the race should not be a problem if the VF and PF are
 exchanging messages in the expected order however in the above case the VF
 sent a E1000_VF_SET_MULTICAST message but did not wait for the reply
 message from the PF. Reviewing the driver code shows that of the five
 messages the PF igb_rcv_msg_from_vf would reply to the VF driver does not
 wait for replies for three (E1000_VF_SET_MULTICAST, E1000_VF_SET_LPE and
 E1000_VF_SET_VLAN). Patching the VF driver (see below) to perform dummy
 reads after sending each of these three messages makes the original link-
 down failure go away.
 
 Have I misunderstood the locking strategy for the mailbox? As far as I can
 see nothing has changed in the newer igb and ibgvf drivers that would
 explain why I'm seeing the VF link failure on the i350 but not on the
 other NICs (I don't see this with the older 82576 in the same system with
 the same drivers). I can only assume it's just very bad luck with timing
 in this particular system and configuration.
 
 Regards,
 James Bulpin
 
 Read (and ignore) replies to VF to PF messages currently unhandled
 
 Signed-off-by: James Bulpin james.bul...@eu.citrix.com
 
 diff -rup igbvf-1.1.5.pristine/src/e1000_vf.c igbvf-
 1.1.5.mboxreply/src/e1000_vf.c
 --- igbvf-1.1.5.pristine/src/e1000_vf.c 2011-08-16 18:38:11.0
 +0100
 +++ igbvf-1.1.5.mboxreply/src/e1000_vf.c2011-11-02
 12:54:13.892369000 

Re: [E1000-devel] igb/i350 VF/PF mailbox locking race leading to VF link staying down

[Rose, Gregory V]

James,

Could you please try this patch?  It should be pretty much identical to yours 
but reuses code a bit more.

If it works for you then we'll go ahead and submit for internal validation and 
testing and then take it from there.

Thanks,

--- igbvf-old/igbvf-2.0.1/src/e1000_vf.c2011-11-03 10:00:52.0 
-0700
+++ igbvf-new/igbvf-2.0.1/src/e1000_vf.c2011-11-03 09:59:10.0 
-0700
@@ -252,6 +252,16 @@ static u32 e1000_hash_mc_addr_vf(struct 
return hash_value;
 }
 
+static void write_and_read_response(struct e1000_hw *hw, u32 *msg, u16 size)
+{
+   struct e1000_mbx_info *mbx = hw-mbx;
+   u32 retmsg[E1000_VFMAILBOX_SIZE];
+   s32 retval = mbx-ops.write_posted(hw, msg, size, 0);
+
+   if (!retval)
+   mbx-ops.read_posted(hw, retmsg, E1000_VFMAILBOX_SIZE, 0);
+}
+
 /**
  *  e1000_update_mc_addr_list_vf - Update Multicast addresses
  *  @hw: pointer to the HW structure
@@ -264,7 +274,6 @@ static u32 e1000_hash_mc_addr_vf(struct 
 void e1000_update_mc_addr_list_vf(struct e1000_hw *hw,
  u8 *mc_addr_list, u32 mc_addr_count)
 {
-   struct e1000_mbx_info *mbx = hw-mbx;
u32 msgbuf[E1000_VFMAILBOX_SIZE];
u16 *hash_list = (u16 *)msgbuf[1];
u32 hash_value;
@@ -298,7 +307,7 @@ void e1000_update_mc_addr_list_vf(struct
mc_addr_list += ETH_ADDR_LEN;
}
 
-   mbx-ops.write_posted(hw, msgbuf, E1000_VFMAILBOX_SIZE, 0);
+   write_and_read_response(hw, msgbuf, E1000_VFMAILBOX_SIZE);
 }
 
 /**
@@ -309,7 +318,6 @@ void e1000_update_mc_addr_list_vf(struct
  **/
 void e1000_vfta_set_vf(struct e1000_hw *hw, u16 vid, bool set)
 {
-   struct e1000_mbx_info *mbx = hw-mbx;
u32 msgbuf[2];
 
msgbuf[0] = E1000_VF_SET_VLAN;
@@ -318,7 +326,7 @@ void e1000_vfta_set_vf(struct e1000_hw *
if (set)
msgbuf[0] |= E1000_VF_SET_VLAN_ADD;
 
-   mbx-ops.write_posted(hw, msgbuf, 2, 0);
+   write_and_read_response(hw, msgbuf, 2);
 }
 
 /** e1000_rlpml_set_vf - Set the maximum receive packet length
@@ -327,13 +335,12 @@ void e1000_vfta_set_vf(struct e1000_hw *
  **/
 void e1000_rlpml_set_vf(struct e1000_hw *hw, u16 max_size)
 {
-   struct e1000_mbx_info *mbx = hw-mbx;
u32 msgbuf[2];
 
msgbuf[0] = E1000_VF_SET_LPE;
msgbuf[1] = max_size;
 
-   mbx-ops.write_posted(hw, msgbuf, 2, 0);
+   write_and_read_response(hw, msgbuf, 2);
 }
 
 /**





igbvf-mbx-patch.patch
Description: igbvf-mbx-patch.patch
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] igb/i350 VF/PF mailbox locking race leading to VF link staying down

[Rose, Gregory V]

Great to hear.  We'll spin a build for our internal validation team.  Barring 
any unforeseen issues it should be coming out in the next release of the igbvf 
driver to sourceforge.  We'll keep you updated if anything comes up.

Thanks!

- Greg

 -Original Message-
 From: James Bulpin [mailto:james.bul...@eu.citrix.com]
 Sent: Thursday, November 03, 2011 11:45 AM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: RE: [E1000-devel] igb/i350 VF/PF mailbox locking race leading to
 VF link staying down
 
 Hi Greg,
 
 That looks a lot more elegant than mine :) . I've tested on my rig and it
 works fine (applied to igbvf 1.1.5).
 
 Thanks,
 James
 
 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: 03 November 2011 17:21
 To: Rose, Gregory V; James Bulpin; e1000-devel@lists.sourceforge.net
 Subject: RE: [E1000-devel] igb/i350 VF/PF mailbox locking race leading to
 VF link staying down
 
 James,
 
 Could you please try this patch?  It should be pretty much identical to
 yours but reuses code a bit more.
 
 If it works for you then we'll go ahead and submit for internal validation
 and testing and then take it from there.
 
 Thanks,
 
 --- igbvf-old/igbvf-2.0.1/src/e1000_vf.c  2011-11-03 10:00:52.0 -
 0700
 +++ igbvf-new/igbvf-2.0.1/src/e1000_vf.c  2011-11-03 09:59:10.0 -
 0700
 @@ -252,6 +252,16 @@ static u32 e1000_hash_mc_addr_vf(struct
   return hash_value;
  }
 
 +static void write_and_read_response(struct e1000_hw *hw, u32 *msg, u16
 +size) {
 + struct e1000_mbx_info *mbx = hw-mbx;
 + u32 retmsg[E1000_VFMAILBOX_SIZE];
 + s32 retval = mbx-ops.write_posted(hw, msg, size, 0);
 +
 + if (!retval)
 + mbx-ops.read_posted(hw, retmsg, E1000_VFMAILBOX_SIZE, 0); }
 +
  /**
   *  e1000_update_mc_addr_list_vf - Update Multicast addresses
   *  @hw: pointer to the HW structure
 @@ -264,7 +274,6 @@ static u32 e1000_hash_mc_addr_vf(struct  void
 e1000_update_mc_addr_list_vf(struct e1000_hw *hw,
 u8 *mc_addr_list, u32 mc_addr_count)  {
 - struct e1000_mbx_info *mbx = hw-mbx;
   u32 msgbuf[E1000_VFMAILBOX_SIZE];
   u16 *hash_list = (u16 *)msgbuf[1];
   u32 hash_value;
 @@ -298,7 +307,7 @@ void e1000_update_mc_addr_list_vf(struct
   mc_addr_list += ETH_ADDR_LEN;
   }
 
 - mbx-ops.write_posted(hw, msgbuf, E1000_VFMAILBOX_SIZE, 0);
 + write_and_read_response(hw, msgbuf, E1000_VFMAILBOX_SIZE);
  }
 
  /**
 @@ -309,7 +318,6 @@ void e1000_update_mc_addr_list_vf(struct
   **/
  void e1000_vfta_set_vf(struct e1000_hw *hw, u16 vid, bool set)  {
 - struct e1000_mbx_info *mbx = hw-mbx;
   u32 msgbuf[2];
 
   msgbuf[0] = E1000_VF_SET_VLAN;
 @@ -318,7 +326,7 @@ void e1000_vfta_set_vf(struct e1000_hw *
   if (set)
   msgbuf[0] |= E1000_VF_SET_VLAN_ADD;
 
 - mbx-ops.write_posted(hw, msgbuf, 2, 0);
 + write_and_read_response(hw, msgbuf, 2);
  }
 
  /** e1000_rlpml_set_vf - Set the maximum receive packet length @@ -327,13
 +335,12 @@ void e1000_vfta_set_vf(struct e1000_hw *
   **/
  void e1000_rlpml_set_vf(struct e1000_hw *hw, u16 max_size)  {
 - struct e1000_mbx_info *mbx = hw-mbx;
   u32 msgbuf[2];
 
   msgbuf[0] = E1000_VF_SET_LPE;
   msgbuf[1] = max_size;
 
 - mbx-ops.write_posted(hw, msgbuf, 2, 0);
 + write_and_read_response(hw, msgbuf, 2);
  }
 
  /**
 
 


--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Question about ixgbevf driver

[Rose, Gregory V]

 -Original Message-
 From: Kirsher, Jeffrey T
 Sent: Wednesday, January 04, 2012 7:58 PM
 To: david.ye...@oracle.com; Rose, Gregory V
 Cc: Brandeburg, Jesse; Allan, Bruce W; Steve Sarvate; e1000-devel
 Subject: Re: Question about ixgbevf driver
 
 Adding e1000-devel mailing list as well as Greg Rose (ixgbevf
 maintainer)...

I've never noticed this before but then I can't say as how I was looking for it 
either.  Let me check it out and I'll get back to you.

- Greg

 
 On Wed, 2012-01-04 at 17:26 -0800, David Yeung wrote:
  Hi  Bruce/Jeffrey/Jesse/,
 
  How do you do?
  We are using the  ixgbevf  driver ( version: 2.4.0-NAPI ) to test the
  VLAN interfaces of Twinville NICs inside the OEL 5 virtual machine, it
  looks like the bi-directional network traffic ran properly on the VLAN
  interfaces of Twinville NICs  inside the OEL 5 virtual machine for
  hours, but the ifconfig command reports strange amount ( it is 0 all
  the time )  of  RX packets and RX bytes of VLAN interfaces of Twinville:
 
  [root@Twinville_VM_156 ~]# ifconfig
  .
  eth6.10   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
 inet addr:192.6.10.156  Bcast:192.6.10.255
 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111332155 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:0 (0.0 b)  TX bytes:2565278846027 (2.3 TiB)
 
  eth6.11   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
 inet addr:192.6.11.156  Bcast:192.6.11.255
 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111289098 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:0 (0.0 b)  TX bytes:2562582125939 (2.3 TiB)
 
  eth6.12   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
 inet addr:192.6.12.156  Bcast:192.6.12.255
 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111287930 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:0 (0.0 b)  TX bytes:2564949229588 (2.3 TiB)
 
  eth6.13   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
 inet addr:192.6.93.156  Bcast:192.6.93.255
 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111070858 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:0 (0.0 b)  TX bytes:2566358628283 (2.3 TiB)
 
  eth6.14   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
 inet addr:192.6.14.156  Bcast:192.6.14.255
 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:111362848 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:0 (0.0 b)  TX bytes:2566349976603 (2.3 TiB)
  .
 
  [root@Twinville_VM_156 ~]# ethtool -i eth6
  driver: ixgbevf
  version: 2.4.0-NAPI
  firmware-version: N/A
  bus-info: :00:09.0
  [root@Powerville_VM_156 ~]#
 
  [root@Twinville_VM_156 ~]# cat   /etc/*release
  Enterprise Linux Enterprise Linux Server release 5.5 (Carthage) Red
  Hat Enterprise Linux Server release 5.5 (Tikanga)
  [root@Twinville_VM_156 ~]#
 
  Do you have any idea about this problem? Any workaround/fix for this
  problem?
 
 
  Thanks,
 
  David
 
 

--
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual 
desktops for less than the cost of PCs and save 60% on VDI infrastructure 
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Question about ixgbevf driver

[Rose, Gregory V]

David,

Just to clarify something.

You're running Oracle Enterprise Linux 5 and the guest OS is Red Hat Enterprise 
Linux 5.5?

We're having trouble reproducing this so I want to make sure we're using the 
right setup.

Thanks,

- Greg

 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: Thursday, January 05, 2012 8:06 AM
 To: Kirsher, Jeffrey T; david.ye...@oracle.com
 Cc: e1000-devel; Allan, Bruce W; Brandeburg, Jesse; Steve Sarvate
 Subject: Re: [E1000-devel] Question about ixgbevf driver
 
  -Original Message-
  From: Kirsher, Jeffrey T
  Sent: Wednesday, January 04, 2012 7:58 PM
  To: david.ye...@oracle.com; Rose, Gregory V
  Cc: Brandeburg, Jesse; Allan, Bruce W; Steve Sarvate; e1000-devel
  Subject: Re: Question about ixgbevf driver
 
  Adding e1000-devel mailing list as well as Greg Rose (ixgbevf
  maintainer)...
 
 I've never noticed this before but then I can't say as how I was looking
 for it either.  Let me check it out and I'll get back to you.
 
 - Greg
 
 
  On Wed, 2012-01-04 at 17:26 -0800, David Yeung wrote:
   Hi  Bruce/Jeffrey/Jesse/,
  
   How do you do?
   We are using the  ixgbevf  driver ( version: 2.4.0-NAPI ) to test the
   VLAN interfaces of Twinville NICs inside the OEL 5 virtual machine, it
   looks like the bi-directional network traffic ran properly on the VLAN
   interfaces of Twinville NICs  inside the OEL 5 virtual machine for
   hours, but the ifconfig command reports strange amount ( it is 0 all
   the time )  of  RX packets and RX bytes of VLAN interfaces of
 Twinville:
  
   [root@Twinville_VM_156 ~]# ifconfig
   .
   eth6.10   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.10.156  Bcast:192.6.10.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111332155 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2565278846027 (2.3 TiB)
  
   eth6.11   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.11.156  Bcast:192.6.11.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111289098 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2562582125939 (2.3 TiB)
  
   eth6.12   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.12.156  Bcast:192.6.12.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111287930 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2564949229588 (2.3 TiB)
  
   eth6.13   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.93.156  Bcast:192.6.93.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111070858 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2566358628283 (2.3 TiB)
  
   eth6.14   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.14.156  Bcast:192.6.14.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111362848 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2566349976603 (2.3 TiB)
   .
  
   [root@Twinville_VM_156 ~]# ethtool -i eth6
   driver: ixgbevf
   version: 2.4.0-NAPI
   firmware-version: N/A
   bus-info: :00:09.0
   [root@Powerville_VM_156 ~]#
  
   [root@Twinville_VM_156 ~]# cat   /etc/*release
   Enterprise Linux Enterprise Linux Server release 5.5 (Carthage) Red
   Hat Enterprise Linux Server release 5.5 (Tikanga)
   [root@Twinville_VM_156 ~]#
  
   Do you have any idea about this problem? Any workaround/fix for this
   problem?
  
  
   Thanks,
  
   David
  
 
 
 --
 
 Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
 infrastructure or vast IT resources to deliver seamless, secure access to
 virtual desktops. With this all-in-one solution, easily deploy virtual
 desktops for less than the cost of PCs and save 60% on VDI infrastructure
 costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
 ___
 E1000-devel mailing list
 E1000

Re: [E1000-devel] Question about ixgbevf driver

[Rose, Gregory V]

OK, thanks for the clarification.  We'll see if we can pick those up on our 
internal server, but just in case is there an external site where we can get 
them?

- Greg

 -Original Message-
 From: David Yeung [mailto:david.ye...@oracle.com]
 Sent: Thursday, January 05, 2012 4:27 PM
 To: Rose, Gregory V
 Cc: Kirsher, Jeffrey T; e1000-devel; Allan, Bruce W; Brandeburg, Jesse;
 Steve Sarvate
 Subject: Re: Question about ixgbevf driver
 
 Greg,
 
 Thank you for your help!
 We are running Oracle VM server release 3.0.3 and the  guest OS is  (
 Oracle ) Enterprise Linux Enterprise Linux Server release 5.5 (Carthage).
 
 
 Thanks,
 
 David
 
 
 On 01/05/12 16:15, Rose, Gregory V wrote:
  David,
 
  Just to clarify something.
 
  You're running Oracle Enterprise Linux 5 and the guest OS is Red Hat
 Enterprise Linux 5.5?
 
  We're having trouble reproducing this so I want to make sure we're using
 the right setup.
 
  Thanks,
 
  - Greg
 
  -Original Message-
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
  Sent: Thursday, January 05, 2012 8:06 AM
  To: Kirsher, Jeffrey T; david.ye...@oracle.com
  Cc: e1000-devel; Allan, Bruce W; Brandeburg, Jesse; Steve Sarvate
  Subject: Re: [E1000-devel] Question about ixgbevf driver
 
  -Original Message-
  From: Kirsher, Jeffrey T
  Sent: Wednesday, January 04, 2012 7:58 PM
  To: david.ye...@oracle.com; Rose, Gregory V
  Cc: Brandeburg, Jesse; Allan, Bruce W; Steve Sarvate; e1000-devel
  Subject: Re: Question about ixgbevf driver
 
  Adding e1000-devel mailing list as well as Greg Rose (ixgbevf
  maintainer)...
 
  I've never noticed this before but then I can't say as how I was
 looking
  for it either.  Let me check it out and I'll get back to you.
 
  - Greg
 
 
  On Wed, 2012-01-04 at 17:26 -0800, David Yeung wrote:
  Hi  Bruce/Jeffrey/Jesse/,
 
  How do you do?
  We are using the  ixgbevf  driver ( version: 2.4.0-NAPI ) to test the
  VLAN interfaces of Twinville NICs inside the OEL 5 virtual machine,
 it
  looks like the bi-directional network traffic ran properly on the
 VLAN
  interfaces of Twinville NICs  inside the OEL 5 virtual machine for
  hours, but the ifconfig command reports strange amount ( it is 0 all
  the time )  of  RX packets and RX bytes of VLAN interfaces of
  Twinville:
 
  [root@Twinville_VM_156 ~]# ifconfig
  .
  eth6.10   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.10.156  Bcast:192.6.10.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111332155 errors:0 dropped:0 overruns:0
  carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2565278846027 (2.3 TiB)
 
  eth6.11   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.11.156  Bcast:192.6.11.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111289098 errors:0 dropped:0 overruns:0
  carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2562582125939 (2.3 TiB)
 
  eth6.12   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.12.156  Bcast:192.6.12.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111287930 errors:0 dropped:0 overruns:0
  carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2564949229588 (2.3 TiB)
 
  eth6.13   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.93.156  Bcast:192.6.93.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111070858 errors:0 dropped:0 overruns:0
  carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2566358628283 (2.3 TiB)
 
  eth6.14   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.14.156  Bcast:192.6.14.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111362848 errors:0 dropped:0 overruns:0
  carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2566349976603 (2.3 TiB)
  .
 
  [root@Twinville_VM_156 ~]# ethtool -i eth6
  driver: ixgbevf
  version: 2.4.0-NAPI
  firmware-version: N/A
  bus-info: :00:09.0
  [root@Powerville_VM_156 ~]#
 
  [root@Twinville_VM_156 ~]# cat   /etc/*release
  Enterprise Linux Enterprise Linux Server release 5.5 (Carthage) Red
  Hat Enterprise Linux

Re: [E1000-devel] Question about ixgbevf driver

[Rose, Gregory V]

Thank you!

 -Original Message-
 From: David Yeung [mailto:david.ye...@oracle.com]
 Sent: Thursday, January 05, 2012 4:38 PM
 To: Rose, Gregory V
 Cc: Kirsher, Jeffrey T; e1000-devel; Allan, Bruce W; Brandeburg, Jesse;
 Steve Sarvate
 Subject: Re: Question about ixgbevf driver
 
 Greg,
 
 The external site is:
 http://edelivery.oracle.com/linux
 
 
 Thanks,
 
 David
 
 On 01/05/12 16:30, Rose, Gregory V wrote:
  OK, thanks for the clarification.  We'll see if we can pick those up on
 our internal server, but just in case is there an external site where we
 can get them?
 
  - Greg
 
  -Original Message-
  From: David Yeung [mailto:david.ye...@oracle.com]
  Sent: Thursday, January 05, 2012 4:27 PM
  To: Rose, Gregory V
  Cc: Kirsher, Jeffrey T; e1000-devel; Allan, Bruce W; Brandeburg, Jesse;
  Steve Sarvate
  Subject: Re: Question about ixgbevf driver
 
  Greg,
 
  Thank you for your help!
  We are running Oracle VM server release 3.0.3 and the  guest OS is  (
  Oracle ) Enterprise Linux Enterprise Linux Server release 5.5
 (Carthage).
 
 
  Thanks,
 
  David
 
 
  On 01/05/12 16:15, Rose, Gregory V wrote:
  David,
 
  Just to clarify something.
 
  You're running Oracle Enterprise Linux 5 and the guest OS is Red Hat
  Enterprise Linux 5.5?
 
  We're having trouble reproducing this so I want to make sure we're
 using
  the right setup.
 
  Thanks,
 
  - Greg
 
  -Original Message-
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
  Sent: Thursday, January 05, 2012 8:06 AM
  To: Kirsher, Jeffrey T; david.ye...@oracle.com
  Cc: e1000-devel; Allan, Bruce W; Brandeburg, Jesse; Steve Sarvate
  Subject: Re: [E1000-devel] Question about ixgbevf driver
 
  -Original Message-
  From: Kirsher, Jeffrey T
  Sent: Wednesday, January 04, 2012 7:58 PM
  To: david.ye...@oracle.com; Rose, Gregory V
  Cc: Brandeburg, Jesse; Allan, Bruce W; Steve Sarvate; e1000-devel
  Subject: Re: Question about ixgbevf driver
 
  Adding e1000-devel mailing list as well as Greg Rose (ixgbevf
  maintainer)...
 
  I've never noticed this before but then I can't say as how I was
  looking
  for it either.  Let me check it out and I'll get back to you.
 
  - Greg
 
 
  On Wed, 2012-01-04 at 17:26 -0800, David Yeung wrote:
  Hi  Bruce/Jeffrey/Jesse/,
 
  How do you do?
  We are using the  ixgbevf  driver ( version: 2.4.0-NAPI ) to test
 the
  VLAN interfaces of Twinville NICs inside the OEL 5 virtual machine,
  it
  looks like the bi-directional network traffic ran properly on the
  VLAN
  interfaces of Twinville NICs  inside the OEL 5 virtual machine for
  hours, but the ifconfig command reports strange amount ( it is 0
 all
  the time )  of  RX packets and RX bytes of VLAN interfaces of
  Twinville:
 
  [root@Twinville_VM_156 ~]# ifconfig
  .
  eth6.10   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
   inet addr:192.6.10.156  Bcast:192.6.10.255
  Mask:255.255.255.0
   UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
   TX packets:111332155 errors:0 dropped:0 overruns:0
  carrier:0
   collisions:0 txqueuelen:0
   RX bytes:0 (0.0 b)  TX bytes:2565278846027 (2.3 TiB)
 
  eth6.11   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
   inet addr:192.6.11.156  Bcast:192.6.11.255
  Mask:255.255.255.0
   UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
   TX packets:111289098 errors:0 dropped:0 overruns:0
  carrier:0
   collisions:0 txqueuelen:0
   RX bytes:0 (0.0 b)  TX bytes:2562582125939 (2.3 TiB)
 
  eth6.12   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
   inet addr:192.6.12.156  Bcast:192.6.12.255
  Mask:255.255.255.0
   UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
   TX packets:111287930 errors:0 dropped:0 overruns:0
  carrier:0
   collisions:0 txqueuelen:0
   RX bytes:0 (0.0 b)  TX bytes:2564949229588 (2.3 TiB)
 
  eth6.13   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
   inet addr:192.6.93.156  Bcast:192.6.93.255
  Mask:255.255.255.0
   UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
   TX packets:111070858 errors:0 dropped:0 overruns:0
  carrier:0
   collisions:0 txqueuelen:0
   RX bytes:0 (0.0 b)  TX bytes:2566358628283 (2.3 TiB)
 
  eth6.14   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
   inet addr:192.6.14.156  Bcast:192.6.14.255
  Mask:255.255.255.0
   UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
   TX packets:111362848 errors:0 dropped:0 overruns

Re: [E1000-devel] Question about ixgbevf driver

[Rose, Gregory V]

David,

We've confirmed the bug and are looking into it.  We'll keep you updated on 
what we find.

Thanks,

- Greg

 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: Thursday, January 05, 2012 8:06 AM
 To: Kirsher, Jeffrey T; david.ye...@oracle.com
 Cc: e1000-devel; Allan, Bruce W; Brandeburg, Jesse; Steve Sarvate
 Subject: Re: [E1000-devel] Question about ixgbevf driver
 
  -Original Message-
  From: Kirsher, Jeffrey T
  Sent: Wednesday, January 04, 2012 7:58 PM
  To: david.ye...@oracle.com; Rose, Gregory V
  Cc: Brandeburg, Jesse; Allan, Bruce W; Steve Sarvate; e1000-devel
  Subject: Re: Question about ixgbevf driver
 
  Adding e1000-devel mailing list as well as Greg Rose (ixgbevf
  maintainer)...
 
 I've never noticed this before but then I can't say as how I was looking
 for it either.  Let me check it out and I'll get back to you.
 
 - Greg
 
 
  On Wed, 2012-01-04 at 17:26 -0800, David Yeung wrote:
   Hi  Bruce/Jeffrey/Jesse/,
  
   How do you do?
   We are using the  ixgbevf  driver ( version: 2.4.0-NAPI ) to test the
   VLAN interfaces of Twinville NICs inside the OEL 5 virtual machine, it
   looks like the bi-directional network traffic ran properly on the VLAN
   interfaces of Twinville NICs  inside the OEL 5 virtual machine for
   hours, but the ifconfig command reports strange amount ( it is 0 all
   the time )  of  RX packets and RX bytes of VLAN interfaces of
 Twinville:
  
   [root@Twinville_VM_156 ~]# ifconfig
   .
   eth6.10   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.10.156  Bcast:192.6.10.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111332155 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2565278846027 (2.3 TiB)
  
   eth6.11   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.11.156  Bcast:192.6.11.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111289098 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2562582125939 (2.3 TiB)
  
   eth6.12   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.12.156  Bcast:192.6.12.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111287930 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2564949229588 (2.3 TiB)
  
   eth6.13   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.93.156  Bcast:192.6.93.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111070858 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2566358628283 (2.3 TiB)
  
   eth6.14   Link encap:Ethernet  HWaddr F2:54:11:05:72:7C
  inet addr:192.6.14.156  Bcast:192.6.14.255
  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:9210  Metric:1
  RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  TX packets:111362848 errors:0 dropped:0 overruns:0
 carrier:0
  collisions:0 txqueuelen:0
  RX bytes:0 (0.0 b)  TX bytes:2566349976603 (2.3 TiB)
   .
  
   [root@Twinville_VM_156 ~]# ethtool -i eth6
   driver: ixgbevf
   version: 2.4.0-NAPI
   firmware-version: N/A
   bus-info: :00:09.0
   [root@Powerville_VM_156 ~]#
  
   [root@Twinville_VM_156 ~]# cat   /etc/*release
   Enterprise Linux Enterprise Linux Server release 5.5 (Carthage) Red
   Hat Enterprise Linux Server release 5.5 (Tikanga)
   [root@Twinville_VM_156 ~]#
  
   Do you have any idea about this problem? Any workaround/fix for this
   problem?
  
  
   Thanks,
  
   David
  
 
 
 --
 
 Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
 infrastructure or vast IT resources to deliver seamless, secure access to
 virtual desktops. With this all-in-one solution, easily deploy virtual
 desktops for less than the cost of PCs and save 60% on VDI infrastructure
 costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174

Re: [E1000-devel] Question about ixgbevf driver

[Rose, Gregory V]

David,

The results being reported are expected.

   Supports auto-negotiation: No   ( We think this should be Yes )

Actually no...  The virtual function (VF) device does not do any 
auto-negotiation.  It requires the physical function (PF) device to do that on 
its behalf.

   Advertised auto-negotiation: No ( We think this should be Yes )

Again, no.  The VF does no auto-negotiation.  It depends upon the PF device for 
that.

   Port: Unknown! (255)  ( We think this should be Twisted Pair )

The VF has no knowledge of or need of such knowledge of the Phy port type.  
Therefore it is in fact unknown.

   Transceiver: Unknown!  ( We think this should be external )

Same as previous - VF has no knowledge of it and shouldn't be concerned.

   Auto-negotiation: off  ( We think this should be on )

And the same here.  The VF does not initiate, advertise or engage in 
auto-negotiation.  It can only report the link speed set by the PF device and 
whether the link is up and it does that.

The 82599 supports up to 63 VF devices.  If each VF was able to control and 
initiate auto-negotiation parameters it would be a real mess to manage.  Our 
controller doesn't allow that and doesn't even allow the VF to even see the 
information for security reasons.

- Greg

 -Original Message-
 From: David Yeung [mailto:david.ye...@oracle.com]
 Sent: Monday, January 09, 2012 11:45 AM
 To: Rose, Gregory V
 Cc: Kirsher, Jeffrey T; e1000-devel; Allan, Bruce W; Brandeburg, Jesse;
 Steve Sarvate
 Subject: Re: Question about ixgbevf driver
 
 Greg,
 
 Thank you for your help!
 In the OEL 5.5 guest domain, its ethtool reports strange results of
 virtual interfaces of Twinville:
   Supports auto-negotiation: No   ( We think this should be Yes )
   Advertised auto-negotiation: No ( We think this should be Yes )
   Port: Unknown! (255)  ( We think this should be Twisted Pair )
   Transceiver: Unknown!  ( We think this should be external )
   Auto-negotiation: off  ( We think this should be on )
 
 Do you see this problem in your guest OS OEL 5.6 and 5.5?
 
 For details:
 ==
 eth5 and eth6 are virtual interfaces of Twinville
 
 [root@Twinville_VM_156 ~]# ethtool eth5
 Settings for eth5:
  Supported ports: [ ]
  Supported link modes:   1baseT/Full
  Supports auto-negotiation: No
  Advertised link modes:  Not reported
  Advertised auto-negotiation: No
  Speed: 1Mb/s
  Duplex: Full
  Port: Unknown! (255)
  PHYAD: 0
  Transceiver: Unknown!
  Auto-negotiation: off
  Current message level: 0x0007 (7)
  Link detected: yes
 [root@Twinville_VM_156 ~]#
 [root@Twinville_VM_156 ~]# ethtool eth6
 Settings for eth6:
  Supported ports: [ ]
  Supported link modes:   1baseT/Full
  Supports auto-negotiation: No
  Advertised link modes:  Not reported
  Advertised auto-negotiation: No
  Speed: 1Mb/s
  Duplex: Full
  Port: Unknown! (255)
  PHYAD: 0
  Transceiver: Unknown!
  Auto-negotiation: off
  Current message level: 0x0007 (7)
  Link detected: yes
 [root@Twinville_VM_162 ~]# ethtool -i eth5
 driver: ixgbevf
 version: 2.4.0-NAPI
 firmware-version: N/A
 bus-info: :00:08.0
 [root@Twinville_VM_162 ~]#
 [root@Twinville_VM_162 ~]# ethtool -i eth6
 driver: ixgbevf
 version: 2.4.0-NAPI
 firmware-version: N/A
 bus-info: :00:09.0
 [root@Twinville_VM_162 ~]#
 [root@Twinville_VM_156 ~]# more  /etc/*release
 ::
 /etc/enterprise-release
 ::
 Enterprise Linux Enterprise Linux Server release 5.5 (Carthage)
 ::
 /etc/redhat-release
 ::
 Red Hat Enterprise Linux Server release 5.5 (Tikanga)
 [root@Twinville_VM_156 ~]#
 
 
 Thanks,
 
 David
 
 On 01/09/12 09:52, Rose, Gregory V wrote:
  David,
 
  We've found that if we use RHEL 5.5 as the guest then the bug still
 occurs but if we upgrade the guest OS to RHEL 5.6 then it does not occur.
 So it does not appear to be a driver bug.  It looks like some issue with
 RHEL 5.5 and OEL 5.5.
 
  We suggest upgrading to 5.6.
 
  Thanks,
 
  - Greg
 
  -Original Message-
  From: Rose, Gregory V
  Sent: Friday, January 06, 2012 4:32 PM
  To: Rose, Gregory V; Kirsher, Jeffrey T; david.ye...@oracle.com
  Cc: e1000-devel; Allan, Bruce W; Brandeburg, Jesse; Steve Sarvate
  Subject: RE: Question about ixgbevf driver
 
  David,
 
  We've confirmed the bug and are looking into it.  We'll keep you
 updated
  on what we find.
 
  Thanks,
 
  - Greg
 
  -Original Message-
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
  Sent: Thursday, January 05, 2012 8:06 AM
  To: Kirsher, Jeffrey T; david.ye...@oracle.com
  Cc: e1000-devel; Allan, Bruce W; Brandeburg, Jesse; Steve Sarvate
  Subject: Re: [E1000-devel] Question about ixgbevf driver
 
  -Original Message

Re: [E1000-devel] VFS vs MTU

[Rose, Gregory V]

 -Original Message-
 From: Steve O'Brien [mailto:obr...@velocent.com]
 Sent: Monday, January 30, 2012 10:05 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] VFS vs MTU
 
 
 Has anyone else encountered this problem?
 
 Sorry, my contact info is:
   Steve O'Brien
   obr...@velocent.com
   Sr. Engineer
   Velocent Systems

If the Intel 10Gig device you're is based on the 82599 controller then there is 
an errata on that device that prevents use of jumbo frames while in SR-IOV 
mode.  You must disable SR-IOV if you want to use jumbo frames.

The driver will not allow any parameter larger than 1500 for those devices when 
SR-IOV mode is enabled and that is why you are getting the invalid parameter 
error message.

The errata will be fixed in future generations of Intel 10gig controllers.  I 
apologize for the inconvenience.

- Greg

 
 -Original Message-
 From: Steve O'Brien [mailto:obr...@velocent.com]
 Sent: Thursday, January 26, 2012 1:00 PM
 To: 'e1000-devel@lists.sourceforge.net'
 Subject: VFS vs MTU
 
 
 Oh, I hope this isn't a dumb question, but I can't set the MTU to 4096
 using SR-IOV.
 
 I'm using the X520-SR2 dual 82599 NIC with RedHat 6.0. I use 'modprobe
 ixgbe max_vfs=4,4`, then try to increase the MTU using 'ifconfig eth2 mtu
 4096', gives me the response `SIOCSIFMTU: Invalid argument`.
 
 The modprobe appears successful, dmesg is:
 
 Intel(R) 10 Gigabit PCI Express Network Driver - version 3.7.17-NAPI
 Copyright (c) 1999-2011 Intel Corporation.
 ixgbe :13:00.0: PCI INT A - GSI 18 (level, low) - IRQ 18 ixgbe
 :13:00.0: setting latency timer to 64
 ixgbe: I/O Virtualization (IOV) set to 4
 ixgbe: :13:00.0: ixgbe_check_options: FCoE Offload feature disabled
 ixgbe :13:00.0: irq 28 for MSI/MSI-X ixgbe :13:00.0: irq 29 for
 MSI/MSI-X ixgbe :13:00.0: (PCI Express:2.5GT/s:Width x8)
 00:1b:21:8e:ae:e4 ixgbe :13:00.0: eth2: MAC: 2, PHY: 15, SFP+: 5, PBA
 No: E68785-003 ixgbe :13:00.0: eth2: Enabled Features: RxQ: 1 TxQ: 1
 LRO ixgbe :13:00.0: eth2: IOV: VF 0 is enabled mac 36:25:DD:F1:C6:DC
 ixgbe :13:00.0: eth2: IOV: VF 1 is enabled mac 3A:98:30:1A:89:9D ixgbe
 :13:00.0: eth2: IOV: VF 2 is enabled mac 42:7F:C3:60:78:60 ixgbe
 :13:00.0: eth2: IOV: VF 3 is enabled mac 46:70:37:00:FC:8C ixgbe
 :13:00.0: eth2: Intel(R) 10 Gigabit Network Connection ixgbe
 :13:00.1: PCI INT B - GSI 19 (level, low) - IRQ 19 ixgbe
 :13:00.1: setting latency timer to 64
 ixgbe: I/O Virtualization (IOV) set to 4
 ixgbe: :13:00.1: ixgbe_check_options: FCoE Offload feature disabled
 ADDRCONF(NETDEV_UP): eth2: link is not ready ixgbe :13:00.0: eth2:
 detected SFP+: 5 ixgbe :13:00.1: irq 30 for MSI/MSI-X ixgbe
 :13:00.1: irq 31 for MSI/MSI-X ixgbe :13:00.1: (PCI
 Express:2.5GT/s:Width x8) 00:1b:21:8e:ae:e5 ixgbe :13:00.1: eth3: MAC:
 2, PHY: 15, SFP+: 6, PBA No: E68785-003 ixgbe :13:00.1: eth3: Enabled
 Features: RxQ: 1 TxQ: 1 LRO ixgbe :13:00.1: eth3: IOV: VF 0 is enabled
 mac 8A:75:79:3D:91:45 ixgbe :13:00.1: eth3: IOV: VF 1 is enabled mac
 EA:DB:6E:21:CE:D2 ixgbe :13:00.1: eth3: IOV: VF 2 is enabled mac
 EE:22:8A:91:7A:AC ixgbe :13:00.1: eth3: IOV: VF 3 is enabled mac
 A2:92:0B:65:4A:24 ixgbe :13:00.1: eth3: Intel(R) 10 Gigabit Network
 Connection
 ADDRCONF(NETDEV_UP): eth3: link is not ready ixgbe :13:00.0: eth2: NIC
 Link is Up 10 Gbps, Flow Control: RX/TX
 ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready ixgbe :13:00.1:
 eth3: detected SFP+: 6 ixgbe :13:00.1: eth3: NIC Link is Up 10 Gbps,
 Flow Control: RX/TX
 ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
 
 
 
 --
 
 Try before you buy = See our experts in action!
 The most comprehensive online learning library for Microsoft developers
 is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
 Metro Style Apps, more. Free future releases when you subscribe now!
 http://p.sf.net/sfu/learndevnow-dev2
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174; Ethernet, visit
 http://communities.intel.com/community/wired

--
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] igb hard-coded mac_table[0] in igb_probe

[Rose, Gregory V]

 -Original Message-
 From: James Bulpin [mailto:james.bul...@eu.citrix.com]
 Sent: Wednesday, February 29, 2012 5:34 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] igb hard-coded mac_table[0] in igb_probe
 
 We've noticed that the igb_probe() function hard-codes mac_table index 0
 when setting the PF MAC entry. Unfortunately this happens after
 igb_set_sriov_capability() has run and used mac_table[0] for the VF0
 initial MAC and therefore the PF MAC silently obliterates the VF0 config.
 In practice this isn't a problem because the VF0 MAC will get re-added to
 the table later on igb_vf_reset_msg() etc.
 
 The only reason this caused a problem for us was because we were using the
 number of free mac_table entries as a guide to how many additional MAC
 filters we could define - to work around this we simply have to allow for
 a spare entry for VF0 when it eventually gets used.
 

This is to support backwards compatibility with legacy drivers that don't use 
or support SR-IOV.  The igb driver is used on several devices that don't have 
SR-IOV features.

However I can see how it's a bit confusing.  In our 10Gig drivers we have ended 
up allocating the VF MAC addresses starting from the top of the MAC address 
filter table.  It would be a good idea to do this for the igb driver also to 
avoid this sort of confusion and to make behavior consistent among our drivers. 
 I'm not sure when the resources for this work would be available though.

- Greg


--
Virtualization  Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing 
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] IO_PAGE_FAULTS with igb or igbvf on AMD IOMMU system

[Rose, Gregory V]

 -Original Message-
 From: Joerg Roedel [mailto:joerg.roe...@amd.com]
 Sent: Wednesday, June 20, 2012 2:49 AM
 To: Duyck, Alexander H
 Cc: Kirsher, Jeffrey T; Brandeburg, Jesse; Allan, Bruce W; Wyborny,
 Carolyn; Skidmore, Donald C; Rose, Gregory V; Waskiewicz Jr, Peter P;
 Ronciak, John; e1000-devel@lists.sourceforge.net; linux-
 ker...@vger.kernel.org
 Subject: Re: IO_PAGE_FAULTS with igb or igbvf on AMD IOMMU system
 
 Hi Alexander,
 
 On Tue, Jun 19, 2012 at 11:19:20AM -0700, Alexander Duyck wrote:
  Based on the faults it would look like accessing the descriptor rings
  is probably triggering the errors.  We allocate the descriptor rings
  using dma_alloc_coherent so the rings should be mapped correctly.
 
 Can this happen before the driver actually allocated the descriptors? As I
 said, the faults appear before any DMA-API call was made for that device
 (hence, domain=0x, because the domain is assigned on the first call to
 the DMA-API for a device).
 
 Also, I don't see the faults every time. One out of ten times
 (estimated) there are not faults. Is it possible that this is a race
 condition, e.g. that the card trys to access its descriptor rings before
 the driver allocated them (or something like that).
 
  The PF and VF will end up being locked out since they are hung on an
  uncompleted DMA transaction.  Normally we recommend that PCIe Advanced
  Error Reporting be enabled if an IOMMU is enabled so the device can be
  reset after triggering a page fault event.
 
  The first thing that pops into my head for possible issues would be
  that maybe the VF pci_dev structure or the device structure isn't
  being correctly initialized when SR-IOV is enabled on the igb
  interface.  Do you know if there are any AMD IOMMU specific values on
  those structures, such as the domain, that are supposed to be
  initialized prior to calling the DMA API calls?  If so, have you tried
  adding debug output to verify if those values are initialized on a VF
 prior to bringing up a VF interface?
 
 Well, when the device appears in the system the IOMMU driver gets notified
 about it using the device_change notifiers. It will then allocate all
 necessary data structures. I also verified that this works correctly while
 debugging this issue. So I am pretty sure the problem isn't there :)
 
  Also have you tried any other SR-IOV capable devices on this system?
  That would be a valuable data point because we could then exclude the
  SR-IOV code as being a possible cause for the issues if other SR-IOV
  devices are working without any issues.
 
 I have another SR-IOV device, but that fails to even enable SR-IOV because
 the BIOS did not let enough MMIO resources left. So I couldn't try it with
 that device. With the 82576 card enabling SR-IOV works fine but results in
 the faults from the VF.

That sounds very suspicious to me.  The 82576 might still seem to work because 
it only has less than 8 VFs, which might be why it isn't reporting the MMIO 
resources issue.  That doesn't mean it would work correctly and I suspect that 
the IO_PAGE_FAULT error is due to an MMIO access, not a DMA access.  MMIO 
resources for devices are page mapped and if your BIOS is broken that might not 
be done correctly.

I have the feeling the issue is the BIOS.  You probably want to contact your 
system vendor and make sure you have the correct BIOS installed or even whether 
they claim that the system is supposed to support SR-IOV.

- Greg

 
 Regards,
 
   Joerg
 
 --
 AMD Operating System Research Center
 
 Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach General
 Managers: Alberto Bozzo
 Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr.
 43632


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] vlan tagging problem with i350 NICs and igb-3.4.8

[Rose, Gregory V]

 -Original Message-
 From: Andy Cress [mailto:andy.cr...@us.kontron.com]
 Sent: Monday, August 06, 2012 7:04 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] vlan tagging problem with i350 NICs and igb-3.4.8
 
 Baseboard: S2600CO with i350 onboard NICs
 
 OS: RHEL6.2
 
 
 
 Current issue = Enabled VLAN tagging, then i350 NICs cannot ping each
 other
 
 This problem occurs in sf igb-3.4.8 but worked with RHEL6 igb-3.0.6-k.
 
 
 
 The network has the same configuration as depicted below, only that Vlan-
 tagging is enabled on the switch for the ports  to which eth0 and
 eth2 cables are connected.
 
 Both the host server and the guest server have IPs for bond0.926 and
 bond0.928 in the ifconfig output, and both of the server can be accessed
 via these IPs.  However, host and guest server cannot ping each other
 regardless of which eth is the active slave on the host or guest.
 


Andy,

I apologize for taking so long to respond to this.  I'm curious if you're 
seeing any reports of spoofed packets in your system log.  I ask because I see 
you're using bonding and there are some bonding configurations that will cause 
the spoofed packets detection to trigger inappropriately.  Can you reproduce 
the problem and then send me the system log?

Thanks,

- Greg Rose
LAD
Intel Corp.

 
 
 This occurs with i350 but works with 82576 NICs.
 
 
 
 *
 
 
 
 The configuration uses Intel SR-IOV technology and VF/PFs with i350 NICs,
 
 as follows:
 
 
 
 eth0:   [VM0]
 
   VF0   vEth0 (vBond0)
 
   VF1  +  + vEth1 (vBond0)
 
|  |
 
   PF_x (bond0) |  |
 
|  |
 
 eth2:  |  |
 
   VFa  (bond0)+
 
|[VM1]
 
+--- vEth_a (vBond_a)
 
   VFb  (bond0)- vEth_b (vBond_a)
 
   PF_y (bond0)
 
 
 
 *
 
 
 
 Original issue = VF eth port confusion
 
 This was broken in RHEL6 igb-3.0.6-k, but resolved in sf igb-3.4.8.
 
 
 
 We are using active-backup bonding on both host server and VMs.
 Normally, when eth0 is the active port, VFs assigned to that port are also
 active. And when we unplug the cable from eth0, eth2 and VFs linked to it
 become active.  This was working fine on other servers (non-i350 NICs).
 
 
 
 However, on new servers with i350 NICs, the working scenario is the
 opposite. If eth0 is the active port, VFs linked to eth2 must be active on
 VM. Otherwise, internal communication (from VM to host or VM to VM) fails.
 
 
 
 EL6 igb 3.0.6-k source:
 http://vault.centos.org/6.2/os/Source/SPackages/igb-3.0.6_k-2.el6_1.src.
 rpm
 
 
 
 


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH 5/5] ixgbe: add driver set_max_vfs support

[Rose, Gregory V]

 -Original Message-
 From: Don Dutile [mailto:ddut...@redhat.com]
 Sent: Wednesday, October 03, 2012 12:03 PM
 To: Duyck, Alexander H
 Cc: Yinghai Lu; Bjorn Helgaas; Greg Kroah-Hartman; linux-
 p...@vger.kernel.org; linux-ker...@vger.kernel.org; yuval...@broadcom.com;
 bhutchi...@solarflare.com; Rose, Gregory V; da...@davemloft.net--no-chain-
 reply-to; Kirsher, Jeffrey T; Brandeburg, Jesse; David S. Miller;
 Fastabend, John R; e1000-devel@lists.sourceforge.net;
 net...@vger.kernel.org
 Subject: Re: [PATCH 5/5] ixgbe: add driver set_max_vfs support
 
 On 10/03/2012 02:47 PM, Alexander Duyck wrote:

[snip]

 
  The ixgbe_set_max_vfs function has several issues.  The two big ones
  are that this function assumes it can just enable/disable SR-IOV
  without any other changes being necessary which is not the case.  I
  would recommend looking at ixgbe_setup_tc for how to do this properly.
  Secondly is the fact that this code will change the PF network device
  and as such sections of the code should be called with the RTNL lock
  held.  In addition I believe you have to disable SR-IOV before
  enabling it again with a different number of VFs.
 
  Below is a link to one of the early patches for igb when we were first
  introducing SR-IOV, and the in-driver sysfs value had been rejected.
  I figure it might be useful as it was also using sysfs to
  enable/disable VFs.  It however doesn't have the correct locking on
  changing the queues and as such will likely throw an error if you were
  to implement it the same way now:
  http://lists.openwall.net/netdev/2009/04/08/34
 
  Thanks,
 
  Alex
 
 Alex,
 Thanks for patch set pointer.
 When I started to work on the ixgbe example use based on the RFC set I
 posted, I ran into the problem you outlined -- the PF uses/consumes all
 the queues  MSI intrs when sriov not enabled at driver load time, which
 required more network shutdown logic that I'm not familiar with... So, I
 was going to defer to Greg to work that magic. :)
 Greg: assume the 2 callback function interface in the RFC patch set I
 sent,
(primarily, just the include/linux/pci.h changes), and you can make
 the
necessary drivers mods from there.  In the meantime, I'll make the
 changes
to my original/v1 RFC to reflect the changes that GKH  Yinghai
 recommended/implemented
for sysfs attribute creation  removal in a v2 posting.
The end result is that the current module parameter setting for
 max_vfs should
continue to work, and the sysfs interface will work when those
 pieces are provided.

OK, I'll start work on it.

Thanks Don,

- Greg

 
 -Don

--
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] SR-IOV problem with Intel 82599EB (not enough MMIO resources for SR-IOV)

[Rose, Gregory V]

 -Original Message-
 From: Jeff Kirsher [mailto:tar...@gmail.com]
 Sent: Thursday, November 08, 2012 3:23 AM
 To: pkill.2012
 Cc: linux-kernel; netdev; kvm; e1000-devel@lists.sourceforge.net; Rose,
 Gregory V
 Subject: Re: SR-IOV problem with Intel 82599EB (not enough MMIO resources
 for SR-IOV)
 
 On 11/08/2012 03:15 AM, pkill.2012 wrote:
  Hello,
 
  I installed kvm and tried to use SR-IOV virtualizaton for
  82599EB(Intel XT-520 T2) dual port card with latest ixgbe
  driver(version:3.11.33) ,
  kernel2.6.32-279.14.1(OS:Centos6.3) ,after configuration and reboot It
  seems that only first port of the card's VFs works,second port of the
  card's VFs didn't work I found some errors in /var/log messages:
 
  Nov  8 14:56:54 12 kernel: ixgbe :07:00.1: not enough MMIO
  resources for SR-IOV Nov  8 14:56:54 12 kernel: ixgbe :07:00.1:
  (unregistered net_device): Failed to enable PCI sriov: -12
 
  How to fix it,any help would be greatly appreciated.

The BIOS in your machine doesn't support SR-IOV.  You'll need to ask the 
manufacturer for a BIOS upgrade, if in fact one is available.  Sometimes 
they're not.

- Greg

 
  below is the related information:
  Server: R710
  OS: centos6.3
  NIC: X520-T2(dual port)
  kernel: 2.6.32-279.14.1.el6.x86_64
  BIOSVersion: 6.3.0(latest)
  BIOS:Inter VT/VT-d or SR-IOV(enabled)
  ixgbe:3.11.33(latest)
  ixgbevf:2.7.12(latest)
  grub config:intel_iommu=on appended
 
  /var/log/messages:
  Nov  8 14:56:54 12 kernel: Intel(R) 10 Gigabit PCI Express Network
  Driver - version 3.11.33 Nov  8 14:56:54 12 kernel: Copyright (c) 1999-
 2012 Intel Corporation.
  Nov  8 14:56:54 12 kernel: ixgbe :07:00.0: PCI INT B - GSI 50
  (level, low) - IRQ 50 Nov  8 14:56:54 12 kernel: ixgbe: I/O
  Virtualization (IOV) set to 2 Nov  8 14:56:54 12 kernel: ixgbe:
  :07:00.0: ixgbe_check_options: FCoE Offload feature enabled Nov  8
  14:56:54 12 kernel: ixgbe :07:00.0: (unregistered net_device):
  SR-IOV enabled with 2 VFs Nov  8 14:56:54 12 kernel: ixgbe
  :07:00.0: FCoE offload feature is not available. Disabling FCoE
  offload feature Nov  8 14:56:54 12 kernel: ixgbe :07:00.0: (PCI
  Express:5.0GT/s:Width x8) 68:05:ca:0c:7a:e2 Nov  8 14:56:54 12 kernel:
  ixgbe :07:00.0: eth4: MAC: 2, PHY: 2, PBA No: G21371-003 Nov  8
  14:56:54 12 kernel: ixgbe :07:00.0: eth4: Enabled Features: RxQ: 1
  TxQ: 1 LRO Nov  8 14:56:54 12 kernel: ixgbe :07:00.0: eth4: IOV:
  VF 0 is enabled mac 0E:15:B9:26:B3:7D Nov  8 14:56:54 12 kernel: ixgbe
  :07:00.0: eth4: IOV: VF 1 is enabled mac 32:69:2D:16:B9:40 Nov  8
  14:56:54 12 kernel: ixgbe :07:00.0: eth4: Intel(R) 10 Gigabit
  Network Connection Nov  8 14:56:54 12 kernel: ixgbe :07:00.1: PCI
  INT A - GSI 40 (level, low) - IRQ 40 Nov  8 14:56:54 12 kernel:
  ixgbe: I/O Virtualization (IOV) set to 2 Nov  8 14:56:54 12 kernel:
  ixgbe: :07:00.1: ixgbe_check_options: FCoE Offload feature enabled
  Nov  8 14:56:54 12 kernel: ixgbe :07:00.1: not enough MMIO resources
 for SR-IOV Nov  8 14:56:54 12 kernel: ixgbe :07:00.1: (unregistered
 net_device): Failed to enable PCI sriov: -12 Nov  8 14:56:54 12 kernel:
 ixgbe :07:00.1: FCoE offload feature is not available. Disabling FCoE
 offload feature Nov  8 14:56:54 12 kernel: ixgbe :07:00.1: (PCI
 Express:5.0GT/s:Width x8) 68:05:ca:0c:7a:e3 Nov  8 14:56:54 12 kernel:
 ixgbe :07:00.1: eth5: MAC: 2, PHY: 2, PBA No: G21371-003 Nov  8
 14:56:54 12 kernel: ixgbe :07:00.1: eth5: Enabled Features:
  RxQ: 16 TxQ: 16 FdirHash RSC
  Nov  8 14:56:54 12 kernel: ixgbe :07:00.1: eth5: Intel(R) 10
  Gigabit Network Connection
 
  # dmesg |grep -E 'DMA|IOMMU'
  ACPI: DMAR bf3b3668 001C0 (v01 DELL   PE_SC3   0001 DELL
 0001)
DMA  0x0001 - 0x1000
DMA320x1000 - 0x0010
DMA zone: 56 pages used for memmap
DMA zone: 102 pages reserved
DMA zone: 3839 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 764849 pages, LIFO batch:31
  Intel-IOMMU: enabled
  DMAR: Host address width 40
  DMAR: DRHD base: 0x00fed9 flags: 0x1 IOMMU fed9: ver 1:0
  cap c90780106f0462 ecap f020fe
  DMAR: RMRR base: 0x00bf4c8000 end: 0x00bf4d
  DMAR: RMRR base: 0x00bf4b1000 end: 0x00bf4b
  DMAR: RMRR base: 0x00bf4a1000 end: 0x00bf4a1fff
  DMAR: RMRR base: 0x00bf4a3000 end: 0x00bf4a3fff
  DMAR: RMRR base: 0x00bf4a5000 end: 0x00bf4a5fff
  DMAR: RMRR base: 0x00bf4a7000 end: 0x00bf4a7fff
  DMAR: RMRR base: 0x00bf4c end: 0x00bf4c0fff
  DMAR: RMRR base: 0x00bf4c2000 end: 0x00bf4c2fff
  DMAR: ATSR flags: 0x0
  DMAR: Device scope device [:00:1a.02] not found
  DMAR: Device scope device [:00:1a.02] not found
  DMAR: Device scope device [:00:1d.02] not found
  DMAR: Device scope device [:00:1d.02] not found IOMMU 0xfed9:
  using Queued invalidation
  IOMMU: Setting RMRR:
  IOMMU: Setting identity map

Re: [E1000-devel] ixgbe: pci_get_device() call without counterpart call of pci_dev_put()

[Rose, Gregory V]

 -Original Message-
 From: Jeff Kirsher [mailto:tar...@gmail.com]
 Sent: Sunday, December 09, 2012 11:18 PM
 To: Elena Gurevich
 Cc: netdev; e1000-devel@lists.sourceforge.net; Rose, Gregory V; Skidmore,
 Donald C
 Subject: Re: ixgbe: pci_get_device() call without counterpart call of
 pci_dev_put()
 
 On 12/09/2012 01:47 AM, Elena Gurevich wrote:
  Hi all,
  I am pioneer in linux device drivers here and using Intel 82599 NIC as
  reference model, During investigation to drivers sources I found the
  suspicious code:
 
  Is code sequence (1)   and (2) the possible device reference count
 leakage
  ???
 
 
  Thanks a lot in advance
  Lena
 
  snipped from ixgbe_main.c  file function
  ixgbe_io_error_detected()
  ---
 
  . . .
  /* Find the pci device of the offending VF */
  vfdev = pci_get_device(PCI_VENDOR_ID_INTEL, device_id, NULL);
  while (vfdev) {
  if (vfdev-devfn == (req_id  0xFF))
  break;
  --  (1) leaves the loop with successful
 get
  call  
 
  vfdev = pci_get_device(PCI_VENDOR_ID_INTEL,
 device_id, vfdev);
  }
  /*
   * There's a slim chance the VF could have been hot plugged,
   * so if it is no longer present we don't need to issue the
   * VFLR.  Just clean up the AER in that case.
   */
  if (vfdev) {
  e_dev_err(Issuing VFLR to VF %d\n, vf);
  pci_write_config_dword(vfdev, 0xA8, 0x8000);
  }
 
  pci_cleanup_aer_uncorrect_error_status(pdev);
  }
 
  /*
   * Even though the error may have occurred on the other port
   * we still need to increment the vf error reference count for
   * both ports because the I/O resume function will be called
   * for both of them.
   */
  adapter-vferr_refcount++;
 
  return PCI_ERS_RESULT_RECOVERED;
    (2) leaves the function
  without put call 
  --  snipped
  ---
 
 Added Greg Rose (ixgbevf maintainer)  Don Skidmore (ixgbe maintainer)
 
 Since the code you have questions about is dealing with the vf, Greg will
 most likely be able to shed some light to your questions.

That's a bug.  I'll generate a patch to add the necessary pci_dev_put().

- Greg

--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] when SR-IOV is used with Intel 82599 NIC, multicast packets are dropped and not delivered to VF(Virtual Function)

[Rose, Gregory V]

We do not enable multicast promiscuous.  The VF drivers do not support any form 
of promiscuous receive behavior.  If you want to enable multicast promiscuous 
for VFs then you'll have to customize the driver for your own use and then 
carry the patch that does so locally.

This command:

promisc on multicast on allmulticast on

has no effect on the VF driver.

Hope that helps, if you have further questions let me know.

Thanks,

- Greg

 -Original Message-
 From: Duyck, Alexander H
 Sent: Wednesday, December 19, 2012 8:48 AM
 To: Rugang Chen
 Cc: Rose, Gregory V; E1000-devel@lists.sourceforge.net
 Subject: Re: when SR-IOV is used with Intel 82599 NIC, multicast packets
 are dropped and not delivered to VF(Virtual Function)
 
 Hi Chen,
 
 I have tried to answer your questions as best I can below.  However I am
 not the maintainer for the ixgbe SR-IOV functionality.  It is Greg Rose
 who is the maintainer so I have added him and the e1000-devel mailing list
 to this email.
 
 Thanks,
 
 Alex
 
 On 12/19/2012 07:08 AM, Rugang Chen wrote:
  Hi Alexander Duyck,
 
 
 
  We are building a Virtualization test bed with Intel 82599 NIC on
  Linux Host and KVM is used. Following is a brief description:
 
 
 
  *Host system:* Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
 
  *NIC: *Intel 82599EB, ixgbe driver version is 3.11.33. See more
  details in attachment lspci.log
 
  *KVM: *QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c)
  2003-2008 Fabrice Bellard (run kvm --version)**
 
  *Virtual Machines:*
 
  * Two Virtual Machines are installed with a same Linux
  kernel(2.6.26.5), and version of ixgbevf driver running on Virtual
  Machines is 2.6.2
 
  * There's a VF (Virtual Function) used as NIC in each Virtual
  Machine, and the two VFs are of the same PF(Physical Function)
 
  *Configuration:*
 
  * Both PF (Intel 8259 NIC) and VF(ixgbevf NIC seen in Virtual
  Machine) are set to promisc on multicast on allmulticast on with
  tool iproute2
 
  * Load ixgbe.ko in Host with parameter max_vfs=16
 
  *Result: *
 
  * When sending multicast packets out from one Virtual Machine A,
  no packet is seen in another Virtual Machine B**
 
  In our testing, multicast MAC address in packets is 01:00:5e:00:00:d2
 
  * On Virtual Machine, when we run ip maddr add
  01:00:5e:00:00:d2 dev ethx(ethx = interface name seen in Virtual
  Machine), we can debug that ixgbe_update_mc_addr_list_vf is called
  **
 
  * Communication of unicast from A to B is OK**
 
 
 
  We checked the ixgbe driver codes(version 3.11.33), and seen following:
  (Function *ixgbe_set_rx_mode*,  file *ixgbe_main.c*)
 
 
 
  if (hw-mac.type !=ixgbe_mac_82598EB) {
 
  vmolr |= IXGBE_READ_REG(hw, IXGBE_VMOLR(VMDQ_P(0))) 
 
   ~(IXGBE_VMOLR_MPE|IXGBE_VMOLR_ROMPE|
 
 IXGBE_VMOLR_ROPE);
 
  IXGBE_WRITE_REG(hw, IXGBE_VMOLR(VMDQ_P(0)), vmolr);
 
}
 
 
 
  We can see thatIXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE are disabled,
  and this may explain why multicast doesn't work.
 
  Checking the previous version of file ixgbe_main.c shipped in kernel
  source, we found that IXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE are
  disabled since the driver was firstly in the kernel.
 
  So our question are:
 
  * Why we disable IXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE, and
  only enable them for 82598EB in latest 3.11.33?
 
 The code you are referencing above only affects the PF specifically.
 The VMDQ_P(0) macro is accessing the offloads register for the PF itself.
 
 The VMOLR_MPE bit would be undesirable as that enables receive of all
 multicast packets.  As far as the VMOLR_ROMPE bit it should be enabled the
 first time a VF has a reset event.  I believe you should see this in
 ixgbe_sriov.c if you grep for VMOLR_ROMPE in the driver.
 
 The 82598EB does not have a VMOLR register and does not support SR-IOV.
  This is why it is excluded in the code you reference above.
 
 
  * Multicast from VF to VF (or between VF and PF) is supported ?
 
 Yes it is my understanding that this is supported.
 
  * If we just enable IXGBE_VMOLR_MPE andIXGBE_VMOLR_ROMPE for
  multicast on 82599EB, will there be any other influence ?
 
 The problem with enabling these by default is that it would mean that the
 PF would always be receiving multicast traffic even for groups that it did
 not add.
 
 
  Thanks for your patience to read the mail. Wait for your answer which
  may be very helpful to us.
 
 
 
  Best regards,
 
  Rugang Chen
 


--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
E1000-devel

Re: [E1000-devel] [Bug 56981] New: [SR-IOV] Intel I350 NIC VF cannot work in a Windows 2008 guest.

[Rose, Gregory V]

Adding Ashish Shah.  IIRC Windows 2008 isn't supported by the VF driver but I 
may be mistaken.  Ashish should know.

- Greg

 -Original Message-
 From: Ren, Yongjie [mailto:yongjie@intel.com]
 Sent: Monday, April 22, 2013 6:16 PM
 To: e1000-devel@lists.sourceforge.net; k...@vger.kernel.org
 Subject: Re: [E1000-devel] [Bug 56981] New: [SR-IOV] Intel I350 NIC VF
 cannot work in a Windows 2008 guest.
 
 Added e1000-devel list to see whether this's a known issue.
 
 Best Regards,
  Yongjie (Jay)
 
 
  -Original Message-
  From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org]
  On Behalf Of bugzilla-dae...@bugzilla.kernel.org
  Sent: Monday, April 22, 2013 10:41 PM
  To: k...@vger.kernel.org
  Subject: [Bug 56981] New: [SR-IOV] Intel I350 NIC VF cannot work in a
  Windows 2008 guest.
 
  https://bugzilla.kernel.org/show_bug.cgi?id=56981
 
 Summary: [SR-IOV] Intel I350 NIC VF cannot work in a
  Windows
  2008 guest.
 Product: Virtualization
 Version: unspecified
  Kernel Version: 3.9.0-rc3
Platform: All
  OS/Version: Linux
Tree: Mainline
  Status: NEW
Severity: normal
Priority: P1
   Component: kvm
  AssignedTo: virtualization_...@kernel-bugs.osdl.org
  ReportedBy: yongjie@intel.com
  Regression: No
 
 
  Environment:
  
  Host OS (ia32/ia32e/IA64):ia32e
  Guest OS (ia32/ia32e/IA64):ia32e
  Guest OS Type (Linux/Windows):Windows
  kvm.git Commit:79558f112fc0352e057f7b5e158e3d88b8b62c60
  qemu-kvm Commit:8912bdea01e8671e59fe0287314379be9c1f40ec
  Host Kernel Version:3.9.0-rc3
  Hardware: Sandy Bridge - EP
 
 
  Bug detailed description:
  --
  The assigned Intel I350 NIC VF (a igbvf) cannot work in a Windows 2008
  guest.
 
  1. If the guest is Windows 7/8/2012 or RHEL6.x , the Intel I350 NIC VF
  can work fine in the guest.
  2. Intel 82576 NIC VF (also igbvf) can work in a Windows 2008 guest.
 
 
  Reproduce steps:
  
  1. ./pcistub.sh -h 0b:10.0   ( to hide a device)
  2. qemu-system-x86_64 -enable-kvm -m 4096 -smp 4 -device
  pci-assign,host=0b:10.0 -net none -hda win2k8.img
 
  Current result:
  
  Intel I350 NIC VF cannot work in win2k8 guest
 
  Expected result:
  
  Intel I350 NIC VF can work in win2k8 guest
 
  --
  Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
  --- You are receiving this mail because: --- You are watching
  the assignee of the bug.
  --
  To unsubscribe from this list: send the line unsubscribe kvm in the
  body of a message to majord...@vger.kernel.org More majordomo info at
  http://vger.kernel.org/majordomo-info.html
 --
 
 Precog is a next-generation analytics platform capable of advanced
 analytics on semi-structured data. The platform includes APIs for building
 apps and a phenomenal toolset for data science. Developers can use our
 toolset for easy data analysis  visualization. Get a free account!
 http://www2.precog.com/precogplatform/slashdotnewsletter
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174; Ethernet, visit
 http://communities.intel.com/community/wired

--
Try New Relic Now  We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app,  servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] 82576 dropping traffic

[Rose, Gregory V]

Greetings Robert!!!  Good to hear from you although I'm sorry to hear that 
you're seeing an issue with the 82576.

I'm forwarding this email to the Sourceforge e1000_devel list where perhaps 
someone may have heard of the issue you describe.

Regards,

- Greg

From: InterNetX - Robert Garrett [mailto:robert.garr...@internetx.com]
Sent: Thursday, May 02, 2013 2:54 AM
To: Rose, Gregory V
Subject: Fwd: 82576 dropping traffic




 Original Message 
Subject:

82576 dropping traffic

Date:

Thu, 02 May 2013 11:52:39 +0200

From:

InterNetX - Robert Garrett 
robert.garr...@internetx.commailto:robert.garr...@internetx.com

Organization:

InternetX

To:

greg.v.r...@intel.commailto:greg.v.r...@intel.com



we are seeing something along the lines of traffic dropping from an

82576 (rev 1)

dropping all connections. This doesnt have to do with SRIOV, just

wondering if you might of heard something.

inbox driver RH 6.4.



--

Mit freundlichen Grüßen



Robert Garrett

Senior System Engineer

Technical Projects  Solutions



InterNetX GmbH

Maximilianstr. 6

93047 Regensburg

Germany



Tel. +49 941 59559-234

Fax  +49 941 59559-245



www.internetx.comhttp://www.internetx.com

www.facebook.com/InterNetXhttp://www.facebook.com/InterNetX

www.twitter.com/InterNetXhttp://www.twitter.com/InterNetX



Geschäftsführer/CEO: Thomas Mörz

Amtsgericht Regensburg, HRB 7142





--

Mit freundlichen Grüßen



Robert Garrett

Senior System Engineer

Technical Projects  Solutions



InterNetX GmbH

Maximilianstr. 6

93047 Regensburg

Germany



Tel. +49 941 59559-234

Fax  +49 941 59559-245



www.internetx.comhttp://www.internetx.com

www.facebook.com/InterNetXhttp://www.facebook.com/InterNetX

www.twitter.com/InterNetXhttp://www.twitter.com/InterNetX



Geschäftsführer/CEO: Thomas Mörz

Amtsgericht Regensburg, HRB 7142


--
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with 2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [non-linux] Cache Coherency Problems on X540?

[Rose, Gregory V]

 -Original Message-
 From: Brandon Falk [mailto:bf...@gamozolabs.com]
 Sent: Thursday, April 23, 2015 12:39 AM
 To: Antoine Kaufmann
 Cc: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] [non-linux] Cache Coherency Problems on X540?
 
 Haha, as you posted this I was just doing testing on no-snoop. By setting
 CTRL_EXT.NS_DIS (no-snoop disable), I no longer have the problem.
 
 Well that was a fun night! At least I found a real fix instead of a some
 cross-my-fingers kludge.
 
 I find it quite offputting that the X540 documentation says in many
 places, and in one case in the DCA Registers descriptions: In most
 applications non snoop should not be enabled. I find it strange that a
 setting that Intel knows themselves is not common is on by default.

I'm glad you found the source of the problem.  I'm not sure of the original 
reasoning for making snoop off by default for the X540 - I'm a SW engineer not 
a HW engineer :^), but at least the solution you found does make sense.

Thanks and regards,

- Greg

 
 Nevertheless, thanks for all the help :P
 
 -B
 
 On Thu, Apr 23, 2015 at 3:11 AM, Antoine Kaufmann 
 antoi...@cs.washington.edu wrote:
 
  As a follow-up on my earlier email about the no snoop setting: If you
  haven't checked that out yet you probably should. (c.f. last post in
https://software.intel.com/en-us/forums/topic/401498)
 
  I think that what you describe below might point to that. From a quick
  look at the 82574L spec it looks like no snoop is disabled by default
  on that card (i.e. the respective PCIe transactions will be snooped by
  the cache), while the x540 spec seems to indicate that no snoop is
  enabled by default (which was also my experience with the 82599).
 
  Not quite sure though about the behavior with mfences though. In any
  case I would rule out the no snoop issue (basically involves setting
  one bit, iirc, and I think I found that in the FreeBSD ixgbe driver).
 
  On Thu, Apr 23 02:53, Brandon Falk wrote:
   This is very strange indeed. The mfence works but the lfence
   does
  not.
   On top of these I have tried *many* other different operations which
   I thought may have an effect. You can find the entire code snippit
   for
  these
   tests at the end of this email. When the code was being tested each
   'section' was individually uncommented and then the result of that
   test
  is
   placed in a comment above the test.
  
   After doing these tests and scratching my head, I decided to do a
   test
  with
   my old 82574L driver. I removed the X540 from the test machine and
   in the same PCIe slot placed in the 82574L card which I have
   previously used
  (also
   using the same ethernet connection). Since the descriptor format for
  legacy
   descriptors is identical in the X540 and 82574L, I used the exact
   same
  code
   (posted below) as I used on the X540. On this card *no fences* were
  needed.
   Indicating that either A. I'm initialing the X540 in a manner that
  somehow
   makes this behaviour possible. B. the X540 (or maybe my specific
   one)
  has a
   bug that is causing a need force these fences. C. Maybe it's not a
   bug
  but
   something that needs to be documented. I still find it very strange
   that
  a
   write fence changes how things operate when no writes are actually
   being done where I'm fencing.
  
   Some other tests I have done:
  
   - Have other processors spin and do mfences while the main core does
   not
  do
   an mfence. This did not make it work, and this is expected as an
   mfence only should locally change behaviour.
   - mfences all over the X540 initialization and prior to doing the DD
   polling. Did not fix the problem.
  
   Some things I could think of that would cause this problem:
  
   - It's just a bug in the X540, or mine specifically. (If I'm not too
   lazy maybe I'll swap my X540s around between machines and try on
 another one).
   - It's a bug in my initialization, but I would find this strange as
   my 82574L driver initializes in almost an identical fashion.
   - It's a bug in my motherboard/CPU, but only on =8x channel PCIe
   cards, which would explain why it didn't affect the 82574L.
  
   - Example code -
  
   ; Get the rx entry
   mov rdx, qword [gs:thread_local.x540_rx_ring_base]
   mov rax, qword [gs:thread_local.x540_rx_head] shl rax, 4
  
   ; XXX: Temporary, used for the loop counter for testing.
   xor ebp, ebp
  
   ; Putting an mfence/lfence/sfence here has no effect on the result.
  
   mov rsi, qword [rdx + rax + 0] ; pointer to packet contents
  
   ; Putting an mfence/lfence/sfence here has no effect on the result.
  
   .lewp:
   ; Putting an mfence/lfence/sfence here has no effect on the result.
  
   ; Wait until a packet is present here by polling the DD bit test
   dword [rdx + rax + 8 + 4], 1
   jz   short .lewp
  
   ; Without anything here, we fail with nothing getting printed.
; Successful
   ;mfence
  
   ; 

Re: [E1000-devel] [non-linux] Cache Coherency Problems on X540?

[Rose, Gregory V]

 -Original Message-
 From: Brandon Falk [mailto:bf...@gamozolabs.com]
 Sent: Wednesday, April 22, 2015 1:10 PM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] [non-linux] Cache Coherency Problems on X540?
 
 I'm writing a custom driver for a custom OS and thus this does not relate
 to the Linux driver.
 
 I have implemented a driver which polls for the DD and EOP flags to get
 set (using legacy receive descriptors) and I have noticed a problem where
 these flags both can get set very slightly before the packet contents have
 actually been DMAed to the host (or at least appear to be).
 
 I'm under the understanding that the X540 device should respect caching
 and thus I should not have to be flushing caches for the packet contents.
 The only thing I could think of would be that I need to add some memory
 barriers as maybe the CPU is speculatively loading the packet contents
 before it is reading the flags, and thus the contents are older than the
 flags?

The Intel Linux driver for the X540 does implement memory barriers and I'm 
pretty sure it's to prevent this sort of problem.  You may want to examine that 
code (understanding at the same time that GPL will prevent you from using it 
directly).

- Greg

--
Greg Rose
FreeBSD/NFV PAE
Network Division
Intel Corporation
Desk - 503-712-5048

Any man who afflicts the human race with ideas must be prepared to see them 
misunderstood.
 
- H. L. Mencken



 
 I just want to verify that this isn't a hardware problem with the X540 and
 that I will need to do a workaround on my end. Ideally the workaround
 actually fixes the problem, rather than a small delay/sleep that makes it
 seem to go away in most cases.
 
 Anyone have any ideas? Adding an clflush, mfence, sfence fixes the
 problem, but an lfence does not. I'm under the suspicion that it's not the
 fences that actually are doing anything, rather the delay of these rather
 long instructions happens to just be a long enough sleep. I just want to
 make sure that I'm fixing the problem correctly and not just hoping the
 sleep delays long enough for the DMA to occur, as that is quite kludgey.
 
 It is a 4 physical processor system, AMD Fam 15h.
 
 -B
--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] [non-linux] Cache Coherency Problems on X540?

[Rose, Gregory V]

Please do check to see if the fence is fixing the problem.  If it is not the 
fence and it turns out it is some other delay then we can follow up.

Thanks,


-Greg

From: Brandon Falk [mailto:bf...@gamozolabs.com]
Sent: Wednesday, April 22, 2015 2:41 PM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] [non-linux] Cache Coherency Problems on X540?

I can see that there is a read memory barrier in the ixgbe_clean_rx_irq() 
functions with the comment This memory barrier is needed to keep us from 
reading any other fields out of the rx_desc until we know the RXD_STAT_DD bit 
is set which is exactly my problem, but that should emit an lfence which if I 
remember correctly did not happen to fix the problem for me.

I'll take a look when I get home. I can make a simple test to try to determine 
if the fence is actually fixing the problem, or if it is just adding a delay 
which makes the problem seem to disappear. Since I have a very lightweight and 
lockless RX implementation, if it is a hardware bug it is unlikely to show up 
in a 'proper' driver given they just happen to do enough logic between the DD 
check and reading packet contents.

My code looks as such:

.poll:


-B

On Wed, Apr 22, 2015 at 5:04 PM, Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com wrote:


 -Original Message-
 From: Brandon Falk [mailto:bf...@gamozolabs.commailto:bf...@gamozolabs.com]
 Sent: Wednesday, April 22, 2015 1:10 PM
 To: 
 e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] [non-linux] Cache Coherency Problems on X540?

 I'm writing a custom driver for a custom OS and thus this does not relate
 to the Linux driver.

 I have implemented a driver which polls for the DD and EOP flags to get
 set (using legacy receive descriptors) and I have noticed a problem where
 these flags both can get set very slightly before the packet contents have
 actually been DMAed to the host (or at least appear to be).

 I'm under the understanding that the X540 device should respect caching
 and thus I should not have to be flushing caches for the packet contents.
 The only thing I could think of would be that I need to add some memory
 barriers as maybe the CPU is speculatively loading the packet contents
 before it is reading the flags, and thus the contents are older than the
 flags?

The Intel Linux driver for the X540 does implement memory barriers and I'm 
pretty sure it's to prevent this sort of problem.  You may want to examine that 
code (understanding at the same time that GPL will prevent you from using it 
directly).

- Greg

--
Greg Rose
FreeBSD/NFV PAE
Network Division
Intel Corporation
Desk - 503-712-5048tel:503-712-5048

Any man who afflicts the human race with ideas must be prepared to see them 
misunderstood.

- H. L. Mencken




 I just want to verify that this isn't a hardware problem with the X540 and
 that I will need to do a workaround on my end. Ideally the workaround
 actually fixes the problem, rather than a small delay/sleep that makes it
 seem to go away in most cases.

 Anyone have any ideas? Adding an clflush, mfence, sfence fixes the
 problem, but an lfence does not. I'm under the suspicion that it's not the
 fences that actually are doing anything, rather the delay of these rather
 long instructions happens to just be a long enough sleep. I just want to
 make sure that I'm fixing the problem correctly and not just hoping the
 sleep delays long enough for the DMA to occur, as that is quite kludgey.

 It is a 4 physical processor system, AMD Fam 15h.

 -B

--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] i40e and RSTP

[Rose, Gregory V]

I don’t see the multicast address for the BPDU in the bridge.  Can you 
explicitly add the BPDU multicast address to the Fortville HW filters using the 
‘ip maddr’ command?  Normally a service will register the associated multicast 
addresses with the network interface but that doesn’t seem to be happening.

Try that and let’s see if it helps.

Thanks,


-Greg

From: Maule Mark [mailto:mark_ma...@yahoo.com]
Sent: Thursday, April 30, 2015 9:04 AM
To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] i40e and RSTP

eth1 is the e40i port attached to the bridge.  Currently I have only one port 
bridged in order to prevent the loop.

eth1  Link encap:Ethernet  HWaddr 00:10:E0:65:2E:65

WNFFEE # bridge fdb
00:10:e0:65:2e:65 dev eth1 permanent
00:21:28:f9:09:2c dev eth1
00:21:28:f9:09:5c dev eth1
00:10:e0:65:2c:af dev eth1

If it matters, I'm using this version of i40e (shipped with the kernel):

#define DRV_VERSION_MAJOR 1
#define DRV_VERSION_MINOR 2
#define DRV_VERSION_BUILD 2

Could not find a i40e README under the kernel src or Documentation directory.




From: Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com
To: Maule Mark mark_ma...@yahoo.commailto:mark_ma...@yahoo.com; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Sent: Thursday, April 30, 2015 10:42 AM
Subject: RE: [E1000-devel] i40e and RSTP


 -Original Message-
 From: Maule Mark [mailto:mark_ma...@yahoo.commailto:mark_ma...@yahoo.com]
 Sent: Thursday, April 30, 2015 8:08 AM
 To: 
 e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] i40e and RSTP

 Hi:
 I have a small 4 node private network where each node has 2 ports, with
 each port directly connected to a port on another node in such a way that
 a ring topology is formed.  Two of the nodes have NIC's controlled by the
 e1000 driver, two of the nodes have NIC's controlled by the i40e
 driver.  The OS is based on Linux 3.8.13.
 I bridge the 2 portss on each node, and am using user space RSTP to manage
 the loop.
 What I am seeing is that the bridge ports on the nodes with i40e based
 NIC's are not receiving BPDU's, and so all ports remain forwarding,
 resulting in a loop.  Using a packet capture program to monitor enet
 frames, it seems that the interfaces for the i40e NIC's never see ethernet
 frames from its direct connected peer destined to
 the 01:80:C2:00:00:00 multicast address.  They do see unicast and
 broadcast frames from the peer.  The direct connected peer (e1000 based
 nic) does see BPDU frames being sent out the i40e based NIC's.
 If I move to a config having all e1000 based NIC's, everything works fine.
 I'm fairly new to networking hw, so I assume I need to turn some
 configuration knobs on the i40e in order to receive the BPDU's.
 I am currently not doing any configuration on the i40e interfaces - just
 modprob'ing i40e to load it, and then standard commands to bring up the
 interface, bridge it, and assign an ip address.
 Any advice would be greatly appreciated.

When running in a bridged environment there is a work around required.  The 
information is in the README that comes with the driver.  Please review and 
make sure that your issue is not related to the one described in the README.  
I've pasted in the relevant section to save you some time.  While you don't 
mention SR-IOV or NPAR modes of operation you may still be seeing a bug related 
to the root cause, which is incorrect source pruning.  Make sure that all 
packets being sent by the i40e driver have a source MAC address in the bridge 
FDB.

If this doesn't help then we'll have to explore other options.

Regards,

- Greg

--
In SR-IOV enabled or NPAR enabled adapters, Physical Function (PF) does not
work in bridge mode. When a bridge is created on the PF device, an emulation
device in the Virtual Machine (VM) connects to this bridge cannot receive any
unicast packets.

To avoid this from occurring, for each emulated device (software Virtual
Station Interface [VSI]) added to the bridge the MAC address of the emulated
device (software Virtual Ethernet Bridge [VEB] VSI) needs to be added manually
to the forwarding database (FDB) filter table using the iproute2 package
bridge tool. This can be done by executing the following command:

  # bridge fdb add MAC ADDR dev PF device interface self permanent

The FDB entry when the emulated device is no longer in use or the guest to
which it is assigned is moved to a different host must be deleted using this


command:


  # bridge fdb del MAC ADDR dev PF device interface


--
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box

Re: [E1000-devel] i40e and RSTP

[Rose, Gregory V]

I really can’t say for sure – as I mentioned a service that requires 
registering a multicast address will usually do that and the address will get 
added to the filters.  Why the RSTP service isn’t registering the multicast 
address is puzzling – or perhaps, it is registering the multicast address but 
on i40e it is not getting instantiated as an L2 filter correctly.  I have heard 
of an issue that will be fixed in the next i40e driver SW release that might 
fix it.  Until then you may have to continue to manually add the multicast 
address as a work around.

Regards,


-Greg

From: Maule Mark [mailto:mark_ma...@yahoo.com]
Sent: Thursday, April 30, 2015 11:59 AM
To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] i40e and RSTP

Sure enough, that did the trick!

Now, the next question is, why do I need to do that on i40e when I don't on 
e1000?  I don't see the BPDU maddr present in the 'ip mroute' or 'bridge fdb' 
output on the e1000 side of the link, which works by default.

e1000 side of the link - 00:21:28:f9:09:5d

[root@pilot1 ~]# ip maddr show dev eth2
3:  eth2
link  33:33:00:00:00:01
link  01:00:5e:00:00:01
link  33:33:ff:f9:09:5d
inet  224.0.0.1
inet6 ff02::1:fff9:95d
inet6 ff02::1

[root@pilot1 ~]# bridge fdb
33:33:00:00:00:01 dev eth1 self permanent
01:00:5e:00:00:01 dev eth1 self permanent
33:33:ff:f9:09:5c dev eth1 self permanent
33:33:00:00:00:01 dev eth2 self permanent
01:00:5e:00:00:01 dev eth2 self permanent
33:33:ff:f9:09:5d dev eth2 self permanent
33:33:00:00:00:01 dev eth3 self permanent
01:00:5e:00:00:01 dev eth3 self permanent
33:33:ff:f9:09:5e dev eth3 self permanent
33:33:00:00:00:01 dev eth0 self permanent
01:00:5e:00:00:01 dev eth0 self permanent
33:33:ff:f9:09:5f dev eth0 self permanent
00:10:e0:65:2c:ae dev eth1
00:10:e0:65:2e:64 dev eth2
00:21:28:f9:09:5c dev eth1 permanent
00:10:e0:65:2e:65 dev eth2
00:21:28:f9:09:5d dev eth2 permanent
00:21:28:f9:09:2c dev eth1



i40e side of the link - 00:10:e0:65:2e:65 - after manually adding with ip maddr

WNFFEE # ip maddr show dev eth1
12: eth1
link  01:00:5e:00:00:01
link  33:33:00:00:02:02
link  33:33:00:00:00:01
link  33:33:ff:65:2e:65
link  00:00:00:00:00:00 static
link  01:80:c2:00:00:00 static
inet  224.0.0.1
inet6 ff02::1:ff65:2e65
inet6 ff02::202
inet6 ff02::1


WNFFEE # bridge fdb
00:10:e0:65:2e:64 dev eth0 permanent
00:10:e0:65:2e:65 dev eth1 permanent
00:21:28:f9:09:2c dev eth1
00:10:e0:65:2c:ae dev eth1
00:21:28:f9:09:5c dev eth1
00:21:28:f9:09:5d dev eth1






From: Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com
To: Maule Mark mark_ma...@yahoo.commailto:mark_ma...@yahoo.com; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Sent: Thursday, April 30, 2015 11:33 AM
Subject: RE: [E1000-devel] i40e and RSTP

I don’t see the multicast address for the BPDU in the bridge.  Can you 
explicitly add the BPDU multicast address to the Fortville HW filters using the 
‘ip maddr’ command?  Normally a service will register the associated multicast 
addresses with the network interface but that doesn’t seem to be happening.

Try that and let’s see if it helps.

Thanks,

-Greg


From: Maule Mark [mailto:mark_ma...@yahoo.com]
Sent: Thursday, April 30, 2015 9:04 AM
To: Rose, Gregory V; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] i40e and RSTP

eth1 is the e40i port attached to the bridge.  Currently I have only one port 
bridged in order to prevent the loop.

eth1  Link encap:Ethernet  HWaddr 00:10:E0:65:2E:65

WNFFEE # bridge fdb
00:10:e0:65:2e:65 dev eth1 permanent
00:21:28:f9:09:2c dev eth1
00:21:28:f9:09:5c dev eth1
00:10:e0:65:2c:af dev eth1

If it matters, I'm using this version of i40e (shipped with the kernel):

#define DRV_VERSION_MAJOR 1
#define DRV_VERSION_MINOR 2
#define DRV_VERSION_BUILD 2

Could not find a i40e README under the kernel src or Documentation directory.



From: Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com
To: Maule Mark mark_ma...@yahoo.commailto:mark_ma...@yahoo.com; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Sent: Thursday, April 30, 2015 10:42 AM
Subject: RE: [E1000-devel] i40e and RSTP


 -Original Message-
 From: Maule Mark [mailto:mark_ma...@yahoo.commailto:mark_ma...@yahoo.com]
 Sent: Thursday, April 30, 2015 8:08 AM
 To: 
 e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] i40e and RSTP

 Hi:
 I have a small 4 node private network where each node has 2 ports

Re: [E1000-devel] i40e and RSTP

[Rose, Gregory V]

 -Original Message-
 From: Maule Mark [mailto:mark_ma...@yahoo.com]
 Sent: Thursday, April 30, 2015 8:08 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] i40e and RSTP
 
 Hi:
 I have a small 4 node private network where each node has 2 ports, with
 each port directly connected to a port on another node in such a way that
 a ring topology is formed.  Two of the nodes have NIC's controlled by the
 e1000 driver, two of the nodes have NIC's controlled by the i40e
 driver.  The OS is based on Linux 3.8.13.
 I bridge the 2 portss on each node, and am using user space RSTP to manage
 the loop.
 What I am seeing is that the bridge ports on the nodes with i40e based
 NIC's are not receiving BPDU's, and so all ports remain forwarding,
 resulting in a loop.  Using a packet capture program to monitor enet
 frames, it seems that the interfaces for the i40e NIC's never see ethernet
 frames from its direct connected peer destined to
 the 01:80:C2:00:00:00 multicast address.  They do see unicast and
 broadcast frames from the peer.  The direct connected peer (e1000 based
 nic) does see BPDU frames being sent out the i40e based NIC's.
 If I move to a config having all e1000 based NIC's, everything works fine.
 I'm fairly new to networking hw, so I assume I need to turn some
 configuration knobs on the i40e in order to receive the BPDU's.
 I am currently not doing any configuration on the i40e interfaces - just
 modprob'ing i40e to load it, and then standard commands to bring up the
 interface, bridge it, and assign an ip address.
 Any advice would be greatly appreciated.

When running in a bridged environment there is a work around required.  The 
information is in the README that comes with the driver.  Please review and 
make sure that your issue is not related to the one described in the README.  
I've pasted in the relevant section to save you some time.  While you don't 
mention SR-IOV or NPAR modes of operation you may still be seeing a bug related 
to the root cause, which is incorrect source pruning.  Make sure that all 
packets being sent by the i40e driver have a source MAC address in the bridge 
FDB.

If this doesn't help then we'll have to explore other options.

Regards,

- Greg

--
In SR-IOV enabled or NPAR enabled adapters, Physical Function (PF) does not 
work in bridge mode. When a bridge is created on the PF device, an emulation 
device in the Virtual Machine (VM) connects to this bridge cannot receive any 
unicast packets. 

To avoid this from occurring, for each emulated device (software Virtual 
Station Interface [VSI]) added to the bridge the MAC address of the emulated
device (software Virtual Ethernet Bridge [VEB] VSI) needs to be added manually 
to the forwarding database (FDB) filter table using the iproute2 package 
bridge tool. This can be done by executing the following command:

  # bridge fdb add MAC ADDR dev PF device interface self permanent

The FDB entry when the emulated device is no longer in use or the guest to 
which it is assigned is moved to a different host must be deleted using this
command:

  # bridge fdb del MAC ADDR dev PF device interface

--
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

[E1000-devel] test - ignore

[Rose, Gregory V]

Test - please ignore

--
Greg Rose
FreeBSD/NFV PAE
Network Division
Intel Corporation
Desk - 503-712-5048

Any man who afflicts the human race with ideas must be prepared to see them 
misunderstood.

- H. L. Mencken

--
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Programming i2c interface on XL710

[Rose, Gregory V]

 -Original Message-
 From: Vladimir Levintovich [mailto:vladim...@silicom.co.il]
 Sent: Monday, May 18, 2015 9:07 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] Programming i2c interface on XL710
 
 Hi guys,
 
 Is someone have an experience with programming i2c interface on XL710?
 
 I need to test QSFP+ 10 Gbs 4X Pluggable Transceiver (FTL410QD2C) , i.e.
 to write something data, read that and also to test Acknowledge.
 Device code of this transceiver is 101 R/W.
 
 According to XL710 data sheet, I need to write/read to/from register
 GLGEN_I2CCMD0 (offset 0x000881E0).
 I perform it using some utility. Here are 2 commands for example:
 find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  write
 0x000881e0 0x20a05a5a  {}' {} ';'
 find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  write
 0x000881e0 0x28a0  {}' {} ';'
 find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  read
 0x000881e0   {}' {} ';'
 
 How do I need to write/read to/from this register properly?

It appears that you've written 0x5a5a to register 0xA0 and then are reading the 
value back.  What makes you think you haven't done it correctly?

 I don't understand what is filed REGADD and PHYADD (chapter 11.2..2.1.14)?

I'm no expert on I2C programming but it appears the REGADD field is the offset 
of one of the registers - in your example above it is 0xA0.  The PHYADD bits 
are refer to the device address - it says it should be 1010b.  It is a bit 
confusing because the field is 3 bits but the default value they mention is 4 
bits.  From your example above it appears you're using the value 000b.  Perhaps 
changing it 101b might help?

Thanks,

- Greg


--
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] fix broken i40e for kernel 3.19

[Rose, Gregory V]

 -Original Message-
 From: Tobias Jungel [mailto:t...@bisdn.de]
 Sent: Thursday, April 09, 2015 1:30 AM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] fix broken i40e for kernel 3.19
 
 this patch fixes the build of i40e on a 3.19 kernel.
 
 Having seen http://sourceforge.net/p/e1000/patches/31/ I guess you already
 have fixed this as well. Nonetheless this might help others as long as the
 code isn't public.
 

Yes, this has already been fixed and will be in the next release if it hasn't 
already.

Thanks,

- Greg

 ---
  src/i40e/i40e_main.c | 7 ++-
  1 file changed, 6 insertions(+), 1 deletion(-)
 
 diff --git a/src/i40e/i40e_main.c b/src/i40e/i40e_main.c index
 5e503e0..f7f82c9 100644
 --- a/src/i40e/i40e_main.c
 +++ b/src/i40e/i40e_main.c
 @@ -8282,7 +8282,7 @@ static void i40e_del_vxlan_port(struct net_device
 *netdev,  #endif /* HAVE_VXLAN_RX_OFFLOAD */  #ifdef
 HAVE_NDO_GET_PHYS_PORT_ID  static int i40e_get_phys_port_id(struct
 net_device *netdev,
 -  struct netdev_phys_port_id *ppid)
 +  struct netdev_phys_item_id *ppid)
  {
   struct i40e_netdev_priv *np = netdev_priv(netdev);
   struct i40e_pf *pf = np-vsi-back;
 @@ -8496,7 +8496,12 @@ static int i40e_ndo_bridge_getlink(struct sk_buff
 *skb, u32 pid, u32 seq,
   if (!veb)
   return 0;
 
 +#if ( LINUX_VERSION_CODE  KERNEL_VERSION(3,19,0) )
   return ndo_dflt_bridge_getlink(skb, pid, seq, dev, veb-
 bridge_mode);
 +#else
 + return ndo_dflt_bridge_getlink(skb, pid, seq, dev, veb-bridge_mode,
 0, 0);
 +#endif
 +
  }
  #endif /* HAVE_BRIDGE_ATTRIBS */
  #endif /* HAVE_FDB_OPS */
 --
 2.1.0
 
 
 --
 
 BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
 Develop your own process in accordance with the BPMN 2 standard
 Learn Process modeling best practices with Bonita BPM through live
 exercises
 http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
 event?utm_
 source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174; Ethernet, visit
 http://communities.intel.com/community/wired

--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Programming i2c interface on XL710

[Rose, Gregory V]

 -Original Message-
 From: Vladimir Levintovich [mailto:vladim...@silicom.co.il]
 Sent: Wednesday, May 20, 2015 7:25 AM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: RE: Programming i2c interface on XL710
 
 Thanks.
 I tryed.
 Here are 2 commands for example:
 find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo write
 0x000881e0 0x05a01234  {}' {} ';'
 find /sys/kernel/debug/i40e/ -name command  -exec sh -c 'echo read
 0x000881e0   {}' {} ';'
 Below is the result from the dmesg buffer:
 [89780.472266] i40e :02:00.1: write: 0x000881e0 = 0x05a00d00
 [89780.474727] i40e :02:00.0: write: 0x000881e0 = 0x05a01234
 [89780.479131] i40e :02:00.1: read: 0x000881e0 = 0xa5a01234
 [89780.481625] i40e :02:00.0: read: 0x000881e0 = 0xa5a01234

It appears to me that you were able to successfully write 01234h to the device 
at bus address 02:00.0 - I rearranged the above to help show it.

 [89780.474727] i40e :02:00.0: write: 0x000881e0 = 0x05a01234
 [89780.481625] i40e :02:00.0: read: 0x000881e0 = 0xa5a01234

But the write to the device at bus address 02:00.1 did not succeed - you wrote 
0D00h but read back 01234h.

 [89780.472266] i40e :02:00.1: write: 0x000881e0 = 0x05a00d00
 [89780.479131] i40e :02:00.1: read: 0x000881e0 = 0xa5a01234

I think the i2c interface is writeable only through the first function of the 
device.  I will check and get back to you with an answer.  But for now it 
appears that you are able to write to the register on the device at bus address 
02:00.0.

- Greg

 
  I have disconnected qsfp Trancievers and I saw at the same result. It
 means I wrote nothing to the QSFP Tranciever.
  Please advise how can I advance?
  Thank you
 
 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: Monday, May 18, 2015 8:30 PM
 To: Vladimir Levintovich; e1000-devel@lists.sourceforge.net
 Subject: RE: Programming i2c interface on XL710
 
 
  -Original Message-
  From: Vladimir Levintovich [mailto:vladim...@silicom.co.il]
  Sent: Monday, May 18, 2015 9:07 AM
  To: e1000-devel@lists.sourceforge.net
  Subject: [E1000-devel] Programming i2c interface on XL710
 
  Hi guys,
 
  Is someone have an experience with programming i2c interface on XL710?
 
  I need to test QSFP+ 10 Gbs 4X Pluggable Transceiver (FTL410QD2C) , i.e.
  to write something data, read that and also to test Acknowledge.
  Device code of this transceiver is 101 R/W.
 
  According to XL710 data sheet, I need to write/read to/from register
  GLGEN_I2CCMD0 (offset 0x000881E0).
  I perform it using some utility. Here are 2 commands for example:
  find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  write
  0x000881e0 0x20a05a5a  {}' {} ';'
  find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  write
  0x000881e0 0x28a0  {}' {} ';'
  find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  read
  0x000881e0   {}' {} ';'
 
  How do I need to write/read to/from this register properly?
 
 It appears that you've written 0x5a5a to register 0xA0 and then are
 reading the value back.  What makes you think you haven't done it
 correctly?
 
  I don't understand what is filed REGADD and PHYADD (chapter
 11.2..2.1.14)?
 
 I'm no expert on I2C programming but it appears the REGADD field is the
 offset of one of the registers - in your example above it is 0xA0.  The
 PHYADD bits are refer to the device address - it says it should be 1010b.
 It is a bit confusing because the field is 3 bits but the default value
 they mention is 4 bits.  From your example above it appears you're using
 the value 000b.  Perhaps changing it 101b might help?
 
 Thanks,
 
 - Greg


--
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Programming i2c interface on XL710

[Rose, Gregory V]

Could you please send the output of 'ethtool -i' for each of the interfaces?  
Also the output of 'dmesg | grep -i i40e' would help.

Thanks,

- Greg

 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: Wednesday, May 20, 2015 1:37 PM
 To: Vladimir Levintovich; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] Programming i2c interface on XL710
 
  -Original Message-
  From: Vladimir Levintovich [mailto:vladim...@silicom.co.il]
  Sent: Wednesday, May 20, 2015 7:25 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: RE: Programming i2c interface on XL710
 
  Thanks.
  I tryed.
  Here are 2 commands for example:
  find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo write
  0x000881e0 0x05a01234  {}' {} ';'
  find /sys/kernel/debug/i40e/ -name command  -exec sh -c 'echo read
  0x000881e0   {}' {} ';'
  Below is the result from the dmesg buffer:
  [89780.472266] i40e :02:00.1: write: 0x000881e0 = 0x05a00d00
  [89780.474727] i40e :02:00.0: write: 0x000881e0 = 0x05a01234
  [89780.479131] i40e :02:00.1: read: 0x000881e0 = 0xa5a01234
  [89780.481625] i40e :02:00.0: read: 0x000881e0 = 0xa5a01234
 
 It appears to me that you were able to successfully write 01234h to the
 device at bus address 02:00.0 - I rearranged the above to help show it.
 
  [89780.474727] i40e :02:00.0: write: 0x000881e0 = 0x05a01234
  [89780.481625] i40e :02:00.0: read: 0x000881e0 = 0xa5a01234
 
 But the write to the device at bus address 02:00.1 did not succeed - you
 wrote 0D00h but read back 01234h.
 
  [89780.472266] i40e :02:00.1: write: 0x000881e0 = 0x05a00d00
  [89780.479131] i40e :02:00.1: read: 0x000881e0 = 0xa5a01234
 
 I think the i2c interface is writeable only through the first function of
 the device.  I will check and get back to you with an answer.  But for now
 it appears that you are able to write to the register on the device at bus
 address 02:00.0.
 
 - Greg
 
 
   I have disconnected qsfp Trancievers and I saw at the same result. It
  means I wrote nothing to the QSFP Tranciever.
   Please advise how can I advance?
   Thank you
 
  -Original Message-
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
  Sent: Monday, May 18, 2015 8:30 PM
  To: Vladimir Levintovich; e1000-devel@lists.sourceforge.net
  Subject: RE: Programming i2c interface on XL710
 
 
   -Original Message-
   From: Vladimir Levintovich [mailto:vladim...@silicom.co.il]
   Sent: Monday, May 18, 2015 9:07 AM
   To: e1000-devel@lists.sourceforge.net
   Subject: [E1000-devel] Programming i2c interface on XL710
  
   Hi guys,
  
   Is someone have an experience with programming i2c interface on XL710?
  
   I need to test QSFP+ 10 Gbs 4X Pluggable Transceiver (FTL410QD2C) ,
 i.e.
   to write something data, read that and also to test Acknowledge.
   Device code of this transceiver is 101 R/W.
  
   According to XL710 data sheet, I need to write/read to/from register
   GLGEN_I2CCMD0 (offset 0x000881E0).
   I perform it using some utility. Here are 2 commands for example:
   find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  write
   0x000881e0 0x20a05a5a  {}' {} ';'
   find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  write
   0x000881e0 0x28a0  {}' {} ';'
   find /sys/kernel/debug/i40e/ -name command -exec sh -c 'echo  read
   0x000881e0   {}' {} ';'
  
   How do I need to write/read to/from this register properly?
 
  It appears that you've written 0x5a5a to register 0xA0 and then are
  reading the value back.  What makes you think you haven't done it
  correctly?
 
   I don't understand what is filed REGADD and PHYADD (chapter
  11.2..2.1.14)?
 
  I'm no expert on I2C programming but it appears the REGADD field is
  the offset of one of the registers - in your example above it is 0xA0.
  The PHYADD bits are refer to the device address - it says it should be
 1010b.
  It is a bit confusing because the field is 3 bits but the default
  value they mention is 4 bits.  From your example above it appears
  you're using the value 000b.  Perhaps changing it 101b might help?
 
  Thanks,
 
  - Greg
 
 
 --
 
 One dashboard for servers and applications across Physical-Virtual-Cloud
 Widest out-of-the-box monitoring support with 50+ applications Performance
 metrics, stats and reports that give you Actionable Insights Deep dive
 visibility with transaction tracing using APM Insight.
 http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174; Ethernet, visit
 http://communities.intel.com/community/wired

--
One dashboard for servers and applications across

Re: [E1000-devel] i40e driver question

[Rose, Gregory V]

I can help with that.

Please look here for the latest driver releas:

http://sourceforge.net/projects/e1000/files/i40e%20stable/

- Greg

 -Original Message-
 From: Nelson, Shannon [mailto:shannon.nel...@intel.com]
 Sent: Thursday, June 11, 2015 6:25 PM
 To: Fujinaka, Todd; Kirsher, Jeffrey T; Alpesh Patel
 Cc: e1000-devel; Brandeburg, Jesse
 Subject: Re: [E1000-devel] i40e driver question
 
 Alpesh,
 
 Your log messages show that you are using driver version 0.4.10-k which is
 extremely old at this point, and will not support the more recent
 hardware/firmware that you have (e8000152d).  Before doing anything else,
 please find a newer driver.
 
 sln
 
  -Original Message-
  From: Fujinaka, Todd [mailto:todd.fujin...@intel.com]
  Sent: Thursday, June 11, 2015 5:26 PM
  To: Kirsher, Jeffrey T; Alpesh Patel
  Cc: e1000-devel; Brandeburg, Jesse
  Subject: Re: [E1000-devel] i40e driver question
 
  Actually, you have direct support through the local Cisco team. Please
  contact them for help.
 
  Todd Fujinaka
  Software Application Engineer
  Networking Division (ND)
  Intel Corporation
  todd.fujin...@intel.com
  (503) 712-4565
 
  -Original Message-
  From: Jeff Kirsher [mailto:jeffrey.t.kirs...@intel.com]
  Sent: Thursday, June 11, 2015 3:55 PM
  To: Alpesh Patel
  Cc: e1000-devel; Brandeburg, Jesse
  Subject: Re: [E1000-devel] i40e driver question
 
  On Wed, 2015-06-10 at 18:57 -0400, Alpesh Patel wrote:
   Hi Jeffrey,
  
   I found your name from lwn.net posting.
  
   I am trying to use the X710 nic and am using the i40e driver
  
   Is that the right driver for that nic?
 
  Yes.  I have added e1000-devel mailing list (as well as a couple of
  i40e
  developers) to help answer your questions.
 
   I see a few dmesg logs (curtailed):
  
   on loading the driver (i40e):
   i40e: Intel(R) Ethernet Connection XL710 Network Driver - version
   0.4.10-k
   i40e: Copyright (c) 2013 - 2014 Intel Corporation.
   i40e :05:00.0: f4.22 a1.1 n04.26 e8000152d i40e :05:00.0:
   MAC
   address: 68:05:ca:32:fd:78 i40e :05:00.0: AQ Querying DCB
   configuration failed: 1 i40e :05:00.0: init_pf_dcb failed: -53
   i40e :05:00.0: irq 188 for MSI/MSI-X ...
   i40e :05:00.0: irq 233 for MSI/MSI-X i40e :05:00.0: Could
   not remove default MAC-VLAN i40e :05:00.0: i40e_ptp_init: added
   PHC
  on
   eth18 i40e :05:00.0: PCI-Express: Speed 8.0GT/s Width x8 i40e
   :05:00.0: Features: PF-id[0] VFs: 32 VSIs: 34 QP: 28 RSS FD_ATR
   FD_SB NTUPLE PTP i40e :05:00.1: f4.22 a1.1 n04.26 e8000152d i40e
   :05:00.1: MAC address: 68:05:ca:32:fd:79 i40e :05:00.1: AQ
   Querying DCB configuration failed: 1 i40e :05:00.1: init_pf_dcb
   failed: -53 i40e :05:00.1: irq 234 for MSI/MSI-X ...
  
  
   and this happens for each of the 4 ports on this nic. I understand
   that this nic uses fortville chip - with 4x10g ports on the nic.
  
   When I connect the port to an ixia interface, the led on ixia is not
   up (it is not lighted with any color)
  
   I manually assigned an ip and brought the interface up - it stays up
   and I am able to ping it, but that's just loopback interface ping.
  
   are the failures/errors in the log above harmless?
  
   any documentation on the driver/parameters? Any configuration or
  other
   information that you can point me to would be helpful.
  
   mii-tool on these interfaces do not work
   [root@ott-ss-dt-14 :05:00.0]# mii-tool eth18 SIOCGMIIPHY on
   'eth18' failed: Operation not supported
  
   any pointers you can provide?
 
  --
  -
  ---
  ___
  E1000-devel mailing list
  E1000-devel@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/e1000-devel
  To learn more about Intel#174; Ethernet, visit
  http://communities.intel.com/community/wired
 
 --
 
 ___
 E1000-devel mailing list
 E1000-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/e1000-devel
 To learn more about Intel#174; Ethernet, visit
 http://communities.intel.com/community/wired

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

   -Original Message-
   From: Stefan Priebe [mailto:s.pri...@profihost.ag]
   Sent: Thursday, August 13, 2015 12:02 PM
   To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
   Subject: Re: [E1000-devel] dropped rx with i40e
  
   Am 13.08.2015 um 20:59 schrieb Rose, Gregory V:
Thanks Stefan, I'll set up a test to replicate your traffic
profile as
   closely as possible and let it run overnight to see if I can repro
   and then update you tomorrow.
   
It does seem that it has nothing to do with load so that makes it
even
   more curious.
  
   May it be related to jumbo frames?

Stefan,

I've got the test up and running now.  Here's the interface config:

6: bond0: BROADCAST,MULTICAST,MASTER,UP,LOWER_UP mtu 9000 qdisc noqueue state 
UP
link/ether 68:05:ca:2f:83:10 brd ff:ff:ff:ff:ff:ff
inet 200.0.0.10/24 brd 200.0.0.255 scope global bond0
   valid_lft forever preferred_lft forever
inet6 fe80::6a05:caff:fe2f:8310/64 scope link
   valid_lft forever preferred_lft forever
9: p4p1: BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP mtu 9000 qdisc mq master bond0 
state UP qlen 1000
link/ether 68:05:ca:2f:83:10 brd ff:ff:ff:ff:ff:ff
inet6 fe80::6a05:caff:fe2f:8310/64 scope link
   valid_lft forever preferred_lft forever
12: p4p4: BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP mtu 9000 qdisc mq master 
bond0 state UP qlen 1000
link/ether 68:05:ca:2f:83:10 brd ff:ff:ff:ff:ff:ff
inet6 fe80::6a05:caff:fe2f:8310/64 scope link
   valid_lft forever preferred_lft forever

p4p1 and p4p4 are the i40e interfaces bonded to the bond0 LACP interface.

Here's the traffic results from a transmitter:

Interim result: 9899.84 10^6bits/s over 5.001 seconds ending at 1439847706.406
Interim result: 9900.15 10^6bits/s over 5.001 seconds ending at 1439847711.407

I have a script watching the dropped packets for the two slaved interfaces p4p1 
and p4p4:

Every 1.0s: ./t1Mon Aug 17 14:43:31 2015

 rx_dropped: 0
 tx_dropped: 0
 rx_fcoe_dropped: 0
 port.rx_dropped: 0
 port.tx_dropped_link_down: 0
 rx_dropped: 0
 tx_dropped: 0
 rx_fcoe_dropped: 0
 port.rx_dropped: 0
 port.tx_dropped_link_down: 0

I'll let this run overnight and get back to you with results tomorrow.

Regards,

- Greg

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

 -Original Message-
 From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
 Sent: Wednesday, August 19, 2015 12:01 AM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Hi,
 
 sad you are not able to reproduce. The good thing i can't reproduce myself
 either ;-(
 
 it just happens out of nothing on the nodes. My current expection is that
 it happens when spikes of packets occur after being idle for some time.
 

OK, my traffic generation was fairly constant.  Let me modify it to send bursts 
of traffic after long idle periods.  Maybe that will help to reproduce.

Something you might try is examining your systems' BIOS settings for sleep 
states and make sure the machines don't go too deep into a sleep state when 
they're idle.  The time it takes to come from sleep state to handling traffic 
bursts can cause some dropped packets.

Thanks,

- Greg

 The good news is after upgrading to the latest intel fw released two days
 ago and to the latest 1.3.38 driver - it works on 10 out of my 18 testing
 hosts.
 
 Currently i've no idea why it does not on those 8.
 
 Stefan
 Am 19.08.2015 um 00:24 schrieb Rose, Gregory V:
 
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Tuesday, August 18, 2015 12:28 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Hi Greg,
 
  could you tell me the output of ethtool -i and ethtool -a and ethtool
  -c and ethtool -k?
 
  OK, I pasted it in below.  I ran traffic overnight and there were no
 dropped packets or other errors.  Everything seemed fine.
 
  - Greg
 
  [root@paelab-gvrose ~]# ethtool -i bond0
  driver: bonding
  version: 3.7.1
  firmware-version: 2
  bus-info:
  supports-statistics: no
  supports-test: no
  supports-eeprom-access: no
  supports-register-dump: no
  supports-priv-flags: no
  [root@paelab-gvrose ~]# ethtool -c bond0 Coalesce parameters for
  bond0:
  Cannot get device coalesce settings: Operation not supported
  [root@paelab-gvrose ~]# ethtool -k bond0 Features for bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
  tx-checksum-ipv4: off [fixed]
  tx-checksum-ip-generic: on
  tx-checksum-ipv6: off [fixed]
  tx-checksum-fcoe-crc: off [fixed]
  tx-checksum-sctp: off [fixed]
  scatter-gather: on
  tx-scatter-gather: on
  tx-scatter-gather-fraglist: off [requested on]
  tcp-segmentation-offload: on
  tx-tcp-segmentation: on
  tx-tcp-ecn-segmentation: on
  tx-tcp6-segmentation: on
  udp-fragmentation-offload: off [fixed]
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: off [fixed]
  tx-ipip-segmentation: off [fixed]
  tx-sit-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-mpls-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: off [fixed]
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  busy-poll: off [fixed]
  [root@paelab-gvrose ~]# ethtool -i p4p1
  driver: i40e
  version: 1.2.47
  firmware-version: f4.40.35115 a1.4 n4.53 e1ce7
  bus-info: :82:00.0
  supports-statistics: yes
  supports-test: yes
  supports-eeprom-access: yes
  supports-register-dump: yes
  supports-priv-flags: yes
  [root@paelab-gvrose ~]# ethtool -c p4p1 Coalesce parameters for p4p1:
  Adaptive RX: on  TX: on
  stats-block-usecs: 0
  sample-interval: 0
  pkt-rate-low: 0
  pkt-rate-high: 0
 
  rx-usecs: 62
  rx-frames: 0
  rx-usecs-irq: 0
  rx-frames-irq: 256
 
  tx-usecs: 122
  tx-frames: 0
  tx-usecs-irq: 0
  tx-frames-irq: 256
 
  rx-usecs-low: 0
  rx-frame-low: 0
  tx-usecs-low: 0
  tx-frame-low: 0
 
  rx-usecs-high: 0
  rx-frame-high: 0
  tx-usecs-high: 0
  tx-frame-high: 0
 
  [root@paelab-gvrose ~]# ethtool -k p4p1 Features for p4p1:
  rx-checksumming: on
  tx-checksumming: on
  tx-checksum-ipv4: on
  tx-checksum-ip-generic: off [fixed]
  tx-checksum-ipv6: on
  tx-checksum-fcoe-crc: off [fixed]
  tx-checksum-sctp: on
  scatter-gather: on
  tx-scatter-gather: on
  tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
  tx-tcp-segmentation: on
  tx-tcp-ecn-segmentation: on
  tx-tcp6-segmentation: on
  udp-fragmentation-offload: off [fixed]
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off [fixed]
  rx-vlan-offload: on
  tx-vlan

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

 -Original Message-
 From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
 Sent: Tuesday, August 18, 2015 12:28 AM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Hi Greg,
 
 could you tell me the output of ethtool -i and ethtool -a and ethtool -c
 and ethtool -k?

OK, I pasted it in below.  I ran traffic overnight and there were no dropped 
packets or other errors.  Everything seemed fine.

- Greg

[root@paelab-gvrose ~]# ethtool -i bond0
driver: bonding
version: 3.7.1
firmware-version: 2
bus-info:
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
[root@paelab-gvrose ~]# ethtool -c bond0
Coalesce parameters for bond0:
Cannot get device coalesce settings: Operation not supported
[root@paelab-gvrose ~]# ethtool -k bond0
Features for bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [requested on]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
[root@paelab-gvrose ~]# ethtool -i p4p1
driver: i40e
version: 1.2.47
firmware-version: f4.40.35115 a1.4 n4.53 e1ce7
bus-info: :82:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[root@paelab-gvrose ~]# ethtool -c p4p1
Coalesce parameters for p4p1:
Adaptive RX: on  TX: on
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 62
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 256

tx-usecs: 122
tx-frames: 0
tx-usecs-irq: 0
tx-frames-irq: 256

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0

[root@paelab-gvrose ~]# ethtool -k p4p1
Features for p4p1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: on
receive-hashing: on
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
[root@paelab-gvrose ~]# ethtool -i p4p4
driver: i40e
version: 1.2.47
firmware-version: f4.40.35115 a1.4 n4.53 e1ce7
bus-info: :82:00.3
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[root@paelab-gvrose ~]# ethtool -c p4p4
Coalesce parameters for p4p4:
Adaptive RX: on  TX: on
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 62
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 256

tx-usecs: 122
tx-frames: 0
tx-usecs-irq: 0
tx-frames-irq: 256

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0

[root@paelab-gvrose ~]# ethtool -k p4p4
Features for p4p4:
rx-checksumming: on
tx-checksumming: on
tx-checksum

Re: [E1000-devel] i40e - Unqualified modules was detected

[Rose, Gregory V]

 -Original Message-
 From: John McDowall [mailto:jmcdow...@paloaltonetworks.com]
 Sent: Friday, August 21, 2015 1:44 PM
 To: e1000-devel@lists.sourceforge.net
 Subject: [E1000-devel] i40e - Unqualified modules was detected
 
 Hi,
 
 I am trying to get a dual 10G X7100 interface up and on boot I am seeing
 the following message:
 

You'll need to use Intel optics - other optical modules are not supported by 
our drivers.

I would go back to whoever sold you the adapter and ask for approved Intel 
optical modules for your adapter.

- Greg

 
 i40e :04:00.0 p1p1: the driver failed to link because an unqualified
 module was detected.
 
 [5.092641] IPv6: ADDRCONF(NETDEV_UP): p1p1: link is not ready
 
 [7.875628] i40e :04:00.1 p1p2: the driver failed to link because
 an unqualified module was detected.
 
 [7.875703] IPv6: ADDRCONF(NETDEV_UP): p1p2: link is not ready
 
 My system is a Dell R610, running CENTOS 7.0 I have upgraded the drivers
 and the flash:
 
 
 [root@localhost ~]# uname -a
 
 Linux localhost.localdomain 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6
 01:06:18 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
 
 [root@localhost ~]# ethtool -i p1p1
 
 driver: i40e
 
 version: 1.3.38
 
 firmware-version: 4.53 0x80001dc0 0.0.0
 
 bus-info: :04:00.0
 
 supports-statistics: yes
 
 supports-test: yes
 
 supports-eeprom-access: yes
 
 supports-register-dump: yes
 
 supports-priv-flags: yes
 
 [root@localhost ~]#
 
 Any ideas of what could be wrong?
 
 Regards
 
 John McDowall
 
 
 


--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

The only cards out in the field should all show a PCI revision ID value of 1.  
If you have any cards that show 0 then they need to be replaced.  But again, 
you shouldn’t have any A0 silicon cards.


-Greg

From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
Sent: Monday, August 24, 2015 10:16 AM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Are there different hw revisions? Currently it seems cards before q4 2014 have 
this problem cards build in 2015 work fine.

Stefan

Excuse my typo sent from my mobile phone.

Am 21.08.2015 um 02:51 schrieb Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com:
Stefan,

Late update on this.  I'm being told know that there are additional parameters 
that may need to be changed.  Please hold until I get this updated information.

Thanks and regards,

- Greg


-Original Message-
From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
Sent: Thursday, August 20, 2015 4:26 PM
To: Stefan Priebe
Cc: e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Stefan,

You will need to run the update utility again but this time edit the
nvmupdate.cfg file to change this:

SKIP OROM: TRUE

to this:

SKIP OROM: FALSE

That should then also update the option ROM for the PXE utility.

Regards,

- Greg

-Original Message-
From: Stefan Priebe [mailto:s.pri...@profihost.ag]
Sent: Wednesday, August 19, 2015 10:40 AM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Correct message:
PXE-m10: The application for the device detected a newer version of
the nvm image than expexted.

So how do i update the application code of the XL710?

Stefan
Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
An additional question since I've updated to latest xl710 firmware
released 17 August the cards show at boot time (bios init/ post)
something like application code detected a newer nvm image than
expected please update application software. But I can't find an
application code update for the xl710 cards.

Stefan

Excuse my typo sent from my mobile phone.

Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com
mailto:gregory.v.r...@intel.com:


-Original Message-
From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
Sent: Wednesday, August 19, 2015 12:01 AM
To: Rose, Gregory V; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
mailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Hi,

sad you are not able to reproduce. The good thing i can't
reproduce myself either ;-(

it just happens out of nothing on the nodes. My current expection
is that it happens when spikes of packets occur after being idle
for some
time.


OK, my traffic generation was fairly constant.  Let me modify it to
send bursts of traffic after long idle periods.  Maybe that will
help to reproduce.

Something you might try is examining your systems' BIOS settings
for sleep states and make sure the machines don't go too deep into
a sleep state when they're idle.  The time it takes to come from
sleep state to handling traffic bursts can cause some dropped
packets.

Thanks,

- Greg

The good news is after upgrading to the latest intel fw released
two days ago and to the latest 1.3.38 driver - it works on 10 out
of my
18 testing hosts.

Currently i've no idea why it does not on those 8.

Stefan
Am 19.08.2015 um 00:24 schrieb Rose, Gregory V:


-Original Message-
From: Stefan Priebe - Profihost AG
[mailto:s.pri...@profihost.ag]
Sent: Tuesday, August 18, 2015 12:28 AM
To: Rose, Gregory V; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
mailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Hi Greg,

could you tell me the output of ethtool -i and ethtool -a and
ethtool -c and ethtool -k?

OK, I pasted it in below.  I ran traffic overnight and there were
no
dropped packets or other errors.  Everything seemed fine.

- Greg

[root@paelab-gvrose ~]# ethtool -i bond0
driver: bonding
version: 3.7.1
firmware-version: 2
bus-info:
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
[root@paelab-gvrose ~]# ethtool -c bond0 Coalesce parameters for
bond0:
Cannot get device coalesce settings: Operation not supported
[root@paelab-gvrose ~]# ethtool -k bond0 Features for bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
  tx-checksum-ipv4: off [fixed]
  tx-checksum-ip-generic: on
  tx-checksum-ipv6: off [fixed]
  tx-checksum-fcoe-crc: off [fixed]
  tx-checksum-sctp: off [fixed]
scatter-gather: on
  tx-scatter-gather: on
  tx-scatter-gather-fraglist: off [requested on]
tcp

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

Stefan,

Unfortunately I haven't been able to get back and look at the issue with 
dropped packets in the last few days.  I do have an update for issue with PXE 
application detecting a newer version of firmware.  It turns out the easiest 
way to do it is with the bootutil tool.

It is available here:

http://www.intel.com/support/network/adapter/pro100/bootagent/sb/cs-008212.htm?wapkw=bootutil

Thanks,

- Greg


 -Original Message-
 From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 Sent: Thursday, August 20, 2015 11:07 PM
 To: Rose, Gregory V
 Cc: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Thanks, will wait or your next mail. Any news regarding the dropped
 packets?
 
 Stefan
 Am 21.08.2015 um 02:51 schrieb Rose, Gregory V:
  Stefan,
 
  Late update on this.  I'm being told know that there are additional
 parameters that may need to be changed.  Please hold until I get this
 updated information.
 
  Thanks and regards,
 
  - Greg
 
  -Original Message-
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
  Sent: Thursday, August 20, 2015 4:26 PM
  To: Stefan Priebe
  Cc: e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Stefan,
 
  You will need to run the update utility again but this time edit the
  nvmupdate.cfg file to change this:
 
  SKIP OROM: TRUE
 
  to this:
 
  SKIP OROM: FALSE
 
  That should then also update the option ROM for the PXE utility.
 
  Regards,
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 19, 2015 10:40 AM
  To: Rose, Gregory V
  Cc: e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Correct message:
  PXE-m10: The application for the device detected a newer version of
  the nvm image than expexted.
 
  So how do i update the application code of the XL710?
 
  Stefan
  Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
  An additional question since I've updated to latest xl710 firmware
  released 17 August the cards show at boot time (bios init/ post)
  something like application code detected a newer nvm image than
  expected please update application software. But I can't find an
  application code update for the xl710 cards.
 
  Stefan
 
  Excuse my typo sent from my mobile phone.
 
  Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
  gregory.v.r...@intel.com
  mailto:gregory.v.r...@intel.com:
 
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 19, 2015 12:01 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Hi,
 
  sad you are not able to reproduce. The good thing i can't
  reproduce myself either ;-(
 
  it just happens out of nothing on the nodes. My current expection
  is that it happens when spikes of packets occur after being idle
  for some
  time.
 
 
  OK, my traffic generation was fairly constant.  Let me modify it
  to send bursts of traffic after long idle periods.  Maybe that
  will help to reproduce.
 
  Something you might try is examining your systems' BIOS settings
  for sleep states and make sure the machines don't go too deep into
  a sleep state when they're idle.  The time it takes to come from
  sleep state to handling traffic bursts can cause some dropped
  packets.
 
  Thanks,
 
  - Greg
 
  The good news is after upgrading to the latest intel fw released
  two days ago and to the latest 1.3.38 driver - it works on 10 out
  of my
  18 testing hosts.
 
  Currently i've no idea why it does not on those 8.
 
  Stefan
  Am 19.08.2015 um 00:24 schrieb Rose, Gregory V:
 
 
  -Original Message-
  From: Stefan Priebe - Profihost AG
  [mailto:s.pri...@profihost.ag]
  Sent: Tuesday, August 18, 2015 12:28 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Hi Greg,
 
  could you tell me the output of ethtool -i and ethtool -a and
  ethtool -c and ethtool -k?
 
  OK, I pasted it in below.  I ran traffic overnight and there
  were no
  dropped packets or other errors.  Everything seemed fine.
 
  - Greg
 
  [root@paelab-gvrose ~]# ethtool -i bond0
  driver: bonding
  version: 3.7.1
  firmware-version: 2
  bus-info:
  supports-statistics: no
  supports-test: no
  supports-eeprom-access: no
  supports-register-dump: no
  supports-priv-flags: no
  [root@paelab-gvrose ~]# ethtool -c bond0 Coalesce parameters for
  bond0:
  Cannot get device coalesce settings: Operation not supported
  [root@paelab-gvrose ~]# ethtool -k bond0 Features for bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
  tx-checksum-ipv4: off [fixed]
  tx-checksum-ip-generic: on
  tx-checksum-ipv6: off [fixed]
  tx-checksum-fcoe-crc: off [fixed

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

So if you swap one of the newer cards with one of the older cards do the packet 
drops continue to follow the older card around?

If so, then please get me the PBA of one of the newer cards and the PBA of one 
of the older ones that fails.

Thanks,


-Greg

From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
Sent: Saturday, August 22, 2015 12:14 AM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

I still have those ugly drops. Generally it seems it takes 20-48 hours before 
they occour but than it doesn't stop.

The strange thing I've 8 newer boxes using same os and hw where I've never seen 
those drops. Are there different xl 710 hw revisions? I already checked hw 
revision of mainboard.

Stefan

Excuse my typo sent from my mobile phone.

Am 21.08.2015 um 02:51 schrieb Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com:
Stefan,

Late update on this.  I'm being told know that there are additional parameters 
that may need to be changed.  Please hold until I get this updated information.

Thanks and regards,

- Greg


-Original Message-
From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
Sent: Thursday, August 20, 2015 4:26 PM
To: Stefan Priebe
Cc: e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Stefan,

You will need to run the update utility again but this time edit the
nvmupdate.cfg file to change this:

SKIP OROM: TRUE

to this:

SKIP OROM: FALSE

That should then also update the option ROM for the PXE utility.

Regards,

- Greg

-Original Message-
From: Stefan Priebe [mailto:s.pri...@profihost.ag]
Sent: Wednesday, August 19, 2015 10:40 AM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Correct message:
PXE-m10: The application for the device detected a newer version of
the nvm image than expexted.

So how do i update the application code of the XL710?

Stefan
Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
An additional question since I've updated to latest xl710 firmware
released 17 August the cards show at boot time (bios init/ post)
something like application code detected a newer nvm image than
expected please update application software. But I can't find an
application code update for the xl710 cards.

Stefan

Excuse my typo sent from my mobile phone.

Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com
mailto:gregory.v.r...@intel.com:


-Original Message-
From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
Sent: Wednesday, August 19, 2015 12:01 AM
To: Rose, Gregory V; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
mailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Hi,

sad you are not able to reproduce. The good thing i can't
reproduce myself either ;-(

it just happens out of nothing on the nodes. My current expection
is that it happens when spikes of packets occur after being idle
for some
time.


OK, my traffic generation was fairly constant.  Let me modify it to
send bursts of traffic after long idle periods.  Maybe that will
help to reproduce.

Something you might try is examining your systems' BIOS settings
for sleep states and make sure the machines don't go too deep into
a sleep state when they're idle.  The time it takes to come from
sleep state to handling traffic bursts can cause some dropped
packets.

Thanks,

- Greg

The good news is after upgrading to the latest intel fw released
two days ago and to the latest 1.3.38 driver - it works on 10 out
of my
18 testing hosts.

Currently i've no idea why it does not on those 8.

Stefan
Am 19.08.2015 um 00:24 schrieb Rose, Gregory V:


-Original Message-
From: Stefan Priebe - Profihost AG
[mailto:s.pri...@profihost.ag]
Sent: Tuesday, August 18, 2015 12:28 AM
To: Rose, Gregory V; 
e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
mailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] dropped rx with i40e

Hi Greg,

could you tell me the output of ethtool -i and ethtool -a and
ethtool -c and ethtool -k?

OK, I pasted it in below.  I ran traffic overnight and there were
no
dropped packets or other errors.  Everything seemed fine.

- Greg

[root@paelab-gvrose ~]# ethtool -i bond0
driver: bonding
version: 3.7.1
firmware-version: 2
bus-info:
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
[root@paelab-gvrose ~]# ethtool -c bond0 Coalesce parameters for
bond0:
Cannot get device coalesce settings: Operation not supported
[root@paelab-gvrose ~]# ethtool -k bond0 Features for bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
  tx-checksum-ipv4: off [fixed]
  tx-checksum-ip-generic

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

Stefan,

You will need to run the update utility again but this time edit the 
nvmupdate.cfg file to change this:

SKIP OROM: TRUE

to this:

SKIP OROM: FALSE

That should then also update the option ROM for the PXE utility.

Regards,

- Greg

 -Original Message-
 From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 Sent: Wednesday, August 19, 2015 10:40 AM
 To: Rose, Gregory V
 Cc: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Correct message:
 PXE-m10: The application for the device detected a newer version of the
 nvm image than expexted.
 
 So how do i update the application code of the XL710?
 
 Stefan
 Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
  An additional question since I've updated to latest xl710 firmware
  released 17 August the cards show at boot time (bios init/ post)
  something like application code detected a newer nvm image than
  expected please update application software. But I can't find an
  application code update for the xl710 cards.
 
  Stefan
 
  Excuse my typo sent from my mobile phone.
 
  Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
  gregory.v.r...@intel.com
  mailto:gregory.v.r...@intel.com:
 
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 19, 2015 12:01 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Hi,
 
  sad you are not able to reproduce. The good thing i can't reproduce
  myself either ;-(
 
  it just happens out of nothing on the nodes. My current expection is
  that it happens when spikes of packets occur after being idle for some
 time.
 
 
  OK, my traffic generation was fairly constant.  Let me modify it to
  send bursts of traffic after long idle periods.  Maybe that will help
  to reproduce.
 
  Something you might try is examining your systems' BIOS settings for
  sleep states and make sure the machines don't go too deep into a
  sleep state when they're idle.  The time it takes to come from sleep
  state to handling traffic bursts can cause some dropped packets.
 
  Thanks,
 
  - Greg
 
  The good news is after upgrading to the latest intel fw released two
  days ago and to the latest 1.3.38 driver - it works on 10 out of my
  18 testing hosts.
 
  Currently i've no idea why it does not on those 8.
 
  Stefan
  Am 19.08.2015 um 00:24 schrieb Rose, Gregory V:
 
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Tuesday, August 18, 2015 12:28 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Hi Greg,
 
  could you tell me the output of ethtool -i and ethtool -a and
  ethtool -c and ethtool -k?
 
  OK, I pasted it in below.  I ran traffic overnight and there were
  no
  dropped packets or other errors.  Everything seemed fine.
 
  - Greg
 
  [root@paelab-gvrose ~]# ethtool -i bond0
  driver: bonding
  version: 3.7.1
  firmware-version: 2
  bus-info:
  supports-statistics: no
  supports-test: no
  supports-eeprom-access: no
  supports-register-dump: no
  supports-priv-flags: no
  [root@paelab-gvrose ~]# ethtool -c bond0 Coalesce parameters for
  bond0:
  Cannot get device coalesce settings: Operation not supported
  [root@paelab-gvrose ~]# ethtool -k bond0 Features for bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
 tx-checksum-ipv4: off [fixed]
 tx-checksum-ip-generic: on
 tx-checksum-ipv6: off [fixed]
 tx-checksum-fcoe-crc: off [fixed]
 tx-checksum-sctp: off [fixed]
  scatter-gather: on
 tx-scatter-gather: on
 tx-scatter-gather-fraglist: off [requested on]
  tcp-segmentation-offload: on
 tx-tcp-segmentation: on
 tx-tcp-ecn-segmentation: on
 tx-tcp6-segmentation: on
  udp-fragmentation-offload: off [fixed]
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: off [fixed]
  tx-ipip-segmentation: off [fixed]
  tx-sit-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-mpls-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: off [fixed]
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  busy-poll: off [fixed]
  [root@paelab-gvrose ~]# ethtool -i p4p1
  driver: i40e
  version: 1.2.47
  firmware-version: f4.40.35115 a1.4 n4.53 e1ce7
  bus-info

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

Stefan,

Late update on this.  I'm being told know that there are additional parameters 
that may need to be changed.  Please hold until I get this updated information.

Thanks and regards,

- Greg

 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: Thursday, August 20, 2015 4:26 PM
 To: Stefan Priebe
 Cc: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Stefan,
 
 You will need to run the update utility again but this time edit the
 nvmupdate.cfg file to change this:
 
 SKIP OROM: TRUE
 
 to this:
 
 SKIP OROM: FALSE
 
 That should then also update the option ROM for the PXE utility.
 
 Regards,
 
 - Greg
 
  -Original Message-
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 19, 2015 10:40 AM
  To: Rose, Gregory V
  Cc: e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Correct message:
  PXE-m10: The application for the device detected a newer version of
  the nvm image than expexted.
 
  So how do i update the application code of the XL710?
 
  Stefan
  Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
   An additional question since I've updated to latest xl710 firmware
   released 17 August the cards show at boot time (bios init/ post)
   something like application code detected a newer nvm image than
   expected please update application software. But I can't find an
   application code update for the xl710 cards.
  
   Stefan
  
   Excuse my typo sent from my mobile phone.
  
   Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
   gregory.v.r...@intel.com
   mailto:gregory.v.r...@intel.com:
  
  
   -Original Message-
   From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
   Sent: Wednesday, August 19, 2015 12:01 AM
   To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
   mailto:e1000-devel@lists.sourceforge.net
   Subject: Re: [E1000-devel] dropped rx with i40e
  
   Hi,
  
   sad you are not able to reproduce. The good thing i can't
   reproduce myself either ;-(
  
   it just happens out of nothing on the nodes. My current expection
   is that it happens when spikes of packets occur after being idle
   for some
  time.
  
  
   OK, my traffic generation was fairly constant.  Let me modify it to
   send bursts of traffic after long idle periods.  Maybe that will
   help to reproduce.
  
   Something you might try is examining your systems' BIOS settings
   for sleep states and make sure the machines don't go too deep into
   a sleep state when they're idle.  The time it takes to come from
   sleep state to handling traffic bursts can cause some dropped
 packets.
  
   Thanks,
  
   - Greg
  
   The good news is after upgrading to the latest intel fw released
   two days ago and to the latest 1.3.38 driver - it works on 10 out
   of my
   18 testing hosts.
  
   Currently i've no idea why it does not on those 8.
  
   Stefan
   Am 19.08.2015 um 00:24 schrieb Rose, Gregory V:
  
  
   -Original Message-
   From: Stefan Priebe - Profihost AG
   [mailto:s.pri...@profihost.ag]
   Sent: Tuesday, August 18, 2015 12:28 AM
   To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
   mailto:e1000-devel@lists.sourceforge.net
   Subject: Re: [E1000-devel] dropped rx with i40e
  
   Hi Greg,
  
   could you tell me the output of ethtool -i and ethtool -a and
   ethtool -c and ethtool -k?
  
   OK, I pasted it in below.  I ran traffic overnight and there were
   no
   dropped packets or other errors.  Everything seemed fine.
  
   - Greg
  
   [root@paelab-gvrose ~]# ethtool -i bond0
   driver: bonding
   version: 3.7.1
   firmware-version: 2
   bus-info:
   supports-statistics: no
   supports-test: no
   supports-eeprom-access: no
   supports-register-dump: no
   supports-priv-flags: no
   [root@paelab-gvrose ~]# ethtool -c bond0 Coalesce parameters for
   bond0:
   Cannot get device coalesce settings: Operation not supported
   [root@paelab-gvrose ~]# ethtool -k bond0 Features for bond0:
   rx-checksumming: off [fixed]
   tx-checksumming: on
  tx-checksum-ipv4: off [fixed]
  tx-checksum-ip-generic: on
  tx-checksum-ipv6: off [fixed]
  tx-checksum-fcoe-crc: off [fixed]
  tx-checksum-sctp: off [fixed]
   scatter-gather: on
  tx-scatter-gather: on
  tx-scatter-gather-fraglist: off [requested on]
   tcp-segmentation-offload: on
  tx-tcp-segmentation: on
  tx-tcp-ecn-segmentation: on
  tx-tcp6-segmentation: on
   udp-fragmentation-offload: off [fixed]
   generic-segmentation-offload: on
   generic-receive-offload: on
   large-receive-offload: off
   rx-vlan-offload: on
   tx-vlan-offload: on
   ntuple-filters: off [fixed]
   receive-hashing: off [fixed]
   highdma: on
   rx-vlan-filter: on
   vlan-challenged: off [fixed]
   tx-lockless: on [fixed]
   netns-local: on [fixed]
   tx-gso-robust: off [fixed]
   tx-fcoe-segmentation: off [fixed

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

 -Original Message-
 From: Christian Ruppert [mailto:id...@qasl.de]
 Sent: Tuesday, July 28, 2015 5:43 AM
 To: Rose, Gregory V
 Cc: e1000-de...@lists.sf.net
 Subject: RE: [E1000-devel] X710 / i40e interface resets
 
 Greg,
 
 On 2015-07-27 18:20, Christian Ruppert wrote:
  On 2015-07-27 18:02, Rose, Gregory V wrote:
  So to make sure I've got your report right... Since updating the FW
  the link flap issue occurs much less frequently and VRRP is not
  causing a switch to a different host but your log is getting spammed
  with ATR messages.  Is that correct?
 
  Exactly!
 
 
  Have you tried turning off ntuple-filters?
 
  I'll try that.
 
 No more:
 [341397.489477] i40e :02:00.0: FD filter space full, new ntuple rules
 will not be added [341398.041918] i40e :02:00.0: FD Filter table
 flushed and FD-SB replayed.
 [341398.049403] i40e :02:00.0: FD Sideband/ntuple is being enabled
 since we have space in the table now [341398.059858] i40e :02:00.0:
 ATR is being enabled since we have space in the table now [341850.732707]
 i40e :02:00.0: ATR re-enabled.
 
 since I disabled ntuple. So that is gone (not fixed though) for now.

OK - I think the messages you're seeing are part of normal operation but they 
are quite noisy.  I know the i40e dev team is trying to reduce the amount of 
message spam to the system log so I'll point this out to them and perhaps 
something can be done to reduce the message volume.

 
 The flaps are still persistent:
 [354195.790717] i40e :02:00.0: TX driver issue detected, PF reset
 issued [354196.551366] i40e :02:00.0: FCoE capability is disabled
 [354196.593458] i40e :02:00.0: PHC enabled [354196.653574] i40e
 :02:00.0: enabling bridge mode: VEPA [354196.672095] i40e :02:00.0
 eth0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None

I've been told that the Tx driver issue detected message is due to a MDD 
(Malicious Driver Detection) event.  We need to find out what is causing that 
so if you could please send me your system log and the output of ethtool intf 
and ethtool -i intf perhaps I can see what might be occurring.  If not then 
we'll have to debug further.

I'll be out the next few days but returning Monday so I'll be able to follow up 
on this issue then.

Thanks,

- Greg


--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

Thanks Stefan.  I think for now you've given us enough data to go on - I've got 
some research to do and then I'll get back to you.

- Greg

 -Original Message-
 From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
 Sent: Wednesday, August 05, 2015 11:32 PM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Am 06.08.2015 um 00:22 schrieb Rose, Gregory V:
  Stefan,
 
  Could you please send me the output of 'ethtool' and 'ethtool -i' for
 each i40e interface that is experiencing the dropped packets issue?
 
 These are around 100 cards. So i won't post the output for all of them.
 As they're all using the same driver and the same firmware - we updated
 all of them i hope it's ok to post the output only from one of them.
 
 # ethtool eth2
 Settings for eth2:
 Supported ports: [ FIBRE ]
 Supported link modes:   1baseT/Full
 Supported pause frame use: Symmetric
 Supports auto-negotiation: No
 Advertised link modes:  Not reported
 Advertised pause frame use: No
 Advertised auto-negotiation: No
 Speed: 1Mb/s
 Duplex: Full
 Port: Direct Attach Copper
 PHYAD: 0
 Transceiver: external
 Auto-negotiation: off
 Supports Wake-on: g
 Wake-on: d
 Current message level: 0x000f (15)
drv probe link timer
 Link detected: yes
 # ethtool -i eth2
 driver: i40e
 version: 1.3.4-k
 firmware-version: f4.33.31377 a1.2 n4.42 e191b
 bus-info: :03:00.0
 supports-statistics: yes
 supports-test: yes
 supports-eeprom-access: yes
 supports-register-dump: yes
 supports-priv-flags: yes
 
   Also, the system log might help also - dmesg can get that.  That'll
 give me something to look at.
 
 As this one is pretty long. i pasted dmesg to pastebin:
 http://pastebin.com/raw.php?i=7Tjp3eDT
 
  By the way, have you tried using ethtool to turn adaptive RX and TX off
 using ethtool to see if that has any impact on the dropped packets?
 
 I tested this one:
 ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0
 
 but it has not helped. Still dropped rx packets. While a 2nd system
 receiving the same load using ixgbe has no dropped packets.
 
  That might be an easy test to run.
 
 Thanks!
 
 Greets,
 Stefan
 
 
  Thanks,
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 05, 2015 11:14 AM
  To: e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
 
  Something i've noticed:
  ixgbe:
  Adaptive RX: off  TX: off
  rx-usecs: 1
  tx-usecs: 0
 
  i40e:
  Adaptive RX: on  TX: on
  rx-usecs: 62
  tx-usecs: 122
 
  Stefan
 
  Am 05.08.2015 um 09:02 schrieb Stefan Priebe - Profihost AG:
  Hello list,
 
  we're using the intel X520 cards with the ixgbe driver since a long
  time for our cloud infrastructure. We never had a problem with
  dropped packets and everything was always fine.
 
  Since a year we started switching to the X710 cards as they're
  better regarding their specs (lower power consumption, lower
  latency, better price).
 
  We've around 100 X710 cards running now and we had a lot of trouble
  with them. Back in 2014 there were a firmware bug, then there were
  driver problems with bonding and so on.
 
  Now we have detected a new problem! We're seeing a lot of rx_dropped
  packets on all X710 cards while all ixgbe based cards are working
 fine.
 
  I've tested the 1.2.48 driver als also the latest 1.3.4-k driver
  from 4.2-rc5.
 
  Can anybody help?
 
  Greets,
  Stefan
 
 
  -
  -
  
  ___
  E1000-devel mailing list
  E1000-devel@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/e1000-devel
  To learn more about Intel#174; Ethernet, visit
  http://communities.intel.com/community/wired

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

Stefan,

I've run into a glitch while in the process of configuring a setup for repo - 
we need to do some reconfig in our lab.  I've put in the appropriate lab 
request and they're usually pretty good about quick response but I may not get 
the test running tonight.  If not I'll get it started ASAP.

Thanks for your patience.

- Greg

 -Original Message-
 From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 Sent: Thursday, August 13, 2015 1:16 PM
 To: Stefan Priebe; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
  -Original Message-
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
  Sent: Thursday, August 13, 2015 12:02 PM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Am 13.08.2015 um 20:59 schrieb Rose, Gregory V:
   Thanks Stefan, I'll set up a test to replicate your traffic profile
   as
  closely as possible and let it run overnight to see if I can repro and
  then update you tomorrow.
  
   It does seem that it has nothing to do with load so that makes it
   even
  more curious.
 
  May it be related to jumbo frames?
 
 Could be - I'll be looking at that angle during my testing and evaluation.
 
 - Greg
 
 
  Stefan
 
  
   - Greg
  
   -Original Message-
   From: Stefan Priebe [mailto:s.pri...@profihost.ag]
   Sent: Thursday, August 13, 2015 11:53 AM
   To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
   Subject: Re: [E1000-devel] dropped rx with i40e
  
   Hi,
  
   sorry for top posting.
  
   I will try to describe the workload as good as i can.
  
   Application is ceph storage (http://ceph.com/).
  
   Workload is TCP Only, Active/Active bond on both ports of the XL710
   card and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So
   overall speed does not seem to matter. Also i can use iperf and get
   a constant speed of 9.8Gb/s in both directions without any rx drops.
  
   The drops don't occur regulary they just happen at a time X and
   then
  stop.
   After some hours it happens again.
   Stefan
  
   Am 13.08.2015 um 17:58 schrieb Rose, Gregory V:
   My apologies but I've been unable to get back to this issue.
  
   After reviewing the thread I don't see anything about steps to
   reproduce
   the problem.  I understand that you're seeing dropped packets with
   the
   Xl710 with various versions of the i40e driver while the X520 with
   the ixgbe driver does not drop packets under the same load.
  
   I don't' see any description of the type of traffic load that is
   causing
   the problem.  That would help me to reproduce the issue.
  
   Keep in mind that dropped packets in and of itself is not a bug.
   It may
   mean that the X520 and the ixgbe driver are more mature and have
   had more tuning and thus work better under the type of traffic
   load you have on your network.  Thus it is important that we
   understand the type of traffic you're seeing on your network so
   that we can work on making the XL710 and i40e driver performance on
   par with the X520 and
  the ixgbe driver.
  
   One other thing.  Below I notice this:
  
   I tested this one:
   ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2
   tx-usecs
   0
  
   I believe that you would be better off using higher values.
   Really low
   values mean the HW interrupt will fire more often - instead you
   should allow the soft IRQ polling to keep processing packets.
  
   - Greg
  
   -Original Message-
   From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
   Sent: Thursday, August 13, 2015 5:41 AM
   To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
   Subject: Re: [E1000-devel] dropped rx with i40e
  
   1.3.12-k from net-next devel does not help either ;-(
  
   Should we open an intel support ticket? We really need a solution.
  
   Stefan
  
   Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG:
   Might this be a memory allocation problem? It happens only after
   some hours running and when the whole memory is filled with
   linux fs
   cache.
  
   Is the i40e driver using kmalloc or vmalloc?
  
   Stefan
   Am 11.08.2015 um 06:03 schrieb Stefan Priebe:
   One more thing to note. It mostly happens after around 8-24
   hours and i could stop it again by rebooting the system/server.
   (can't prove
   it)
  
   Stefan
   Am 06.08.2015 um 22:59 schrieb Rose, Gregory V:
   Thanks Stefan.  I think for now you've given us enough data to
   go on
   - I've got some research to do and then I'll get back to you.
  
   - Greg
  
   -Original Message-
   From: Stefan Priebe - Profihost AG
   [mailto:s.pri...@profihost.ag]
   Sent: Wednesday, August 05, 2015 11:32 PM
   To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
   Subject: Re: [E1000-devel] dropped rx with i40e
  
   Am 06.08.2015 um 00:22 schrieb Rose, Gregory V:
   Stefan,
  
   Could you please send me the output of 'ethtool' and
   'ethtool -
  i

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

Thanks Stefan, I'll set up a test to replicate your traffic profile as closely 
as possible and let it run overnight to see if I can repro and then update you 
tomorrow.

It does seem that it has nothing to do with load so that makes it even more 
curious.

- Greg

 -Original Message-
 From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 Sent: Thursday, August 13, 2015 11:53 AM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Hi,
 
 sorry for top posting.
 
 I will try to describe the workload as good as i can.
 
 Application is ceph storage (http://ceph.com/).
 
 Workload is TCP Only, Active/Active bond on both ports of the XL710 card
 and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So overall speed
 does not seem to matter. Also i can use iperf and get a constant speed of
 9.8Gb/s in both directions without any rx drops.
 
 The drops don't occur regulary they just happen at a time X and then stop.
 After some hours it happens again.
 Stefan
 
 Am 13.08.2015 um 17:58 schrieb Rose, Gregory V:
  My apologies but I've been unable to get back to this issue.
 
  After reviewing the thread I don't see anything about steps to reproduce
 the problem.  I understand that you're seeing dropped packets with the
 Xl710 with various versions of the i40e driver while the X520 with the
 ixgbe driver does not drop packets under the same load.
 
  I don't' see any description of the type of traffic load that is causing
 the problem.  That would help me to reproduce the issue.
 
  Keep in mind that dropped packets in and of itself is not a bug.  It may
 mean that the X520 and the ixgbe driver are more mature and have had more
 tuning and thus work better under the type of traffic load you have on
 your network.  Thus it is important that we understand the type of traffic
 you're seeing on your network so that we can work on making the XL710 and
 i40e driver performance on par with the X520 and the ixgbe driver.
 
  One other thing.  Below I notice this:
 
  I tested this one:
  ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0
 
  I believe that you would be better off using higher values.  Really low
 values mean the HW interrupt will fire more often - instead you should
 allow the soft IRQ polling to keep processing packets.
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Thursday, August 13, 2015 5:41 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  1.3.12-k from net-next devel does not help either ;-(
 
  Should we open an intel support ticket? We really need a solution.
 
  Stefan
 
  Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG:
  Might this be a memory allocation problem? It happens only after
  some hours running and when the whole memory is filled with linux fs
 cache.
 
  Is the i40e driver using kmalloc or vmalloc?
 
  Stefan
  Am 11.08.2015 um 06:03 schrieb Stefan Priebe:
  One more thing to note. It mostly happens after around 8-24 hours
  and i could stop it again by rebooting the system/server. (can't
  prove
  it)
 
  Stefan
  Am 06.08.2015 um 22:59 schrieb Rose, Gregory V:
  Thanks Stefan.  I think for now you've given us enough data to go
  on
  - I've got some research to do and then I'll get back to you.
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 05, 2015 11:32 PM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Am 06.08.2015 um 00:22 schrieb Rose, Gregory V:
  Stefan,
 
  Could you please send me the output of 'ethtool' and 'ethtool -i'
  for
  each i40e interface that is experiencing the dropped packets issue?
 
  These are around 100 cards. So i won't post the output for all of
  them.
  As they're all using the same driver and the same firmware - we
  updated all of them i hope it's ok to post the output only from
  one
  of them.
 
  # ethtool eth2
  Settings for eth2:
Supported ports: [ FIBRE ]
Supported link modes:   1baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: No
Advertised link modes:  Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 1Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 0
Transceiver: external
Auto-negotiation: off
Supports Wake-on: g
Wake-on: d
Current message level: 0x000f (15)
   drv probe link timer
Link detected: yes
  # ethtool -i eth2
  driver: i40e
  version: 1.3.4-k
  firmware-version: f4.33.31377 a1.2 n4.42 e191b
  bus-info: :03:00.0
  supports-statistics: yes

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

 -Original Message-
 From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 Sent: Thursday, August 13, 2015 12:02 PM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 Am 13.08.2015 um 20:59 schrieb Rose, Gregory V:
  Thanks Stefan, I'll set up a test to replicate your traffic profile as
 closely as possible and let it run overnight to see if I can repro and
 then update you tomorrow.
 
  It does seem that it has nothing to do with load so that makes it even
 more curious.
 
 May it be related to jumbo frames?

Could be - I'll be looking at that angle during my testing and evaluation.

- Greg

 
 Stefan
 
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
  Sent: Thursday, August 13, 2015 11:53 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Hi,
 
  sorry for top posting.
 
  I will try to describe the workload as good as i can.
 
  Application is ceph storage (http://ceph.com/).
 
  Workload is TCP Only, Active/Active bond on both ports of the XL710
  card and jumbo frames (MTU 9000). Traffic peak was 400MBit/s - So
  overall speed does not seem to matter. Also i can use iperf and get a
  constant speed of 9.8Gb/s in both directions without any rx drops.
 
  The drops don't occur regulary they just happen at a time X and then
 stop.
  After some hours it happens again.
  Stefan
 
  Am 13.08.2015 um 17:58 schrieb Rose, Gregory V:
  My apologies but I've been unable to get back to this issue.
 
  After reviewing the thread I don't see anything about steps to
  reproduce
  the problem.  I understand that you're seeing dropped packets with
  the
  Xl710 with various versions of the i40e driver while the X520 with
  the ixgbe driver does not drop packets under the same load.
 
  I don't' see any description of the type of traffic load that is
  causing
  the problem.  That would help me to reproduce the issue.
 
  Keep in mind that dropped packets in and of itself is not a bug.  It
  may
  mean that the X520 and the ixgbe driver are more mature and have had
  more tuning and thus work better under the type of traffic load you
  have on your network.  Thus it is important that we understand the
  type of traffic you're seeing on your network so that we can work on
  making the XL710 and i40e driver performance on par with the X520 and
 the ixgbe driver.
 
  One other thing.  Below I notice this:
 
  I tested this one:
  ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs
  0
 
  I believe that you would be better off using higher values.  Really
  low
  values mean the HW interrupt will fire more often - instead you
  should allow the soft IRQ polling to keep processing packets.
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Thursday, August 13, 2015 5:41 AM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  1.3.12-k from net-next devel does not help either ;-(
 
  Should we open an intel support ticket? We really need a solution.
 
  Stefan
 
  Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG:
  Might this be a memory allocation problem? It happens only after
  some hours running and when the whole memory is filled with linux
  fs
  cache.
 
  Is the i40e driver using kmalloc or vmalloc?
 
  Stefan
  Am 11.08.2015 um 06:03 schrieb Stefan Priebe:
  One more thing to note. It mostly happens after around 8-24 hours
  and i could stop it again by rebooting the system/server. (can't
  prove
  it)
 
  Stefan
  Am 06.08.2015 um 22:59 schrieb Rose, Gregory V:
  Thanks Stefan.  I think for now you've given us enough data to
  go on
  - I've got some research to do and then I'll get back to you.
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe - Profihost AG
  [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 05, 2015 11:32 PM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Am 06.08.2015 um 00:22 schrieb Rose, Gregory V:
  Stefan,
 
  Could you please send me the output of 'ethtool' and 'ethtool -
 i'
  for
  each i40e interface that is experiencing the dropped packets
 issue?
 
  These are around 100 cards. So i won't post the output for all
  of
  them.
  As they're all using the same driver and the same firmware - we
  updated all of them i hope it's ok to post the output only from
  one
  of them.
 
  # ethtool eth2
  Settings for eth2:
 Supported ports: [ FIBRE ]
 Supported link modes:   1baseT/Full
 Supported pause frame use: Symmetric
 Supports auto-negotiation: No
 Advertised link modes:  Not reported
 Advertised pause frame use: No
 Advertised auto-negotiation: No
 Speed: 1Mb/s
 Duplex

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

My apologies but I've been unable to get back to this issue.

After reviewing the thread I don't see anything about steps to reproduce the 
problem.  I understand that you're seeing dropped packets with the Xl710 with 
various versions of the i40e driver while the X520 with the ixgbe driver does 
not drop packets under the same load.

I don't' see any description of the type of traffic load that is causing the 
problem.  That would help me to reproduce the issue.

Keep in mind that dropped packets in and of itself is not a bug.  It may mean 
that the X520 and the ixgbe driver are more mature and have had more tuning 
and thus work better under the type of traffic load you have on your network.  
Thus it is important that we understand the type of traffic you're seeing on 
your network so that we can work on making the XL710 and i40e driver 
performance on par with the X520 and the ixgbe driver.

One other thing.  Below I notice this:

 I tested this one:
 ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs 0

I believe that you would be better off using higher values.  Really low values 
mean the HW interrupt will fire more often - instead you should allow the soft 
IRQ polling to keep processing packets.

- Greg

 -Original Message-
 From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
 Sent: Thursday, August 13, 2015 5:41 AM
 To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 1.3.12-k from net-next devel does not help either ;-(
 
 Should we open an intel support ticket? We really need a solution.
 
 Stefan
 
 Am 12.08.2015 um 10:29 schrieb Stefan Priebe - Profihost AG:
  Might this be a memory allocation problem? It happens only after some
  hours running and when the whole memory is filled with linux fs cache.
 
  Is the i40e driver using kmalloc or vmalloc?
 
  Stefan
  Am 11.08.2015 um 06:03 schrieb Stefan Priebe:
  One more thing to note. It mostly happens after around 8-24 hours and
  i could stop it again by rebooting the system/server. (can't prove
  it)
 
  Stefan
  Am 06.08.2015 um 22:59 schrieb Rose, Gregory V:
  Thanks Stefan.  I think for now you've given us enough data to go on
  - I've got some research to do and then I'll get back to you.
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 05, 2015 11:32 PM
  To: Rose, Gregory V; e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Am 06.08.2015 um 00:22 schrieb Rose, Gregory V:
  Stefan,
 
  Could you please send me the output of 'ethtool' and 'ethtool -i'
  for
  each i40e interface that is experiencing the dropped packets issue?
 
  These are around 100 cards. So i won't post the output for all of
 them.
  As they're all using the same driver and the same firmware - we
  updated all of them i hope it's ok to post the output only from one
 of them.
 
  # ethtool eth2
  Settings for eth2:
   Supported ports: [ FIBRE ]
   Supported link modes:   1baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: No
   Advertised link modes:  Not reported
   Advertised pause frame use: No
   Advertised auto-negotiation: No
   Speed: 1Mb/s
   Duplex: Full
   Port: Direct Attach Copper
   PHYAD: 0
   Transceiver: external
   Auto-negotiation: off
   Supports Wake-on: g
   Wake-on: d
   Current message level: 0x000f (15)
  drv probe link timer
   Link detected: yes
  # ethtool -i eth2
  driver: i40e
  version: 1.3.4-k
  firmware-version: f4.33.31377 a1.2 n4.42 e191b
  bus-info: :03:00.0
  supports-statistics: yes
  supports-test: yes
  supports-eeprom-access: yes
  supports-register-dump: yes
  supports-priv-flags: yes
 
Also, the system log might help also - dmesg can get that.
  That'll
  give me something to look at.
 
  As this one is pretty long. i pasted dmesg to pastebin:
  http://pastebin.com/raw.php?i=7Tjp3eDT
 
  By the way, have you tried using ethtool to turn adaptive RX and
  TX off
  using ethtool to see if that has any impact on the dropped packets?
 
  I tested this one:
  ethtool -C eth3 adaptive-rx off adaptive-tx off rx-usecs 2 tx-usecs
  0
 
  but it has not helped. Still dropped rx packets. While a 2nd system
  receiving the same load using ixgbe has no dropped packets.
 
  That might be an easy test to run.
 
  Thanks!
 
  Greets,
  Stefan
 
 
  Thanks,
 
  - Greg
 
  -Original Message-
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
  Sent: Wednesday, August 05, 2015 11:14 AM
  To: e1000-devel@lists.sourceforge.net
  Subject: Re: [E1000-devel] dropped rx with i40e
 
 
  Something i've noticed:
  ixgbe:
  Adaptive RX: off  TX: off
  rx-usecs: 1
  tx-usecs: 0
 
  i40e:
  Adaptive RX: on  TX: on
  rx

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

So to make sure I've got your report right... Since updating the FW the link 
flap issue occurs much less frequently and VRRP is not causing a switch to a 
different host but your log is getting spammed with ATR messages.  Is that 
correct?

Have you tried turning off ntuple-filters?

As for the link flap I don't think we've seen a lot of customers running the 
XL710 at 1Gbps speeds, most are using the 10Gbps or 40Gbps speeds.  I'll do 
some research and see if there are any other reports of issues with the XL710 
running at 1Gbps speed and get back to you.

- Greg

 -Original Message-
 From: Christian Ruppert [mailto:id...@qasl.de]
 Sent: Monday, July 27, 2015 8:47 AM
 To: e1000-de...@lists.sf.net
 Subject: [E1000-devel] X710 / i40e interface resets
 
 Hi List,
 
 after having some serious issues with the X520 ones I wanted to give the
 X710 a try. At first I had some trouble again. Started to flap sometimes
 and later flapping until VRRP switched to a different host. The Kernel log
 was spammed with:
 Jul 24 12:02:10 somehost kernel: [ 1446.325468] i40e :02:00.0: TX
 driver issue detected, PF reset issued Jul 24 12:02:11 somehost kernel: [
 1447.154720] i40e :02:00.0: FCoE capability is disabled Jul 24
 12:02:11 somehost kernel: [ 1447.196823] i40e :02:00.0: PHC enabled
 Jul 24 12:02:11 somehost kernel: [ 1447.255115] i40e :02:00.0:
 enabling bridge mode: VEPA
 Jul 24 12:02:11 somehost kernel: [ 1447.267922] i40e :02:00.0 eth0:
 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
 
 and:
 Jul 24 12:16:08 somehost kernel: [ 2285.524925] i40e :02:00.0: FD
 filter space full, new ntuple rules will not be added Jul 24 12:16:09
 somehost kernel: [ 2286.348784] i40e :02:00.0: FD Filter table flushed
 and FD-SB replayed.
 Jul 24 12:16:09 somehost kernel: [ 2286.356063] i40e :02:00.0: FD
 Sideband/ntuple is being enabled since we have space in the table now Jul
 24 12:16:09 somehost kernel: [ 2286.365777] i40e :02:00.0: ATR is
 being enabled since we have space in the table now
 
 Then I read something about updating the NVM/preboot (here on the ML)
 which I did then. After having some issues with flashing (the update
 utility doesn't like CONFIG_STRICT_DEVMEM=y, there's also no warning about
 it. disabling it did help but not entirely) I decided to flash from a
 Windows host (I had to setup one...) as it seemed somewhat easier and
 safer, for now.
 The preboot util still complained about having an older NVM than expected
 but somehow it worked and it's a more recent version now than it was
 before. So here is the initialization of the 20.2 version/revision and
 also using the most recent 1.2.48 driver on a
 3.18.17 Kernel:
 
 Jul 24 11:38:13 somehost kernel: [2.251951] i40e: Intel(R) Ethernet
 Connection XL710 Network Driver - version 1.2.48
 Jul 24 11:38:13 somehost kernel: [2.259620] i40e: Copyright (c) 2013
 - 2015 Intel Corporation.
 Jul 24 11:38:13 somehost kernel: [2.283072] i40e :02:00.0:
 f4.33.31377 a1.2 n4.42 e191b
 Jul 24 11:38:13 somehost kernel: [2.528098] i40e :02:00.0: FCoE
 capability is disabled
 Jul 24 11:38:13 somehost kernel: [2.541226] i40e :02:00.0: MAC
 address: 68:05:ca:33:17:50
 Jul 24 11:38:13 somehost kernel: [2.545296] i40e :02:00.0: SAN
 MAC: 68:05:ca:33:17:52
 Jul 24 11:38:13 somehost kernel: [2.559708] i40e :02:00.0: fcoe
 queues = 0
 Jul 24 11:38:13 somehost kernel: [2.588853] i40e :02:00.0:
 enabling bridge mode: VEPA
 Jul 24 11:38:13 somehost kernel: [2.623476] i40e :02:00.0: PHC
 enabled
 Jul 24 11:38:13 somehost kernel: [2.639801] i40e :02:00.0:
 PCI-Express: Speed 8.0GT/s Width x8
 Jul 24 11:38:13 somehost kernel: [2.653156] i40e :02:00.0:
 Features: PF-id[0] VFs: 64 VSIs: 66 QP: 8 RX: 1BUF RSS FD_ATR FD_SB NTUPLE
 DCB PTP
 Jul 24 11:38:13 somehost kernel: [2.672182] i40e :02:00.1:
 f4.33.31377 a1.2 n4.42 e191b
 Jul 24 11:38:13 somehost kernel: [2.921075] i40e :02:00.1: FCoE
 capability is disabled
 Jul 24 11:38:13 somehost kernel: [2.935064] i40e :02:00.1: MAC
 address: 68:05:ca:33:17:51
 Jul 24 11:38:13 somehost kernel: [2.944833] i40e :02:00.1: SAN
 MAC: 68:05:ca:33:17:53
 Jul 24 11:38:13 somehost kernel: [2.971041] i40e :02:00.1: fcoe
 queues = 0
 Jul 24 11:38:13 somehost kernel: [3.004728] i40e :02:00.1:
 enabling bridge mode: VEPA
 Jul 24 11:38:13 somehost kernel: [3.290178] i40e :02:00.1: PHC
 enabled
 Jul 24 11:38:13 somehost kernel: [3.310646] i40e :02:00.1:
 PCI-Express: Speed 8.0GT/s Width x8
 Jul 24 11:38:13 somehost kernel: [3.331079] i40e :02:00.1:
 Features: PF-id[1] VFs: 64 VSIs: 66 QP: 8 RX: 1BUF RSS FD_ATR FD_SB NTUPLE
 DCB PTP
 Jul 24 11:38:13 somehost kernel: [4.279754] i40e :02:00.1
 rename5: renamed from eth3
 Jul 24 11:38:13 somehost kernel: [4.315533] i40e :02:00.0 eth0:
 renamed from eth1
 Jul 24 11:38:13 somehost kernel: [4.387610] 

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

 -Original Message-
 From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 Sent: Monday, August 24, 2015 11:14 AM
 To: Rose, Gregory V
 Cc: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 IS there a way to check this under linux withou the need to pull out the
 card?

Yes, use lspci.  Here's an example (as root):

# lspci -m | grep Eth

Sample output from one of my 4 port adapters:

82:00.0 Ethernet controller Intel Corporation Ethernet Controller X710 for 
10GbE SFP+ -r01 Intel Corporation Ethernet Converged Network Adapter X710-4
82:00.1 Ethernet controller Intel Corporation Ethernet Controller X710 for 
10GbE SFP+ -r01 Intel Corporation Ethernet Converged Network Adapter X710
82:00.2 Ethernet controller Intel Corporation Ethernet Controller X710 for 
10GbE SFP+ -r01 Intel Corporation Ethernet Converged Network Adapter X710
82:00.3 Ethernet controller Intel Corporation Ethernet Controller X710 for 
10GbE SFP+ -r01 Intel Corporation Ethernet Converged Network Adapter X710

And the '-r01' is the chip revision number.  That's what you should be seeing, 
if you don't then let me know.

Thanks,

- Greg

 
 Mit freundlichen Grüßen
Stefan Priebe
 Bachelor of Science in Computer Science (BSCS) Vorstand (CTO)
 
 ---
 Profihost AG
 Expo Plaza 1
 30539 Hannover
 Deutschland
 
 Tel.: +49 (511) 5151 8181 | Fax.: +49 (511) 5151 8282
 URL: http://www.profihost.com | E-Mail: i...@profihost.com
 
 Sitz der Gesellschaft: Hannover, USt-IdNr. DE813460827
 Registergericht: Amtsgericht Hannover, Register-Nr.: HRB 202350
 Vorstand: Cristoph Bluhm, Sebastian Bluhm, Stefan Priebe
 Aufsichtsrat: Prof. Dr. iur. Winfried Huck (Vorsitzender)
 
 Am 24.08.2015 um 19:31 schrieb Rose, Gregory V:
  The only cards out in the field should all show a PCI revision ID
  value of 1.  If you have any cards that show 0 then they need to be
 replaced.
  But again, you shouldn’t have any A0 silicon cards.
 
  -Greg
 
  *From:*Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  *Sent:* Monday, August 24, 2015 10:16 AM
  *To:* Rose, Gregory V
  *Cc:* e1000-devel@lists.sourceforge.net
  *Subject:* Re: [E1000-devel] dropped rx with i40e
 
  Are there different hw revisions? Currently it seems cards before q4
  2014 have this problem cards build in 2015 work fine.
 
  Stefan
 
  Excuse my typo sent from my mobile phone.
 
 
  Am 21.08.2015 um 02:51 schrieb Rose, Gregory V
  gregory.v.r...@intel.com
  mailto:gregory.v.r...@intel.com:
 
  Stefan,
 
  Late update on this.  I'm being told know that there are additional
  parameters that may need to be changed.  Please hold until I get
  this updated information.
 
  Thanks and regards,
 
  - Greg
 
 
  -Original Message-
 
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 
  Sent: Thursday, August 20, 2015 4:26 PM
 
  To: Stefan Priebe
 
  Cc: e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
 
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Stefan,
 
  You will need to run the update utility again but this time
  edit the
 
  nvmupdate.cfg file to change this:
 
  SKIP OROM: TRUE
 
  to this:
 
  SKIP OROM: FALSE
 
  That should then also update the option ROM for the PXE utility.
 
  Regards,
 
  - Greg
 
  -Original Message-
 
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 
  Sent: Wednesday, August 19, 2015 10:40 AM
 
  To: Rose, Gregory V
 
  Cc: e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
 
  Subject: Re: [E1000-devel] dropped rx with i40e
 
  Correct message:
 
  PXE-m10: The application for the device detected a newer
  version of
 
  the nvm image than expexted.
 
  So how do i update the application code of the XL710?
 
  Stefan
 
  Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
 
  An additional question since I've updated to latest
  xl710 firmware
 
  released 17 August the cards show at boot time (bios
  init/ post)
 
  something like application code detected a newer nvm
  image than
 
  expected please update application software. But I can't
  find an
 
  application code update for the xl710 cards.
 
  Stefan
 
  Excuse my typo sent from my mobile phone.
 
  Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
 
  gregory.v.r...@intel.com
  mailto:gregory.v.r...@intel.com
 
  mailto:gregory.v.r...@intel.com:
 
  -Original Message

Re: [E1000-devel] i40e RSS configuration

[Rose, Gregory V]

I’ll carry that suggestion back to the development team, it seems like a good 
idea to me and would have saved us some time.

Thanks and regards,


-Greg

From: Trevor Highland [mailto:trevor.highl...@gmail.com]
Sent: Monday, August 24, 2015 5:45 PM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] i40e RSS configuration

Thanks for clarifying that this is a limitation of the card. It would be very 
helpful to update the driver so that only supported hashing configurations are 
accepted.

Trevor

On Mon, Aug 24, 2015 at 7:36 PM, Rose, Gregory V 
gregory.v.r...@intel.commailto:gregory.v.r...@intel.com wrote:
Trevor,

The X710 and XL710 devices do not work the same as the NICs supported by the 
ixgbe driver so the configuration you’re attempting with the i40e driver does 
not work as you would expect on the devices supported by ixgbe.  Intel has 
received requests from other parties requesting support for the configuration 
you’re attempting and the request is under consideration but at this time we 
can’t make any commitment as to when that sort of support might be available.

Regards,


-Greg

From: Trevor Highland 
[mailto:trevor.highl...@gmail.commailto:trevor.highl...@gmail.com]
Sent: Monday, August 24, 2015 4:51 PM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.netmailto:e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] i40e RSS configuration

I am running using the 1.3.38 driver. We have also tried using the driver 
provided DPDK. I can reproduce this behavior with the following steps. The 
traffic we are generating has randomized IP address from several /24 networks.

root@x:~# lspci | grep 710
82:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
82:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
82:00.2 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
82:00.3 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.2 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.3 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)

root@x:~# modprobe i40e
root@x:~# ethtool -N p2p4 rx-flow-hash tcp4 sd
root@x:~# ethtool -N p2p4 rx-flow-hash udp4 sd
root@x:~# ifconfig p2p4 up
root@x:~#ethtool -S p2p4
NIC statistics:
 rx_packets: 842749216
 tx_packets: 8
 rx_bytes: 503124766615
 tx_bytes: 648
 rx_errors: 0
 tx_errors: 0
 rx_dropped: 0
 tx_dropped: 0
 collisions: 0
 rx_length_errors: 0
 rx_crc_errors: 0
 rx_unicast: 842749231
 tx_unicast: 0
 rx_multicast: 0
 tx_multicast: 6
 rx_broadcast: 0
 tx_broadcast: 0
 rx_unknown_protocol: 0
 fcoe_bad_fccrc: 0
 rx_fcoe_dropped: 0
 rx_fcoe_packets: 0
 rx_fcoe_dwords: 0
 fcoe_ddp_count: 0
 fcoe_last_error: 0
 tx_fcoe_packets: 0
 tx_fcoe_dwords: 0
 tx-0.tx_packets: 0
 tx-0.tx_bytes: 0
 rx-0.rx_packets: 842749216
 rx-0.rx_bytes: 503124766615
 tx-1.tx_packets: 0
 tx-1.tx_bytes: 0
 rx-1.rx_packets: 0
 rx-1.rx_bytes: 0
 tx-2.tx_packets: 8
 tx-2.tx_bytes: 648
 rx-2.rx_packets: 0
 rx-2.rx_bytes: 0
 tx-3.tx_packets: 0
 tx-3.tx_bytes: 0
 rx-3.rx_packets: 0
 rx-3.rx_bytes: 0
 tx-4.tx_packets: 0
 tx-4.tx_bytes: 0
 rx-4.rx_packets: 0
 rx-4.rx_bytes: 0
 tx-5.tx_packets: 0
 tx-5.tx_bytes: 0
 rx-5.rx_packets: 0
 rx-5.rx_bytes: 0
 tx-6.tx_packets: 0
 tx-6.tx_bytes: 0
 rx-6.rx_packets: 0
 rx-6.rx_bytes: 0
 tx-7.tx_packets: 0
 tx-7.tx_bytes: 0
 rx-7.rx_packets: 0
 rx-7.rx_bytes: 0
 tx-8.tx_packets: 0
 tx-8.tx_bytes: 0
 rx-8.rx_packets: 0
 rx-8.rx_bytes: 0
  removed subsequent output

root@x:~# ethtool -N p2p4 rx-flow-hash tcp4 sdfn
root@x:~# ethtool -N p2p4 rx-flow-hash udp4 sdfn
root@x:~# ethtool -S p2p4
NIC statistics:
 rx_packets: 995860297
 tx_packets: 8
 rx_bytes: 594919177484
 tx_bytes: 648
 rx_errors: 0
 tx_errors: 0
 rx_dropped: 0
 tx_dropped: 0
 collisions: 0
 rx_length_errors: 0
 rx_crc_errors: 0
 rx_unicast: 995860307
 tx_unicast: 0
 rx_multicast: 0
 tx_multicast: 6
 rx_broadcast: 0
 tx_broadcast: 0
 rx_unknown_protocol: 0
 fcoe_bad_fccrc: 0
 rx_fcoe_dropped: 0
 rx_fcoe_packets: 0
 rx_fcoe_dwords: 0
 fcoe_ddp_count: 0
 fcoe_last_error: 0
 tx_fcoe_packets: 0
 tx_fcoe_dwords: 0
 tx-0.tx_packets: 0
 tx-0.tx_bytes: 0
 rx-0.rx_packets: 877750096
 rx-0.rx_bytes: 523937667302
 tx-1.tx_packets: 0
 tx-1.tx_bytes: 0
 rx-1.rx_packets: 1909700
 rx-1.rx_bytes: 1136954121
 tx-2.tx_packets: 8
 tx

Re: [E1000-devel] i40e RSS configuration

[Rose, Gregory V]

Trevor,

The X710 and XL710 devices do not work the same as the NICs supported by the 
ixgbe driver so the configuration you’re attempting with the i40e driver does 
not work as you would expect on the devices supported by ixgbe.  Intel has 
received requests from other parties requesting support for the configuration 
you’re attempting and the request is under consideration but at this time we 
can’t make any commitment as to when that sort of support might be available.

Regards,


-Greg

From: Trevor Highland [mailto:trevor.highl...@gmail.com]
Sent: Monday, August 24, 2015 4:51 PM
To: Rose, Gregory V
Cc: e1000-devel@lists.sourceforge.net
Subject: Re: [E1000-devel] i40e RSS configuration

I am running using the 1.3.38 driver. We have also tried using the driver 
provided DPDK. I can reproduce this behavior with the following steps. The 
traffic we are generating has randomized IP address from several /24 networks.

root@x:~# lspci | grep 710
82:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
82:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
82:00.2 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
82:00.3 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.2 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)
83:00.3 Ethernet controller: Intel Corporation Ethernet 10G 2P X710 Adapter 
(rev 01)

root@x:~# modprobe i40e
root@x:~# ethtool -N p2p4 rx-flow-hash tcp4 sd
root@x:~# ethtool -N p2p4 rx-flow-hash udp4 sd
root@x:~# ifconfig p2p4 up
root@x:~#ethtool -S p2p4
NIC statistics:
 rx_packets: 842749216
 tx_packets: 8
 rx_bytes: 503124766615
 tx_bytes: 648
 rx_errors: 0
 tx_errors: 0
 rx_dropped: 0
 tx_dropped: 0
 collisions: 0
 rx_length_errors: 0
 rx_crc_errors: 0
 rx_unicast: 842749231
 tx_unicast: 0
 rx_multicast: 0
 tx_multicast: 6
 rx_broadcast: 0
 tx_broadcast: 0
 rx_unknown_protocol: 0
 fcoe_bad_fccrc: 0
 rx_fcoe_dropped: 0
 rx_fcoe_packets: 0
 rx_fcoe_dwords: 0
 fcoe_ddp_count: 0
 fcoe_last_error: 0
 tx_fcoe_packets: 0
 tx_fcoe_dwords: 0
 tx-0.tx_packets: 0
 tx-0.tx_bytes: 0
 rx-0.rx_packets: 842749216
 rx-0.rx_bytes: 503124766615
 tx-1.tx_packets: 0
 tx-1.tx_bytes: 0
 rx-1.rx_packets: 0
 rx-1.rx_bytes: 0
 tx-2.tx_packets: 8
 tx-2.tx_bytes: 648
 rx-2.rx_packets: 0
 rx-2.rx_bytes: 0
 tx-3.tx_packets: 0
 tx-3.tx_bytes: 0
 rx-3.rx_packets: 0
 rx-3.rx_bytes: 0
 tx-4.tx_packets: 0
 tx-4.tx_bytes: 0
 rx-4.rx_packets: 0
 rx-4.rx_bytes: 0
 tx-5.tx_packets: 0
 tx-5.tx_bytes: 0
 rx-5.rx_packets: 0
 rx-5.rx_bytes: 0
 tx-6.tx_packets: 0
 tx-6.tx_bytes: 0
 rx-6.rx_packets: 0
 rx-6.rx_bytes: 0
 tx-7.tx_packets: 0
 tx-7.tx_bytes: 0
 rx-7.rx_packets: 0
 rx-7.rx_bytes: 0
 tx-8.tx_packets: 0
 tx-8.tx_bytes: 0
 rx-8.rx_packets: 0
 rx-8.rx_bytes: 0
  removed subsequent output

root@x:~# ethtool -N p2p4 rx-flow-hash tcp4 sdfn
root@x:~# ethtool -N p2p4 rx-flow-hash udp4 sdfn
root@x:~# ethtool -S p2p4
NIC statistics:
 rx_packets: 995860297
 tx_packets: 8
 rx_bytes: 594919177484
 tx_bytes: 648
 rx_errors: 0
 tx_errors: 0
 rx_dropped: 0
 tx_dropped: 0
 collisions: 0
 rx_length_errors: 0
 rx_crc_errors: 0
 rx_unicast: 995860307
 tx_unicast: 0
 rx_multicast: 0
 tx_multicast: 6
 rx_broadcast: 0
 tx_broadcast: 0
 rx_unknown_protocol: 0
 fcoe_bad_fccrc: 0
 rx_fcoe_dropped: 0
 rx_fcoe_packets: 0
 rx_fcoe_dwords: 0
 fcoe_ddp_count: 0
 fcoe_last_error: 0
 tx_fcoe_packets: 0
 tx_fcoe_dwords: 0
 tx-0.tx_packets: 0
 tx-0.tx_bytes: 0
 rx-0.rx_packets: 877750096
 rx-0.rx_bytes: 523937667302
 tx-1.tx_packets: 0
 tx-1.tx_bytes: 0
 rx-1.rx_packets: 1909700
 rx-1.rx_bytes: 1136954121
 tx-2.tx_packets: 8
 tx-2.tx_bytes: 648
 rx-2.rx_packets: 1895315
 rx-2.rx_bytes: 1145483514
 tx-3.tx_packets: 0
 tx-3.tx_bytes: 0
 rx-3.rx_packets: 1881385
 rx-3.rx_bytes: 1152641588
 tx-4.tx_packets: 0
 tx-4.tx_bytes: 0
 rx-4.rx_packets: 1907359
 rx-4.rx_bytes: 1158220856
 tx-5.tx_packets: 0
 tx-5.tx_bytes: 0
 rx-5.rx_packets: 1855043
 rx-5.rx_bytes: 1118706052
 tx-6.tx_packets: 0
 tx-6.tx_bytes: 0
 rx-6.rx_packets: 1914796
 rx-6.rx_bytes: 1134671275
 tx-7.tx_packets: 0
 tx-7.tx_bytes: 0
 rx-7.rx_packets: 1890944
 rx-7.rx_bytes: 1132133104
 tx-8.tx_packets: 0
 tx-8.tx_bytes: 0
 rx-8.rx_packets: 1892025
 rx-8.rx_bytes: 1125067639

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

> -Original Message-
> From: Christian Ruppert [mailto:id...@qasl.de]
> Sent: Monday, November 09, 2015 1:17 AM
> To: Rose, Gregory V
> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> Subject: RE: [E1000-devel] X710 / i40e interface resets
> 

> 
> The system is running about a month without any resets but having TSO and
> GSO disabled. Does it have a performance impact with TSO/GSO disabled, or
> any other disadvantages? We haven't done any benchmarks yet.

Yes, I'm certain there are performance disadvantages with certain traffic 
profiles.  However, I've been pulled off of this issue and working others for 
the last few weeks.  I do hope to get back to this.

Thanks,

- Greg

> 
> --
> Regards,
> Christian Ruppert

--
Presto, an open source distributed SQL query engine for big data, initially
developed by Facebook, enables you to easily query your data on Hadoop in a 
more interactive manner. Teradata is also now providing full enterprise
support for Presto. Download a free open source copy now.
http://pubads.g.doubleclick.net/gampad/clk?id=250295911=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

> -Original Message-
> From: Christian Ruppert [mailto:id...@qasl.de]
> Sent: Wednesday, October 07, 2015 10:54 AM
> To: Rose, Gregory V
> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> Subject: RE: [E1000-devel] X710 / i40e interface resets
> 
> On 2015-10-07 18:49, Rose, Gregory V wrote:
> >> -Original Message-
> >> From: Christian Ruppert [mailto:id...@qasl.de]
> >> Sent: Wednesday, October 07, 2015 8:00 AM
> >> To: Rose, Gregory V
> >> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> >> Subject: RE: [E1000-devel] X710 / i40e interface resets
> >>
> >

[snip]

> > Maybe, but first let me ask you if you're using the VEPA mode work
> > around for bonding?
> >
> 
> No, just the basic bonding in 802.3ad mode. We don't use VEPA at all.

Can you send me the output of the command "bridge link show"?

Thanks,

- Greg

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

> -Original Message-
> From: Christian Ruppert [mailto:id...@qasl.de]
> Sent: Thursday, October 15, 2015 4:02 AM
> To: Rose, Gregory V
> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> Subject: RE: [E1000-devel] X710 / i40e interface resets
> 
> On 2015-10-08 17:05, Rose, Gregory V wrote:
> >> -Original Message-
> >> From: Christian Ruppert [mailto:id...@qasl.de]
> >> Sent: Thursday, October 08, 2015 7:34 AM
> >> To: Rose, Gregory V
> >> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> >> Subject: RE: [E1000-devel] X710 / i40e interface resets
> > Please try turning off TSO/GSO to see if that helps.
> 
> That seems to help. No more resets. Do you want me to enable TSO or GSO
> again to see which one exactly cause those resets? Or do you want me to do
> anything else?
> 

Sorry for the delayed response but I've been out of the office for the last 
week.

Yes, please do try disabling TSO and GSO separately - I suspect that internally 
to the driver it won't make a difference but if it does then it would be good 
to know.

Thanks!

- Greg


--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

> -Original Message-
> From: Christian Ruppert [mailto:id...@qasl.de]
> Sent: Thursday, October 08, 2015 7:34 AM
> To: Rose, Gregory V
> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> Subject: RE: [E1000-devel] X710 / i40e interface resets
> 
> On 2015-10-08 16:23, Rose, Gregory V wrote:
> >> -Original Message-
> >> From: Christian Ruppert [mailto:id...@qasl.de]
> >> Sent: Wednesday, October 07, 2015 10:54 AM
> >> To: Rose, Gregory V
> >> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> >> Subject: RE: [E1000-devel] X710 / i40e interface resets
> >>
> >> On 2015-10-07 18:49, Rose, Gregory V wrote:
> >> >> -Original Message-
> >> >> From: Christian Ruppert [mailto:id...@qasl.de]
> >> >> Sent: Wednesday, October 07, 2015 8:00 AM
> >> >> To: Rose, Gregory V
> >> >> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> >> >> Subject: RE: [E1000-devel] X710 / i40e interface resets
> >> >>
> >> >
> >
> > [snip]
> >
> >> > Maybe, but first let me ask you if you're using the VEPA mode work
> >> > around for bonding?
> >> >
> >>
> >> No, just the basic bonding in 802.3ad mode. We don't use VEPA at all.
> >
> > Can you send me the output of the command "bridge link show"?
> 
> 3: eth0 state UP : <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 master
> bond0 hwmode VEPA
> 5: eth1 state UP : <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 master
> bond0 hwmode VEPA
> 
> hm, now that's interesting...

That is exactly what we want to see.  The XL710/X710 controllers have an 
internal switch for when SR-IOV mode is used so that all the VFs on that 
controller can communicate with each other across the bridge.  If the hwmode 
had been VEB then that would indicate a problem. However, you're in the correct 
internal switch mode for LACP operation.

Please try turning off TSO/GSO to see if that helps.

If not can I get you to enter a bug ticket at the Intel SF site?

Please include the following:

Results of 'ethtool -i' on each X710 interface.
Results of 'ehtool' on each X710 interface.
Results of 'ip link show' and 'ip addr show'.
The system log obtained via 'dmesg' *after* the error has occurred.

That should get us started.

Thanks!

- Greg

> 
> >
> > Thanks,
> >
> > - Greg
> 
> --
> Regards,
> Christian Ruppert

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

> -Original Message-
> From: Christian Ruppert [mailto:id...@qasl.de]
> Sent: Wednesday, October 07, 2015 12:59 AM
> To: Rose, Gregory V
> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> Subject: RE: [E1000-devel] X710 / i40e interface resets
> 
> Is there anything else I can do to help to get that debugged/fixed?
> We upgraded the connection from 1G to 10G btw (it was 1G before).

We'll need to get a setup to reproduce the problem so we can debug it.  Can you 
provide that information?  Or perhaps you've provided it before and I'm just 
not seeing it?  If that's the case can you point me to it again?

Thanks,

- Greg

> 
> On 2015-09-28 16:53, Christian Ruppert wrote:
> > And there we go:
> >
> > [77571.409182] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [77572.245943] bond0: link status up again after 0 ms for interface
> > eth1
> > [94412.738896] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [94413.569379] bond0: link status up again after 0 ms for interface
> > eth1
> > [94516.443592] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [94516.718341] bond0: link status up again after 0 ms for interface
> > eth1
> > [96850.580882] i40e :02:00.0: TX driver issue detected, PF reset
> > issued
> > [96851.036998] bond0: link status up again after 0 ms for interface
> > eth0
> > [96939.474122] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [96940.217799] bond0: link status up again after 0 ms for interface
> > eth1
> > [96945.007365] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [96945.225029] bond0: link status up again after 0 ms for interface
> > eth1
> > [96965.362669] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [96966.255350] bond0: link status up again after 0 ms for interface
> > eth1
> > [99844.991331] i40e :02:00.0: TX driver issue detected, PF reset
> > issued
> > [99845.360288] bond0: link status up again after 0 ms for interface
> > eth0
> > [159908.054220] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [159909.165679] bond0: link status up again after 0 ms for interface
> > eth1
> > [329531.138706] i40e :02:00.1: TX driver issue detected, PF reset
> > issued
> > [329532.167421] bond0: link status up again after 0 ms for interface
> > eth1
> >
> > On 2015-09-25 16:19, Christian Ruppert wrote:
> >> Hi,
> >>
> >> the card came back and I had to flash it again but it worked perfectly
> >> fine this time :)
> >> So I setup a new Kernel (3.18.20) and the most recent i40e driver
> >> (1.3.39.1) and removed the previous workaround to disable ntuples on
> >> both interfaces. So basically a vanilla setup.
> >> That host is up for several hours no and there was no spam so far and
> >> only one reset yet:
> >> Sep 25 12:38:54 somehost kernel: [77571.409182] i40e :02:00.1: TX
> >> driver issue detected, PF reset issued
> >> Sep 25 12:38:55 somehost kernel: [77572.245943] bond0: link status up
> >> again after 0 ms for interface eth1
> >>
> >> and here's the requested dmesg again:
> >> [0.00] Initializing cgroup subsys cpuset
> >> [0.00] Initializing cgroup subsys cpu
> >> [0.00] Initializing cgroup subsys cpuacct
> >> [0.00] Linux version 3.18.20 (fud@somehost) (gcc version 4.7.2
> >> (Debian 4.7.2-5) ) #1 SMP Wed Aug 26 14:25:15 CEST 2015
> >> [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-3.18.20
> >> root=UUID=ea7a13a2-a96b-4165-801a-119b0f0d9b8f ro ipv6.disable=1
> >> intel_pstate=disable earlyprintk=ttyS0,115200,keep intel_iommu=off
> >> [0.00] e820: BIOS-provided physical RAM map:
> >> [0.00] BIOS-e820: [mem 0x-0x00099bff]
> >> usable
> >> [0.00] BIOS-e820: [mem 0x00099c00-0x0009]
> >> reserved
> >> [0.00] BIOS-e820: [mem 0x000e-0x000f]
> >> reserved
> >> [0.00] BIOS-e820: [mem 0x0010-0xbecc8fff]
> >> usable
> >> [0.00] BIOS-e820: [mem 0xbecc9000-0xbed0bfff]
> >> ACPI NVS
> >> [0.00] BIOS-e820: [mem 0xbed0c000-0xce491fff]
> >> usable
> >> [0.00] BIOS-e820: [mem 0xce492000-0xce534fff]
> >> reserved
> >> [0.00] BIOS-e820: [mem 0x00

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

> -Original Message-
> From: Christian Ruppert [mailto:id...@qasl.de]
> Sent: Wednesday, October 07, 2015 8:00 AM
> To: Rose, Gregory V
> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> Subject: RE: [E1000-devel] X710 / i40e interface resets
> 
> On 2015-10-07 16:14, Rose, Gregory V wrote:
> >> -Original Message-
> >> From: Christian Ruppert [mailto:id...@qasl.de]
> >> Sent: Wednesday, October 07, 2015 12:59 AM
> >> To: Rose, Gregory V
> >> Cc: Fujinaka, Todd; e1000-de...@lists.sf.net
> >> Subject: RE: [E1000-devel] X710 / i40e interface resets
> >>
> >> Is there anything else I can do to help to get that debugged/fixed?
> >> We upgraded the connection from 1G to 10G btw (it was 1G before).
> >
> > We'll need to get a setup to reproduce the problem so we can debug it.
> >  Can you provide that information?  Or perhaps you've provided it
> > before and I'm just not seeing it?  If that's the case can you point
> > me to it again?
> >
> > Thanks,
> >
> > - Greg
> 
> Hi Greg,
> 
> in this case it's a Supermicro 5017C-MTF (X9SCL/X9SCM) using a X710 10GE
> card, both ports connected to 10GE switch ports (LACP).
> WAN->Router->Swich->Host
> The host in this case acts as a loadbalancer using e.g. Varnish and
> HAProxy and doing mostly HTTP traffic (also SSL termination).
> Do you need anything else?

Maybe, but first let me ask you if you're using the VEPA mode work around for 
bonding?

- Greg

> 
> >
> >>
> >> On 2015-09-28 16:53, Christian Ruppert wrote:
> >> > And there we go:
> >> >
> >> > [77571.409182] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [77572.245943] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [94412.738896] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [94413.569379] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [94516.443592] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [94516.718341] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [96850.580882] i40e :02:00.0: TX driver issue detected, PF reset
> >> > issued
> >> > [96851.036998] bond0: link status up again after 0 ms for interface
> >> > eth0
> >> > [96939.474122] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [96940.217799] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [96945.007365] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [96945.225029] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [96965.362669] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [96966.255350] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [99844.991331] i40e :02:00.0: TX driver issue detected, PF reset
> >> > issued
> >> > [99845.360288] bond0: link status up again after 0 ms for interface
> >> > eth0
> >> > [159908.054220] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [159909.165679] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> > [329531.138706] i40e :02:00.1: TX driver issue detected, PF reset
> >> > issued
> >> > [329532.167421] bond0: link status up again after 0 ms for interface
> >> > eth1
> >> >
> >> > On 2015-09-25 16:19, Christian Ruppert wrote:
> >> >> Hi,
> >> >>
> >> >> the card came back and I had to flash it again but it worked
> perfectly
> >> >> fine this time :)
> >> >> So I setup a new Kernel (3.18.20) and the most recent i40e driver
> >> >> (1.3.39.1) and removed the previous workaround to disable ntuples on
> >> >> both interfaces. So basically a vanilla setup.
> >> >> That host is up for several hours no and there was no spam so far
> and
> >> >> only one reset yet:
> >> >> Sep 25 12:38:54 somehost kernel: [77571.409182] i40e :02:00.1:
> TX
> >> >> driver issue detected, PF reset issued
> >> >> Sep 25 12:38:55 somehost kernel: [77572.245943] bond0: link status
> up
> >> >> again after 0 ms 

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

 -Original Message-
 From: Christian Ruppert [mailto:id...@qasl.de]
 Sent: Thursday, August 27, 2015 4:54 AM
 To: Fujinaka, Todd
 Cc: Rose, Gregory V; e1000-de...@lists.sf.net
 Subject: RE: [E1000-devel] X710 / i40e interface resets
 
 Well, that's it for now anyway...
 
 ...
 Power cycle is required to complete the update process.
 Tool execution completed with the following status: All operations
 completed successfully
 
 The card doesn't even appear in lspci anymore... :/ I used a Win7 to
 flash it since a co-worker already broke one when flashing via Linux. No
 idea how to unbrick it.

Christian,

I'm very sorry to hear that.  It is pretty rare for this sort of thing to 
happen but not unheard of.  Please contact supp...@intel.com to get help 
replacing the part.

- Greg

 
 
 On 2015-08-27 01:25, Fujinaka, Todd wrote:
  Attachments don't work on this mailing list (though they may have
  gotten to Greg). Please file a bug on e1000.sourceforge.net and attach
  the files there.
 
  Todd Fujinaka
  Software Application Engineer
  Networking Division (ND)
  Intel Corporation
  todd.fujin...@intel.com
  (503) 712-4565
 
  -Original Message-
  From: Christian Ruppert [mailto:id...@qasl.de]
  Sent: Wednesday, August 26, 2015 10:24 AM
  To: Rose, Gregory V
  Cc: e1000-de...@lists.sf.net
  Subject: Re: [E1000-devel] X710 / i40e interface resets
 
  On 2015-08-26 01:19, Rose, Gregory V wrote:
  -Original Message-
  From: Christian Ruppert [mailto:id...@qasl.de]
  Sent: Friday, August 21, 2015 7:06 AM
  To: Rose, Gregory V
  Cc: e1000-de...@lists.sf.net
  Subject: RE: [E1000-devel] X710 / i40e interface resets
 
  On 2015-07-28 18:51, Rose, Gregory V wrote:
   -Original Message-
   From: Christian Ruppert [mailto:id...@qasl.de]
   Sent: Tuesday, July 28, 2015 5:43 AM
   To: Rose, Gregory V
   Cc: e1000-de...@lists.sf.net
   Subject: RE: [E1000-devel] X710 / i40e interface resets
  
 
  [snip]
 
   I've been told that the Tx driver issue detected message is due to
   a MDD (Malicious Driver Detection) event.  We need to find out what
   is causing that so if you could please send me your system log and
   the output of ethtool intf and ethtool -i intf perhaps I can
   see what might be occurring.  If not then we'll have to debug
 further.
 
  See the initial Mail for ethtool -k and so on. Also:
  # ethtool eth0
  Settings for eth0:
Supported ports: [ FIBRE ]
Supported link modes:   1000baseT/Full
1baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: No
Advertised link modes:  1000baseT/Full
1baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 1Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: external
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x000f (15)
   drv probe link timer
Link detected: yes
  # ethtool -i eth0
  driver: i40e
  version: 1.2.48
  firmware-version: f4.33.31377 a1.2 n4.42 e191b
  bus-info: :02:00.0
  supports-statistics: yes
  supports-test: yes
  supports-eeprom-access: yes
  supports-register-dump: yes
  supports-priv-flags: yes
 
  I also need the output of 'dmesg' - and I need to figure out how to
  determine what specific MDD event is occurring.  I'll get back to you
  with instructions on how to do that when I get the system log from
  you.
 
  See the attached file. It's a dmesg from boot + some minutes.
  I'll upgrade the card's firmware tomorrow I guess and try out the new
  drivers btw.
 
 
  Thanks,
 
  - Greg
 
  --
  Regards,
  Christian Ruppert
 
 --
 Regards,
 Christian Ruppert

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] dropped rx with i40e

[Rose, Gregory V]

 -Original Message-
 From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
 Sent: Wednesday, August 26, 2015 5:34 AM
 To: Rose, Gregory V
 Cc: e1000-devel@lists.sourceforge.net
 Subject: Re: [E1000-devel] dropped rx with i40e
 
 I have a reproducer!! It happens under cache pressure.
 
 I can reproduce this with the following:
 
 1.) start iperf -s on the system wird X710
 2.) Start sending a lot of data to it i used 20 other servers using iperf
 -c TARGETIP -d -t 600
 3.) read a lot of big files (i used find /DIRWITHBIGFILES ... -exec dd
 if=FILE of=/dev/zero bs=4M)
 
 I can trigger the drops with this code 100%.

Great, I'll try to set up a similar environment today.  Finding 20 other 
servers won't work for me but I think I can certainly emulate a similar traffic 
profile.

I'll let you know how it goes.

Thanks,

- Greg

 
 Mit freundlichen Grüßen
   Stefan Priebe
 Bachelor of Science in Computer Science (BSCS) Vorstand (CTO)
 
 ---
 Profihost AG
 Expo Plaza 1
 30539 Hannover
 Deutschland
 
 Tel.: +49 (511) 5151 8181 | Fax.: +49 (511) 5151 8282
 URL: http://www.profihost.com | E-Mail: i...@profihost.com
 
 Sitz der Gesellschaft: Hannover, USt-IdNr. DE813460827
 Registergericht: Amtsgericht Hannover, Register-Nr.: HRB 202350
 Vorstand: Cristoph Bluhm, Sebastian Bluhm, Stefan Priebe
 Aufsichtsrat: Prof. Dr. iur. Winfried Huck (Vorsitzender)
 
 Am 24.08.2015 um 19:40 schrieb Rose, Gregory V:
  So if you swap one of the newer cards with one of the older cards do
  the packet drops continue to follow the older card around?
 
 
 
  If so, then please get me the PBA of one of the newer cards and the
  PBA of one of the older ones that fails.
 
 
 
  Thanks,
 
 
 
  -Greg
 
 
 
  *From:*Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
  *Sent:* Saturday, August 22, 2015 12:14 AM
  *To:* Rose, Gregory V
  *Cc:* e1000-devel@lists.sourceforge.net
  *Subject:* Re: [E1000-devel] dropped rx with i40e
 
 
 
  I still have those ugly drops. Generally it seems it takes 20-48 hours
  before they occour but than it doesn't stop.
 
 
 
  The strange thing I've 8 newer boxes using same os and hw where I've
  never seen those drops. Are there different xl 710 hw revisions? I
  already checked hw revision of mainboard.
 
  Stefan
 
 
 
  Excuse my typo sent from my mobile phone.
 
 
  Am 21.08.2015 um 02:51 schrieb Rose, Gregory V
  gregory.v.r...@intel.com
  mailto:gregory.v.r...@intel.com:
 
  Stefan,
 
  Late update on this.  I'm being told know that there are additional
  parameters that may need to be changed.  Please hold until I get
  this updated information.
 
  Thanks and regards,
 
  - Greg
 
 
  -Original Message-
 
  From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
 
  Sent: Thursday, August 20, 2015 4:26 PM
 
  To: Stefan Priebe
 
  Cc: e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
 
  Subject: Re: [E1000-devel] dropped rx with i40e
 
 
 
  Stefan,
 
 
 
  You will need to run the update utility again but this time
  edit the
 
  nvmupdate.cfg file to change this:
 
 
 
  SKIP OROM: TRUE
 
 
 
  to this:
 
 
 
  SKIP OROM: FALSE
 
 
 
  That should then also update the option ROM for the PXE utility.
 
 
 
  Regards,
 
 
 
  - Greg
 
 
 
  -Original Message-
 
  From: Stefan Priebe [mailto:s.pri...@profihost.ag]
 
  Sent: Wednesday, August 19, 2015 10:40 AM
 
  To: Rose, Gregory V
 
  Cc: e1000-devel@lists.sourceforge.net
  mailto:e1000-devel@lists.sourceforge.net
 
  Subject: Re: [E1000-devel] dropped rx with i40e
 
 
 
  Correct message:
 
  PXE-m10: The application for the device detected a newer
  version of
 
  the nvm image than expexted.
 
 
 
  So how do i update the application code of the XL710?
 
 
 
  Stefan
 
  Am 19.08.2015 um 19:04 schrieb Stefan Priebe - Profihost AG:
 
  An additional question since I've updated to latest
  xl710 firmware
 
  released 17 August the cards show at boot time (bios
  init/ post)
 
  something like application code detected a newer nvm
  image than
 
  expected please update application software. But I can't
  find an
 
  application code update for the xl710 cards.
 
 
 
  Stefan
 
 
 
  Excuse my typo sent from my mobile phone.
 
 
 
  Am 19.08.2015 um 18:20 schrieb Rose, Gregory V
 
  gregory.v.r...@intel.com
  mailto:gregory.v.r...@intel.com
 
  mailto:gregory.v.r...@intel.com

Re: [E1000-devel] X710 / i40e interface resets

[Rose, Gregory V]

 -Original Message-
 From: Christian Ruppert [mailto:id...@qasl.de]
 Sent: Friday, August 21, 2015 7:06 AM
 To: Rose, Gregory V
 Cc: e1000-de...@lists.sf.net
 Subject: RE: [E1000-devel] X710 / i40e interface resets
 
 On 2015-07-28 18:51, Rose, Gregory V wrote:
  -Original Message-
  From: Christian Ruppert [mailto:id...@qasl.de]
  Sent: Tuesday, July 28, 2015 5:43 AM
  To: Rose, Gregory V
  Cc: e1000-de...@lists.sf.net
  Subject: RE: [E1000-devel] X710 / i40e interface resets
 

[snip]

  I've been told that the Tx driver issue detected message is due to a
  MDD (Malicious Driver Detection) event.  We need to find out what is
  causing that so if you could please send me your system log and the
  output of ethtool intf and ethtool -i intf perhaps I can see what
  might be occurring.  If not then we'll have to debug further.
 
 See the initial Mail for ethtool -k and so on. Also:
 # ethtool eth0
 Settings for eth0:
   Supported ports: [ FIBRE ]
   Supported link modes:   1000baseT/Full
   1baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: No
   Advertised link modes:  1000baseT/Full
   1baseT/Full
   Advertised pause frame use: No
   Advertised auto-negotiation: No
   Speed: 1Mb/s
   Duplex: Full
   Port: FIBRE
   PHYAD: 0
   Transceiver: external
   Auto-negotiation: off
   Supports Wake-on: d
   Wake-on: d
   Current message level: 0x000f (15)
  drv probe link timer
   Link detected: yes
 # ethtool -i eth0
 driver: i40e
 version: 1.2.48
 firmware-version: f4.33.31377 a1.2 n4.42 e191b
 bus-info: :02:00.0
 supports-statistics: yes
 supports-test: yes
 supports-eeprom-access: yes
 supports-register-dump: yes
 supports-priv-flags: yes

I also need the output of 'dmesg' - and I need to figure out how to determine 
what specific MDD event is occurring.  I'll get back to you with instructions 
on how to do that when I get the system log from you.

Thanks,

- Greg


--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel#174; Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Vlan Table in I40e XL710

[Rose, Gregory V]

> -Original Message-
> From: Shankar Krishnamurthy [mailto:shankar.krishnamur...@citrix.com]
> Sent: Tuesday, September 15, 2015 2:57 PM
> To: e1000-devel@lists.sourceforge.net
> Subject: [E1000-devel] Vlan Table in I40e XL710
> 
> Hi fortville Experts,
> 
> Was trying to use VLAN Table (for filtering) in conjunction with MACVLAN
> table mentioned in "7.4.8.3 L2 Filtering Algorithm" (also cut n' pasted
> below)
> 
> My test scenario:
> 1. In SRIOV mode, gave one VF to VM. For that, I added the corresponding
> MAC into MACVLAN table with vlan=25
> 2. Added VLAN table with vlan=25. (For p.VPE=0 I used
> 'i40e_aqc_opc_set_vsi_promiscuous_modes' opcode with vlan promiscuous.)
> 

Why?  You're setting a VLAN filter with the tag value of 25.  Why do 
promiscuous mode in addition?

> Now the packets are coming into this VM.
> 
> 1. Now, I delete the vlan=25 entry in the VLAN table (expecting that
> packets stop coming in)
> 2. Packets are still coming in.

Turn off promiscuous mode.

- Greg

> 
> But I am confused with this behavior. According the algorithm below,
> Namely:
> " EFilter = MFilter and (VFilter or p.VPE} and AddressType == Unicast;"
> 
> The packets should have stopped coming in. What am I missing?
> 
> Is there any global setting /register that I need to toggle for the VLAN
> table to be effective?
> 
> Is VSI setup needs additional flags? Not sure
> 
> thanks,
> Shankar.
> 
> I40e Driver Version: 1.2.37
> 
> 7.4.8.3 L2 Filtering Algorithm
> The following pseudo code describes the algorithm used to determine if a
> packet passes the L2 filtering
> element.
> // Global parameters
> // Define Lookup tables
> MacVlan Table: Array of {MAC (MAC Address), VLAN (VLAN tag)}
> Ethernet Controller XL710 - Internal Switch
> 728
> Mac Table: Array of {MAC (MAC Address)}
> VLAN table: Array of {VLAN (VLAN tag)}
> HashMacVlan Table: Array of {HashMAC (Hash Values), VLAN (VLAN tag),
> AddressType}
> HashMac Table: Array of {HashMAC (Hash Values) , AddressType}
> EtherType Table : Array of {Etype (Ethertypes Values)}
> MacEtherType Table: Array of {MAC (MAC Address), Etype (Ethertype value)}
> // Define Virtual ports modes
> Port.PUE // Promiscuous Unicast Enable
> Port.PME // Promiscuous Multicast Enable
> Port.BAM // Broadcast Enable
> Port.VPE // Promiscuous VLAN enable
> Port.MaxSize: Max Packet size
> L2_function(Packet)
> // Variables
> MFilter: = False // MAC Filtering
> VFilter: = False // VLAN Filtering
> EFilter: = False // Exclusive Filtering
> NEPMFilter = False // Non Exclusive Perfect MAC Filtering
> NEPFilter = False // Non Exclusive Perfect Filtering
> NEIPMFilter = False // Non Exclusive Imperfect MAC Filtering
> NEIPFilter = False // Non Exclusive Imperfect Filtering
> Pass = False // Final decision.
> //Define packet parameters
> DA = Packet.DA //Destination Address of the packet
> VID = Packet.VLAN ID // Vlan tag of the packet
> Etype = Packet.Ethertype // Ethertype of the packet
> AddressType //Type of address of the DA. Can be Unicast, Multicast or
> Broadcast.
> HDA = HashFunction(DA).
> PSize: Packet size. This do not include any tag or other header removed by
> previous stages or to be added by following stages.
> 
> // Exclusive Filters
> For Each entry e in MacVlan Table
> If (DA == e.MAC and (VID == e.VLAN or (VID == NULL and e.VLAN == 0))
> MFilter = True;
> For Each entry e in Mac Table
> If (DA == e.MAC) MFilter = True;
> For Each entry e in Ethertype Table
> If (Etype == e.Etype) MFilter = True;
> For Each entry e in MacEthertype Table
> If (DA == e.MAC and Etype == e.Etype) MFilter = True;
> For Each entry e in Vlan Table
> If (VID == e.VLAN or (VID == NULL and e.VLAN == 0)) VFilter = True;
> // VLAN filters are ANDed with the previous filters unless promiscuous
> VLAN is enabled
> EFilter = MFilter and (VFilter or p.VPE} and AddressType == Unicast;
> // Non Exclusive Perfect Filters
> If (AddressType == Unicast and p.PUE == 1) NEPMFilter = True;
> If (AddressType == Multicast and (p.PME == 1 or MFilter)) NEPMFilter =
> True;
> If (AddressType == Broadcast and (p.BAM == 1 or MFilter)) NEPMFilter =
> True;
> // VLAN filters are ANDed with the previous filters unless promiscuous
> VLAN is enabled
> NEPFilter = NEPMFilter and (VFilter or p.VPE};
> // Non Exclusive Imperfect Filters
> For Each entry e in HashMACVlan Table
> If (HDA == e.HashMAC and (VID == e.VLAN or (VID == NULL and e.VLAN == 0))
> and AddressType == e.AddressType) NEIPMFilter = True;
> For Each entry e in HashMAC Table
> If (HDA == e.HashMAC and AddressType == e.AddressType) NEIPMFilter = True;
> // VLAN filters are ANDed with the previous filters unless promiscuous
> VLAN is enabled
> NIEPFilter = NIEPMFilter and (VFilter or p.VPE};
> // Packet size filtering is done at the queue level, so it is not part of
> the switch algorithm
> Pass = (EFilter or NEPFilter or NEIPFilter)
> Return Pass;
> }
> 
> -Original Message-
> From: Ronciak, John [mailto:john.ronc...@intel.com]
> Sent: Tuesday, September 15, 

Re: [E1000-devel] new ixgbe driver for kernel 4.1 and newer?

[Rose, Gregory V]

The 4.1 kernel should have an up to date driver included in the kernel.

- Greg

> -Original Message-
> From: Stefan Priebe - Profihost AG [mailto:s.pri...@profihost.ag]
> Sent: Monday, September 21, 2015 8:03 AM
> To: e1000-devel@lists.sourceforge.net
> Subject: [E1000-devel] new ixgbe driver for kernel 4.1 and newer?
> 
> Hi,
> 
> the current ixgbe driver available is not compatible with kernel 4.1 and
> newer.
> 
> Does anybody know when there is a newer version compatible with kernel
> 4.1?
> 
> Stefan
> 
> --
> 
> ___
> E1000-devel mailing list
> E1000-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/e1000-devel
> To learn more about Intel Ethernet, visit
> http://communities.intel.com/community/wired

--
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Vlan Table in I40e XL710

[Rose, Gregory V]

From: Shankar Krishnamurthy [mailto:shankar.krishnamur...@citrix.com]
Sent: Tuesday, September 15, 2015 11:00 PM
To: Rose, Gregory V; Shankar Krishnamurthy; e1000-devel@lists.sourceforge.net
Subject: RE: [E1000-devel] Vlan Table in I40e XL710

Thanks for response.

> Why?  You're setting a VLAN filter with the tag value of 25.  Why do 
> promiscuous mode in addition?
--> I was confirming that I am _not_ setting p.VPE (essentially disabling VLAN 
promiscuous ). We are in same page.
Still the packets are not going through.

To set and clear a VLAN filter should not require any call to the 
i40e_aq_set_and_clr_vsi_vlan_promiscuous() function at all.

Use the MAC+VLAN filters.


-Greg

'set' is just the opcode name ' i40e_aqc_set_vsi_promiscuous_modes' - same is 
used to 'clear' I guess.

The following code does this when 'set=0' is passed.

i40e_status i40e_aq_set_and_clr_vsi_vlan_promiscuous(struct i40e_hw *hw,
u16 seid, bool set, struct i40e_asq_cmd_details 
*cmd_details)
{
struct i40e_aq_desc desc;
struct i40e_aqc_set_vsi_promiscuous_modes *cmd =
(struct i40e_aqc_set_vsi_promiscuous_modes *)
i40e_status status;
u16 flags = 0;

i40e_fill_default_direct_cmd_desc(,
i40e_aqc_opc_set_vsi_promiscuous_modes);

if (set)
flags |= I40E_AQC_SET_VSI_PROMISC_VLAN;

cmd->promiscuous_flags = CPU_TO_LE16(flags);

cmd->valid_flags = CPU_TO_LE16(I40E_AQC_SET_VSI_PROMISC_VLAN);

cmd->seid = CPU_TO_LE16(seid);
status = i40e_asq_send_command(hw, , NULL, 0, cmd_details);

return status;
}

In the contrary, this is not the way to set p.VPE=0 (as described in L2 
filtering algorithm) or someother/additional action needs to be taken, please 
let me know

thanks,
Shankar.

-Original Message-
From: Rose, Gregory V [mailto:gregory.v.r...@intel.com]
Sent: Tuesday, September 15, 2015 3:43 PM
To: Shankar Krishnamurthy 
<shankar.krishnamur...@citrix.com<mailto:shankar.krishnamur...@citrix.com>>; 
e1000-devel@lists.sourceforge.net<mailto:e1000-devel@lists.sourceforge.net>
Subject: RE: [E1000-devel] Vlan Table in I40e XL710

> -Original Message-
> From: Shankar Krishnamurthy [mailto:shankar.krishnamur...@citrix.com]
> Sent: Tuesday, September 15, 2015 2:57 PM
> To: 
> e1000-devel@lists.sourceforge.net<mailto:e1000-devel@lists.sourceforge.net>
> Subject: [E1000-devel] Vlan Table in I40e XL710
>
> Hi fortville Experts,
>
> Was trying to use VLAN Table (for filtering) in conjunction with
> MACVLAN table mentioned in "7.4.8.3 L2 Filtering Algorithm" (also cut
> n' pasted
> below)
>
> My test scenario:
> 1. In SRIOV mode, gave one VF to VM. For that, I added the
> corresponding MAC into MACVLAN table with vlan=25 2. Added VLAN table
> with vlan=25. (For p.VPE=0 I used
> 'i40e_aqc_opc_set_vsi_promiscuous_modes' opcode with vlan
> promiscuous.)
>

Why?  You're setting a VLAN filter with the tag value of 25.  Why do 
promiscuous mode in addition?

> Now the packets are coming into this VM.
>
> 1. Now, I delete the vlan=25 entry in the VLAN table (expecting that
> packets stop coming in) 2. Packets are still coming in.

Turn off promiscuous mode.

- Greg

>
> But I am confused with this behavior. According the algorithm below,
> Namely:
> " EFilter = MFilter and (VFilter or p.VPE} and AddressType == Unicast;"
>
> The packets should have stopped coming in. What am I missing?
>
> Is there any global setting /register that I need to toggle for the
> VLAN table to be effective?
>
> Is VSI setup needs additional flags? Not sure
>
> thanks,
> Shankar.
>
> I40e Driver Version: 1.2.37
>
> 7.4.8.3 L2 Filtering Algorithm
> The following pseudo code describes the algorithm used to determine if
> a packet passes the L2 filtering element.
> // Global parameters
> // Define Lookup tables
> MacVlan Table: Array of {MAC (MAC Address), VLAN (VLAN tag)} Ethernet
> Controller XL710 - Internal Switch
> 728
> Mac Table: Array of {MAC (MAC Address)} VLAN table: Array of {VLAN
> (VLAN tag)} HashMacVlan Table: Array of {HashMAC (Hash Values), VLAN
> (VLAN tag), AddressType} HashMac Table: Array of {HashMAC (Hash
> Values) , AddressType} EtherType Table : Array of {Etype (Ethertypes
> Values)} MacEtherType Table: Array of {MAC (MAC Address), Etype
> (Ethertype value)} // Define Virtual ports modes Port.PUE //
> Promiscuous Unicast Enable Port.PME // Promiscuous Multicast Enable
> Port.BAM // Broadcast Enable Port.VPE // Promiscuous VLAN enable
> Port.MaxSize: Max Packet size
> L2_function(Packet)
> // Variables
> MFilter: = False // MAC Filtering
> VFilter: = False // VLAN Filtering
> EFilter: = False // Excl

Re: [E1000-devel] Can we support XL710 VF by modifying current Intel 82599 VF driver

[Rose, Gregory V]

I really think you'd be better off going from the FreeBSD driver.  The two 
devices are quite dissimilar.

- Greg

> -Original Message-
> From: Lu, Shengcong (Nokia - CN/Hangzhou) [mailto:shengcong...@nokia.com]
> Sent: Thursday, December 03, 2015 6:54 PM
> To: e1000-devel@lists.sourceforge.net
> Subject: [E1000-devel] Can we support XL710 VF by modifying current Intel
> 82599 VF driver
> 
> Hi,
> 
> We have an own OS called DMX and it's totally different from Linux or
> FreeBSD. There we have Intel 82599 drivers.
> 
> To write new driver in DMX OS is not small work. So we are thinking if
> it's possible to make some modification based on 82599 VF driver to
> support XL710 VF.
> 
> Can I get any answer help here? Thanks in advance!
> 
> BR, Lu
> 
> 


--
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] i40e and bonding

[Rose, Gregory V]

oremove_wake_function+0x0/0x40
> [] ? worker_thread+0x0/0x2a0
> [] ? kthread+0x9e/0xc0
> [] ? child_rip+0xa/0x20
> [] ? kthread+0x0/0xc0
> [] ? child_rip+0x0/0x20
> ---[ end trace 2c401f7c7f160a8e ]---
> [ cut here ]
> WARNING: at drivers/net/i40e/i40e_main.c:1539
> i40e_vsi_setup_queue_map+0x30c/0x330 [i40e]() (Tainted: G W -- ---
> - )
> Hardware name: PowerEdge R430
> Modules linked in: bonding ipv6 joydev sg microcode ipmi_devintf iTCO_wdt
> iTCO_vendor_support dcdbas tg3 power_meter acpi_ipmi ipmi_si
> ipmi_msghandler i40e configfs ptp pps_core sb_edac edac_core lpc_ich
> mfd_core shpchp ext4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif xhci_hcd
> ahci megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last
> unloaded: scsi_wait_scan]
> Pid: 131, comm: events/0 Tainted: G W --  2.6.32-
> 573.8.1.el6.x86_64 #1
> Call Trace:
> [] ? warn_slowpath_common+0x91/0xe0
> [] ? warn_slowpath_null+0x1a/0x20
> [] ? i40e_vsi_setup_queue_map+0x30c/0x330 [i40e]
> [] ? i40e_vsi_config_tc+0x1e4/0x320 [i40e]
> [] ? i40e_handle_lldp_event+0x2b0/0x650 [i40e]
> [] ? i40e_service_task+0xcf8/0x1180 [i40e]
> [] ? i40e_service_task+0x0/0x1180 [i40e]
> [] ? worker_thread+0x170/0x2a0
> [] ? autoremove_wake_function+0x0/0x40
> [] ? worker_thread+0x0/0x2a0
> [] ? kthread+0x9e/0xc0
> [] ? child_rip+0xa/0x20
> [] ? kthread+0x0/0xc0
> [] ? child_rip+0x0/0x20
> ---[ end trace 2c401f7c7f160a8f ]---
> i40e :04:00.0: veb bw config failed, aq_err=14
> i40e :04:00.0: Failed configuring TC for VEB seid=288
> i40e :04:00.0: AQ command Config VSI BW allocation per TC failed = 14
> i40e :04:00.0: Failed configuring TC map 255 for VSI 516
> i40e :04:00.0: Failed configuring TC for VSI seid=516
> i40e :04:00.1: veb bw config failed, aq_err=14
> i40e :04:00.1: Failed configuring TC for VEB seid=289
> i40e :04:00.1: AQ command Config VSI BW allocation per TC failed = 14
> i40e :04:00.1: Failed configuring TC map 255 for VSI 517
> i40e :04:00.1: Failed configuring TC for VSI seid=517
> alloc irq_desc for 320 on node 0
> alloc kstat_irqs on node 0
> 
> 
> --
> Andy
> 
> 
> 
> 
> 
> On 01/12/2015, 18:59, "Rose, Gregory V" <gregory.v.r...@intel.com> wrote:
> 
> >Please provide the output of 'ethtool -i' and 'ethtool' for each
> XL710/X710 interface in the system as well as the output of 'dmesg'
> *after* the error has occurred.
> >
> >Thanks,
> >
> >- Greg
> >
> >> -Original Message-
> >> From: Andy Fletcher [mailto:andy.fletc...@ukdedicated.com]
> >> Sent: Friday, November 27, 2015 9:16 AM
> >> To: e1000-devel@lists.sourceforge.net
> >> Subject: [E1000-devel] i40e and bonding
> >>
> >> Hello,
> >>
> >> We’re trying to get the XL710 working with the Linux bonding driver,
> but
> >> the driver simply refuses to enslave the 10G interfaces:
> >>
> >> # ifenslave bond1 p1p1
> >>
> >> Master ‘bond1', Slave 'p1p1': Error: Enslave failed strace on this
> shows:
> >> ioctl(3, SIOCBONDENSLAVE, 0x7ffd6c35c560) = -1 EINVAL (Invalid
> argument)
> >>
> >>
> >> Furthermore, if moving from our distro (RHEL6) driver (1.2.9-k) to
> >> anything from the i40e stable, this causes a kernel panic on bringing
> up
> >> the bonded interface.
> >>
> >> The driver notes say that bonding is supported, so it’s not clear why
> this
> >> is failing. Can anyone confirm if this is actually supported?
> >>
> >> Kernel panic:
> >> <6>bond1: Enslaving p2p1 as an active interface with an up link
> >> <1>BUG: unable to handle kernel NULL pointer dereference at
> >> 001c
> >> <1>IP: [] i40e_tx_map+0x77/0x4b0 [i40e] <4>PGD 0
> >> <4>Oops: 0002 [#1] SMP
> >> <4>last sysfs file: /sys/devices/virtual/net/bond1/broadcast
> >> <4>CPU 18
> >> <4>Modules linked in: bonding i40e(U) ipv6 microcode sg joydev
> >> ipmi_devintf iTCO_wdt iTCO_vendor_support dcdbas tg3 power_meter
> acpi_ipmi
> >> ipmi_si ipmi_msghandler configfs ptp pps_core sb_edac edac_core lpc_ich
> >> mfd_core shpchp ext4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif
> >> megaraid_sas ahci wmi dm_mirror dm_region_hash dm_log dm_mod [last
> >> unloaded: i40e] <4>
> >>
> >>
> >>
> >> --
> >> Andy
> This message and any attachments may be private or confidential. If you
> have received this message in error then please remove it from your
> syste

Re: [E1000-devel] i40e and bonding

[Rose, Gregory V]

Please provide the output of 'ethtool -i' and 'ethtool' for each XL710/X710 
interface in the system as well as the output of 'dmesg' *after* the error has 
occurred.

Thanks,

- Greg

> -Original Message-
> From: Andy Fletcher [mailto:andy.fletc...@ukdedicated.com]
> Sent: Friday, November 27, 2015 9:16 AM
> To: e1000-devel@lists.sourceforge.net
> Subject: [E1000-devel] i40e and bonding
> 
> Hello,
> 
> We’re trying to get the XL710 working with the Linux bonding driver, but
> the driver simply refuses to enslave the 10G interfaces:
> 
> # ifenslave bond1 p1p1
> 
> Master ‘bond1', Slave 'p1p1': Error: Enslave failed strace on this shows:
> ioctl(3, SIOCBONDENSLAVE, 0x7ffd6c35c560) = -1 EINVAL (Invalid argument)
> 
> 
> Furthermore, if moving from our distro (RHEL6) driver (1.2.9-k) to
> anything from the i40e stable, this causes a kernel panic on bringing up
> the bonded interface.
> 
> The driver notes say that bonding is supported, so it’s not clear why this
> is failing. Can anyone confirm if this is actually supported?
> 
> Kernel panic:
> <6>bond1: Enslaving p2p1 as an active interface with an up link
> <1>BUG: unable to handle kernel NULL pointer dereference at
> 001c
> <1>IP: [] i40e_tx_map+0x77/0x4b0 [i40e] <4>PGD 0
> <4>Oops: 0002 [#1] SMP
> <4>last sysfs file: /sys/devices/virtual/net/bond1/broadcast
> <4>CPU 18
> <4>Modules linked in: bonding i40e(U) ipv6 microcode sg joydev
> ipmi_devintf iTCO_wdt iTCO_vendor_support dcdbas tg3 power_meter acpi_ipmi
> ipmi_si ipmi_msghandler configfs ptp pps_core sb_edac edac_core lpc_ich
> mfd_core shpchp ext4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif
> megaraid_sas ahci wmi dm_mirror dm_region_hash dm_log dm_mod [last
> unloaded: i40e] <4>
> 
> 
> 
> --
> Andy
> 
> 
> 
> 
> 
> This message and any attachments may be private or confidential. If you
> have received this message in error then please remove it from your
> systems and notify us. All communications are subject to:
> http://www.ukdedicated.com/email/
> 
> UKDedicated Ltd - Registered in England and Wales - No. 04625539. VAT No.
> GB814020091. Registered Office: 3 Centro, Boundary Way, Hemel Hempstead,
> HP2 7SU
> --
> 
> ___
> E1000-devel mailing list
> E1000-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/e1000-devel
> To learn more about Intel Ethernet, visit
> http://communities.intel.com/community/wired
--
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] X710 (i40e) direct-attach-cable problem

[Rose, Gregory V]

The FW and SW drivers are tightly coupled on X710 - it is best to update both 
at the same time.

Thanks,

- Greg

> -Original Message-
> From: Sven Anders [mailto:and...@anduras.de]
> Sent: Tuesday, November 24, 2015 2:10 AM
> To: e1000-devel@lists.sourceforge.net
> Subject: Re: [E1000-devel] X710 (i40e) direct-attach-cable problem
> 
> Am 23.11.2015 um 17:29 schrieb Sven Anders:
> > Hello!
> >
> > I have a problem with my Intel Dual Ethernet controller (Intel
> > Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)).
> > I want to use a direct-attached cable to connect my server with a HP
> > switch.
> >
> > It works ok, if I attach the cable before booting/starting the server.
> > The "ip link" command show "UP,LOWER_UP". Everything is ok.
> >
> > But, if I attach the cable after booting the server, the connection is
> > "DOWN" and will only come up, if I run an "ethtool -t eth2" test on it.
> > A single "ethtool -r eth2" does not help.
> >
> > I see the same behavior, if I connect the two ports of the Intel card
> > directly, so the HP switch does not cause it.
> >
> > I tried the following Linux kernels:
> >  - 4.1.12 (with i40e in version 1.3.2)
> >  - 4.3 (with i40e in version 1.3.9)
> >
> >> ethtool -i eth2
> > driver: i40e
> > version: 1.3.9-k
> > firmware-version: f4.22.27454 a1.1 n4.25 e143f
> > bus-info: :03:00.0
> > supports-statistics: yes
> > supports-test: yes
> > supports-eeprom-access: yes
> > supports-register-dump: yes
> > supports-priv-flags: yes
> >
> > The cable I use is:
> >  INTEL(R) ETHERNET SFP+ TWINAXIAL CABLE, XDACBL3M
> >  Version: G45737-003   Date: 09/02/2015
> >
> > I this error known or any ideas, what causes it?!
> 
> Hello!
> 
> To answer by own question...
> 
> I tried the Intel driver (1.3.47) and had the same problems.
> Then I tried Windows 2012R2 and it worked. So the hardware wasn't faulty.
> 
> As a last resort I tried the newest firmware and IT WORKED!
> 
> (New firmware-version: f4.40.35115 a1.4 n4.53 e1dc0 )
> 
> Sorry for the noise, but I didn't expect the firmware causing this.
> 
> Regards
>  Sven Anders
> --
>  Sven Anders  () UTF-8 Ribbon Campaign
>  /\ Support plain text e-
> mail  ANDURAS intranet security AG  Messestrasse 3 - 94036 Passau -
> Germany
>  Web: www.anduras.de - Tel: +49 (0)851-4 90 50-0 - Fax: +49 (0)851-4 90
> 50-55
> 
> Those who would give up essential Liberty, to purchase a little temporary
> Safety, deserve neither Liberty nor Safety.
>   - Benjamin Franklin
> 
> 
> --
> 
> Go from Idea to Many App Stores Faster with Intel(R) XDK Give your users
> amazing mobile app experiences with Intel(R) XDK.
> Use one codebase in this all-in-one HTML5 development environment.
> Design, debug & build mobile apps & 2D/3D high-impact games for multiple
> OSs.
> http://pubads.g.doubleclick.net/gampad/clk?id=254741551=/4140
> ___
> E1000-devel mailing list
> E1000-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/e1000-devel
> To learn more about Intel Ethernet, visit
> http://communities.intel.com/community/wired

--
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Connectivity Problems with i40e-1.4.25 Driver

[Rose, Gregory V]

You should first upgrade the FW on your X710 adapters using this tool:

https://downloadcenter.intel.com/download/25791

And then retry the test.

- Greg

-Original Message-
From: Reuben Farrelly [mailto:reuben-sourceforge-e1...@reub.net] 
Sent: Friday, March 11, 2016 5:32 AM
To: e1000-devel@lists.sourceforge.net
Subject: [E1000-devel] Connectivity Problems with i40e-1.4.25 Driver

Hi,

I have an Intel X710DA2 NIC in a backup server I maintain at home.  The server 
runs Gentoo Linux x86_64 and is kept pretty up to date - has been running 
gentoo 4.4.x kernels for some time now.

The server is used to back up around 1000G worth of data each week. This takes 
the form of a wake on LAN packet sent to the system which then wakes up, boots 
up and pulls down the data via scp before shutting down again.  The data takes 
the form of an 880G tar file and some other smaller tarballs.

The servers are in slightly different locations but are connected via a 
back-to-back 10G OM4 fibre with Intel optics end to end.  The remote end has an 
X520 card and is running ESXi 6.

Using the standard in-tree kernel i40e driver 1.3.46 I am able to get fairly 
good throughputs and able to transfer this data in about 90 minutes without 
stopping.

However with the latest released i40e driver version 1.4.25 (from
https://sourceforge.net/projects/e1000/files/i40e%20stable/) the backup job 
always fails.  Somewhere after the transfer starts - perhaps 100G or so in - 
the connectivity falls over and the scp session doing the transfer fails .  The 
job can be restarted again but it then fails again somewhere mid way through 
the transfer.

The backup job logs this:

/bin/tar: Removing leading `/' from member names Total bytes written: 90357760 
(87MiB, 2.6MiB/s)
packet_write_wait: Connection to 2001:44b8::::2: Broken pipe lost 
connection
/bin/tar: Removing leading `/' from member names
packet_write_wait: Connection to 2001:44b8::::2: Broken pipe

This problem *doesn't* occur with the in-tree kernel driver which leads me to 
believe there is an issue with the 1.4 kernel that is posted online.

Here's the dmesg output from the in-tree kernel driver:

i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.3.46-k
i40e: Copyright (c) 2013 - 2014 Intel Corporation.
i40e :07:00.0: fw 5.0.40043 api 1.5 nvm 5.02 0x80002282 0.0.0 i40e 
:07:00.0: The driver for the device detected a newer version of the NVM 
image than expected. Please install the most recent version of the network 
driver.
i40e :07:00.0: MAC address: 68:05:ca:30:53:d0 i40e :07:00.0: 
PCI-Express: Speed 8.0GT/s Width x8 i40e :07:00.0: Features: PF-id[0] VSIs: 
66 QP: 8 RX: PS RSS FD_ATR FD_SB NTUPLE PTP VEPA i40e :07:00.1: fw 
5.0.40043 api 1.5 nvm 5.02 0x80002282 0.0.0 i40e :07:00.1: The driver for 
the device detected a newer version of the NVM image than expected. Please 
install the most recent version of the network driver.
i40e :07:00.1: MAC address: 68:05:ca:30:53:d1 i40e :07:00.1: 
PCI-Express: Speed 8.0GT/s Width x8 i40e :07:00.1: Features: PF-id[1] VSIs: 
66 QP: 8 RX: PS RSS FD_ATR FD_SB NTUPLE PTP VEPA i40e :07:00.1 enp7s0f1: 
renamed from eth1 i40e :07:00.0 enp7s0f0: renamed from eth0 i40e 
:07:00.0 enp7s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow
Control: None
i40e :07:00.0 enp7s0f0: changing MTU from 1500 to 9000

The connectivity is straight IP, no other traffic on the card, no FC, no VLANs, 
nothing.  Just IP over 10G Ethernet with scp and an MTU of 9000.

Can someone suggest what we can do to narrow this down and ideally fix this in 
the 1.4 driver?

I can run this with the in-tree kernel driver with is fine for now but I guess 
this still should really be fixed in the latest released driver anyway.

Thanks,
Reuben


--
Transform Data into Opportunity.
Accelerate data analysis in your applications with Intel Data Analytics 
Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

--
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel Ethernet, visit 
http://communities.intel.com/community/wired

Re: [E1000-devel] Connectivity Problems with i40e-1.4.25 Driver

[Rose, Gregory V]

Reuben,

My apologies.  I read message from the driver below completely backwards from 
its actual meaning.

In fact you are using the newest NVM and the 1.4.25 driver is also the newest 
driver.  So let's restart this conversation.

Please send me the output of dmesg after your system is up and running and then 
after the error occurs.  Also, I need the output of 'ethtool -i' and 'ethtool' 
for each X710 interface on the system.  And then the output of 'ip addr show'.

Thanks,

- Greg

-Original Message-
From: Rose, Gregory V 
Sent: Monday, March 14, 2016 9:49 AM
To: 'Reuben Farrelly' <reuben-sourceforge-e1...@reub.net>; 
e1000-devel@lists.sourceforge.net
Subject: RE: [E1000-devel] Connectivity Problems with i40e-1.4.25 Driver

You should first upgrade the FW on your X710 adapters using this tool:

https://downloadcenter.intel.com/download/25791

And then retry the test.

- Greg

-Original Message-
From: Reuben Farrelly [mailto:reuben-sourceforge-e1...@reub.net] 
Sent: Friday, March 11, 2016 5:32 AM
To: e1000-devel@lists.sourceforge.net
Subject: [E1000-devel] Connectivity Problems with i40e-1.4.25 Driver

Hi,

I have an Intel X710DA2 NIC in a backup server I maintain at home.  The server 
runs Gentoo Linux x86_64 and is kept pretty up to date - has been running 
gentoo 4.4.x kernels for some time now.

The server is used to back up around 1000G worth of data each week. This takes 
the form of a wake on LAN packet sent to the system which then wakes up, boots 
up and pulls down the data via scp before shutting down again.  The data takes 
the form of an 880G tar file and some other smaller tarballs.

The servers are in slightly different locations but are connected via a 
back-to-back 10G OM4 fibre with Intel optics end to end.  The remote end has an 
X520 card and is running ESXi 6.

Using the standard in-tree kernel i40e driver 1.3.46 I am able to get fairly 
good throughputs and able to transfer this data in about 90 minutes without 
stopping.

However with the latest released i40e driver version 1.4.25 (from
https://sourceforge.net/projects/e1000/files/i40e%20stable/) the backup job 
always fails.  Somewhere after the transfer starts - perhaps 100G or so in - 
the connectivity falls over and the scp session doing the transfer fails .  The 
job can be restarted again but it then fails again somewhere mid way through 
the transfer.

The backup job logs this:

/bin/tar: Removing leading `/' from member names Total bytes written: 90357760 
(87MiB, 2.6MiB/s)
packet_write_wait: Connection to 2001:44b8::::2: Broken pipe lost 
connection
/bin/tar: Removing leading `/' from member names
packet_write_wait: Connection to 2001:44b8::::2: Broken pipe

This problem *doesn't* occur with the in-tree kernel driver which leads me to 
believe there is an issue with the 1.4 kernel that is posted online.

Here's the dmesg output from the in-tree kernel driver:

i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.3.46-k
i40e: Copyright (c) 2013 - 2014 Intel Corporation.
i40e :07:00.0: fw 5.0.40043 api 1.5 nvm 5.02 0x80002282 0.0.0 i40e 
:07:00.0: The driver for the device detected a newer version of the NVM 
image than expected. Please install the most recent version of the network 
driver.
i40e :07:00.0: MAC address: 68:05:ca:30:53:d0 i40e :07:00.0: 
PCI-Express: Speed 8.0GT/s Width x8 i40e :07:00.0: Features: PF-id[0] VSIs: 
66 QP: 8 RX: PS RSS FD_ATR FD_SB NTUPLE PTP VEPA i40e :07:00.1: fw 
5.0.40043 api 1.5 nvm 5.02 0x80002282 0.0.0 i40e :07:00.1: The driver for 
the device detected a newer version of the NVM image than expected. Please 
install the most recent version of the network driver.
i40e :07:00.1: MAC address: 68:05:ca:30:53:d1 i40e :07:00.1: 
PCI-Express: Speed 8.0GT/s Width x8 i40e :07:00.1: Features: PF-id[1] VSIs: 
66 QP: 8 RX: PS RSS FD_ATR FD_SB NTUPLE PTP VEPA i40e :07:00.1 enp7s0f1: 
renamed from eth1 i40e :07:00.0 enp7s0f0: renamed from eth0 i40e 
:07:00.0 enp7s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow
Control: None
i40e :07:00.0 enp7s0f0: changing MTU from 1500 to 9000

The connectivity is straight IP, no other traffic on the card, no FC, no VLANs, 
nothing.  Just IP over 10G Ethernet with scp and an MTU of 9000.

Can someone suggest what we can do to narrow this down and ideally fix this in 
the 1.4 driver?

I can run this with the in-tree kernel driver with is fine for now but I guess 
this still should really be fixed in the latest released driver anyway.

Thanks,
Reuben


--
Transform Data into Opportunity.
Accelerate data analysis in your applications with Intel Data Analytics 
Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140
___
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourcefor