Hi Looks like there is a bug in API "ixgbevf_remove_mac_addr()". For deleting a given MAC it deletes all MAC (including permanent MAC) and while adding (it does after few statements), it skips to add permanent MAC.
ixgbevf_remove_mac_addr(struct rte_eth_dev *dev, uint32_t index) { ....... /* * The IXGBE_VF_SET_MACVLAN command of the ixgbe-pf driver does * not support the deletion of a given MAC address. * Instead, it imposes to delete all MAC addresses, then to add again * all MAC addresses with the exception of the one to be deleted. */ (void) ixgbevf_set_uc_addr_vf(hw, 0, NULL); ... 2. Following modification at "ixgbevf_remove_mac_addr()} also helped to make ping works $ git diff dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c --- a/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c +++ b/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c @@ -3530,10 +3557,7 @@ ixgbevf_remove_mac_addr(struct rte_eth_dev *dev, uint32_t index) /* Skip NULL MAC addresses */ if (is_zero_ether_addr(mac_addr)) continue; - /* Skip the permanent MAC address */ - if (memcmp(perm_addr, mac_addr, sizeof(struct ether_addr)) == 0) - continue; On Mon, Apr 25, 2016 at 7:05 PM, santosh <santosh.iitg at gmail.com> wrote: > Hi Ivan > > ixgbevf_set_default_mac_addr() could not find in our code base. > put traces at other places as suggested by you. > Log at "eth_ixgbevf_dev_init" never displayed > Rest logs displayed as shown below. I re-build the driver module and > loaded on our virtual router and rebooted the system. > > 1. > Logs at the time of boot up: > ----------------------------------------- > > INIT: Initializing NIC port 0 RX queue 0 ... > INIT: Initializing NIC port 0 TX queue 0 ... > Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2 > ..... > .... > > Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2 > RUNTIME: Detected port 0 status changed to UP. > > 2. > a. > Configured a new MAC at Virtual Router CLI > > root at mx86-bgl-2-r1# set interfaces ge-0/0/0 mac 00:50:56:a0:a0:c3 > > [edit] > root at mx86-bgl-2-r1# commit > commit complete > > [edit] > root at mx86-bgl-2-r1# show interfaces > ge-0/0/0 { > mac 00:50:56:a0:a0:c3; > unit 0 { > family inet { > address 10.1.1.102/24; > } > } > } > > [edit] > root at mx86-bgl-2-r1# run ping 10.1.1.101 > PING 10.1.1.101 (10.1.1.101): 56 data bytes > 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.142 ms > 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.465 ms > ^C > --- 10.1.1.101 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max/stddev = 2.142/5.303/8.465/3.162 ms > > [edit] > root at mx86-bgl-2-r1# > > b. > Logs at console for above config: > ------------------------------------------------- > > root at localhost:/home/pfe# Santosh ixgbevf_add_mac_addr portid=0 > mac=00:50:56:A0:A0:C3 > > root at localhost:/home/pfe# > > > 3. > a. Deleted above config MAC > root at mx86-bgl-2-r1# delete interfaces ge-0/0/0 mac > > [edit] > root at mx86-bgl-2-r1# commit > commit complete > > [edit] > root at mx86-bgl-2-r1# show interfaces > ge-0/0/0 { > unit 0 { > family inet { > address 10.1.1.102/24; > } > } > } > > [edit] > root at mx86-bgl-2-r1# run ping 10.1.1.101 > PING 10.1.1.101 (10.1.1.101): 56 data bytes > ^C > --- 10.1.1.101 ping statistics --- > 3 packets transmitted, 0 packets received, 100% packet loss > > [edit] > root at mx86-bgl-2-r1# > > b. > Logs at console for above cofig > ------------------------- > > > root at localhost:/home/pfe# ixgbevf_remove_mac_addr santosh portid=0 > index=1 mac=00:50:56:A0:A0:C3 > > > > Thanks > Santosh > > > > > > > > > > > > > > > On Fri, Apr 22, 2016 at 12:51 PM, Ivan Boule <ivan.boule at 6wind.com> wrote: >> Hi Santosh, >> >> My job at 6WIND does not consist in answering to DPDK questions. In general, >> I have other priorities, including vacations... >> In the meantime, nobody prevents you to add traces in the code to really >> understand what happens, as suggested in my last answer. >> >> Regards, >> Ivan >> >> >> On 04/21/2016 07:55 AM, santosh wrote: >>> >>> Hi Ivan and team, >>> >>> Please respond to my last mail and let me know if there is any >>> alternate way to handle this. >>> Our release is in pending due to this issue. >>> >>> >>> Thanks & Regards >>> Santosh >>> >>> On Wed, Apr 20, 2016 at 2:35 PM, santosh <santosh.iitg at gmail.com> wrote: >>>> >>>> Hi Ivan, >>>> >>>> Thanks for your response. >>>> >>>> Let me explain you the issue that we are facing on our virtual router >>>> in VMware environment. >>>> >>>> 1. We are using ixgbe driver , SRIOV enabled . >>>> root at localhost:~# lspci >>>> .... "Intel Corporation 82599 Ethernet Controller Virtual >>>> Function" >>>> >>>> 2. "mx86-bgl-1-r1" is our router under testing and R2 is a standard >>>> router. >>>> >>>> mx86-bgl-1-r1 is connected to R2 >>>> mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101) >>>> >>>> 2. This time ping test passes >>>> >>>> root at mx86-bgl-1-r1# show interfaces >>>> ge-0/0/0 { >>>> unit 0 { >>>> family inet { >>>> address 10.1.1.103/24; >>>> } >>>> } >>>> } >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# >>>> root at mx86-bgl-1-r1# run ping 10.1.1.101 >>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms >>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms >>>> >>>> >>>> 3. Default MAC address of interface ge-0/0/0 is as shown below: >>>> >>>> root at mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep >>>> current >>>> Current address: 02:06:0a:0e:ff:f0, Hardware address: >>>> 02:06:0a:0e:ff:f0 >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# >>>> >>>> >>>> 4. Now I am changing MAC. Ping works this time also >>>> >>>> root at mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0 >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# commit >>>> commit complete >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep >>>> current >>>> Current address: 02:06:0a:0a:ff:f0, Hardware address: >>>> 02:06:0a:0e:ff:f0 >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# show interfaces >>>> ge-0/0/0 { >>>> mac 02:06:0a:0a:ff:f0; >>>> unit 0 { >>>> family inet { >>>> address 10.1.1.103/24; >>>> } >>>> } >>>> } >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# >>>> root at mx86-bgl-1-r1# run ping 10.1.1.101 >>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms >>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms >>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms >>>> >>>> >>>> 5. I am deleting the above configured MAC. >>>> In this case newly configured MAC as shown above will be deleted >>>> and Default MAC (02:06:0a:0e:ff:f0) >>>> will be applied. Ping fails at this step. "return statement >>>> added by you is not allowing to configure default MAC. >>>> >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# commit >>>> commit complete >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# show interfaces >>>> ge-0/0/0 { >>>> unit 0 { >>>> family inet { >>>> address 10.1.1.103/24; >>>> } >>>> } >>>> } >>>> >>>> [edit] >>>> >>>> root at mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep >>>> current // Displays value from router's local database not from >>>> NIC. >>>> Current address: 02:06:0a:0e:ff:f0, Hardware address: >>>> 02:06:0a:0e:ff:f0 >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# run ping 10.1.1.101 >>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>> ^C >>>> 2 packets transmitted, 0 packets received, 100% packet loss >>>> >>>> [edit] >>>> root at mx86-bgl-1-r1# >>>> >>>> >>>> 7. LOGs: (I have added a debug log just before the return statement >>>> that you place) >>>> >>>> Log o/p in failure case >>>> .... >>>> Santosh ixgbevf_add_mac_addr returning >>>> .... >>>> >>>> code changes: >>>> ----------------------------- >>>> ixgbevf_add_mac_addr(....) { >>>> ... >>>> if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) >>>> == 0) { >>>> PMD_DRV_LOG(DEBUG, "Existing MAC \n"); >>>> printf("Santosh %s returning \n",__FUNCTION__); >>>> return; >>>> } >>>> >>>> >>>> 8. If I remove the above "return" statement, build the driver, loaded >>>> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete >>>> on our router then ping passes. >>>> >>>> root at mx86-bgl-1-r1# run ping 10.1.1.101 >>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms >>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms >>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms >>>> ^C >>>> --- 10.1.1.101 ping statistics --- >>>> 3 packets transmitted, 3 packets received, 0% packet loss >>>> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms >>>> >>>> >>>> 9. Please let me know whether is it fine to remove that return >>>> statement added by you in "ixgbevf_add_mac_addr" ? >>>> >>>> >>>> >>>> Thanks & Regards, >>>> Santosh >>>> >>>> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule at 6wind.com> >>>> wrote: >>>>> >>>>> Hi Santosh, >>>>> >>>>> I do not get exactly what you attempt to do on a VF. >>>>> Are you first deleting the so-called permanent MAC address by a call to >>>>> the >>>>> function ixgbevf_remove_mac_addr() ? This operation is not allowed. >>>>> Can you explain exactly the sequence of operations that are done, so >>>>> that I >>>>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr, >>>>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr() >>>>> prevents them to be successfully performed. >>>>> >>>>> Ivan >>>>> >>>>> PS : please, can you CC your emails to dev at dpdk.org >>>>> >>>>> >>>>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg at gmail.com>: >>>>>> >>>>>> >>>>>> Hi Ivan, >>>>>> >>>>>> Your following code changes causing issue in Vmware environment. >>>>>> >>>>>> ----------------------------------- ------------------- >>>>>> ------------------------------ >>>>>> + /* >>>>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent >>>>>> + * operation. Trap this case to avoid exhausting the [very limited] >>>>>> + * set of PF resources used to store VF MAC addresses. >>>>>> + */ >>>>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == >>>>>> 0) >>>>>> + return; >>>>>> diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes); >>>>>> -------------------------------------------------------------------- >>>>>> ------------- ------- >>>>>> >>>>>> >>>>>> Issue: >>>>>> At CLI , we have provision to add /del MAC of an interface. >>>>>> During MAC delete, existing MAC is deleted and default MAC is applied. >>>>>> This default MAC is not being applied in VMware environment >>>>>> successfully due to "return" statement >>>>>> in your above code changes. As a result traffic is stopped completely. >>>>>> If I remove above >>>>>> "return" statement then traffic continues to flow after MAC delete. >>>>>> >>>>>> Please let me know your suggestion to handle this scenario . If I >>>>>> remove "return" what will be the consequences ? >>>>>> >>>>>> If removing "return" statement is not good idea then what are other >>>>>> way to handle MAC delete scenario ? we have only 1 VF per PF in our >>>>>> setup as of now. >>>>>> >>>>>> >>>>>> Thanks >>>>>> Santosh >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Ivan BOULE >>>>> 6WIND >>>>> Software Engineer >>>>> >>>>> Tel: +33 1 39 30 92 47 >>>>> Mob: + 33 6 77 25 26 38 >>>>> Fax: +33 1 39 30 92 11 >>>>> ivan.boule at 6wind.com >>>>> www.6wind.com >>>>> Join the Multicore Packet Processing Forum: >>>>> www.multicorepacketprocessing.com >>>>> >>>>> Ce courriel ainsi que toutes les pi?ces jointes, est uniquement destin? >>>>> ? >>>>> son ou ses destinataires. Il contient des informations confidentielles >>>>> qui >>>>> sont la propri?t? de 6WIND. Toute r?v?lation, distribution ou copie des >>>>> informations qu'il contient est strictement interdite. Si vous avez re?u >>>>> ce >>>>> message par erreur, veuillez imm?diatement le signaler ? l'?metteur et >>>>> d?truire toutes les donn?es re?ues. >>>>> >>>>> This e-mail message, including any attachments, is for the sole use of >>>>> the >>>>> intended recipient(s) and contains information that is confidential and >>>>> proprietary to 6WIND. All unauthorized review, use, disclosure or >>>>> distribution is prohibited. If you are not the intended recipient, >>>>> please >>>>> contact the sender by reply e-mail and destroy all copies of the >>>>> original >>>>> message. >>>>> >> >> >> -- >> Ivan Boule >> 6WIND Development Engineer