On 1/15/2020 8:43 PM, Thomas Monjalon wrote: > 15/01/2020 19:35, Ferruh Yigit: >> On 1/15/2020 6:49 AM, 方统浩50450 wrote: >>> Hi Ferruh, thanks for your message. >>> >>> >>> We developed a ethtool-dpdk which is secondary process based dpdk 17.08 >>> version. Our device >>> support hotplug detach, but hotplug deatch is failed when we use >>> ethtool-dpdk.We found the >>> secondary process will change the shared memory when initializing.Secondary >>> process calls >>> "rte_eth_dev_pci_allocate" function and enters "rte_eth_copy_pci_info" >>> function. >>> (rte_eth_dev_pci_generic_probe -> rte_eth_dev_pci_allocate -> >>> rte_eth_copy_pci_info) >>> Then it sets the value of struct "rte_eth_dev_data.dev_flags" to zero.In >>> our platform, this value >>> is equal to 0x0003.(RTE_ETH_DEV_DETACHABLE | RTE_ETH_DEV_INTR_LSC),but >>> after reset >>> the "dev_flags", the value changed to 0x0002.(RTE_ETH_DEV_DETACHABLE).So, >>> our device hotplug >>> detach is failed.I found the similar problem in other dpdk version, include >>> dpdk 19.11.Even though >>> the deivce hotplug detach is discarded,but i think the shared memory >>> changed is unexpected by primary >>> process. >> >> I agree this is the problem. >> In the driver code, 'rte_eth_copy_pci_info' is called only by primary >> process, >> but the generic code is faulty. >> >> And in 19.11 additionally 'eth_dev_pci_specific_init' also seems has same >> problem. >> >>> Our driver is ixgbe, i think this problem has a little relationship with >>> driver, Secondary process >>> enters "rte_eth_copy_pci_info" by "rte_eth_dev_pci_allocate".And I agree >>> your opinion, the helper >>> function should simple on what it does.I have two ways to fix this problem, >>> one is add an if-statement >>> >>> in "rte_eth_dev_pci_allocate" function to forbid secondary process enters >>> "rte_eth_copy_pci_info" function, >>> another way is add an if-statement in "rte_eth_copy_pci_info" function to >>> forbid secondary process change >>> shared memory.And First way need to ensure the "rte_eth_copy_pci_info" >>> function won't be called anywhere else. >>> I think the second way is simple and lower risk. >> >> Yes these are the two options. >> >> I agree adding check in the 'rte_eth_copy_pci_info' covers all cases and >> safer. >> BUT my concern was adding decision making to simple/leaf function and make it >> harder to debug/use, instead of giving what primary/secondary process should >> call decision in higher level. >> >> But I just recognized that some PMDs are calling 'rte_eth_copy_pci_info' on >> secondary process, like mlx4 or szedata2, and most probably this is not their >> intention. >> And 'eth_dev->intr_handle' set in 'rte_eth_copy_pci_info', not calling this >> function may have side affect of 'eth_dev->intr_handle' not set in secondary. >> >> With above considerations I am OK to your proposal to cover all cases, >> Thomas, >> Andrew, any concern? > > Do you mean drivers need to be fixed? >
either it or 'rte_eth_copy_pci_info'. Right now 'rte_eth_copy_pci_info' updates the shared memory, calling it in secondary overwrites the memory set by primary. Options Fang mentioned: 1) Don't call 'rte_eth_copy_pci_info' from secondary process path, this requires fixing 'rte_eth_dev_pci_allocate', 'eth_dev_pci_specific_init' and possibly some drivers. 2) Add a check inside the 'rte_eth_copy_pci_info' to prevent updating shared memory if it is secondary process. Fang's patch does (2), and I am OK with it as well after latest findings.