Re: PCI MSI issue with reinserting a driver

2021-04-06 Thread John Garry
On 03/02/2021 17:23, Marc Zyngier wrote: On 2021-02-02 15:46, John Garry wrote: On 02/02/2021 14:48, Marc Zyngier wrote: Not sure. I also now notice an error for the SAS PCI driver on D06 when nr_cpus < 16, which means number of MSI vectors allocated < 32, so looks the same problem. There

Re: PCI MSI issue with reinserting a driver

2021-02-04 Thread John Garry
On 03/02/2021 17:23, Marc Zyngier wrote: Before:  In free path, we have: its_irq_domain_free(nvecs = 27)    bitmap_release_region(count order = 5 == 32bits) Current:  In free path, we have: its_irq_domain_free(nvecs = 1) for free each 27 vecs    bitmap_release_region(count

Re: PCI MSI issue with reinserting a driver

2021-02-03 Thread Marc Zyngier
On 2021-02-02 15:46, John Garry wrote: On 02/02/2021 14:48, Marc Zyngier wrote: Not sure. I also now notice an error for the SAS PCI driver on D06 when nr_cpus < 16, which means number of MSI vectors allocated < 32, so looks the same problem. There we try to allocate 16 + max(nr cpus, 16)

Re: PCI MSI issue with reinserting a driver

2021-02-02 Thread John Garry
On 02/02/2021 14:48, Marc Zyngier wrote: Not sure. I also now notice an error for the SAS PCI driver on D06 when nr_cpus < 16, which means number of MSI vectors allocated < 32, so looks the same problem. There we try to allocate 16 + max(nr cpus, 16) MSI. Anyway, let me have a look today

Re: PCI MSI issue with reinserting a driver

2021-02-02 Thread Marc Zyngier
On 2021-02-02 12:38, John Garry wrote: Here's my suspicion: two of the interrupts are mapped in the low-level domain (the ITS, I'd expect in your case), but they have never been mapped at the higher level. On teardown, we only get rid of the 30 that were actually mapped, and leave the last two

Re: PCI MSI issue with reinserting a driver

2021-02-02 Thread John Garry
Here's my suspicion: two of the interrupts are mapped in the low-level domain (the ITS, I'd expect in your case), but they have never been mapped at the higher level. On teardown, we only get rid of the 30 that were actually mapped, and leave the last two dangling in the ITS domain, and thus the

Re: PCI MSI issue with reinserting a driver

2021-02-02 Thread John Garry
On 01/02/2021 18:50, Marc Zyngier wrote: Hi Marc, Just a heads-up, by chance I noticed that I can't re-insert a specific driver on v5.11-rc6: [ 64.356023] hisi_dma :7b:00.0: Adding to iommu group 31 [ 64.368627] hisi_dma :7b:00.0: enabling device ( -> 0002) [ 64.384156]

Re: PCI MSI issue with reinserting a driver

2021-02-01 Thread Marc Zyngier
Hi John, On Mon, 01 Feb 2021 18:34:59 +, John Garry wrote: > > Just a heads-up, by chance I noticed that I can't re-insert a specific > driver on v5.11-rc6: > > [ 64.356023] hisi_dma :7b:00.0: Adding to iommu group 31 > [ 64.368627] hisi_dma :7b:00.0: enabling device ( ->

PCI MSI issue with reinserting a driver

2021-02-01 Thread John Garry
Just a heads-up, by chance I noticed that I can't re-insert a specific driver on v5.11-rc6: [ 64.356023] hisi_dma :7b:00.0: Adding to iommu group 31 [ 64.368627] hisi_dma :7b:00.0: enabling device ( -> 0002) [ 64.384156] hisi_dma :7b:00.0: Failed to allocate MSI vectors! [