Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
Provide the private MSI setup functions in bus-driver layer can't apply to all Non-PCI MSI devices, because we can not guarantee Non-PCI MSI devices are always on a bus. The existing HPET, DMAR device both have no bus bind. Yes, that's why I was not sure of bus-driver or device-driver model. I'm working on a new MSI setup framework, as you mentioned before, in device-driver model. I abstracted a new virtual device (called struct msi_dev), this msi_dev will manage all MSI info, Will this struct msi_dev will be part of struct device? and a new bus named msi_bus, also introduced a new driver msi_driver, msi_bus is responsible for binding msi_dev and msi_driver. All MSI devices will be classified into different MSI device types, like MSI_TYPE_PCI, MSI_TYPE_HPET, MSI_TYPE_DMAR, etc.. Each MSI type device should provide a private struct msi_driver. msi_driver should contain the type specific MSI ops functions to help setup and enable MSI device, request MSI irq. I almost finish the first draft, and will post out next week in plan :) Will be looking forward to next version. Hi Bharat, I'm sorry I had to delay to send out the new version :(. I found some risks in the new MSI framework, i.e. DMAR MSI initialized the MSI before the linux device-driver tree be built. And we also found some problems during test. So I think I need more time to review and test. Thanks! Yijing. Thanks -Bharat Thanks! Yijing. Thanks -Bharat My patchset is just a RFC draft, I will update it later, all we want to do is make kernel support Non-PCI MSI devices. Thanks! Yijing. Thanks Arnab -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ . -- Thanks! Yijing -- To unsubscribe from this list: send the line unsubscribe linux-pci in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html . -- Thanks! Yijing -- To unsubscribe from this list: send the line unsubscribe linux-pci in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
The key difference between PCI device and Non-PCI MSI is the interfaces to access hardware MSI registers. for instance, currently, msi_chip-setup_irq() to setup MSI irq and configure the MSI address/data registers, so we need to provide device specific write_msi_msg() interface, then when we call msi_chip-setup_irq(), the device MSI registers can be configured appropriately. What if we can register/override the setup_irq() from bus-driver (not sure, but may be device-driver itself). Example PCI bus-driver will provide setup_irq() (or the part of setup_irq which set address and data in h/w) by PCI bus, which configure address/data in h/w as per PCI standard. We in Freescale will be using MSI for the devices behind a new-bus (which is not PCI based), We have a separate bus driver for same. And this new bus driver register/provide its own address/data write function which is based on that specific bus protocol. Hi Bharat, I'm glad to know your MSI device working mode. Provide the private MSI setup functions in bus-driver layer can't apply to all Non-PCI MSI devices, because we can not guarantee Non-PCI MSI devices are always on a bus. The existing HPET, DMAR device both have no bus bind. I'm working on a new MSI setup framework, as you mentioned before, in device-driver model. I abstracted a new virtual device (called struct msi_dev), this msi_dev will manage all MSI info, and a new bus named msi_bus, also introduced a new driver msi_driver, msi_bus is responsible for binding msi_dev and msi_driver. All MSI devices will be classified into different MSI device types, like MSI_TYPE_PCI, MSI_TYPE_HPET, MSI_TYPE_DMAR, etc.. Each MSI type device should provide a private struct msi_driver. msi_driver should contain the type specific MSI ops functions to help setup and enable MSI device, request MSI irq. I almost finish the first draft, and will post out next week in plan :) Thanks! Yijing. Thanks -Bharat My patchset is just a RFC draft, I will update it later, all we want to do is make kernel support Non-PCI MSI devices. Thanks! Yijing. Thanks Arnab -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ . -- Thanks! Yijing -- To unsubscribe from this list: send the line unsubscribe linux-pci in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 03/11] PCI/MSI: Refactor pci_dev_msi_enabled()
On 2014/8/20 13:57, bharat.bhus...@freescale.com wrote:
-Original Message-
From: linux-pci-ow...@vger.kernel.org
[mailto:linux-pci-ow...@vger.kernel.org]
On Behalf Of Yijing Wang
Sent: Saturday, July 26, 2014 8:39 AM
To: linux-ker...@vger.kernel.org
Cc: Xinwei Hu; Wuyun; Bjorn Helgaas; linux-...@vger.kernel.org;
paul.mu...@huawei.com; James E.J. Bottomley; Marc Zyngier; linux-arm-
ker...@lists.infradead.org; Russell King; linux-a...@vger.kernel.org; Basu
Arnab-B45036; virtualization@lists.linux-foundation.org; Hanjun Guo; Yijing
Wang
Subject: [RFC PATCH 03/11] PCI/MSI: Refactor pci_dev_msi_enabled()
Pci_dev_msi_enabled() is used to check whether device MSI/MSIX enabled.
Refactor
this function to suuport checking only device MSI or MSIX enabled.
s/support/support
From code it looks like you added one more parameter to pci_dev_msi_enabled()
to check for a specific type, which earlier it was checking for both MSI and
MSIX enable. While the description is not clear to me, Am I missing something
?
Right~
Thanks
-Bharat
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
arch/cris/arch-v32/drivers/pci/bios.c |2 +-
arch/frv/mb93090-mb00/pci-vdk.c |2 +-
arch/ia64/pci/pci.c |4 ++--
arch/powerpc/kernel/eeh_driver.c |2 +-
arch/x86/pci/common.c |5 +++--
drivers/block/nvme-core.c |4 ++--
drivers/dma/ioat/dma.c|2 +-
drivers/firewire/ohci.c |2 +-
drivers/gpu/drm/i915/i915_dma.c |4 ++--
drivers/misc/mei/hw-me.c |2 +-
drivers/misc/mei/hw-txe.c |2 +-
drivers/misc/mei/pci-me.c |4 ++--
drivers/misc/mei/pci-txe.c|4 ++--
drivers/misc/mic/host/mic_debugfs.c |4 ++--
drivers/misc/mic/host/mic_intr.c |8
drivers/ntb/ntb_hw.c |2 +-
drivers/pci/irq.c |4 ++--
drivers/pci/msi.c | 15 +--
drivers/pci/pci.c |6 +++---
drivers/pci/pcie/portdrv_core.c |4 ++--
drivers/scsi/esas2r/esas2r_init.c |4 ++--
drivers/scsi/esas2r/esas2r_ioctl.c|4 ++--
drivers/scsi/hpsa.c |4 ++--
drivers/staging/crystalhd/crystalhd_lnx.c |2 +-
drivers/xen/xen-pciback/pciback_ops.c | 12 ++--
include/linux/pci.h | 12 ++--
virt/kvm/assigned-dev.c |2 +-
27 files changed, 67 insertions(+), 55 deletions(-)
diff --git a/arch/cris/arch-v32/drivers/pci/bios.c b/arch/cris/arch-
v32/drivers/pci/bios.c
index 64a5fb9..d9d8332 100644
--- a/arch/cris/arch-v32/drivers/pci/bios.c
+++ b/arch/cris/arch-v32/drivers/pci/bios.c
@@ -93,7 +93,7 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
if ((err = pcibios_enable_resources(dev, mask)) 0)
return err;
-if (!dev-msi_enabled)
+if (!pci_dev_msi_enabled(dev, MSI_TYPE))
pcibios_enable_irq(dev);
return 0;
}
diff --git a/arch/frv/mb93090-mb00/pci-vdk.c
b/arch/frv/mb93090-mb00/pci-vdk.c
index efa5d65..b96c128 100644
--- a/arch/frv/mb93090-mb00/pci-vdk.c
+++ b/arch/frv/mb93090-mb00/pci-vdk.c
@@ -409,7 +409,7 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
if ((err = pci_enable_resources(dev, mask)) 0)
return err;
-if (!dev-msi_enabled)
+if (!pci_dev_msi_enabled(dev, MSI_TYPE))
pcibios_enable_irq(dev);
return 0;
}
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c index 291a582..da8ddff
100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -568,7 +568,7 @@ pcibios_enable_device (struct pci_dev *dev, int mask)
if (ret 0)
return ret;
-if (!dev-msi_enabled)
+if (!pci_dev_msi_enabled(dev, MSI_TYPE))
return acpi_pci_irq_enable(dev);
return 0;
}
@@ -577,7 +577,7 @@ void
pcibios_disable_device (struct pci_dev *dev) {
BUG_ON(atomic_read(dev-enable_cnt));
-if (!dev-msi_enabled)
+if (!pci_dev_msi_enabled(dev, MSI_TYPE))
acpi_pci_irq_disable(dev);
}
diff --git a/arch/powerpc/kernel/eeh_driver.c
b/arch/powerpc/kernel/eeh_driver.c
index 420da61..e3f2074 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -123,7 +123,7 @@ static void eeh_disable_irq(struct pci_dev *dev)
* effectively disabled by the DMA Stopped state
* when an EEH error occurs.
*/
-if (dev-msi_enabled || dev-msix_enabled)
+if (pci_dev_msi_enabled(dev, MSI_TYPE | MSIX_TYPE))
return;
if (!irq_has_action(dev-irq))
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c index
059a76c..4597940 100644
--- a/arch/x86/pci
Re: [RFC PATCH 09/11] PCI/MSI: refactor PCI MSI driver
@@ -1025,21 +1059,52 @@ int pci_msi_enabled(void)
}
EXPORT_SYMBOL(pci_msi_enabled);
-void pci_msi_init_pci_dev(struct pci_dev *dev)
+static struct msi_ops pci_msi = {
+.msi_set_enable = msi_set_enable,
+.msi_setup_entry = msi_setup_entry,
+.msix_setup_entries = msix_setup_entries,
+.msi_mask_irq = default_msi_mask_irq,
+.msix_mask_irq = default_msix_mask_irq,
+.msi_read_message = __read_msi_msg,
+.msi_write_message = __write_msi_msg,
+.msi_set_intx = pci_intx_for_msi,
+};
Ahh, want to be sure I am understanding this correctly. So if I have a
non-pci driver xyz which wants to use separate ops then I need to have a
all these functions in that driver. Something like driver/xyz/msi.c
Yes, because different MSI device has different MSI hardware registers, so
every MSI type should provide its own msi_ops, or its own msi_driver in my new
proposal.
Thanks
-Bharat
+
+struct msi_irqs *alloc_msi_irqs(void *data, struct msi_ops *ops)
{
-INIT_LIST_HEAD(dev-msi_list);
+struct msi_irqs *msi;
+
+msi = kzalloc(sizeof(struct msi_irqs), GFP_KERNEL);
+if (!msi)
+return NULL;
+INIT_LIST_HEAD(msi-msi_list);
+msi-data = data;
+msi-ops = ops;
+return msi;
+}
+
+void pci_msi_init_pci_dev(struct pci_dev *dev)
+{
/* Disable the msi hardware to avoid screaming interrupts
* during boot. This is the power on reset default so
* usually this should be a noop.
*/
dev-msi_cap = pci_find_capability(dev, PCI_CAP_ID_MSI);
-if (dev-msi_cap)
-msi_set_enable(dev, 0);
-
dev-msix_cap = pci_find_capability(dev, PCI_CAP_ID_MSIX);
-if (dev-msix_cap)
-msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_ENABLE, 0);
+
+if (dev-msi_cap || dev-msix_cap) {
+dev-msi = alloc_msi_irqs(dev, pci_msi);
+if (!dev-msi)
+return;
+
+dev-msi-node = dev_to_node(dev-dev);
+if (dev-msi_cap)
+msi_set_enable(dev-msi, 0, MSI_TYPE);
+
+if (dev-msix_cap)
+msi_set_enable(dev-msi, 0, MSIX_TYPE);
+}
}
/**
@@ -1060,13 +1125,13 @@ int pci_enable_msi_range(struct pci_dev *dev, int
minvec, int maxvec)
int rc;
struct msi_desc *entry;
-if (dev-current_state != PCI_D0)
+if (dev-current_state != PCI_D0 || !dev-msi)
return -EINVAL;
-WARN_ON(!!dev-msi_enabled);
+WARN_ON(!!pci_dev_msi_enabled(dev, MSI_TYPE));
/* Check whether driver already requested MSI-X irqs */
-if (dev-msix_enabled) {
+if (pci_dev_msi_enabled(dev, MSIX_TYPE)) {
dev_info(dev-dev,
can't enable MSI (MSI-X already enabled)\n);
return -EINVAL;
@@ -1095,7 +1160,7 @@ int pci_enable_msi_range(struct pci_dev *dev, int
minvec,
int maxvec)
} while (rc);
do {
-rc = msi_capability_init(dev, nvec);
+rc = msi_capability_init(dev-msi, nvec);
if (rc 0) {
return rc;
} else if (rc 0) {
@@ -1107,14 +1172,14 @@ int pci_enable_msi_range(struct pci_dev *dev, int
minvec, int maxvec)
rc = populate_msi_sysfs(dev);
if (rc) {
-msi_set_enable(dev, 0);
-pci_intx_for_msi(dev, 1);
-dev-msi_enabled = 0;
-free_msi_irqs(dev);
+msi_set_enable(dev-msi, 0, MSI_TYPE);
+pci_intx_for_msi(dev-msi, 1);
+dev-msi-msi_enabled = 0;
+free_msi_irqs(dev-msi);
return rc;
}
-entry = list_entry(dev-msi_list.next, struct msi_desc, list);
+entry = list_entry(dev-msi-msi_list.next, struct msi_desc, list);
dev-irq = entry-irq;
return nvec;
}
@@ -1158,3 +1223,5 @@ int pci_enable_msix_range(struct pci_dev *dev, struct
msix_entry *entries,
return nvec;
}
EXPORT_SYMBOL(pci_enable_msix_range);
+
+
diff --git a/include/linux/msi.h b/include/linux/msi.h
index 5a672d3..fc8f3e8 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -83,15 +83,15 @@ struct msi_desc {
* implemented as weak symbols so that they /can/ be overriden by
* architecture specific code if needed.
*/
-int arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc *desc);
+int arch_setup_msi_irq(struct msi_irqs *msi, struct msi_desc *desc);
void arch_teardown_msi_irq(unsigned int irq);
-int arch_setup_msi_irqs(struct pci_dev *dev, int nvec, int type);
-void arch_teardown_msi_irqs(struct pci_dev *dev);
-int arch_msi_check_device(struct pci_dev* dev, int nvec, int type);
-void arch_restore_msi_irqs(struct pci_dev *dev);
+int arch_setup_msi_irqs(struct msi_irqs *msi, int nvec, int type);
+void arch_teardown_msi_irqs(struct msi_irqs *msi);
+int arch_msi_check_device(struct msi_irqs *msi, int nvec, int type);
+void arch_restore_msi_irqs(struct msi_irqs *msi);
Re: [RFC PATCH 10/11] PCI/MSI: Split the generic MSI code into new file
+int msi_capability_init(struct msi_irqs *msi, int nvec)
+{
+struct msi_desc *entry;
+int ret;
+unsigned mask;
+
+msi_set_enable(msi, 0, MSI_TYPE); /* Disable MSI during set up */
+
+/* MSI Entry Initialization */
+entry = msi_setup_entry(msi);
+if (!entry)
+return -ENOMEM;
+
+/* All MSIs are unmasked by default, Mask them all */
Will this be true for non-pci devices as well?
In my opinion, yes, I think all msi devices should be masked during the setup.
Of course, mask and unmask functions will be override by private mask/unmask
functions.
Thanks
-Bharat
+mask = msi_mask(entry-msi_attrib.multi_cap);
+msi_mask_irq(entry, mask, mask);
+
+/* Configure MSI capability structure */
+ret = arch_setup_msi_irqs(msi, nvec, MSI_TYPE);
+if (ret)
+goto err;
+
+/* Set MSI enabled bits */
+msi_set_intx(msi, 0);
+msi_set_enable(msi, 1, MSI_TYPE);
+msi-msi_enabled = 1;
+
+return 0;
+
+err:
+msi_mask_irq(entry, mask, ~mask);
+free_msi_irqs(msi);
+return ret;
+}
+
+static void msix_program_entries(struct msi_irqs *msi,
+ struct msix_entry *entries)
+{
+struct msi_desc *entry;
+int i = 0;
+
+list_for_each_entry(entry, msi-msi_list, list) {
+entries[i].vector = entry-irq;
+irq_set_msi_desc(entry-irq, entry);
+i++;
+}
+}
+
+/**
+ * msix_capability_init - configure device's MSI-X capability
+ * @dev: pointer to the pci_dev data structure of MSI-X device function
+ * @entries: pointer to an array of struct msix_entry entries
+ * @nvec: number of @entries
+ *
+ * Setup the MSI-X capability structure of device function with a
+ * single MSI-X irq. A return of zero indicates the successful setup of
+ * requested MSI-X entries with allocated irqs or non-zero for otherwise.
+ **/
+int msix_capability_init(struct msi_irqs *msi, void __iomem *base,
+struct msix_entry *entries, int nvec)
+{
+int ret;
+
+/* Ensure MSI-X is disabled while it is set up */
+msi_set_enable(msi, 0, MSIX_TYPE);
+
+ret = msix_setup_entries(msi, base, entries, nvec);
+if (ret)
+return ret;
+
+ret = arch_setup_msi_irqs(msi, nvec, MSIX_TYPE);
+if (ret)
+goto out_avail;
+
+msix_program_entries(msi, entries);
+
+/* Set MSI-X enabled bits and unmask the function */
+msi_set_intx(msi, 0);
+msi-msix_enabled = 1;
+
+msi_set_enable(msi, 1, MSIX_TYPE);
+
+return 0;
+
+out_avail:
+if (ret 0) {
+/*
+ * If we had some success, report the number of irqs
+ * we succeeded in setting up.
+ */
+struct msi_desc *entry;
+int avail = 0;
+
+list_for_each_entry(entry, msi-msi_list, list) {
+if (entry-irq != 0)
+avail++;
+}
+if (avail != 0)
+ret = avail;
+}
+
+free_msi_irqs(msi);
+
+return ret;
+}
+
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 893503f..1a10488 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -2,10 +2,10 @@
# PCI configuration
#
config PCI_MSI
-bool Message Signaled Interrupts (MSI and MSI-X)
-depends on PCI
+bool PCI Message Signaled Interrupts (MSI and MSI-X)
+depends on PCI MSI
help
- This allows device drivers to enable MSI (Message Signaled
+ This allows PCI device drivers to enable MSI (Message Signaled
Interrupts). Message Signaled Interrupts enable a device to
generate an interrupt using an inbound Memory Write on its
PCI bus instead of asserting a device IRQ pin.
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index f0c5989..df7223c 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -26,121 +26,8 @@ static int pci_msi_enable = 1;
#define msix_table_size(flags) ((flags PCI_MSIX_FLAGS_QSIZE) + 1)
-
-/* Arch hooks */
-
-int __weak arch_setup_msi_irq(struct msi_irqs *msi, struct msi_desc *desc)
-{
-struct pci_dev *dev = msi-data; //TO BE DONE: rework msi_chip to
support
Non-PCI
-struct msi_chip *chip = dev-bus-msi;
-int err;
-
-if (!chip || !chip-setup_irq)
-return -EINVAL;
-
-err = chip-setup_irq(chip, dev, desc);
-if (err 0)
-return err;
-
-irq_set_chip_data(desc-irq, chip);
-
-return 0;
-}
-
-void __weak arch_teardown_msi_irq(unsigned int irq)
-{
-struct msi_chip *chip = irq_get_chip_data(irq);
-
-if (!chip || !chip-teardown_irq)
-return;
-
-chip-teardown_irq(chip, irq);
-}
-
-int __weak arch_msi_check_device(struct msi_irqs *msi, int nvec, int type)
-{
-struct pci_dev *dev = msi-data; //TO BE DONE: rework
Re: [RFC PATCH 11/11] x86/MSI: Refactor x86 MSI code
On 2014/8/20 14:20, bharat.bhus...@freescale.com wrote:
-Original Message-
From: linux-pci-ow...@vger.kernel.org
[mailto:linux-pci-ow...@vger.kernel.org]
On Behalf Of Yijing Wang
Sent: Saturday, July 26, 2014 8:39 AM
To: linux-ker...@vger.kernel.org
Cc: Xinwei Hu; Wuyun; Bjorn Helgaas; linux-...@vger.kernel.org;
paul.mu...@huawei.com; James E.J. Bottomley; Marc Zyngier; linux-arm-
ker...@lists.infradead.org; Russell King; linux-a...@vger.kernel.org; Basu
Arnab-B45036; virtualization@lists.linux-foundation.org; Hanjun Guo; Yijing
Wang
Subject: [RFC PATCH 11/11] x86/MSI: Refactor x86 MSI code
Please provide description about what this refactoring is? Also does other
architecture also need similar refactoring ?
Sorry, I will update all description in my new proposal.
I provided another patchset to decouple MSI driver and arch MSI code,
link: http://marc.info/?l=linux-pcim=140782732604433w=2
Based that, there are few changes related to arch MSI code.
I will rebas the patchset based that~
Thanks
-Bharat
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
arch/x86/include/asm/io_apic.h |2 +-
arch/x86/include/asm/irq_remapping.h |4 +-
arch/x86/include/asm/pci.h |6 ++--
arch/x86/include/asm/x86_init.h | 10 +++---
arch/x86/kernel/apic/io_apic.c | 23 +++
arch/x86/kernel/x86_init.c | 12
drivers/iommu/amd_iommu.c| 16 ++
drivers/iommu/intel_irq_remapping.c |9 --
drivers/iommu/irq_remapping.c| 51
-
drivers/iommu/irq_remapping.h|6 ++--
drivers/msi/msi.c|3 +-
11 files changed, 72 insertions(+), 70 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 90f97b4..692a90f 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -158,7 +158,7 @@ extern int native_setup_ioapic_entry(int, struct
IO_APIC_route_entry *,
struct io_apic_irq_attr *);
extern void eoi_ioapic_irq(unsigned int irq, struct irq_cfg *cfg);
-extern void native_compose_msi_msg(struct pci_dev *pdev,
+extern void native_compose_msi_msg(struct msi_irqs *msi,
unsigned int irq, unsigned int dest,
struct msi_msg *msg, u8 hpet_id);
extern void native_eoi_ioapic_pin(int apic, int pin, int vector);
diff --git a/arch/x86/include/asm/irq_remapping.h
b/arch/x86/include/asm/irq_remapping.h
index b7747c4..a10003d 100644
--- a/arch/x86/include/asm/irq_remapping.h
+++ b/arch/x86/include/asm/irq_remapping.h
@@ -47,7 +47,7 @@ extern int setup_ioapic_remapped_entry(int irq,
int vector,
struct io_apic_irq_attr *attr);
extern void free_remapped_irq(int irq);
-extern void compose_remapped_msi_msg(struct pci_dev *pdev,
+extern void compose_remapped_msi_msg(struct msi_irqs *msi,
unsigned int irq, unsigned int dest,
struct msi_msg *msg, u8 hpet_id);
extern int setup_hpet_msi_remapped(unsigned int irq, unsigned int id);
@@ -77,7 +77,7 @@ static inline int setup_ioapic_remapped_entry(int irq,
return -ENODEV;
}
static inline void free_remapped_irq(int irq) { }
-static inline void compose_remapped_msi_msg(struct pci_dev *pdev,
+static inline void compose_remapped_msi_msg(struct msi_irqs *msi,
unsigned int irq, unsigned int dest,
struct msi_msg *msg, u8 hpet_id)
{
diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
index 0892ea0..04c9ef6 100644
--- a/arch/x86/include/asm/pci.h
+++ b/arch/x86/include/asm/pci.h
@@ -96,10 +96,10 @@ extern void pci_iommu_alloc(void);
#ifdef CONFIG_PCI_MSI
/* implemented in arch/x86/kernel/apic/io_apic. */
struct msi_desc;
-int native_setup_msi_irqs(struct pci_dev *dev, int nvec, int type);
+int native_setup_msi_irqs(struct msi_irqs *msi, int nvec, int type);
void native_teardown_msi_irq(unsigned int irq);
-void native_restore_msi_irqs(struct pci_dev *dev);
-int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
+void native_restore_msi_irqs(struct msi_irqs *msi);
+int setup_msi_irq(struct msi_irqs *msi, struct msi_desc *msidesc,
unsigned int irq_base, unsigned int irq_offset);
#else
#define native_setup_msi_irqs NULL
diff --git a/arch/x86/include/asm/x86_init.h
b/arch/x86/include/asm/x86_init.h
index e45e4da..8e42f17 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -170,18 +170,18 @@ struct x86_platform_ops {
void (*apic_post_init)(void);
};
-struct pci_dev;
+struct msi_irqs;
struct msi_msg;
struct msi_desc;
struct x86_msi_ops
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
We in Freescale will be using MSI for the devices behind a new-bus (which is
not PCI based), We have a separate bus driver for same. And this new bus
driver
register/provide its own address/data write function which is based on that
specific bus protocol.
Hi Bharat, I'm glad to know your MSI device working mode.
Provide the private MSI setup functions in bus-driver layer can't apply to
all
Non-PCI MSI devices, because we can not guarantee Non-PCI MSI devices are
always
on a bus. The existing HPET, DMAR device both have no bus bind.
Yes, that's why I was not sure of bus-driver or device-driver model.
I'm working on a
new MSI setup framework, as you mentioned before, in device-driver model.
I abstracted a new virtual device (called struct msi_dev), this msi_dev will
manage all MSI info,
Will this struct msi_dev will be part of struct device?
struct msi_dev contains the struct device
piece code:
struct msi_dev {
u8 type;
u8 enabled;
u8 nvec;
u8 nvec_retry;
char *id;
void __iomem *base;
struct msix_entry *entries;
struct list_head msi_list;
struct device dev;
void *msi_data;
struct msi_driver *driver;
const struct attribute_group **irq_groups;
};
struct msi_driver {
const char *name;
char *id;
void (*msi_set_enable)(struct msi_dev *dev, int enable);
int (*msi_setup_entry)(struct msi_dev *dev, struct msi_desc *entry);
int (*msix_setup_entries)(struct msi_dev *dev, struct msi_desc *entry, int
index);
u32 (*msi_mask_irq)(struct msi_desc *desc, u32 mask, u32 flag);
u32 (*msix_mask_irq)(struct msi_desc *desc, u32 flag);
void (*msi_read_message)(struct msi_desc *desc, struct msi_msg *msg);
void (*msi_write_message)(struct msi_desc *desc, struct msi_msg *msg);
void (*msi_set_legacy_irq)(struct msi_dev *dev, int enable);
struct device_driver driver;
};
Thanks!
Yijing.
and a new bus named msi_bus, also introduced a new driver
msi_driver, msi_bus is responsible for binding msi_dev and msi_driver.
All MSI devices will be classified into different MSI device types, like
MSI_TYPE_PCI, MSI_TYPE_HPET, MSI_TYPE_DMAR, etc..
Each MSI type device should provide a private struct msi_driver. msi_driver
should contain the type specific MSI ops functions to help setup and enable
MSI
device, request MSI irq.
I almost finish the first draft, and will post out next week in plan :)
Will be looking forward to next version.
Thanks
-Bharat
Thanks!
Yijing.
Thanks
-Bharat
My patchset is just a RFC draft, I will update it later, all we want
to do is make kernel support Non-PCI MSI devices.
Thanks!
Yijing.
Thanks
Arnab
--
To unsubscribe from this list: send the line unsubscribe
linux-kernel in the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
.
--
Thanks!
Yijing
--
To unsubscribe from this list: send the line unsubscribe linux-pci
in the body of a message to majord...@vger.kernel.org More majordomo
info at http://vger.kernel.org/majordomo-info.html
.
--
Thanks!
Yijing
--
To unsubscribe from this list: send the line unsubscribe linux-pci in the
body
of a message to majord...@vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
.
--
Thanks!
Yijing
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 03/11] PCI/MSI: Refactor pci_dev_msi_enabled()
On 2014/8/6 6:35, Stuart Yoder wrote: On Fri, Jul 25, 2014 at 10:08 PM, Yijing Wang wangyij...@huawei.com wrote: Pci_dev_msi_enabled() is used to check whether device MSI/MSIX enabled. Refactor this function to suuport checking only device MSI or MSIX enabled. Signed-off-by: Yijing Wang wangyij...@huawei.com So this patch refactors things so that checks like this: - if (!dev-msi_enabled) are moved into a function: + if (!pci_dev_msi_enabled(dev, MSI_TYPE)) Can you explain a bit more why this needed. Is it just cleanup? Hi Stuart, it's not just cleanup, because [RFC PATCH 08/11] PCI/MSI: Introduce new struct msi_irqs and struct msi_ops introduced struct msi_irqs, so the code will change to if (!dev-msi_irqs-msi_enabled) I think driver should not need to know the details of MSI members. So I try to rework the pci_dev_msi_enabled() to hide the detailed MSI info. Thanks! Yijing. . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
On 2014/8/1 21:52, Arnd Bergmann wrote:
On Wednesday 30 July 2014, Yijing Wang wrote:
On 2014/7/29 22:08, Arnd Bergmann wrote:
On Saturday 26 July 2014 11:08:37 Yijing Wang wrote:
The new data struct for generic MSI driver.
struct msi_irqs {
u8 msi_enabled:1; /* Enable flag */
u8 msix_enabled:1;
struct list_head msi_list; /* MSI desc list */
void *data; /* help to find the MSI device */
struct msi_ops *ops; /* MSI device specific hook */
};
struct msi_irqs is used to manage MSI related informations. Every device
supports
MSI should contain this data struct and allocate it.
I think you should have a stronger association with the 'struct
device' here. Can you replace the 'void *data' with 'struct device *dev'?
Actually, I used the struct device *dev in my first draft, finally, I
replaced
it with void *data, because some MSI devices don't have a struct device *dev,
like the existing hpet device, dmar msi device, and OF device, like the ARM
consolidator.
Of course, we can make the MSI devices have their own struct device, and
register to
device tree, eg, add a class device named MSI_DEV. But I'm not sure whether
it is appropriate.
It doesn't have to be in the (OF) device tree, but I think it absolutely makes
sense to use the 'struct device' infrastructure here, as almost everything
uses
a device, and the ones that don't do that today can be easily changed.
I will try to use struct device infrastructure, thanks for your suggestion. :)
The other part I'm not completely sure about is how you want to
have MSIs map into normal IRQ descriptors. At the moment, all
MSI users are based on IRQ numbers, but this has known scalability problems.
Hmmm, I still use the IRQ number to map the MSIs to IRQ description.
I'm sorry, I don't understand you meaning.
What are the scalability problems you mentioned ?
For device drivers, they always process interrupt in two steps.
If irq is the legacy interrupt, drivers will first
use the irq_of_parse_and_map() or pci_enable_device() to parse and get the
IRQ number.
Then drivers will call the request_irq() to register the interrupt handler.
If irq is MSIs, first call pci_enable_msi/x() to get the IRQ number and then
call
request_irq() to register interrupt handler.
The method you describe here makes sense for PCI devices that are required to
support
legacy interrupts and may or may not support MSI on a given system, but not
so much
for platform devices for which we know exactly whether we want to use MSI
or legacy interrupts.
In particular if you have a device that can only do MSI, the entire
pci_enable_msi
step is pointless: all we need to do is program the correct MSI target
address/message
pair into the device and register the handler.
Yes, I almost agree if we won't change the existing hundreds of drivers, what
I worried about is some drivers may want to know the IRQ numbers, and use the
IRQ
number to process different things, as I mentioned in another reply.
But we can also provide the interface which integrate MSI configuration and
request_irq(),
if most drivers don't care the IRQ number.
I wonder if we can do the interface in a way that
hides the interrupt number from generic device drivers and just
passes a 'struct irq_desc'. Note that there are long-term plans to
get rid of IRQ numbers entirely, but those plans have existed for
a long time already without anybody seriously addressing the device
driver interfaces so far, so it might never really happen.
Maybe this is a huge work, now hundreds drivers use the IRQ number, so maybe
we can consider
this in a separate title.
Sorry for being unclear here: I did suggest changing all drivers now. What I
meant
is that we use a different API for non-PCI devices that works without IRQ
numbers.
I don't think we should touch the PCI interfaces at this point.
OK, I got it.
What I'd envision as the API from the device driver perspective is something
as simple like this:
struct msi_desc *msi_request(struct msi_chip *chip, irq_handler_t handler,
unsigned long flags, const char *name, struct device
*dev);
which would get an msi descriptor that is valid for this device (dev)
connected to a particular msi_chip, and associate a handler function
with it. The device driver can call that function and retrieve the
address/message pair from the msi_desc in order to store it in its own
device specific registers. The request_irq() can be handled internally
to msi_request().
This is a huge change for device drivers, and some device drivers don't know
which msi_chip
their MSI irq deliver to. I'm reworking the msi_chip, and try to use
msi_chip to eliminate
all arch_msi_xxx() under every arch in kernel. And the important point is
how to create the
binding for the MSI device to the target msi_chip.
Which drivers are you thinking of? Again, I wouldn't expect
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
The method you describe here makes sense for PCI devices that are required to support legacy interrupts and may or may not support MSI on a given system, but not so much for platform devices for which we know exactly whether we want to use MSI or legacy interrupts. In particular if you have a device that can only do MSI, the entire pci_enable_msi step is pointless: all we need to do is program the correct MSI target address/message pair into the device and register the handler. Yes, I almost agree if we won't change the existing hundreds of drivers, what I worried about is some drivers may want to know the IRQ numbers, and use the IRQ number to process different things, as I mentioned in another reply. But we can also provide the interface which integrate MSI configuration and request_irq(), if most drivers don't care the IRQ number. The driver would still have the option of getting the IRQ number for now: With the interface I imagine, you would get a 'struct msi_desc' pointer, from which you can look up the 'struct irq_desc' pointer (either embedded in msi_desc, or using a pointer from a member of msi_desc), and you can already get the interrupt number from the irq_desc. My point was that a well-written driver already does not care about the interrupt number: the only information a driver needs in the interrupt handler is a pointer to its own context, which we already derive from the irq_desc. Agree, I will try to introduce this similar interface in next version, thanks! The main interface that currently requires the irq number is free_irq(), but I would argue that we can just add a wrapper that takes the msi_desc pointer as its first argument so the driver does not have to worry about it. We can add additional wrappers like that as needed. OK This is a huge change for device drivers, and some device drivers don't know which msi_chip their MSI irq deliver to. I'm reworking the msi_chip, and try to use msi_chip to eliminate all arch_msi_xxx() under every arch in kernel. And the important point is how to create the binding for the MSI device to the target msi_chip. Which drivers are you thinking of? Again, I wouldn't expect to change any PCI drivers, but only platform drivers that do native MSI, so we only have to change drivers that do not support any MSI at all yet and that need to be changed anyway in order to add support. I mean platform device drivers, because we can find the target msi_chip by some platform interfaces(like the existing of_pci_find_msi_chip_by_node()). So we no need to explicitly provide the msi_chip as the function argument. Right, that works too. I was thinking we might need an interface that allows us to pick a particular msi_chip if there are several alternatives (e.g. one in the GIC and one in the PCI host), but you are right: we should normally be able to hardwire that information in DT or elsewhere, and just need the 'struct device pointer' which should probably be the first argument here. As you pointed out, it's common to have multiple MSIs for a single device, so we also need a context to pass around, so my suggestion would become something like: struct msi_desc *msi_request(struct device *dev, irq_handler_t handler, unsigned long flags, const char *name, void *data); It's possible that we have to add one or two more arguments here. Good suggestion, thanks! A degenerate case of this would be a system where a PCI device sends its MSI into the host controller, that generates a legacy interrupt and that in turn gets sent to an irqchip which turns it back into an MSI for the GICv3. This would of course be very inefficient, but I think we should be able to express this with both the binding and the in-kernel framework just to be on the safe side. Yes, the best way to tell the kernel which msi_chip should deliver to is describe the binding in DTS file. If a real degenerate case found, we can update the platform interface which is responsible for getting the match msi_chip in future. Ok. Arnd . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
On 2014/8/4 22:45, Arnd Bergmann wrote: On Monday 04 August 2014, Yijing Wang wrote: I have another question is some drivers will request more than one MSI/MSI-X IRQ, and the driver will use them to process different things. Eg. network driver generally uses one of them to process trivial network thins, and others to transmit/receive data. So, in this case, it seems to driver need to touch the IRQ numbers. wr-linux:~ # cat /proc/interrupts CPU0 CPU1 CPU2 CPU17 CPU18 CPU19 CPU20 CPU21 CPU22 CPU23 .. 100: 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0 101: 2 0 0 0 0 0 302830488 0 0 0 IR-PCI-MSI-edge eth0-TxRx-0 102:110 0 0 0 0 360675897 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-1 103:109 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-2 104:107 0 0 9678933 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-3 105:107 0 0 0 357838258 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-4 106:115 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-5 107:114 0 0 0 0 0 0 337866096 0 0 IR-PCI-MSI-edge eth0-TxRx-6 108: 373801199 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-7 I think in this example, you just need to request eight interrupts, and pass a different data pointer each time, pointing to the napi_struct of each of the NIC queues. The driver has no need to deal with the IRQ number at all, and I would be surprised if it cared today. Yes, you are right, this is not a stumbling block. :) Arnd . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
MSI was introduced in PCI Spec 2.2. Currently, kernel MSI driver codes are bonding with PCI device. Because MSI has a lot advantages in design. More and more non-PCI devices want to use MSI as their default interrupt. The existing MSI device include HPET. HPET driver provide its own MSI code to initialize and process MSI interrupts. In the latest GIC v3 spec, legacy device can deliver MSI by the help of a relay device named consolidator. Consolidator can translate the legacy interrupts connected to it to MSI/MSI-X. And new non-PCI device will be designed to support MSI in future. So make the MSI driver code be generic will help the non-PCI device use MSI more simply. As per my understanding the GICv3 provides a service that will convert writes to a specified address to IRQs delivered to the core and as you mention above MSIs are part of the PCI spec. So I can see a strong case for non-PCI devices to want MSI like functionality without being fully compliant with the requirements of the MSI spec. In GICv3, MBI is named for the service, but there is no more detailed information about it, only we can know is MBI is analogous to MSI, MBI devices must have address/data registers, but other registers like enable/mask/ctrl are not mandatory requirement. I don't know whether the MBI spec will be release, but anyway I think MSI refactoring is make sense, there are some existing Non-PCI MSI device like hpet, dmar. For simplicity, let name MSI and MBI to MSI temporarily. My question is do we necessarily want to rework so much of the PCI-MSI layer to support non PCI devices? Or will it be sufficient to create a framework to allow non PCI devices to hook up with a device that can convert their writes to an IRQ to the core. As I understand it, the msi_chip is (almost) such a framework. The only problem being that it makes some PCI specific assumptions (such as PCI specific writes from within msi_chip functions). Won't it be sufficient to make the msi_chip framework generic enough to be used by non-PCI devices and let each bus/device manage any additional requirements (such as configuration flow, bit definitions etc) that it places on message based interrupts? msi_chip framework is important to support that, but I think maybe it's not enough, msi_chip is only responsible for IRQ allocation, teardown, etc.. The key difference between PCI device and Non-PCI MSI is the interfaces to access hardware MSI registers. for instance, currently, msi_chip-setup_irq() to setup MSI irq and configure the MSI address/data registers, so we need to provide device specific write_msi_msg() interface, then when we call msi_chip-setup_irq(), the device MSI registers can be configured appropriately. My patchset is just a RFC draft, I will update it later, all we want to do is make kernel support Non-PCI MSI devices. Thanks! Yijing. Thanks Arnab -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
On 2014/8/1 21:16, Arnd Bergmann wrote: On Wednesday 30 July 2014, Yijing Wang wrote: The other part I'm not completely sure about is how you want to have MSIs map into normal IRQ descriptors. At the moment, all MSI users are based on IRQ numbers, but this has known scalability problems. Hmmm, I still use the IRQ number to map the MSIs to IRQ description. I'm sorry, I don't understand you meaning. What are the scalability problems you mentioned ? We have soft limitation of nr_irqs or hard limitation NR_IRQS, we couldn't allocate as much irq number as we need in some cases, such as to support MSI-x. Oh, yes, this is a potential issue. Gerry, thanks for you explanation. :) This should no longer be an issue, as arm64 uses CONFIG_SPARSE_IRQ and the number of interrupts is not limited in any form. My point was more that the device driver should not need to care about the interrupt number: it gets made up on the spot when the MSI is needed, and then it is only used to request the IRQ. This can be simplified into one interface at the device driver level, even though the internal still use numbers somewhere. If we ever remove IRQ numbers from the driver API, this part doesn't need to get touched again. Hi Arnd, I have another question is some drivers will request more than one MSI/MSI-X IRQ, and the driver will use them to process different things. Eg. network driver generally uses one of them to process trivial network thins, and others to transmit/receive data. So, in this case, it seems to driver need to touch the IRQ numbers. wr-linux:~ # cat /proc/interrupts CPU0 CPU1 CPU2 CPU17 CPU18 CPU19 CPU20 CPU21 CPU22 CPU23 .. 100: 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0 101: 2 0 0 0 0 0 302830488 0 0 0 IR-PCI-MSI-edge eth0-TxRx-0 102:110 0 0 0 0 360675897 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-1 103:109 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-2 104:107 0 0 9678933 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-3 105:107 0 0 0 357838258 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-4 106:115 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-5 107:114 0 0 0 0 0 0 337866096 0 0 IR-PCI-MSI-edge eth0-TxRx-6 108: 373801199 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth0-TxRx-7 Thanks! Yijing. . -- Thanks! Yijing ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
On 2014/7/30 14:47, Jiang Liu wrote:
On 2014/7/30 10:45, Yijing Wang wrote:
On 2014/7/29 22:08, Arnd Bergmann wrote:
On Saturday 26 July 2014 11:08:37 Yijing Wang wrote:
The series is a draft of generic MSI driver that supports PCI
and Non-PCI device which have MSI capability. If you're not interested
it, sorry for the noise.
I've finally managed to take some time to look at the series. Overall,
the concept looks good to me, and the patches look very well implemented.
The part I'm not sure about is the interface we want to end up with
at the end of the series. More on that below
Hi Arnd,
Thanks for your review and comments very much!
Please refer the inline comments.
The series is based on Linux-3.16-rc1.
MSI was introduced in PCI Spec 2.2. Currently, kernel MSI
driver codes are bonding with PCI device. Because MSI has a lot
advantages in design. More and more non-PCI devices want to
use MSI as their default interrupt. The existing MSI device
include HPET. HPET driver provide its own MSI code to initialize
and process MSI interrupts. In the latest GIC v3 spec, legacy device
can deliver MSI by the help of a relay device named consolidator.
Consolidator can translate the legacy interrupts connected to it
to MSI/MSI-X. And new non-PCI device will be designed to
support MSI in future. So make the MSI driver code be generic will
help the non-PCI device use MSI more simply.
The new data struct for generic MSI driver.
struct msi_irqs {
u8 msi_enabled:1; /* Enable flag */
u8 msix_enabled:1;
struct list_head msi_list; /* MSI desc list */
void *data; /* help to find the MSI device */
struct msi_ops *ops; /* MSI device specific hook */
};
struct msi_irqs is used to manage MSI related informations. Every device
supports
MSI should contain this data struct and allocate it.
I think you should have a stronger association with the 'struct
device' here. Can you replace the 'void *data' with 'struct device *dev'?
Actually, I used the struct device *dev in my first draft, finally, I
replaced
it with void *data, because some MSI devices don't have a struct device *dev,
like the existing hpet device, dmar msi device, and OF device, like the ARM
consolidator.
Of course, we can make the MSI devices have their own struct device, and
register to
device tree, eg, add a class device named MSI_DEV. But I'm not sure whether
it is appropriate.
The other part I'm not completely sure about is how you want to
have MSIs map into normal IRQ descriptors. At the moment, all
MSI users are based on IRQ numbers, but this has known scalability problems.
Hmmm, I still use the IRQ number to map the MSIs to IRQ description.
I'm sorry, I don't understand you meaning.
What are the scalability problems you mentioned ?
We have soft limitation of nr_irqs or hard limitation NR_IRQS,
we couldn't allocate as much irq number as we need in some cases,
such as to support MSI-x.
Oh, yes, this is a potential issue. Gerry, thanks for you explanation. :)
For device drivers, they always process interrupt in two steps.
If irq is the legacy interrupt, drivers will first
use the irq_of_parse_and_map() or pci_enable_device() to parse and get the
IRQ number.
Then drivers will call the request_irq() to register the interrupt handler.
If irq is MSIs, first call pci_enable_msi/x() to get the IRQ number and then
call
request_irq() to register interrupt handler.
I wonder if we can do the interface in a way that
hides the interrupt number from generic device drivers and just
passes a 'struct irq_desc'. Note that there are long-term plans to
get rid of IRQ numbers entirely, but those plans have existed for
a long time already without anybody seriously addressing the device
driver interfaces so far, so it might never really happen.
Maybe this is a huge work, now hundreds drivers use the IRQ number, so maybe
we can consider
this in a separate title.
struct msi_ops {
struct msi_desc *(*msi_setup_entry)(struct msi_irqs *msi, struct
msi_desc *entry);
int msix_setup_entries(struct msi_irqs *msi, struct msix_entry
*entries);
u32 (*msi_mask_irq)(struct msi_desc *desc, u32 mask, u32 flag);
u32 (*msix_mask_irq)(struct msi_desc *desc, u32 flag);
void (*msi_read_message)(struct msi_desc *desc, struct msi_msg
*msg);
void (*msi_write_message)(struct msi_desc *desc, struct msi_msg
*msg);
void (*msi_set_intx)(struct msi_irqs *msi, int enable);
};
struct msi_ops provides several hook functions, generic MSI driver will
call
the hook functions to access device specific registers. PCI devices will
share
the same msi_ops, because they have the same way to access MSI hardware
registers.
Generic MSI layer export msi_capability_init() and msix_capability_init()
functions
to drivers. msi/x_capability_init() will initialize MSI capability data
struct
Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
On 2014/7/29 22:08, Arnd Bergmann wrote:
On Saturday 26 July 2014 11:08:37 Yijing Wang wrote:
The series is a draft of generic MSI driver that supports PCI
and Non-PCI device which have MSI capability. If you're not interested
it, sorry for the noise.
I've finally managed to take some time to look at the series. Overall,
the concept looks good to me, and the patches look very well implemented.
The part I'm not sure about is the interface we want to end up with
at the end of the series. More on that below
Hi Arnd,
Thanks for your review and comments very much!
Please refer the inline comments.
The series is based on Linux-3.16-rc1.
MSI was introduced in PCI Spec 2.2. Currently, kernel MSI
driver codes are bonding with PCI device. Because MSI has a lot
advantages in design. More and more non-PCI devices want to
use MSI as their default interrupt. The existing MSI device
include HPET. HPET driver provide its own MSI code to initialize
and process MSI interrupts. In the latest GIC v3 spec, legacy device
can deliver MSI by the help of a relay device named consolidator.
Consolidator can translate the legacy interrupts connected to it
to MSI/MSI-X. And new non-PCI device will be designed to
support MSI in future. So make the MSI driver code be generic will
help the non-PCI device use MSI more simply.
The new data struct for generic MSI driver.
struct msi_irqs {
u8 msi_enabled:1; /* Enable flag */
u8 msix_enabled:1;
struct list_head msi_list; /* MSI desc list */
void *data; /* help to find the MSI device */
struct msi_ops *ops; /* MSI device specific hook */
};
struct msi_irqs is used to manage MSI related informations. Every device
supports
MSI should contain this data struct and allocate it.
I think you should have a stronger association with the 'struct
device' here. Can you replace the 'void *data' with 'struct device *dev'?
Actually, I used the struct device *dev in my first draft, finally, I replaced
it with void *data, because some MSI devices don't have a struct device *dev,
like the existing hpet device, dmar msi device, and OF device, like the ARM
consolidator.
Of course, we can make the MSI devices have their own struct device, and
register to
device tree, eg, add a class device named MSI_DEV. But I'm not sure whether it
is appropriate.
The other part I'm not completely sure about is how you want to
have MSIs map into normal IRQ descriptors. At the moment, all
MSI users are based on IRQ numbers, but this has known scalability problems.
Hmmm, I still use the IRQ number to map the MSIs to IRQ description.
I'm sorry, I don't understand you meaning.
What are the scalability problems you mentioned ?
For device drivers, they always process interrupt in two steps.
If irq is the legacy interrupt, drivers will first
use the irq_of_parse_and_map() or pci_enable_device() to parse and get the IRQ
number.
Then drivers will call the request_irq() to register the interrupt handler.
If irq is MSIs, first call pci_enable_msi/x() to get the IRQ number and then
call
request_irq() to register interrupt handler.
I wonder if we can do the interface in a way that
hides the interrupt number from generic device drivers and just
passes a 'struct irq_desc'. Note that there are long-term plans to
get rid of IRQ numbers entirely, but those plans have existed for
a long time already without anybody seriously addressing the device
driver interfaces so far, so it might never really happen.
Maybe this is a huge work, now hundreds drivers use the IRQ number, so maybe we
can consider
this in a separate title.
struct msi_ops {
struct msi_desc *(*msi_setup_entry)(struct msi_irqs *msi, struct
msi_desc *entry);
int msix_setup_entries(struct msi_irqs *msi, struct msix_entry
*entries);
u32 (*msi_mask_irq)(struct msi_desc *desc, u32 mask, u32 flag);
u32 (*msix_mask_irq)(struct msi_desc *desc, u32 flag);
void (*msi_read_message)(struct msi_desc *desc, struct msi_msg *msg);
void (*msi_write_message)(struct msi_desc *desc, struct msi_msg
*msg);
void (*msi_set_intx)(struct msi_irqs *msi, int enable);
};
struct msi_ops provides several hook functions, generic MSI driver will call
the hook functions to access device specific registers. PCI devices will
share
the same msi_ops, because they have the same way to access MSI hardware
registers.
Generic MSI layer export msi_capability_init() and msix_capability_init()
functions
to drivers. msi/x_capability_init() will initialize MSI capability data
struct msi_desc
and alloc the irq, then write the msi address/data value to hardware
registers.
This series only did compile test, we will test it in x86 and arm platform
later.
For the generic drivers, I don't see much point in differentiating between
MSI and MSI-X, as I believe the difference is something internal to the PCI
[RFC PATCH 00/11] Refactor MSI to support Non-PCI device
Hi all,
The series is a draft of generic MSI driver that supports PCI
and Non-PCI device which have MSI capability. If you're not interested
it, sorry for the noise.
The series is based on Linux-3.16-rc1.
MSI was introduced in PCI Spec 2.2. Currently, kernel MSI
driver codes are bonding with PCI device. Because MSI has a lot
advantages in design. More and more non-PCI devices want to
use MSI as their default interrupt. The existing MSI device
include HPET. HPET driver provide its own MSI code to initialize
and process MSI interrupts. In the latest GIC v3 spec, legacy device
can deliver MSI by the help of a relay device named consolidator.
Consolidator can translate the legacy interrupts connected to it
to MSI/MSI-X. And new non-PCI device will be designed to
support MSI in future. So make the MSI driver code be generic will
help the non-PCI device use MSI more simply.
The new data struct for generic MSI driver.
struct msi_irqs {
u8 msi_enabled:1; /* Enable flag */
u8 msix_enabled:1;
struct list_head msi_list; /* MSI desc list */
void *data; /* help to find the MSI device */
struct msi_ops *ops; /* MSI device specific hook */
};
struct msi_irqs is used to manage MSI related informations. Every device
supports
MSI should contain this data struct and allocate it.
struct msi_ops {
struct msi_desc *(*msi_setup_entry)(struct msi_irqs *msi, struct
msi_desc *entry);
int msix_setup_entries(struct msi_irqs *msi, struct msix_entry
*entries);
u32 (*msi_mask_irq)(struct msi_desc *desc, u32 mask, u32 flag);
u32 (*msix_mask_irq)(struct msi_desc *desc, u32 flag);
void (*msi_read_message)(struct msi_desc *desc, struct msi_msg *msg);
void (*msi_write_message)(struct msi_desc *desc, struct msi_msg *msg);
void (*msi_set_intx)(struct msi_irqs *msi, int enable);
};
struct msi_ops provides several hook functions, generic MSI driver will call
the hook functions to access device specific registers. PCI devices will share
the same msi_ops, because they have the same way to access MSI hardware
registers.
Generic MSI layer export msi_capability_init() and msix_capability_init()
functions
to drivers. msi/x_capability_init() will initialize MSI capability data struct
msi_desc
and alloc the irq, then write the msi address/data value to hardware registers.
This series only did compile test, we will test it in x86 and arm platform
later.
Any comments are welcome.
Thanks!
Yijing.
Yijing Wang (11):
PCI/MSI: Use pci_dev-msi_cap instead of msi_desc-msi_attrib.pos
PCI/MSI: Use new MSI type macro instead of PCI MSI flags
PCI/MSI: Refactor pci_dev_msi_enabled()
PCI/MSI: Move MSIX table address mapping out of msix_capability_init
PCI/MSI: Move populate_msi_sysfs() out of msi_capability_init()
PCI/MSI: Save MSI irq in PCI MSI layer
PCI/MSI: Mask MSI-X entry in msix_setup_entries()
PCI/MSI: Introduce new struct msi_irqs and struct msi_ops
PCI/MSI: refactor PCI MSI driver
PCI/MSI: Split the generic MSI code into new file
x86/MSI: Refactor x86 MSI code
arch/cris/arch-v32/drivers/pci/bios.c |2 +-
arch/frv/mb93090-mb00/pci-vdk.c |2 +-
arch/ia64/pci/pci.c |4 +-
arch/mips/pci/msi-octeon.c|8 +-
arch/powerpc/kernel/eeh_driver.c |2 +-
arch/powerpc/kernel/msi.c |2 +-
arch/powerpc/platforms/pseries/msi.c |8 +-
arch/s390/pci/pci.c |2 +-
arch/x86/include/asm/io_apic.h|2 +-
arch/x86/include/asm/irq_remapping.h |4 +-
arch/x86/include/asm/pci.h|6 +-
arch/x86/include/asm/x86_init.h | 10 +-
arch/x86/kernel/apic/io_apic.c| 25 +-
arch/x86/kernel/x86_init.c| 12 +-
arch/x86/pci/common.c |5 +-
arch/x86/pci/xen.c| 24 +-
drivers/Kconfig |1 +
drivers/Makefile |1 +
drivers/block/nvme-core.c |4 +-
drivers/dma/ioat/dma.c|2 +-
drivers/firewire/ohci.c |2 +-
drivers/gpu/drm/i915/i915_dma.c |4 +-
drivers/iommu/amd_iommu.c | 16 +-
drivers/iommu/intel_irq_remapping.c |9 +-
drivers/iommu/irq_remapping.c | 53 ++--
drivers/iommu/irq_remapping.h |6 +-
drivers/irqchip/irq-armada-370-xp.c |2 +-
drivers/misc/mei/hw-me.c |2 +-
drivers/misc/mei/hw-txe.c |2 +-
drivers/misc/mei/pci-me.c |4 +-
drivers/misc/mei/pci-txe.c|4 +-
drivers/misc/mic/host/mic_debugfs.c |4 +-
drivers/misc/mic/host/mic_intr.c |8 +-
drivers/msi/Kconfig |8 +
drivers/msi/Makefile |1 +
drivers/msi/msi.c
[RFC PATCH 01/11] PCI/MSI: Use pci_dev-msi_cap instead of msi_desc-msi_attrib.pos
PCI devices save the msi and msix capability offset in pci_dev-msi_cap
and pci_dev-msix_cap. When we access PCI device MSI and MSIX
registers, we can use msi_cap and msix_cap in pci_dev directly.
Remove the pos member in msi_attrib.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
arch/mips/pci/msi-octeon.c |4 ++--
drivers/pci/host/pcie-designware.c |2 +-
drivers/pci/msi.c |2 --
include/linux/msi.h|1 -
4 files changed, 3 insertions(+), 6 deletions(-)
diff --git a/arch/mips/pci/msi-octeon.c b/arch/mips/pci/msi-octeon.c
index ab0c5d1..6a6a99f 100644
--- a/arch/mips/pci/msi-octeon.c
+++ b/arch/mips/pci/msi-octeon.c
@@ -73,7 +73,7 @@ int arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc
*desc)
* wants. Most devices only want 1, which will give
* configured_private_bits and request_private_bits equal 0.
*/
- pci_read_config_word(dev, desc-msi_attrib.pos + PCI_MSI_FLAGS,
+ pci_read_config_word(dev, dev-msi_cap + PCI_MSI_FLAGS,
control);
/*
@@ -176,7 +176,7 @@ msi_irq_allocated:
/* Update the number of IRQs the device has available to it */
control = ~PCI_MSI_FLAGS_QSIZE;
control |= request_private_bits 4;
- pci_write_config_word(dev, desc-msi_attrib.pos + PCI_MSI_FLAGS,
+ pci_write_config_word(dev, dev-msi_cap + PCI_MSI_FLAGS,
control);
irq_set_msi_desc(irq, desc);
diff --git a/drivers/pci/host/pcie-designware.c
b/drivers/pci/host/pcie-designware.c
index 1eaf4df..04339cd 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -335,7 +335,7 @@ static int dw_msi_setup_irq(struct msi_chip *chip, struct
pci_dev *pdev,
return -EINVAL;
}
- pci_read_config_word(pdev, desc-msi_attrib.pos+PCI_MSI_FLAGS,
+ pci_read_config_word(pdev, pdev-msi_cap + PCI_MSI_FLAGS,
msg_ctr);
msgvec = (msg_ctrPCI_MSI_FLAGS_QSIZE) 4;
if (msgvec == 0)
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 5a40516..e67acd1 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -595,7 +595,6 @@ static struct msi_desc *msi_setup_entry(struct pci_dev *dev)
entry-msi_attrib.entry_nr = 0;
entry-msi_attrib.maskbit = !!(control PCI_MSI_FLAGS_MASKBIT);
entry-msi_attrib.default_irq = dev-irq; /* Save IOAPIC IRQ */
- entry-msi_attrib.pos = dev-msi_cap;
entry-msi_attrib.multi_cap = (control PCI_MSI_FLAGS_QMASK) 1;
if (control PCI_MSI_FLAGS_64BIT)
@@ -699,7 +698,6 @@ static int msix_setup_entries(struct pci_dev *dev, void
__iomem *base,
entry-msi_attrib.is_64 = 1;
entry-msi_attrib.entry_nr = entries[i].entry;
entry-msi_attrib.default_irq = dev-irq;
- entry-msi_attrib.pos = dev-msix_cap;
entry-mask_base= base;
list_add_tail(entry-list, dev-msi_list);
diff --git a/include/linux/msi.h b/include/linux/msi.h
index 8103f32..ce88c5b 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -29,7 +29,6 @@ struct msi_desc {
__u8multi_cap : 3; /* log2 num of messages supported */
__u8maskbit : 1;/* mask-pending bit supported ? */
__u8is_64 : 1;/* Address size: 0=32bit 1=64bit */
- __u8pos;/* Location of the msi capability */
__u16 entry_nr; /* specific enabled entry */
unsigned default_irq; /* default pre-assigned irq */
} msi_attrib;
--
1.7.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 06/11] PCI/MSI: Save MSI irq in PCI MSI layer
Save MSI irq in PCI MSI layer, this is preparation
for generic MSI.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
drivers/pci/msi.c |5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 21b16e0..f96dd38 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -650,8 +650,6 @@ static int msi_capability_init(struct pci_dev *dev, int
nvec)
pci_intx_for_msi(dev, 0);
msi_set_enable(dev, 1);
dev-msi_enabled = 1;
-
- dev-irq = entry-irq;
return 0;
}
@@ -1059,6 +1057,7 @@ int pci_enable_msi_range(struct pci_dev *dev, int minvec,
int maxvec)
{
int nvec;
int rc;
+ struct msi_desc *entry;
if (dev-current_state != PCI_D0)
return -EINVAL;
@@ -1114,6 +1113,8 @@ int pci_enable_msi_range(struct pci_dev *dev, int minvec,
int maxvec)
return rc;
}
+ entry = list_entry(dev-msi_list.next, struct msi_desc, list);
+ dev-irq = entry-irq;
return nvec;
}
EXPORT_SYMBOL(pci_enable_msi_range);
--
1.7.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 04/11] PCI/MSI: Move MSIX table address mapping out of msix_capability_init
Move MSIX table address mapping work to PCI MSIX layer.
Some Non-PCI MSI device will do their address mapping work before
enable MSIX capability or their MSIX table address is within
device address block. So Move address mapping stuff out of the
generic MSIX core. This is prepartion for generic MSI drvier.
Suggested-by: Yun Wu wuyun...@huawei.com
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
drivers/pci/msi.c | 25 +
1 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index d5c8e56..116383c 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -668,8 +668,8 @@ static void __iomem *msix_map_region(struct pci_dev *dev,
unsigned nr_entries)
u32 table_offset;
u8 bir;
- pci_read_config_dword(dev, dev-msix_cap + PCI_MSIX_TABLE,
- table_offset);
+ pci_read_config_dword(dev, dev-msix_cap + PCI_MSIX_TABLE,
+ table_offset);
bir = (u8)(table_offset PCI_MSIX_TABLE_BIR);
table_offset = PCI_MSIX_TABLE_OFFSET;
phys_addr = pci_resource_start(dev, bir) + table_offset;
@@ -734,22 +734,14 @@ static void msix_program_entries(struct pci_dev *dev,
* single MSI-X irq. A return of zero indicates the successful setup of
* requested MSI-X entries with allocated irqs or non-zero for otherwise.
**/
-static int msix_capability_init(struct pci_dev *dev,
+static int msix_capability_init(struct pci_dev *dev, void __iomem *base,
struct msix_entry *entries, int nvec)
{
int ret;
- u16 control;
- void __iomem *base;
/* Ensure MSI-X is disabled while it is set up */
msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_ENABLE, 0);
- pci_read_config_word(dev, dev-msix_cap + PCI_MSIX_FLAGS, control);
- /* Request Map MSI-X table region */
- base = msix_map_region(dev, msix_table_size(control));
- if (!base)
- return -ENOMEM;
-
ret = msix_setup_entries(dev, base, entries, nvec);
if (ret)
return ret;
@@ -948,6 +940,8 @@ int pci_enable_msix(struct pci_dev *dev, struct msix_entry
*entries, int nvec)
{
int status, nr_entries;
int i, j;
+ void __iomem *base;
+ u16 control;
if (!entries || !dev-msix_cap || dev-current_state != PCI_D0)
return -EINVAL;
@@ -978,7 +972,14 @@ int pci_enable_msix(struct pci_dev *dev, struct msix_entry
*entries, int nvec)
dev_info(dev-dev, can't enable MSI-X (MSI IRQ already
assigned)\n);
return -EINVAL;
}
- status = msix_capability_init(dev, entries, nvec);
+
+ /* Request Map MSI-X table region */
+ pci_read_config_word(dev, dev-msix_cap + PCI_MSIX_FLAGS, control);
+ base = msix_map_region(dev, msix_table_size(control));
+ if (!base)
+ return -ENOMEM;
+
+ status = msix_capability_init(dev, base, entries, nvec);
return status;
}
EXPORT_SYMBOL(pci_enable_msix);
--
1.7.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 05/11] PCI/MSI: Move populate_msi_sysfs() out of msi_capability_init()
Because some Non-PCI devices don't need to create sysfs object,
so move populate_msi_sysfs() out of generic MSI function
msi/x_capability_init().
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
drivers/pci/msi.c | 31 ++-
1 files changed, 18 insertions(+), 13 deletions(-)
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 116383c..21b16e0 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -646,13 +646,6 @@ static int msi_capability_init(struct pci_dev *dev, int
nvec)
return ret;
}
- ret = populate_msi_sysfs(dev);
- if (ret) {
- msi_mask_irq(entry, mask, ~mask);
- free_msi_irqs(dev);
- return ret;
- }
-
/* Set MSI enabled bits */
pci_intx_for_msi(dev, 0);
msi_set_enable(dev, 1);
@@ -760,10 +753,6 @@ static int msix_capability_init(struct pci_dev *dev, void
__iomem *base,
msix_program_entries(dev, entries);
- ret = populate_msi_sysfs(dev);
- if (ret)
- goto out_free;
-
/* Set MSI-X enabled bits and unmask the function */
pci_intx_for_msi(dev, 0);
dev-msix_enabled = 1;
@@ -789,7 +778,6 @@ out_avail:
ret = avail;
}
-out_free:
free_msi_irqs(dev);
return ret;
@@ -939,7 +927,7 @@ EXPORT_SYMBOL(pci_msix_vec_count);
int pci_enable_msix(struct pci_dev *dev, struct msix_entry *entries, int nvec)
{
int status, nr_entries;
- int i, j;
+ int i, j, ret;
void __iomem *base;
u16 control;
@@ -980,6 +968,14 @@ int pci_enable_msix(struct pci_dev *dev, struct msix_entry
*entries, int nvec)
return -ENOMEM;
status = msix_capability_init(dev, base, entries, nvec);
+ if (!status) {
+ ret = populate_msi_sysfs(dev);
+ if (ret) {
+ dev-msix_enabled = 0;
+ pci_intx_for_msi(dev, 1);
+ free_msi_irqs(dev);
+ }
+ }
return status;
}
EXPORT_SYMBOL(pci_enable_msix);
@@ -1109,6 +1105,15 @@ int pci_enable_msi_range(struct pci_dev *dev, int
minvec, int maxvec)
}
} while (rc);
+ rc = populate_msi_sysfs(dev);
+ if (rc) {
+ msi_set_enable(dev, 0);
+ pci_intx_for_msi(dev, 1);
+ dev-msi_enabled = 0;
+ free_msi_irqs(dev);
+ return rc;
+ }
+
return nvec;
}
EXPORT_SYMBOL(pci_enable_msi_range);
--
1.7.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 02/11] PCI/MSI: Use new MSI type macro instead of PCI MSI flags
Add new MSI type marco(MSI_TYPE and MSIX_TYPE) to support
the future generic MSI driver. The coming generic MSI driver
will be used by PCI and Non-PCI devices that have MSI capability.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
arch/mips/pci/msi-octeon.c |4 ++--
arch/powerpc/kernel/msi.c|2 +-
arch/powerpc/platforms/pseries/msi.c |8
arch/s390/pci/pci.c |2 +-
arch/x86/kernel/apic/io_apic.c |2 +-
arch/x86/pci/xen.c | 24
drivers/iommu/irq_remapping.c|2 +-
drivers/irqchip/irq-armada-370-xp.c |2 +-
drivers/pci/msi.c| 10 +-
include/linux/msi.h |3 +++
10 files changed, 31 insertions(+), 28 deletions(-)
diff --git a/arch/mips/pci/msi-octeon.c b/arch/mips/pci/msi-octeon.c
index 6a6a99f..8105610 100644
--- a/arch/mips/pci/msi-octeon.c
+++ b/arch/mips/pci/msi-octeon.c
@@ -192,14 +192,14 @@ int arch_setup_msi_irqs(struct pci_dev *dev, int nvec,
int type)
/*
* MSI-X is not supported.
*/
- if (type == PCI_CAP_ID_MSIX)
+ if (type == MSIX_TYPE)
return -EINVAL;
/*
* If an architecture wants to support multiple MSI, it needs to
* override arch_setup_msi_irqs()
*/
- if (type == PCI_CAP_ID_MSI nvec 1)
+ if (type == MSI_TYPE nvec 1)
return 1;
list_for_each_entry(entry, dev-msi_list, list) {
diff --git a/arch/powerpc/kernel/msi.c b/arch/powerpc/kernel/msi.c
index 8bbc12d..05b3133 100644
--- a/arch/powerpc/kernel/msi.c
+++ b/arch/powerpc/kernel/msi.c
@@ -21,7 +21,7 @@ int arch_msi_check_device(struct pci_dev* dev, int nvec, int
type)
}
/* PowerPC doesn't support multiple MSI yet */
- if (type == PCI_CAP_ID_MSI nvec 1)
+ if (type == MSI_TYPE nvec 1)
return 1;
if (ppc_md.msi_check_device) {
diff --git a/arch/powerpc/platforms/pseries/msi.c
b/arch/powerpc/platforms/pseries/msi.c
index 0c882e8..e2f27d6 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -339,7 +339,7 @@ static int rtas_msi_check_device(struct pci_dev *pdev, int
nvec, int type)
{
int quota, rc;
- if (type == PCI_CAP_ID_MSIX)
+ if (type == MSIX_TYPE)
rc = check_req_msix(pdev, nvec);
else
rc = check_req_msi(pdev, nvec);
@@ -406,14 +406,14 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int
nvec_in, int type)
if (!pdn)
return -ENODEV;
- if (type == PCI_CAP_ID_MSIX check_msix_entries(pdev))
+ if (type == MSIX_TYPE check_msix_entries(pdev))
return -EINVAL;
/*
* Firmware currently refuse any non power of two allocation
* so we round up if the quota will allow it.
*/
- if (type == PCI_CAP_ID_MSIX) {
+ if (type == MSIX_TYPE) {
int m = roundup_pow_of_two(nvec);
int quota = msi_quota_for_device(pdev, m);
@@ -427,7 +427,7 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int
nvec_in, int type)
* return MSI-Xs.
*/
again:
- if (type == PCI_CAP_ID_MSI) {
+ if (type == MSI_TYPE) {
if (pdn-force_32bit_msi) {
rc = rtas_change_msi(pdn, RTAS_CHANGE_32MSI_FN, nvec);
if (rc 0) {
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 9ddc51e..fe3a40c 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -407,7 +407,7 @@ int arch_setup_msi_irqs(struct pci_dev *pdev, int nvec, int
type)
struct msi_msg msg;
int rc, irq;
- if (type == PCI_CAP_ID_MSI nvec 1)
+ if (type == MSI_TYPE nvec 1)
return 1;
msi_vecs = min(nvec, ZPCI_MSI_VEC_MAX);
msi_vecs = min_t(unsigned int, msi_vecs, CONFIG_PCI_NR_MSI);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 81e08ef..b833042 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3069,7 +3069,7 @@ int native_setup_msi_irqs(struct pci_dev *dev, int nvec,
int type)
int node, ret;
/* Multiple MSI vectors only supported with interrupt remapping */
- if (type == PCI_CAP_ID_MSI nvec 1)
+ if (type == MSI_TYPE nvec 1)
return 1;
node = dev_to_node(dev-dev);
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 905956f..c19a8de 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -162,14 +162,14 @@ static int xen_setup_msi_irqs(struct pci_dev *dev, int
nvec, int type)
struct msi_desc *msidesc;
int *v;
- if (type == PCI_CAP_ID_MSI nvec 1)
+ if (type == MSI_TYPE nvec 1)
return 1;
v = kzalloc(sizeof(int) * max(1, nvec), GFP_KERNEL
[RFC PATCH 11/11] x86/MSI: Refactor x86 MSI code
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
arch/x86/include/asm/io_apic.h |2 +-
arch/x86/include/asm/irq_remapping.h |4 +-
arch/x86/include/asm/pci.h |6 ++--
arch/x86/include/asm/x86_init.h | 10 +++---
arch/x86/kernel/apic/io_apic.c | 23 +++
arch/x86/kernel/x86_init.c | 12
drivers/iommu/amd_iommu.c| 16 ++
drivers/iommu/intel_irq_remapping.c |9 --
drivers/iommu/irq_remapping.c| 51 -
drivers/iommu/irq_remapping.h|6 ++--
drivers/msi/msi.c|3 +-
11 files changed, 72 insertions(+), 70 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index 90f97b4..692a90f 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -158,7 +158,7 @@ extern int native_setup_ioapic_entry(int, struct
IO_APIC_route_entry *,
struct io_apic_irq_attr *);
extern void eoi_ioapic_irq(unsigned int irq, struct irq_cfg *cfg);
-extern void native_compose_msi_msg(struct pci_dev *pdev,
+extern void native_compose_msi_msg(struct msi_irqs *msi,
unsigned int irq, unsigned int dest,
struct msi_msg *msg, u8 hpet_id);
extern void native_eoi_ioapic_pin(int apic, int pin, int vector);
diff --git a/arch/x86/include/asm/irq_remapping.h
b/arch/x86/include/asm/irq_remapping.h
index b7747c4..a10003d 100644
--- a/arch/x86/include/asm/irq_remapping.h
+++ b/arch/x86/include/asm/irq_remapping.h
@@ -47,7 +47,7 @@ extern int setup_ioapic_remapped_entry(int irq,
int vector,
struct io_apic_irq_attr *attr);
extern void free_remapped_irq(int irq);
-extern void compose_remapped_msi_msg(struct pci_dev *pdev,
+extern void compose_remapped_msi_msg(struct msi_irqs *msi,
unsigned int irq, unsigned int dest,
struct msi_msg *msg, u8 hpet_id);
extern int setup_hpet_msi_remapped(unsigned int irq, unsigned int id);
@@ -77,7 +77,7 @@ static inline int setup_ioapic_remapped_entry(int irq,
return -ENODEV;
}
static inline void free_remapped_irq(int irq) { }
-static inline void compose_remapped_msi_msg(struct pci_dev *pdev,
+static inline void compose_remapped_msi_msg(struct msi_irqs *msi,
unsigned int irq, unsigned int dest,
struct msi_msg *msg, u8 hpet_id)
{
diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
index 0892ea0..04c9ef6 100644
--- a/arch/x86/include/asm/pci.h
+++ b/arch/x86/include/asm/pci.h
@@ -96,10 +96,10 @@ extern void pci_iommu_alloc(void);
#ifdef CONFIG_PCI_MSI
/* implemented in arch/x86/kernel/apic/io_apic. */
struct msi_desc;
-int native_setup_msi_irqs(struct pci_dev *dev, int nvec, int type);
+int native_setup_msi_irqs(struct msi_irqs *msi, int nvec, int type);
void native_teardown_msi_irq(unsigned int irq);
-void native_restore_msi_irqs(struct pci_dev *dev);
-int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
+void native_restore_msi_irqs(struct msi_irqs *msi);
+int setup_msi_irq(struct msi_irqs *msi, struct msi_desc *msidesc,
unsigned int irq_base, unsigned int irq_offset);
#else
#define native_setup_msi_irqs NULL
diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h
index e45e4da..8e42f17 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -170,18 +170,18 @@ struct x86_platform_ops {
void (*apic_post_init)(void);
};
-struct pci_dev;
+struct msi_irqs;
struct msi_msg;
struct msi_desc;
struct x86_msi_ops {
- int (*setup_msi_irqs)(struct pci_dev *dev, int nvec, int type);
- void (*compose_msi_msg)(struct pci_dev *dev, unsigned int irq,
+ int (*setup_msi_irqs)(struct msi_irqs *msi, int nvec, int type);
+ void (*compose_msi_msg)(struct msi_irqs *msi, unsigned int irq,
unsigned int dest, struct msi_msg *msg,
u8 hpet_id);
void (*teardown_msi_irq)(unsigned int irq);
- void (*teardown_msi_irqs)(struct pci_dev *dev);
- void (*restore_msi_irqs)(struct pci_dev *dev);
+ void (*teardown_msi_irqs)(struct msi_irqs *msi);
+ void (*restore_msi_irqs)(struct msi_irqs *msi);
int (*setup_hpet_msi)(unsigned int irq, unsigned int id);
u32 (*msi_mask_irq)(struct msi_desc *desc, u32 mask, u32 flag);
u32 (*msix_mask_irq)(struct msi_desc *desc, u32 flag);
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index b833042..3cb4a6a 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -2939,7 +2939,7 @@ void arch_teardown_hwirq
[RFC PATCH 03/11] PCI/MSI: Refactor pci_dev_msi_enabled()
Pci_dev_msi_enabled() is used to check whether device
MSI/MSIX enabled. Refactor this function to suuport
checking only device MSI or MSIX enabled.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
arch/cris/arch-v32/drivers/pci/bios.c |2 +-
arch/frv/mb93090-mb00/pci-vdk.c |2 +-
arch/ia64/pci/pci.c |4 ++--
arch/powerpc/kernel/eeh_driver.c |2 +-
arch/x86/pci/common.c |5 +++--
drivers/block/nvme-core.c |4 ++--
drivers/dma/ioat/dma.c|2 +-
drivers/firewire/ohci.c |2 +-
drivers/gpu/drm/i915/i915_dma.c |4 ++--
drivers/misc/mei/hw-me.c |2 +-
drivers/misc/mei/hw-txe.c |2 +-
drivers/misc/mei/pci-me.c |4 ++--
drivers/misc/mei/pci-txe.c|4 ++--
drivers/misc/mic/host/mic_debugfs.c |4 ++--
drivers/misc/mic/host/mic_intr.c |8
drivers/ntb/ntb_hw.c |2 +-
drivers/pci/irq.c |4 ++--
drivers/pci/msi.c | 15 +--
drivers/pci/pci.c |6 +++---
drivers/pci/pcie/portdrv_core.c |4 ++--
drivers/scsi/esas2r/esas2r_init.c |4 ++--
drivers/scsi/esas2r/esas2r_ioctl.c|4 ++--
drivers/scsi/hpsa.c |4 ++--
drivers/staging/crystalhd/crystalhd_lnx.c |2 +-
drivers/xen/xen-pciback/pciback_ops.c | 12 ++--
include/linux/pci.h | 12 ++--
virt/kvm/assigned-dev.c |2 +-
27 files changed, 67 insertions(+), 55 deletions(-)
diff --git a/arch/cris/arch-v32/drivers/pci/bios.c
b/arch/cris/arch-v32/drivers/pci/bios.c
index 64a5fb9..d9d8332 100644
--- a/arch/cris/arch-v32/drivers/pci/bios.c
+++ b/arch/cris/arch-v32/drivers/pci/bios.c
@@ -93,7 +93,7 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
if ((err = pcibios_enable_resources(dev, mask)) 0)
return err;
- if (!dev-msi_enabled)
+ if (!pci_dev_msi_enabled(dev, MSI_TYPE))
pcibios_enable_irq(dev);
return 0;
}
diff --git a/arch/frv/mb93090-mb00/pci-vdk.c b/arch/frv/mb93090-mb00/pci-vdk.c
index efa5d65..b96c128 100644
--- a/arch/frv/mb93090-mb00/pci-vdk.c
+++ b/arch/frv/mb93090-mb00/pci-vdk.c
@@ -409,7 +409,7 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
if ((err = pci_enable_resources(dev, mask)) 0)
return err;
- if (!dev-msi_enabled)
+ if (!pci_dev_msi_enabled(dev, MSI_TYPE))
pcibios_enable_irq(dev);
return 0;
}
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 291a582..da8ddff 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -568,7 +568,7 @@ pcibios_enable_device (struct pci_dev *dev, int mask)
if (ret 0)
return ret;
- if (!dev-msi_enabled)
+ if (!pci_dev_msi_enabled(dev, MSI_TYPE))
return acpi_pci_irq_enable(dev);
return 0;
}
@@ -577,7 +577,7 @@ void
pcibios_disable_device (struct pci_dev *dev)
{
BUG_ON(atomic_read(dev-enable_cnt));
- if (!dev-msi_enabled)
+ if (!pci_dev_msi_enabled(dev, MSI_TYPE))
acpi_pci_irq_disable(dev);
}
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 420da61..e3f2074 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -123,7 +123,7 @@ static void eeh_disable_irq(struct pci_dev *dev)
* effectively disabled by the DMA Stopped state
* when an EEH error occurs.
*/
- if (dev-msi_enabled || dev-msix_enabled)
+ if (pci_dev_msi_enabled(dev, MSI_TYPE | MSIX_TYPE))
return;
if (!irq_has_action(dev-irq))
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 059a76c..4597940 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -662,14 +662,15 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
if ((err = pci_enable_resources(dev, mask)) 0)
return err;
- if (!pci_dev_msi_enabled(dev))
+ if (!pci_dev_msi_enabled(dev, MSI_TYPE | MSIX_TYPE))
return pcibios_enable_irq(dev);
return 0;
}
void pcibios_disable_device (struct pci_dev *dev)
{
- if (!pci_dev_msi_enabled(dev) pcibios_disable_irq)
+ if (!pci_dev_msi_enabled(dev, MSI_TYPE | MSIX_TYPE)
+pcibios_disable_irq)
pcibios_disable_irq(dev);
}
diff --git a/drivers/block/nvme-core.c b/drivers/block/nvme-core.c
index 02351e2..f96b90f 100644
--- a/drivers/block/nvme-core.c
+++ b/drivers/block/nvme-core.c
@@ -2325,9 +2325,9 @@ static int nvme_dev_map(struct nvme_dev *dev)
static void nvme_dev_unmap(struct nvme_dev *dev
[RFC PATCH 07/11] PCI/MSI: Mask MSI-X entry in msix_setup_entries()
Save the MSI-X entry initial mask status in
msix_setup_entries(), also mask the entry.
This is preparation for generic MSI.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
drivers/pci/msi.c | 21 +++--
1 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index f96dd38..41c33da 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -672,7 +672,7 @@ static int msix_setup_entries(struct pci_dev *dev, void
__iomem *base,
struct msix_entry *entries, int nvec)
{
struct msi_desc *entry;
- int i;
+ int i, offset;
for (i = 0; i nvec; i++) {
entry = alloc_msi_entry(dev);
@@ -691,6 +691,15 @@ static int msix_setup_entries(struct pci_dev *dev, void
__iomem *base,
entry-msi_attrib.default_irq = dev-irq;
entry-mask_base= base;
+ msix_clear_and_set_ctrl(dev, 0,
+ PCI_MSIX_FLAGS_MASKALL | PCI_MSIX_FLAGS_ENABLE);
+ offset = entries[i].entry * PCI_MSIX_ENTRY_SIZE +
+ PCI_MSIX_ENTRY_VECTOR_CTRL;
+ entry-masked = readl(entry-mask_base + offset);
+ msix_mask_irq(entry, 1);
+ msix_clear_and_set_ctrl(dev,
+ PCI_MSIX_FLAGS_MASKALL | PCI_MSIX_FLAGS_ENABLE,
0);
+
list_add_tail(entry-list, dev-msi_list);
}
@@ -704,13 +713,8 @@ static void msix_program_entries(struct pci_dev *dev,
int i = 0;
list_for_each_entry(entry, dev-msi_list, list) {
- int offset = entries[i].entry * PCI_MSIX_ENTRY_SIZE +
- PCI_MSIX_ENTRY_VECTOR_CTRL;
-
entries[i].vector = entry-irq;
irq_set_msi_desc(entry-irq, entry);
- entry-masked = readl(entry-mask_base + offset);
- msix_mask_irq(entry, 1);
i++;
}
}
@@ -746,16 +750,13 @@ static int msix_capability_init(struct pci_dev *dev, void
__iomem *base,
* MSI-X registers. We need to mask all the vectors to prevent
* interrupts coming in before they're fully set up.
*/
- msix_clear_and_set_ctrl(dev, 0,
- PCI_MSIX_FLAGS_MASKALL | PCI_MSIX_FLAGS_ENABLE);
-
msix_program_entries(dev, entries);
/* Set MSI-X enabled bits and unmask the function */
pci_intx_for_msi(dev, 0);
dev-msix_enabled = 1;
- msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL, 0);
+ msix_clear_and_set_ctrl(dev, 0, PCI_MSIX_FLAGS_ENABLE);
return 0;
--
1.7.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 08/11] PCI/MSI: Introduce new struct msi_irqs and struct msi_ops
Currently, MSI driver is bonding with PCI everywhere.
Now introduce a new struct msi_irqs to manage all MSI
related informations in a MSI support device. In addition,
we introduce struct msi_ops to hook all device specific
MSI operations. Then MSI driver can be decoupled with
PCI.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
include/linux/msi.h | 30 +-
include/linux/pci.h |7 +--
2 files changed, 30 insertions(+), 7 deletions(-)
diff --git a/include/linux/msi.h b/include/linux/msi.h
index 3ad8416..5a672d3 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -10,6 +10,34 @@ struct msi_msg {
u32 data; /* 16 bits of msi message data */
};
+struct msi_ops;
+
+struct msi_irqs {
+ u8 msi_enabled:1;
+ u8 msix_enabled:1;
+ int node;
+ struct list_head msi_list;
+ void *data;
+ struct msi_ops *ops;
+};
+
+struct msix_entry {
+ u32 vector; /* kernel uses to write allocated vector */
+ u16 entry; /* driver uses to specify entry, OS writes */
+};
+
+struct msi_ops {
+ void (*msi_set_enable)(struct msi_irqs *msi, int enable, int type);
+ struct msi_desc *(*msi_setup_entry)(struct msi_irqs *msi);
+ int (*msix_setup_entries)(struct msi_irqs *msi, void __iomem *base,
+ struct msix_entry *entries, int nvec);
+ u32 (*msi_mask_irq)(struct msi_desc *desc, u32 mask, u32 flag);
+ u32 (*msix_mask_irq)(struct msi_desc *desc, u32 flag);
+ void (*msi_read_message)(struct msi_desc *desc, struct msi_msg *msg);
+ void (*msi_write_message)(struct msi_desc *desc, struct msi_msg *msg);
+ void (*msi_set_intx)(struct msi_irqs *msi, int enable);
+};
+
/* Helper functions */
struct irq_data;
struct msi_desc;
@@ -42,7 +70,7 @@ struct msi_desc {
void __iomem *mask_base;
u8 mask_pos;
};
- struct pci_dev *dev;
+ struct msi_irqs *msi;
/* Last set MSI message */
struct msi_msg msg;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index c6c01ae..c7bca1c 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -32,8 +32,8 @@
#include uapi/linux/pci.h
#include linux/pci_ids.h
-
#include linux/msi.h
+
/*
* The PCI interface treats multi-function devices as independent
* devices. The slot/function address of each device is encoded
@@ -1182,11 +1182,6 @@ enum pci_dma_burst_strategy {
strategy_parameter byte boundaries */
};
-struct msix_entry {
- u32 vector; /* kernel uses to write allocated vector */
- u16 entry; /* driver uses to specify entry, OS writes */
-};
-
#ifdef CONFIG_PCI_MSI
int pci_msi_vec_count(struct pci_dev *dev);
--
1.7.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC PATCH 10/11] PCI/MSI: Split the generic MSI code into new file
MSI interrupt will not only used in PCI device, more
and more Non-PCI device also want to use MSI. ARM
GIC v3 spec says in ARM platform with GIC v3 controller,
Non-PCI device can also be design to support MSI to
simplify interrupt wires, for the existing Non-PCI
device, consolidator is designed and used to translate
legacy interrupt to MSI. So for support Non-PCI MSI
device, generic MSI driver is needed. Split the generic
MSI code into new location, drivers/msi/msi.c. Then
MSI driver does not depend PCI anymore.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
drivers/Kconfig |1 +
drivers/Makefile |1 +
drivers/msi/Kconfig |8 +
drivers/msi/Makefile |1 +
drivers/msi/msi.c| 540 ++
drivers/pci/Kconfig |6 +-
drivers/pci/msi.c| 500 ---
include/linux/msi.h | 31 +++-
8 files changed, 617 insertions(+), 471 deletions(-)
create mode 100644 drivers/msi/Kconfig
create mode 100644 drivers/msi/Makefile
create mode 100644 drivers/msi/msi.c
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 0e87a34..4d05749 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -176,4 +176,5 @@ source drivers/powercap/Kconfig
source drivers/mcb/Kconfig
+source drivers/msi/Kconfig
endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index f98b50d..47ae3d1 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -158,3 +158,4 @@ obj-$(CONFIG_NTB) += ntb/
obj-$(CONFIG_FMC) += fmc/
obj-$(CONFIG_POWERCAP) += powercap/
obj-$(CONFIG_MCB) += mcb/
+obj-$(CONFIG_MSI) += msi/
diff --git a/drivers/msi/Kconfig b/drivers/msi/Kconfig
new file mode 100644
index 000..739bd13
--- /dev/null
+++ b/drivers/msi/Kconfig
@@ -0,0 +1,8 @@
+config MSI
+ bool Message Signaled Interrupts (MSI and MSI-X)
+ default y
+ help
+ This allows device drivers to use generic MSI(Message
+ Signaled Interrupt). Message Signaled Interrupts enable
+ a device to generate an interrupt using an inbound Memory
+ Write to a specific target address.
diff --git a/drivers/msi/Makefile b/drivers/msi/Makefile
new file mode 100644
index 000..39cb026
--- /dev/null
+++ b/drivers/msi/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_MSI) += msi.o
diff --git a/drivers/msi/msi.c b/drivers/msi/msi.c
new file mode 100644
index 000..3fbd539
--- /dev/null
+++ b/drivers/msi/msi.c
@@ -0,0 +1,540 @@
+/*
+ * File: msi.c
+ * Purpose:Message Signaled Interrupt (MSI)
+ *
+ * Copyright (C) 2014 Huawei Ltd.
+ * Copyright (C) Yijing Wang wangyij...@huawei.com
+ */
+#include linux/err.h
+#include linux/mm.h
+#include linux/irq.h
+#include linux/interrupt.h
+#include linux/export.h
+#include linux/ioport.h
+#include linux/proc_fs.h
+#include linux/msi.h
+#include linux/smp.h
+#include linux/errno.h
+#include linux/io.h
+#include linux/slab.h
+#include linux/device.h
+#include linux/pci.h
+
+/* Arch hooks */
+
+int __weak arch_setup_msi_irq(struct msi_irqs *msi, struct msi_desc *desc)
+{
+ struct pci_dev *dev = msi-data;
+ struct msi_chip *chip = dev-bus-msi; //TO BE DONE: rework msi_chip to
support Non-PCI MSI
+ int err;
+
+ if (!chip || !chip-setup_irq)
+ return -EINVAL;
+
+ err = chip-setup_irq(chip, dev, desc);
+ if (err 0)
+ return err;
+
+ irq_set_chip_data(desc-irq, chip);
+ return 0;
+}
+
+void __weak arch_teardown_msi_irq(unsigned int irq)
+{
+ struct msi_chip *chip = irq_get_chip_data(irq);
+
+ if (!chip || !chip-teardown_irq)
+ return;
+
+ chip-teardown_irq(chip, irq);
+}
+
+int __weak arch_msi_check_device(struct msi_irqs *msi, int nvec, int type)
+{
+ struct pci_dev *dev = msi-data;
+ struct msi_chip *chip = dev-bus-msi; //TO BE DONE: rework msi_chip to
support Non-PCI MSI
+
+ if (!chip || !chip-check_device)
+ return 0;
+
+ return chip-check_device(chip, dev, nvec, type);
+}
+
+int __weak arch_setup_msi_irqs(struct msi_irqs *msi, int nvec, int type)
+{
+ struct msi_desc *entry;
+ int ret;
+
+ /*
+* If an architecture wants to support multiple MSI, it needs to
+* override arch_setup_msi_irqs()
+*/
+ if (type == MSI_TYPE nvec 1)
+ return 1;
+
+ list_for_each_entry(entry, msi-msi_list, list) {
+ ret = arch_setup_msi_irq(msi, entry);
+ if (ret 0)
+ return ret;
+ if (ret 0)
+ return -ENOSPC;
+ }
+ return 0;
+}
+
+
+void __weak arch_teardown_msi_irqs(struct msi_irqs *msi)
+{
+ return default_teardown_msi_irqs(msi);
+}
+
+/*
+ * We have a default implementation available as a separate non-weak
+ * function, as it is used by the Xen x86 PCI code
+ */
+void default_teardown_msi_irqs
[RFC PATCH 09/11] PCI/MSI: refactor PCI MSI driver
Use struct msi_ops to hook PCI MSI operations,
and use struct msi_irqs to refactor PCI MSI drvier.
Signed-off-by: Yijing Wang wangyij...@huawei.com
---
drivers/pci/msi.c | 351 ++-
include/linux/msi.h | 14 +-
include/linux/pci.h | 11 +-
3 files changed, 222 insertions(+), 154 deletions(-)
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 41c33da..f0c5989 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -29,8 +29,9 @@ static int pci_msi_enable = 1;
/* Arch hooks */
-int __weak arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc *desc)
+int __weak arch_setup_msi_irq(struct msi_irqs *msi, struct msi_desc *desc)
{
+ struct pci_dev *dev = msi-data; //TO BE DONE: rework msi_chip to
support Non-PCI
struct msi_chip *chip = dev-bus-msi;
int err;
@@ -56,8 +57,9 @@ void __weak arch_teardown_msi_irq(unsigned int irq)
chip-teardown_irq(chip, irq);
}
-int __weak arch_msi_check_device(struct pci_dev *dev, int nvec, int type)
+int __weak arch_msi_check_device(struct msi_irqs *msi, int nvec, int type)
{
+ struct pci_dev *dev = msi-data; //TO BE DONE: rework msi_chip to
support Non-PCI
struct msi_chip *chip = dev-bus-msi;
if (!chip || !chip-check_device)
@@ -66,7 +68,7 @@ int __weak arch_msi_check_device(struct pci_dev *dev, int
nvec, int type)
return chip-check_device(chip, dev, nvec, type);
}
-int __weak arch_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+int __weak arch_setup_msi_irqs(struct msi_irqs *msi, int nvec, int type)
{
struct msi_desc *entry;
int ret;
@@ -78,8 +80,8 @@ int __weak arch_setup_msi_irqs(struct pci_dev *dev, int nvec,
int type)
if (type == MSI_TYPE nvec 1)
return 1;
- list_for_each_entry(entry, dev-msi_list, list) {
- ret = arch_setup_msi_irq(dev, entry);
+ list_for_each_entry(entry, msi-msi_list, list) {
+ ret = arch_setup_msi_irq(msi, entry);
if (ret 0)
return ret;
if (ret 0)
@@ -93,11 +95,11 @@ int __weak arch_setup_msi_irqs(struct pci_dev *dev, int
nvec, int type)
* We have a default implementation available as a separate non-weak
* function, as it is used by the Xen x86 PCI code
*/
-void default_teardown_msi_irqs(struct pci_dev *dev)
+void default_teardown_msi_irqs(struct msi_irqs *msi)
{
struct msi_desc *entry;
- list_for_each_entry(entry, dev-msi_list, list) {
+ list_for_each_entry(entry, msi-msi_list, list) {
int i, nvec;
if (entry-irq == 0)
continue;
@@ -110,22 +112,22 @@ void default_teardown_msi_irqs(struct pci_dev *dev)
}
}
-void __weak arch_teardown_msi_irqs(struct pci_dev *dev)
+void __weak arch_teardown_msi_irqs(struct msi_irqs *msi)
{
- return default_teardown_msi_irqs(dev);
+ return default_teardown_msi_irqs(msi);
}
-static void default_restore_msi_irq(struct pci_dev *dev, int irq)
+static void default_restore_msi_irq(struct msi_irqs *msi, int irq)
{
struct msi_desc *entry;
entry = NULL;
- if (dev-msix_enabled) {
- list_for_each_entry(entry, dev-msi_list, list) {
+ if (msi-msix_enabled) {
+ list_for_each_entry(entry, msi-msi_list, list) {
if (irq == entry-irq)
break;
}
- } else if (pci_dev_msi_enabled(dev, MSI_TYPE)) {
+ } else if (msi-msi_enabled) {
entry = irq_get_msi_desc(irq);
}
@@ -133,20 +135,9 @@ static void default_restore_msi_irq(struct pci_dev *dev,
int irq)
write_msi_msg(irq, entry-msg);
}
-void __weak arch_restore_msi_irqs(struct pci_dev *dev)
+void __weak arch_restore_msi_irqs(struct msi_irqs *msi)
{
- return default_restore_msi_irqs(dev);
-}
-
-static void msi_set_enable(struct pci_dev *dev, int enable)
-{
- u16 control;
-
- pci_read_config_word(dev, dev-msi_cap + PCI_MSI_FLAGS, control);
- control = ~PCI_MSI_FLAGS_ENABLE;
- if (enable)
- control |= PCI_MSI_FLAGS_ENABLE;
- pci_write_config_word(dev, dev-msi_cap + PCI_MSI_FLAGS, control);
+ return default_restore_msi_irqs(msi);
}
static void msix_clear_and_set_ctrl(struct pci_dev *dev, u16 clear, u16 set)
@@ -159,6 +150,25 @@ static void msix_clear_and_set_ctrl(struct pci_dev *dev,
u16 clear, u16 set)
pci_write_config_word(dev, dev-msix_cap + PCI_MSIX_FLAGS, ctrl);
}
+static void msi_set_enable(struct msi_irqs *msi, int enable, int type)
+{
+ u16 control;
+ struct pci_dev *dev = msi-data;
+
+ if (type == MSI_TYPE) {
+ pci_read_config_word(dev, dev-msi_cap + PCI_MSI_FLAGS,
control);
+ control = ~PCI_MSI_FLAGS_ENABLE;
+ if (enable)
+ control
