Re: [PATCH v11 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
Thank you very much for this.
zhenhua
From My iPhone
> 在 2015年5月30日，00:22，Joerg Roedel 写道：
>
>> On Mon, May 11, 2015 at 05:52:44PM +0800, Li, Zhen-Hua wrote:
>> Li, Zhen-Hua (10):
>> iommu/vt-d: New function to attach domain with id
>> iommu/vt-d: Items required for kdump
>> iommu/vt-d: Function to get existing context entry
>> iommu/vt-d: functions to copy data from old mem
>> iommu/vt-d: Add functions to load and save old re
>> iommu/vt-d: datatypes and functions used for kdump
>> iommu/vt-d: enable kdump support in iommu module
>> iommu/vt-d: assign new page table for dma_map
>> iommu/vt-d: Copy functions for irte
>> iommu/vt-d: Use old irte in kdump kernel
>>
>> drivers/iommu/intel-iommu.c | 536
>> ++--
>> drivers/iommu/intel_irq_remapping.c | 96 ++-
>> include/linux/intel-iommu.h | 16 ++
>> 3 files changed, 623 insertions(+), 25 deletions(-)
>
> Applied with a few minor changes, thanks.
>
N�Р骒r��yb�X�肚�v�^�)藓{.n�+�伐�{��赙zXФ�≤�}��财�z�:+v�����赙zZ+��+zf＂�h���~i���z� �wア�?�ㄨ��&�)撷f��^j谦y�m��@A�a囤�
0鹅h���i
Re: [PATCH v11 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
Thank you very much for this.
zhenhua
From My iPhone
在 2015年5月30日，00:22，Joerg Roedel j...@8bytes.org 写道：
On Mon, May 11, 2015 at 05:52:44PM +0800, Li, Zhen-Hua wrote:
Li, Zhen-Hua (10):
iommu/vt-d: New function to attach domain with id
iommu/vt-d: Items required for kdump
iommu/vt-d: Function to get existing context entry
iommu/vt-d: functions to copy data from old mem
iommu/vt-d: Add functions to load and save old re
iommu/vt-d: datatypes and functions used for kdump
iommu/vt-d: enable kdump support in iommu module
iommu/vt-d: assign new page table for dma_map
iommu/vt-d: Copy functions for irte
iommu/vt-d: Use old irte in kdump kernel
drivers/iommu/intel-iommu.c | 536
++--
drivers/iommu/intel_irq_remapping.c | 96 ++-
include/linux/intel-iommu.h | 16 ++
3 files changed, 623 insertions(+), 25 deletions(-)
Applied with a few minor changes, thanks.
N�Р骒r��yb�X�肚�v�^�)藓{.n�+�伐�{��赙zXФ�≤�}��财�z�j:+v�����赙zZ+��+zf＂�h���~i���z� �wア�?�ㄨ���)撷f��^j谦y�m��@A�a囤�
0鹅h���i
Re: [PATCH v11 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
ng the old ones. > 3. Remove functions: > intel_iommu_did_to_domain_values_entry > intel_iommu_get_dids_from_old_kernel > device_to_domain_id > copy_page_addr > copy_page_table > copy_context_entry > copy_context_entry_table > 4. Add new function device_to_existing_context_entry. > > Changelog[v8]: > 1. Add a missing __iommu_flush_cache in function copy_page_table. > > Changelog[v7]: > 1. Use __iommu_flush_cache to flush the data to hardware. > > Changelog[v6]: > 1. Use "unsigned long" as type of physical address. > 2. Use new function unmap_device_dma to unmap the old dma. > 3. Some small incorrect bits order for aw shift. > > Changelog[v5]: > 1. Do not disable and re-enable traslation and interrupt remapping. > 2. Use old root entry table. > 3. Use old interrupt remapping table. > 4. New functions to copy data from old kernel, and save to old kernel mem. > 5. New functions to save updated root entry table and irte table. > 6. Use intel_unmap to unmap the old dma; > 7. Allocate new pages while driver is being loaded. > > Changelog[v4]: > 1. Cut off the patches that move some defines and functions to new files. > 2. Reduce the numbers of patches to five, make it more easier to read. > 3. Changed the name of functions, make them consistent with current context > get/set functions. > 4. Add change to function __iommu_attach_domain. > > Changelog[v3]: > 1. Commented-out "#define DEBUG 1" to eliminate debug messages. > 2. Updated the comments about changes in each version. > 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad > struct as recommended by Baoquan He [b...@redhat.com] > init_iova_domain(>iovad, DMA_32BIT_PFN); > > Changelog[v2]: > The following series implements a fix for: > A kdump problem about DMA that has been discussed for a long time. That is, > when a kernel panics and boots into the kdump kernel, DMA started by the > panicked kernel is not stopped before the kdump kernel is booted and the > kdump kernel disables the IOMMU while this DMA continues. This causes the > IOMMU to stop translating the DMA addresses as IOVAs and begin to treat > them as physical memory addresses -- which causes the DMA to either: > (1) generate DMAR errors or > (2) generate PCI SERR errors or > (3) transfer data to or from incorrect areas of memory. Often this > causes the dump to fail. > > Changelog[v1]: > The original version. > > Changed in this version: > 1. Do not disable and re-enable traslation and interrupt remapping. > 2. Use old root entry table. > 3. Use old interrupt remapping table. > 4. Use "unsigned long" as physical address. > 5. Use intel_unmap to unmap the old dma; > > Baoquan He helps testing this patchset. > Takao Indoh gives valuable suggestions. > > Li, Zhen-Hua (10): > iommu/vt-d: New function to attach domain with id > iommu/vt-d: Items required for kdump > iommu/vt-d: Function to get existing context entry > iommu/vt-d: functions to copy data from old mem > iommu/vt-d: Add functions to load and save old re > iommu/vt-d: datatypes and functions used for kdump > iommu/vt-d: enable kdump support in iommu module > iommu/vt-d: assign new page table for dma_map > iommu/vt-d: Copy functions for irte > iommu/vt-d: Use old irte in kdump kernel > > drivers/iommu/intel-iommu.c | 536 ++-- > drivers/iommu/intel_irq_remapping.c | 96 ++- > include/linux/intel-iommu.h | 16 ++ > 3 files changed, 623 insertions(+), 25 deletions(-) > > -- > 2.0.0-rc0 > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v11 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
. Changelog[v7]: 1. Use __iommu_flush_cache to flush the data to hardware. Changelog[v6]: 1. Use unsigned long as type of physical address. 2. Use new function unmap_device_dma to unmap the old dma. 3. Some small incorrect bits order for aw shift. Changelog[v5]: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. New functions to copy data from old kernel, and save to old kernel mem. 5. New functions to save updated root entry table and irte table. 6. Use intel_unmap to unmap the old dma; 7. Allocate new pages while driver is being loaded. Changelog[v4]: 1. Cut off the patches that move some defines and functions to new files. 2. Reduce the numbers of patches to five, make it more easier to read. 3. Changed the name of functions, make them consistent with current context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out #define DEBUG 1 to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(domain-iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use unsigned long as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He b...@redhat.com helps testing this patchset. Takao Indoh indou.ta...@jp.fujitsu.com gives valuable suggestions. Li, Zhen-Hua (10): iommu/vt-d: New function to attach domain with id iommu/vt-d: Items required for kdump iommu/vt-d: Function to get existing context entry iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel drivers/iommu/intel-iommu.c | 536 ++-- drivers/iommu/intel_irq_remapping.c | 96 ++- include/linux/intel-iommu.h | 16 ++ 3 files changed, 623 insertions(+), 25 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 01/10] iommu/vt-d: New function to attach domain with id
Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter "did".
Bill Sumner:
(In older versions) Add new 'did' parameter to iommu_attach_domain();
The caller of this function.
Li, Zhenhua:
New function iommu_attach_domain_with_id(), instead of updating function
iommu_attach_domain();
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 68d43be..cb9d6cc 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1552,6 +1552,16 @@ static int iommu_attach_domain(struct dmar_domain
*domain,
return num;
}
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+ struct intel_iommu *iommu,
+ int domain_number)
+{
+ if (domain_number >= 0)
+ return domain_number;
+
+ return iommu_attach_domain(domain, iommu);
+}
+
static int iommu_attach_vm_domain(struct dmar_domain *domain,
struct intel_iommu *iommu)
{
@@ -2220,6 +2230,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to "no domain_id supplied" */
domain = find_domain(dev);
if (domain)
@@ -2253,7 +2264,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain->id = iommu_attach_domain(domain, iommu);
+ domain->id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
return NULL;
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 54 -
include/linux/intel-iommu.h | 3 +++
2 files changed, 56 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 0b97c15..3a5d446 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -371,6 +371,10 @@ static struct context_entry
*device_to_existing_context_entry(
struct intel_iommu *iommu,
u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
+
/*
* A structure used to store the address allocated by ioremap();
* The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
@@ -382,7 +386,6 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
-
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
@@ -4935,3 +4938,52 @@ int __iommu_free_mapped_mem(void)
return 0;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+ memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+
+ if (index < -1 || index >= ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu->root_entry_old_virt;
+ from = iommu->root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index ced1fac..e7cac12 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -340,6 +340,9 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+ void __iomem*root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 58 ++---
1 file changed, 50 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 91545bf..3cc1027 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -396,6 +396,9 @@ static int copy_root_entry_table(struct intel_iommu *iommu);
static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
+static void unmap_device_dma(struct dmar_domain *domain,
+ struct device *dev,
+ struct intel_iommu *iommu);
static void iommu_check_pre_te_status(struct intel_iommu *iommu);
static u8 g_translation_pre_enabled;
@@ -3115,6 +3118,7 @@ static struct iova *intel_alloc_iova(struct device *dev,
static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
{
struct dmar_domain *domain;
+ struct intel_iommu *iommu;
int ret;
domain = get_domain_for_dev(dev, DEFAULT_DOMAIN_ADDRESS_WIDTH);
@@ -3124,14 +3128,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR "Domain context map for %s failed",
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+ iommu = domain_get_iommu(domain);
+ if (iommu->pre_enabled_trans) {
+ unmap_device_dma(domain, dev, iommu);
+
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err("Allocating domain for %s failed",
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err("Domain context map for %s failed",
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5168,6 +5188,28 @@ static int intel_iommu_load_translation_tables(struct
intel_iommu *iommu)
return ret;
}
+static void unmap_device_dma(struct dmar_domain *domain,
+ struct device *dev,
+ struct intel_iommu *iommu)
+{
+ struct context_entry *ce;
+ struct iova *iova;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+ struct pci_dev *pdev;
+
+ pdev = to_pci_dev(dev);
+ ce = iommu_context_addr(iommu, pdev->bus->number, pdev->devfn, 1);
+ phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(>iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
static void iommu_check_pre_te_status(struct intel_iommu *iommu)
{
u32 sts;
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 102
include/linux/intel-iommu.h | 6 +++
2 files changed, 108 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 07e6118..0b97c15 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -371,6 +371,17 @@ static struct context_entry
*device_to_existing_context_entry(
struct intel_iommu *iommu,
u8 bus, u8 devfn);
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
/*
* This domain is a statically identity mapping domain.
@@ -4833,3 +4844,94 @@ static struct context_entry
*device_to_existing_context_entry(
return ret;
}
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from >> VTD_PAGE_SHIFT;
+ offset = from & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to >> VTD_PAGE_SHIFT;
+ offset = to & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(&__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+ iounmap(mem_entry->mem);
+ list_del(_entry->list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(&__iommu_mem_list_lock);
+ return 0;
+}
+
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 796ef96..ced1fac 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -380,4 +380,10 @@ extern int dmar_ir_support(void);
extern const struct attribute_group *intel_iommu_groups[];
+extern int __iommu_load_from_oldmem(void *to, unsi
[PATCH v11 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
iommu_context_addr
free_context_table
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Use the did and gaw from old context entry;
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 95 +++--
1 file changed, 83 insertions(+), 12 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 28c3c64..91545bf 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -397,6 +397,7 @@ static int copy_root_entry_table(struct intel_iommu *iommu);
static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
static void iommu_check_pre_te_status(struct intel_iommu *iommu);
+static u8 g_translation_pre_enabled;
/*
* This domain is a statically identity mapping domain.
@@ -794,6 +795,9 @@ static inline struct context_entry
*iommu_context_addr(struct intel_iommu *iommu
phy_addr = virt_to_phys((void *)context);
*entry = phy_addr | 1;
__iommu_flush_cache(iommu, entry, sizeof(*entry));
+
+ if (iommu->pre_enabled_trans)
+ __iommu_update_old_root_entry(iommu, bus);
}
return [devfn];
}
@@ -887,13 +891,15 @@ static void clear_context_table(struct intel_iommu
*iommu, u8 bus, u8 devfn)
static void free_context_table(struct intel_iommu *iommu)
{
+ struct root_entry *root = NULL;
int i;
unsigned long flags;
struct context_entry *context;
spin_lock_irqsave(>lock, flags);
if (!iommu->root_entry) {
- goto out;
+ spin_unlock_irqrestore(>lock, flags);
+ return;
}
for (i = 0; i < ROOT_ENTRY_NR; i++) {
context = iommu_context_addr(iommu, i, 0, 0);
@@ -908,10 +914,23 @@ static void free_context_table(struct intel_iommu *iommu)
free_pgtable_page(context);
}
+
+ if (iommu->pre_enabled_trans) {
+ iommu->root_entry_old_phys = 0;
+ root = iommu->root_entry_old_virt;
+ iommu->root_entry_old_virt = NULL;
+ }
+
free_pgtable_page(iommu->root_entry);
iommu->root_entry = NULL;
-out:
+
spin_unlock_irqrestore(>lock, flags);
+
+ /* We put this out of spin_unlock is because iounmap() may
+* cause error if surrounded by spin_lock and unlock;
+*/
+ if (iommu->pre_enabled_trans)
+ iounmap(root);
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2333,6 +2352,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to "no domain_id supplied" */
+ struct context_entry *ce = NULL;
domain = find_domain(dev);
if (domain)
@@ -2366,6 +2386,20 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+ if (iommu->pre_enabled_trans) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+ if (ce) {
+ did = context_domain_id(ce);
+ gaw = agaw_to_width(context_address_width(ce));
+ }
+ }
+
domain->id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
@@ -2897,6 +2931,7 @@ static int __init init_dmars(void)
goto free_g_iommus;
}
+ g_translation_pre_enabled = 0; /* To know whether to skip RMRR */
for_each_active_iommu(iommu, drhd) {
g_iommus[iommu->seq_id] = iommu;
@@ -2904,14 +2939,30 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root & context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+ iommu_check_pre_te_status(iommu);
+ if (iommu->pre_enabled_trans) {
+ pr_info("IOMMU Copying translate tables from panicked
kernel\n");
+ ret = intel_iommu_load_translation_tables(iommu);
+
[PATCH v11 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
Use old root entry table, and load the old data to root_entry as cache.
Malloc new context table and copy old context table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Create new function iommu_check_pre_te_status() to check status.
Update the caller of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that loads root entry table.
Use new function to copy old context entry tables and page tables.
Use "unsigned long" for physical address.
Remove the functions to copy page table in Bill's version.
Remove usage of dve and ppap in Bill's version.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 121
include/linux/intel-iommu.h | 3 ++
2 files changed, 124 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 3a5d446..28c3c64 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -386,6 +386,18 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
+
+static void iommu_check_pre_te_status(struct intel_iommu *iommu);
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
@@ -4987,3 +4999,112 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+ u32 bus;/* Index: root-entry-table */
+ struct root_entry *re; /* Virt(iterator: new table) */
+ unsigned long context_old_phys; /* Phys(context table entry) */
+ struct context_entry *context_new_virt; /* Virt(new context_entry) */
+
+ /*
+* A new root entry table has been allocated ,
+* we need copy re from old kernel to the new allocated one.
+*/
+
+ if (!iommu->root_entry_old_phys)
+ return -ENOMEM;
+
+ for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+ if (!root_present(re))
+ continue;
+
+ context_old_phys = get_context_phys_from_root(re);
+
+ if (!context_old_phys)
+ continue;
+
+ context_new_virt =
+ (struct context_entry *)alloc_pgtable_page(iommu->node);
+
+ if (!context_new_virt)
+ return -ENOMEM;
+
+ __iommu_load_from_oldmem(context_new_virt,
+ context_old_phys,
+ VTD_PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+ set_root_value(re, virt_to_phys(context_new_virt));
+ }
+
+ return 0;
+}
+
+/*
+ * Interface to the "load translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct intel_iommu *iommu)
+{
+ unsigned long long q; /* quadword scratch */
+ int ret = 0;/* Integer return code */
+ unsigned long flags;
+
+ q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+ if (!q)
+ return -1;
+
+ spin_lock_irqsave(>lock, flags);
+
+ /* Load the root-entry table from the old kernel
+* foreach context_entry_table in root_entry
+* Copy each entry table from old kernel
+*/
+ if (!iommu->root_entry) {
+ iommu->root_entry =
+ (struct root_entry *)alloc_pgtable_page(iommu->node);
+ if (!iommu->root_entry) {
+ spin_unlock_irqrestore(>lock, flags);
+ return -ENOMEM;
+ }
+ }
+
+ iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+ if (!iommu->root_entry_old_phys) {
+ pr_err("Could not read old root entry address.");
+ return -1;
+ }
+
+
[PATCH v11 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 68 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 72 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index 5709ae9..c2a4406 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -17,6 +17,10 @@
#include "irq_remapping.h"
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+static void iommu_check_pre_ir_status(struct intel_iommu *iommu);
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1299,3 +1303,67 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ memcpy(iommu->ir_table->base,
+ iommu->ir_table->base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu->ir_table->base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu->ir_table->base_old_virt;
+ from = iommu->ir_table->base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+
+static void iommu_check_pre_ir_status(struct intel_iommu *iommu)
+{
+ u32 sts;
+
+ sts = readl(iommu->reg + DMAR_GSTS_REG);
+ if (sts & DMA_GSTS_IRES) {
+ pr_info("IR is enabled prior to OS.\n");
+ iommu->pre_enabled_ir = 1;
+ }
+}
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 90e122e..d7a0b5d 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -301,6 +301,8 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
};
#endif
@@ -342,6 +344,8 @@ struct intel_iommu {
/* whether translation is enabled prior to OS*/
u8 pre_enabled_trans;
+ /* whether interrupt remapping is enabled prior to OS*/
+ u8 pre_enabled_ir;
void __iomem*root_entry_old_virt; /* mapped from old root entry */
unsigned long root_entry_old_phys; /* root entry in old kernel */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in second kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 28
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index c2a4406..46d80ad 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -197,6 +197,10 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(>low, irte_modified->low);
set_64bit(>high, irte_modified->high);
+
+ if (iommu->pre_enabled_ir)
+ __iommu_update_old_irte(iommu, index);
+
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -258,6 +262,9 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu->ir_table->bitmap, index,
irq_iommu->irte_mask);
+ if (iommu->pre_enabled_ir)
+ __iommu_update_old_irte(iommu, -1);
+
return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
}
@@ -657,11 +664,13 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
+ iommu_check_pre_ir_status(iommu);
+
/*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
+* Here we do not disable intr remapping,
+* if already enabled prior to OS handover.
*/
- iommu_disable_irq_remapping(iommu);
+ /* iommu_disable_irq_remapping(iommu); */
dmar_disable_qi(iommu);
}
@@ -697,7 +706,18 @@ static int __init intel_enable_irq_remapping(void)
* Setup Interrupt-remapping for all the DRHD's now.
*/
for_each_iommu(iommu, drhd) {
- iommu_set_irq_remapping(iommu, eim);
+ if (iommu->pre_enabled_ir) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+ iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+ iommu->ir_table->base_old_virt = ioremap_cache(
+ iommu->ir_table->base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+ iommu_set_irq_remapping(iommu, eim);
+
setup = true;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 02/10] iommu/vt-d: Items required for kdump
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Remove the structure dve which is not used in new version.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 72 +
1 file changed, 72 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index cb9d6cc..1e7ceb5 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -190,6 +190,31 @@ struct root_entry {
};
#define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
+static inline bool root_present(struct root_entry *root)
+{
+ return (root->lo & 1);
+}
+
+static inline void set_root_value(struct root_entry *root, unsigned long value)
+{
+ root->lo &= ~VTD_PAGE_MASK;
+ root->lo |= value & VTD_PAGE_MASK;
+}
+
+static inline struct context_entry *
+get_context_addr_from_root(struct root_entry *root)
+{
+ return (struct context_entry *)
+ (root_present(root)?phys_to_virt(
+ root->lo & VTD_PAGE_MASK) :
+ NULL);
+}
+
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root->lo & VTD_PAGE_MASK) : 0;
+}
/*
* low 64 bits:
@@ -211,6 +236,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context->lo & 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c->hi >> 8) & 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context->lo |= 1;
@@ -296,6 +347,27 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte & ~VTD_PAGE_MASK);
}
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *in the old kernel to the new ones.
+ *
+ * In other kinds of kernel, for example, a kernel started by kexec,
+ * do the same thing as crashdump kernel.
+ */
+
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 03/10] iommu/vt-d: Function to get existing context entry
Interface for when a new domain in the old kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 23 +++
1 file changed, 23 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1e7ceb5..07e6118 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -367,6 +367,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
* do the same thing as crashdump kernel.
*/
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn);
+
/*
* This domain is a statically identity mapping domain.
@@ -4810,3 +4814,22 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n",
vtisochctrl);
}
+
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn)
+{
+ struct root_entry *root;
+ struct context_entry *context;
+ struct context_entry *ret = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(>lock, flags);
+ root = >root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context && context_present(context+devfn))
+ ret = [devfn];
+ spin_unlock_irqrestore(>lock, flags);
+ return ret;
+}
+
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
pe of physical address. 2. Use new function unmap_device_dma to unmap the old dma. 3. Some small incorrect bits order for aw shift. Changelog[v5]: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. New functions to copy data from old kernel, and save to old kernel mem. 5. New functions to save updated root entry table and irte table. 6. Use intel_unmap to unmap the old dma; 7. Allocate new pages while driver is being loaded. Changelog[v4]: 1. Cut off the patches that move some defines and functions to new files. 2. Reduce the numbers of patches to five, make it more easier to read. 3. Changed the name of functions, make them consistent with current context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out "#define DEBUG 1" to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(>iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use "unsigned long" as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He helps testing this patchset. Takao Indoh gives valuable suggestions. Li, Zhen-Hua (10): iommu/vt-d: New function to attach domain with id iommu/vt-d: Items required for kdump iommu/vt-d: Function to get existing context entry iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel drivers/iommu/intel-iommu.c | 536 ++-- drivers/iommu/intel_irq_remapping.c | 96 ++- include/linux/intel-iommu.h | 16 ++ 3 files changed, 623 insertions(+), 25 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 01/10] iommu/vt-d: New function to attach domain with id
Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter did.
Bill Sumner:
(In older versions) Add new 'did' parameter to iommu_attach_domain();
The caller of this function.
Li, Zhenhua:
New function iommu_attach_domain_with_id(), instead of updating function
iommu_attach_domain();
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 68d43be..cb9d6cc 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1552,6 +1552,16 @@ static int iommu_attach_domain(struct dmar_domain
*domain,
return num;
}
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+ struct intel_iommu *iommu,
+ int domain_number)
+{
+ if (domain_number = 0)
+ return domain_number;
+
+ return iommu_attach_domain(domain, iommu);
+}
+
static int iommu_attach_vm_domain(struct dmar_domain *domain,
struct intel_iommu *iommu)
{
@@ -2220,6 +2230,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to no domain_id supplied */
domain = find_domain(dev);
if (domain)
@@ -2253,7 +2264,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain-id = iommu_attach_domain(domain, iommu);
+ domain-id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
return NULL;
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu-root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 54 -
include/linux/intel-iommu.h | 3 +++
2 files changed, 56 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 0b97c15..3a5d446 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -371,6 +371,10 @@ static struct context_entry
*device_to_existing_context_entry(
struct intel_iommu *iommu,
u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
+
/*
* A structure used to store the address allocated by ioremap();
* The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
@@ -382,7 +386,6 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
-
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
@@ -4935,3 +4938,52 @@ int __iommu_free_mapped_mem(void)
return 0;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+ memcpy(iommu-root_entry, iommu-root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu-root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+
+ if (index -1 || index = ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu-root_entry_old_virt;
+ from = iommu-root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index ced1fac..e7cac12 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -340,6 +340,9 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+ void __iomem*root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 102
include/linux/intel-iommu.h | 6 +++
2 files changed, 108 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 07e6118..0b97c15 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -371,6 +371,17 @@ static struct context_entry
*device_to_existing_context_entry(
struct intel_iommu *iommu,
u8 bus, u8 devfn);
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
/*
* This domain is a statically identity mapping domain.
@@ -4833,3 +4844,94 @@ static struct context_entry
*device_to_existing_context_entry(
return ret;
}
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from VTD_PAGE_SHIFT;
+ offset = from (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to VTD_PAGE_SHIFT;
+ offset = to (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, __iommu_remapped_mem, list) {
+ iounmap(mem_entry-mem);
+ list_del(mem_entry-list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(__iommu_mem_list_lock);
+ return 0;
+}
+
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 796ef96..ced1fac 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -380,4 +380,10 @@ extern int dmar_ir_support(void);
extern const struct attribute_group *intel_iommu_groups[];
+extern int __iommu_load_from_oldmem(void *to, unsigned long from
[PATCH v11 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
iommu_context_addr
free_context_table
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Use the did and gaw from old context entry;
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 95 +++--
1 file changed, 83 insertions(+), 12 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 28c3c64..91545bf 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -397,6 +397,7 @@ static int copy_root_entry_table(struct intel_iommu *iommu);
static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
static void iommu_check_pre_te_status(struct intel_iommu *iommu);
+static u8 g_translation_pre_enabled;
/*
* This domain is a statically identity mapping domain.
@@ -794,6 +795,9 @@ static inline struct context_entry
*iommu_context_addr(struct intel_iommu *iommu
phy_addr = virt_to_phys((void *)context);
*entry = phy_addr | 1;
__iommu_flush_cache(iommu, entry, sizeof(*entry));
+
+ if (iommu-pre_enabled_trans)
+ __iommu_update_old_root_entry(iommu, bus);
}
return context[devfn];
}
@@ -887,13 +891,15 @@ static void clear_context_table(struct intel_iommu
*iommu, u8 bus, u8 devfn)
static void free_context_table(struct intel_iommu *iommu)
{
+ struct root_entry *root = NULL;
int i;
unsigned long flags;
struct context_entry *context;
spin_lock_irqsave(iommu-lock, flags);
if (!iommu-root_entry) {
- goto out;
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return;
}
for (i = 0; i ROOT_ENTRY_NR; i++) {
context = iommu_context_addr(iommu, i, 0, 0);
@@ -908,10 +914,23 @@ static void free_context_table(struct intel_iommu *iommu)
free_pgtable_page(context);
}
+
+ if (iommu-pre_enabled_trans) {
+ iommu-root_entry_old_phys = 0;
+ root = iommu-root_entry_old_virt;
+ iommu-root_entry_old_virt = NULL;
+ }
+
free_pgtable_page(iommu-root_entry);
iommu-root_entry = NULL;
-out:
+
spin_unlock_irqrestore(iommu-lock, flags);
+
+ /* We put this out of spin_unlock is because iounmap() may
+* cause error if surrounded by spin_lock and unlock;
+*/
+ if (iommu-pre_enabled_trans)
+ iounmap(root);
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2333,6 +2352,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to no domain_id supplied */
+ struct context_entry *ce = NULL;
domain = find_domain(dev);
if (domain)
@@ -2366,6 +2386,20 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+ if (iommu-pre_enabled_trans) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+ if (ce) {
+ did = context_domain_id(ce);
+ gaw = agaw_to_width(context_address_width(ce));
+ }
+ }
+
domain-id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
@@ -2897,6 +2931,7 @@ static int __init init_dmars(void)
goto free_g_iommus;
}
+ g_translation_pre_enabled = 0; /* To know whether to skip RMRR */
for_each_active_iommu(iommu, drhd) {
g_iommus[iommu-seq_id] = iommu;
@@ -2904,14 +2939,30 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+ iommu_check_pre_te_status(iommu);
+ if (iommu-pre_enabled_trans) {
+ pr_info(IOMMU Copying translate tables from panicked
kernel\n);
+ ret = intel_iommu_load_translation_tables(iommu);
+ if (ret
[PATCH v11 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
Use old root entry table, and load the old data to root_entry as cache.
Malloc new context table and copy old context table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Create new function iommu_check_pre_te_status() to check status.
Update the caller of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that loads root entry table.
Use new function to copy old context entry tables and page tables.
Use unsigned long for physical address.
Remove the functions to copy page table in Bill's version.
Remove usage of dve and ppap in Bill's version.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 121
include/linux/intel-iommu.h | 3 ++
2 files changed, 124 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 3a5d446..28c3c64 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -386,6 +386,18 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
+
+static void iommu_check_pre_te_status(struct intel_iommu *iommu);
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
@@ -4987,3 +4999,112 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+ u32 bus;/* Index: root-entry-table */
+ struct root_entry *re; /* Virt(iterator: new table) */
+ unsigned long context_old_phys; /* Phys(context table entry) */
+ struct context_entry *context_new_virt; /* Virt(new context_entry) */
+
+ /*
+* A new root entry table has been allocated ,
+* we need copy re from old kernel to the new allocated one.
+*/
+
+ if (!iommu-root_entry_old_phys)
+ return -ENOMEM;
+
+ for (bus = 0, re = iommu-root_entry; bus 256; bus += 1, re += 1) {
+ if (!root_present(re))
+ continue;
+
+ context_old_phys = get_context_phys_from_root(re);
+
+ if (!context_old_phys)
+ continue;
+
+ context_new_virt =
+ (struct context_entry *)alloc_pgtable_page(iommu-node);
+
+ if (!context_new_virt)
+ return -ENOMEM;
+
+ __iommu_load_from_oldmem(context_new_virt,
+ context_old_phys,
+ VTD_PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+ set_root_value(re, virt_to_phys(context_new_virt));
+ }
+
+ return 0;
+}
+
+/*
+ * Interface to the load translation tables set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct intel_iommu *iommu)
+{
+ unsigned long long q; /* quadword scratch */
+ int ret = 0;/* Integer return code */
+ unsigned long flags;
+
+ q = dmar_readq(iommu-reg + DMAR_RTADDR_REG);
+ if (!q)
+ return -1;
+
+ spin_lock_irqsave(iommu-lock, flags);
+
+ /* Load the root-entry table from the old kernel
+* foreach context_entry_table in root_entry
+* Copy each entry table from old kernel
+*/
+ if (!iommu-root_entry) {
+ iommu-root_entry =
+ (struct root_entry *)alloc_pgtable_page(iommu-node);
+ if (!iommu-root_entry) {
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return -ENOMEM;
+ }
+ }
+
+ iommu-root_entry_old_phys = q VTD_PAGE_MASK;
+ if (!iommu-root_entry_old_phys) {
+ pr_err(Could not read old root entry address.);
+ return -1;
+ }
+
+ iommu-root_entry_old_virt = ioremap_cache(iommu
[PATCH v11 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 68 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 72 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index 5709ae9..c2a4406 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -17,6 +17,10 @@
#include irq_remapping.h
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+static void iommu_check_pre_ir_status(struct intel_iommu *iommu);
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1299,3 +1303,67 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ memcpy(iommu-ir_table-base,
+ iommu-ir_table-base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu-ir_table-base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ if (index -1 || index = INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu-ir_table-base_old_virt;
+ from = iommu-ir_table-base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+
+static void iommu_check_pre_ir_status(struct intel_iommu *iommu)
+{
+ u32 sts;
+
+ sts = readl(iommu-reg + DMAR_GSTS_REG);
+ if (sts DMA_GSTS_IRES) {
+ pr_info(IR is enabled prior to OS.\n);
+ iommu-pre_enabled_ir = 1;
+ }
+}
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 90e122e..d7a0b5d 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -301,6 +301,8 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
};
#endif
@@ -342,6 +344,8 @@ struct intel_iommu {
/* whether translation is enabled prior to OS*/
u8 pre_enabled_trans;
+ /* whether interrupt remapping is enabled prior to OS*/
+ u8 pre_enabled_ir;
void __iomem*root_entry_old_virt; /* mapped from old root entry */
unsigned long root_entry_old_phys; /* root entry in old kernel */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in second kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 28
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index c2a4406..46d80ad 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -197,6 +197,10 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(irte-low, irte_modified-low);
set_64bit(irte-high, irte_modified-high);
+
+ if (iommu-pre_enabled_ir)
+ __iommu_update_old_irte(iommu, index);
+
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -258,6 +262,9 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu-ir_table-bitmap, index,
irq_iommu-irte_mask);
+ if (iommu-pre_enabled_ir)
+ __iommu_update_old_irte(iommu, -1);
+
return qi_flush_iec(iommu, index, irq_iommu-irte_mask);
}
@@ -657,11 +664,13 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
+ iommu_check_pre_ir_status(iommu);
+
/*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
+* Here we do not disable intr remapping,
+* if already enabled prior to OS handover.
*/
- iommu_disable_irq_remapping(iommu);
+ /* iommu_disable_irq_remapping(iommu); */
dmar_disable_qi(iommu);
}
@@ -697,7 +706,18 @@ static int __init intel_enable_irq_remapping(void)
* Setup Interrupt-remapping for all the DRHD's now.
*/
for_each_iommu(iommu, drhd) {
- iommu_set_irq_remapping(iommu, eim);
+ if (iommu-pre_enabled_ir) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu-reg + DMAR_IRTA_REG);
+ iommu-ir_table-base_old_phys = q VTD_PAGE_MASK;
+ iommu-ir_table-base_old_virt = ioremap_cache(
+ iommu-ir_table-base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+ iommu_set_irq_remapping(iommu, eim);
+
setup = true;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 58 ++---
1 file changed, 50 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 91545bf..3cc1027 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -396,6 +396,9 @@ static int copy_root_entry_table(struct intel_iommu *iommu);
static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
+static void unmap_device_dma(struct dmar_domain *domain,
+ struct device *dev,
+ struct intel_iommu *iommu);
static void iommu_check_pre_te_status(struct intel_iommu *iommu);
static u8 g_translation_pre_enabled;
@@ -3115,6 +3118,7 @@ static struct iova *intel_alloc_iova(struct device *dev,
static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
{
struct dmar_domain *domain;
+ struct intel_iommu *iommu;
int ret;
domain = get_domain_for_dev(dev, DEFAULT_DOMAIN_ADDRESS_WIDTH);
@@ -3124,14 +3128,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR Domain context map for %s failed,
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+ iommu = domain_get_iommu(domain);
+ if (iommu-pre_enabled_trans) {
+ unmap_device_dma(domain, dev, iommu);
+
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err(Allocating domain for %s failed,
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err(Domain context map for %s failed,
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5168,6 +5188,28 @@ static int intel_iommu_load_translation_tables(struct
intel_iommu *iommu)
return ret;
}
+static void unmap_device_dma(struct dmar_domain *domain,
+ struct device *dev,
+ struct intel_iommu *iommu)
+{
+ struct context_entry *ce;
+ struct iova *iova;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+ struct pci_dev *pdev;
+
+ pdev = to_pci_dev(dev);
+ ce = iommu_context_addr(iommu, pdev-bus-number, pdev-devfn, 1);
+ phys_addr = context_address_root(ce) VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(domain-iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
static void iommu_check_pre_te_status(struct intel_iommu *iommu)
{
u32 sts;
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
new function unmap_device_dma to unmap the old dma. 3. Some small incorrect bits order for aw shift. Changelog[v5]: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. New functions to copy data from old kernel, and save to old kernel mem. 5. New functions to save updated root entry table and irte table. 6. Use intel_unmap to unmap the old dma; 7. Allocate new pages while driver is being loaded. Changelog[v4]: 1. Cut off the patches that move some defines and functions to new files. 2. Reduce the numbers of patches to five, make it more easier to read. 3. Changed the name of functions, make them consistent with current context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out #define DEBUG 1 to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(domain-iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use unsigned long as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He b...@redhat.com helps testing this patchset. Takao Indoh indou.ta...@jp.fujitsu.com gives valuable suggestions. Li, Zhen-Hua (10): iommu/vt-d: New function to attach domain with id iommu/vt-d: Items required for kdump iommu/vt-d: Function to get existing context entry iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel drivers/iommu/intel-iommu.c | 536 ++-- drivers/iommu/intel_irq_remapping.c | 96 ++- include/linux/intel-iommu.h | 16 ++ 3 files changed, 623 insertions(+), 25 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 02/10] iommu/vt-d: Items required for kdump
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Remove the structure dve which is not used in new version.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 72 +
1 file changed, 72 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index cb9d6cc..1e7ceb5 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -190,6 +190,31 @@ struct root_entry {
};
#define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
+static inline bool root_present(struct root_entry *root)
+{
+ return (root-lo 1);
+}
+
+static inline void set_root_value(struct root_entry *root, unsigned long value)
+{
+ root-lo = ~VTD_PAGE_MASK;
+ root-lo |= value VTD_PAGE_MASK;
+}
+
+static inline struct context_entry *
+get_context_addr_from_root(struct root_entry *root)
+{
+ return (struct context_entry *)
+ (root_present(root)?phys_to_virt(
+ root-lo VTD_PAGE_MASK) :
+ NULL);
+}
+
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root-lo VTD_PAGE_MASK) : 0;
+}
/*
* low 64 bits:
@@ -211,6 +236,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context-lo 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c-lo 1) 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c-lo 2) 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c-lo VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c-hi 0) 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c-hi 8) 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context-lo |= 1;
@@ -296,6 +347,27 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte ~VTD_PAGE_MASK);
}
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *in the old kernel to the new ones.
+ *
+ * In other kinds of kernel, for example, a kernel started by kexec,
+ * do the same thing as crashdump kernel.
+ */
+
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v11 03/10] iommu/vt-d: Function to get existing context entry
Interface for when a new domain in the old kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 23 +++
1 file changed, 23 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1e7ceb5..07e6118 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -367,6 +367,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
* do the same thing as crashdump kernel.
*/
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn);
+
/*
* This domain is a statically identity mapping domain.
@@ -4810,3 +4814,22 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n,
vtisochctrl);
}
+
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn)
+{
+ struct root_entry *root;
+ struct context_entry *context;
+ struct context_entry *ret = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(iommu-lock, flags);
+ root = iommu-root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context context_present(context+devfn))
+ ret = context[devfn];
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return ret;
+}
+
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
The hardware will do some verification, but not completely. If people think
the OS should also do this, then it should be another patchset, I think.
Thanks
Zhenhua
> 在 2015年4月3日，17:21，Dave Young 写道：
>
>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>> Hi Dave,
>>
>> There may be some possibilities that the old iommu data is corrupted by
>> some other modules. Currently we do not have a better solution for the
>> dmar faults.
>>
>> But I think when this happens, we need to fix the module that corrupted
>> the old iommu data. I once met a similar problem in normal kernel, the
>> queue used by the qi_* functions was written again by another module.
>> The fix was in that module, not in iommu module.
>
> It is too late, there will be no chance to save vmcore then.
>
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
>
> So I think the tables at least need some verifycation before being used.
>
>>
>>
>> Thanks
>> Zhenhua
>>
>> On 04/03/2015 04:40 PM, Dave Young wrote:
To fix this problem, we modifies the behaviors of the intel vt-d in the
crashdump kernel:
For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table, copy data from the old ones that
used by the old kernel.
>>>
>>> Have not read all the patches, but I have a question, not sure this has been
>>> answered before. Old memory is not reliable, what if the old memory get
>>> corrupted
>>> before panic? Is it safe to continue using it in 2nd kernel, I worry that
>>> it will
>>> cause problems.
>>>
>>> Hope I'm wrong though.
>>>
>>> Thanks
>>> Dave
>>
N�Р骒r��yb�X�肚�v�^�)藓{.n�+�伐�{��赙zXФ�≤�}��财�z�:+v�����赙zZ+��+zf＂�h���~i���z� �wア�?�ㄨ��&�)撷f��^j谦y�m��@A�a囤�
0鹅h���i
Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
The hardware will do some verification, but not completely. If people think
the OS should also do this, then it should be another patchset, I think.
Thanks
Zhenhua
在 2015年4月3日，17:21，Dave Young dyo...@redhat.com 写道：
On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
Hi Dave,
There may be some possibilities that the old iommu data is corrupted by
some other modules. Currently we do not have a better solution for the
dmar faults.
But I think when this happens, we need to fix the module that corrupted
the old iommu data. I once met a similar problem in normal kernel, the
queue used by the qi_* functions was written again by another module.
The fix was in that module, not in iommu module.
It is too late, there will be no chance to save vmcore then.
Also if it is possible to continue corrupt other area of oldmem because
of using old iommu tables then it will cause more problems.
So I think the tables at least need some verifycation before being used.
Thanks
Zhenhua
On 04/03/2015 04:40 PM, Dave Young wrote:
To fix this problem, we modifies the behaviors of the intel vt-d in the
crashdump kernel:
For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table, copy data from the old ones that
used by the old kernel.
Have not read all the patches, but I have a question, not sure this has been
answered before. Old memory is not reliable, what if the old memory get
corrupted
before panic? Is it safe to continue using it in 2nd kernel, I worry that
it will
cause problems.
Hope I'm wrong though.
Thanks
Dave
N�Р骒r��yb�X�肚�v�^�)藓{.n�+�伐�{��赙zXФ�≤�}��财�z�j:+v�����赙zZ+��+zf＂�h���~i���z� �wア�?�ㄨ���)撷f��^j谦y�m��@A�a囤�
0鹅h���i
[PATCH v9 01/10] iommu/vt-d: New function to attach domain with id
Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter "did".
Bill Sumner:
(In older versions) Add new 'did' parameter to iommu_attach_domain();
The caller of this function.
Li, Zhenhua:
New function iommu_attach_domain_with_id(), instead of updating funtion
iommu_attach_domain();
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ae4c1a8..76674a1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1558,6 +1558,16 @@ static int iommu_attach_domain(struct dmar_domain
*domain,
return num;
}
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+ struct intel_iommu *iommu,
+ int domain_number)
+{
+ if (domain_number >= 0)
+ return domain_number;
+
+ return iommu_attach_domain(domain, iommu);
+}
+
static int iommu_attach_vm_domain(struct dmar_domain *domain,
struct intel_iommu *iommu)
{
@@ -2225,6 +2235,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to "no domain_id supplied" */
domain = find_domain(dev);
if (domain)
@@ -2258,7 +2269,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain->id = iommu_attach_domain(domain, iommu);
+ domain->id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
return NULL;
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 02/10] iommu/vt-d: Items required for kdump
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Remove the structure dve which is not used in new version.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 56 +
1 file changed, 56 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 76674a1..577d5de 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -211,6 +212,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -231,6 +238,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context->lo & 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c->hi >> 8) & 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context->lo |= 1;
@@ -316,6 +349,29 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte & ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *in the old kernel to the new ones.
+ *
+ */
+
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 102
include/linux/intel-iommu.h | 9
2 files changed, 111 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f7dbe70..7f3484a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -374,6 +374,17 @@ static struct context_entry
*device_to_existing_context_entry(
u8 bus, u8 devfn);
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4822,4 +4833,95 @@ static struct context_entry
*device_to_existing_context_entry(
return ret;
}
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from >> VTD_PAGE_SHIFT;
+ offset = from & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to >> VTD_PAGE_SHIFT;
+ offset = to & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(&__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+ iounmap(mem_entry->mem);
+ list_del(_entry->list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(&__iommu_mem_list_lock);
+ return 0;
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
#include
#include
#include
+#include
#include
#include
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
extern const struct attribute_group *intel_iommu_groups[];
+#ifdef CONFIG_CR
[PATCH v9 03/10] iommu/vt-d: Function to get old context entry
Interface for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 27 +++
1 file changed, 27 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 577d5de..f7dbe70 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -369,6 +369,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
*
*/
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn);
+
#endif /* CONFIG_CRASH_DUMP */
@@ -4796,3 +4800,26 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n",
vtisochctrl);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn)
+{
+ struct root_entry *root;
+ struct context_entry *context;
+ struct context_entry *ret;
+ unsigned long flags;
+
+ ret = NULL;
+ spin_lock_irqsave(>lock, flags);
+ root = >root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context && context_present(context+devfn))
+ ret = [devfn];
+ spin_unlock_irqrestore(>lock, flags);
+ return ret;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 52 +
include/linux/intel-iommu.h | 5 +
2 files changed, 57 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 7f3484a..1cb9780 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -373,6 +373,9 @@ static struct context_entry
*device_to_existing_context_entry(
struct intel_iommu *iommu,
u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
/*
* A structure used to store the address allocated by ioremap();
@@ -4924,4 +4927,53 @@ int __iommu_free_mapped_mem(void)
return 0;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+ memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+
+ if (index < -1 || index >= ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu->root_entry_old_virt;
+ from = iommu->root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 43 +++--
1 file changed, 37 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index 20c060b..4e5a02d 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -199,6 +199,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(>low, irte_modified->low);
set_64bit(>high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, index);
+#endif
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -260,6 +265,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu->ir_table->bitmap, index,
irq_iommu->irte_mask);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, -1);
+#endif
+
return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
}
@@ -662,11 +672,20 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
- /*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
-*/
- iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /* Do notdisable irq and then re-enable again. */
+ } else {
+#endif
+ /*
+* Disable intr remapping and queued invalidation,
+* if already enabled prior to OS handover.
+*/
+ iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+ }
+#endif
dmar_disable_qi(iommu);
}
@@ -702,7 +721,19 @@ static int __init intel_enable_irq_remapping(void)
* Setup Interrupt-remapping for all the DRHD's now.
*/
for_each_iommu(iommu, drhd) {
- iommu_set_irq_remapping(iommu, eim);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+ iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+ iommu->ir_table->base_old_virt = ioremap_cache(
+ iommu->ir_table->base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+#endif
+ iommu_set_irq_remapping(iommu, eim);
setup = 1;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 62 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 66 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index 14de1ab..20c060b 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -17,6 +18,11 @@
#include "irq_remapping.h"
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1302,3 +1308,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ memcpy(iommu->ir_table->base,
+ iommu->ir_table->base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu->ir_table->base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu->ir_table->base_old_virt;
+ from = iommu->ir_table->base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 54 ++---
1 file changed, 46 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 312f06b..6b691d4 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
#include
#include
#include
+#include
#include "irq_remapping.h"
@@ -3121,14 +3122,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR "Domain context map for %s failed",
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unmap_device_dma(domain, dev);
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err("Allocating domain for %s failed",
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+#endif
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err("Domain context map for %s failed",
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5177,4 +5194,25 @@ static int intel_iommu_load_translation_tables(struct
dmar_drhd_unit *drhd)
return ret;
}
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+ struct intel_iommu *iommu;
+ struct context_entry *ce;
+ struct iova *iova;
+ u8 bus, devfn;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+
+ iommu = device_to_iommu(dev, , );
+ ce = device_to_context_entry(iommu, bus, devfn);
+ phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(>iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
Use old root entry table, and load the old data to root_entry as cache.
Malloc new context table and copy old context table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Update the caller of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that loads root entry table.
Use new function to copy old context entry tables and page tables.
Use "unsigned long" for physical address.
Remove the functions to copy page table in Bill's version.
Remove usage of dve and ppap in Bill's version.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 113
1 file changed, 113 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1cb9780..44f3369 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,18 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd);
+
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4976,4 +4988,105 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+ u32 bus;/* Index: root-entry-table */
+ struct root_entry *re; /* Virt(iterator: new table) */
+ unsigned long context_old_phys; /* Phys(context table entry) */
+ struct context_entry *context_new_virt; /* Virt(new context_entry) */
+
+ /*
+* A new root entry table has been allocated ,
+* we need copy re from old kernel to the new allocated one.
+*/
+
+ if (!iommu->root_entry_old_phys)
+ return -ENOMEM;
+
+ for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+ if (!root_present(re))
+ continue;
+
+ context_old_phys = get_context_phys_from_root(re);
+
+ if (!context_old_phys)
+ continue;
+
+ context_new_virt =
+ (struct context_entry *)alloc_pgtable_page(iommu->node);
+
+ if (!context_new_virt)
+ return -ENOMEM;
+
+ __iommu_load_from_oldmem(context_new_virt,
+ context_old_phys,
+ VTD_PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+ set_root_value(re, virt_to_phys(context_new_virt));
+ }
+
+ return 0;
+}
+
+/*
+ * Interface to the "load translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
+{
+ struct intel_iommu *iommu; /* Virt(iommu hardware registers) */
+ unsigned long long q; /* quadword scratch */
+ int ret = 0;/* Integer return code */
+ unsigned long flags;
+
+ iommu = drhd->iommu;
+ q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+ if (!q)
+ return -1;
+
+ spin_lock_irqsave(>lock, flags);
+
+ /* Load the root-entry table from the old kernel
+* foreach context_entry_table in root_entry
+* Copy each entry table from old kernel
+*/
+ if (!iommu->root_entry) {
+ iommu->root_entry =
+ (struct root_entry *)alloc_pgtable_page(iommu->node);
+ if (!iommu->root_entry) {
+ spin_unlock_irqrestore(>lock, flags);
+ return -ENOMEM;
+ }
+ }
+
+ iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+ if (!iommu->root_entry_old_phys) {
+ pr_err("Could not read old root entry address.");
+ return -1;
+ }
+
+ iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
[PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
device_to_context_entry
free_context_table
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Use the did and gaw from old context entry;
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 118 ++--
1 file changed, 103 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 44f3369..312f06b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -841,6 +841,11 @@ static struct context_entry *
device_to_context_entry(struct intel_iommu *iommu,
set_root_value(root, phy_addr);
set_root_present(root);
__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_root_entry(iommu, bus);
+#endif
}
spin_unlock_irqrestore(>lock, flags);
return [devfn];
@@ -892,7 +897,8 @@ static void free_context_table(struct intel_iommu *iommu)
spin_lock_irqsave(>lock, flags);
if (!iommu->root_entry) {
- goto out;
+ spin_unlock_irqrestore(>lock, flags);
+ return;
}
for (i = 0; i < ROOT_ENTRY_NR; i++) {
root = >root_entry[i];
@@ -900,10 +906,23 @@ static void free_context_table(struct intel_iommu *iommu)
if (context)
free_pgtable_page(context);
}
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ iommu->root_entry_old_phys = 0;
+ root = iommu->root_entry_old_virt;
+ iommu->root_entry_old_virt = NULL;
+ }
+#endif
free_pgtable_page(iommu->root_entry);
iommu->root_entry = NULL;
-out:
+
spin_unlock_irqrestore(>lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ iounmap(root);
+#endif
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2322,6 +2341,9 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+ struct context_entry *ce = NULL;
+#endif /* CONFIG_CRASH_DUMP */
domain = find_domain(dev);
if (domain)
@@ -2355,6 +2377,22 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+ if (ce) {
+ did = context_domain_id(ce);
+ gaw = agaw_to_width(context_address_width(ce));
+ }
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
domain->id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
@@ -2889,14 +2927,33 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root & context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ pr_info("IOMMU Copying translate tables from panicked
kernel\n");
+ ret = intel_iommu_load_translation_tables(drhd);
+ if (ret) {
+ pr_err("IOMMU: Copy translate tables failed\n");
+
+ /* Best to stop trying */
+ goto free_iommu;
+ }
+ pr_info("IOMMU: root_cache:0x%12.12llx
phys:0x%12.12llx\n",
+ (u64)iommu->root_entry,
+ (u64)iommu->root_entry_old_phys);
+ } else {
+#endif /* CONFIG_CRASH_DUMP */
+ /*
+* TBD:
+* we could share the same root & context tables
+* among all IOMMU's. Need to Split it later.
+
[PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
pdated root entry table and irte table. 6. Use intel_unmap to unmap the old dma; 7. Allocate new pages while driver is being loaded. Changelog[v4]: 1. Cut off the patches that move some defines and functions to new files. 2. Reduce the numbers of patches to five, make it more easier to read. 3. Changed the name of functions, make them consistent with current context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out "#define DEBUG 1" to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(>iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use "unsigned long" as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He helps testing this patchset. Takao Indoh gives valuable suggestions. Li, Zhen-Hua (10): iommu/vt-d: New function to attach domain with id iommu/vt-d: Items required for kdump iommu/vt-d: Function to get old context entry iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel drivers/iommu/intel-iommu.c | 535 ++-- drivers/iommu/intel_irq_remapping.c | 105 ++- include/linux/intel-iommu.h | 18 ++ 3 files changed, 628 insertions(+), 30 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 01/10] iommu/vt-d: New function to attach domain with id
Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter did.
Bill Sumner:
(In older versions) Add new 'did' parameter to iommu_attach_domain();
The caller of this function.
Li, Zhenhua:
New function iommu_attach_domain_with_id(), instead of updating funtion
iommu_attach_domain();
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ae4c1a8..76674a1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1558,6 +1558,16 @@ static int iommu_attach_domain(struct dmar_domain
*domain,
return num;
}
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+ struct intel_iommu *iommu,
+ int domain_number)
+{
+ if (domain_number = 0)
+ return domain_number;
+
+ return iommu_attach_domain(domain, iommu);
+}
+
static int iommu_attach_vm_domain(struct dmar_domain *domain,
struct intel_iommu *iommu)
{
@@ -2225,6 +2235,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to no domain_id supplied */
domain = find_domain(dev);
if (domain)
@@ -2258,7 +2269,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain-id = iommu_attach_domain(domain, iommu);
+ domain-id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
return NULL;
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
and irte table. 6. Use intel_unmap to unmap the old dma; 7. Allocate new pages while driver is being loaded. Changelog[v4]: 1. Cut off the patches that move some defines and functions to new files. 2. Reduce the numbers of patches to five, make it more easier to read. 3. Changed the name of functions, make them consistent with current context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out #define DEBUG 1 to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(domain-iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use unsigned long as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He b...@redhat.com helps testing this patchset. Takao Indoh indou.ta...@jp.fujitsu.com gives valuable suggestions. Li, Zhen-Hua (10): iommu/vt-d: New function to attach domain with id iommu/vt-d: Items required for kdump iommu/vt-d: Function to get old context entry iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel drivers/iommu/intel-iommu.c | 535 ++-- drivers/iommu/intel_irq_remapping.c | 105 ++- include/linux/intel-iommu.h | 18 ++ 3 files changed, 628 insertions(+), 30 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 54 ++---
1 file changed, 46 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 312f06b..6b691d4 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
#include asm/irq_remapping.h
#include asm/cacheflush.h
#include asm/iommu.h
+#include linux/dma-mapping.h
#include irq_remapping.h
@@ -3121,14 +3122,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR Domain context map for %s failed,
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unmap_device_dma(domain, dev);
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err(Allocating domain for %s failed,
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+#endif
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err(Domain context map for %s failed,
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5177,4 +5194,25 @@ static int intel_iommu_load_translation_tables(struct
dmar_drhd_unit *drhd)
return ret;
}
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+ struct intel_iommu *iommu;
+ struct context_entry *ce;
+ struct iova *iova;
+ u8 bus, devfn;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+
+ iommu = device_to_iommu(dev, bus, devfn);
+ ce = device_to_context_entry(iommu, bus, devfn);
+ phys_addr = context_address_root(ce) VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(domain-iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
Use old root entry table, and load the old data to root_entry as cache.
Malloc new context table and copy old context table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Update the caller of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that loads root entry table.
Use new function to copy old context entry tables and page tables.
Use unsigned long for physical address.
Remove the functions to copy page table in Bill's version.
Remove usage of dve and ppap in Bill's version.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 113
1 file changed, 113 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1cb9780..44f3369 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,18 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd);
+
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4976,4 +4988,105 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+ u32 bus;/* Index: root-entry-table */
+ struct root_entry *re; /* Virt(iterator: new table) */
+ unsigned long context_old_phys; /* Phys(context table entry) */
+ struct context_entry *context_new_virt; /* Virt(new context_entry) */
+
+ /*
+* A new root entry table has been allocated ,
+* we need copy re from old kernel to the new allocated one.
+*/
+
+ if (!iommu-root_entry_old_phys)
+ return -ENOMEM;
+
+ for (bus = 0, re = iommu-root_entry; bus 256; bus += 1, re += 1) {
+ if (!root_present(re))
+ continue;
+
+ context_old_phys = get_context_phys_from_root(re);
+
+ if (!context_old_phys)
+ continue;
+
+ context_new_virt =
+ (struct context_entry *)alloc_pgtable_page(iommu-node);
+
+ if (!context_new_virt)
+ return -ENOMEM;
+
+ __iommu_load_from_oldmem(context_new_virt,
+ context_old_phys,
+ VTD_PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+ set_root_value(re, virt_to_phys(context_new_virt));
+ }
+
+ return 0;
+}
+
+/*
+ * Interface to the load translation tables set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
+{
+ struct intel_iommu *iommu; /* Virt(iommu hardware registers) */
+ unsigned long long q; /* quadword scratch */
+ int ret = 0;/* Integer return code */
+ unsigned long flags;
+
+ iommu = drhd-iommu;
+ q = dmar_readq(iommu-reg + DMAR_RTADDR_REG);
+ if (!q)
+ return -1;
+
+ spin_lock_irqsave(iommu-lock, flags);
+
+ /* Load the root-entry table from the old kernel
+* foreach context_entry_table in root_entry
+* Copy each entry table from old kernel
+*/
+ if (!iommu-root_entry) {
+ iommu-root_entry =
+ (struct root_entry *)alloc_pgtable_page(iommu-node);
+ if (!iommu-root_entry) {
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return -ENOMEM;
+ }
+ }
+
+ iommu-root_entry_old_phys = q VTD_PAGE_MASK;
+ if (!iommu-root_entry_old_phys) {
+ pr_err(Could not read old root entry address.);
+ return -1;
+ }
+
+ iommu-root_entry_old_virt = ioremap_cache(iommu-root_entry_old_phys,
+ VTD_PAGE_SIZE
[PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
device_to_context_entry
free_context_table
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Use the did and gaw from old context entry;
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 118 ++--
1 file changed, 103 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 44f3369..312f06b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -841,6 +841,11 @@ static struct context_entry *
device_to_context_entry(struct intel_iommu *iommu,
set_root_value(root, phy_addr);
set_root_present(root);
__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_root_entry(iommu, bus);
+#endif
}
spin_unlock_irqrestore(iommu-lock, flags);
return context[devfn];
@@ -892,7 +897,8 @@ static void free_context_table(struct intel_iommu *iommu)
spin_lock_irqsave(iommu-lock, flags);
if (!iommu-root_entry) {
- goto out;
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return;
}
for (i = 0; i ROOT_ENTRY_NR; i++) {
root = iommu-root_entry[i];
@@ -900,10 +906,23 @@ static void free_context_table(struct intel_iommu *iommu)
if (context)
free_pgtable_page(context);
}
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ iommu-root_entry_old_phys = 0;
+ root = iommu-root_entry_old_virt;
+ iommu-root_entry_old_virt = NULL;
+ }
+#endif
free_pgtable_page(iommu-root_entry);
iommu-root_entry = NULL;
-out:
+
spin_unlock_irqrestore(iommu-lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ iounmap(root);
+#endif
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2322,6 +2341,9 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to no domain_id supplied */
+#ifdef CONFIG_CRASH_DUMP
+ struct context_entry *ce = NULL;
+#endif /* CONFIG_CRASH_DUMP */
domain = find_domain(dev);
if (domain)
@@ -2355,6 +2377,22 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+ if (ce) {
+ did = context_domain_id(ce);
+ gaw = agaw_to_width(context_address_width(ce));
+ }
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
domain-id = iommu_attach_domain_with_id(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
@@ -2889,14 +2927,33 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ pr_info(IOMMU Copying translate tables from panicked
kernel\n);
+ ret = intel_iommu_load_translation_tables(drhd);
+ if (ret) {
+ pr_err(IOMMU: Copy translate tables failed\n);
+
+ /* Best to stop trying */
+ goto free_iommu;
+ }
+ pr_info(IOMMU: root_cache:0x%12.12llx
phys:0x%12.12llx\n,
+ (u64)iommu-root_entry,
+ (u64)iommu-root_entry_old_phys);
+ } else {
+#endif /* CONFIG_CRASH_DUMP */
+ /*
+* TBD:
+* we could share the same root context tables
+* among all IOMMU's. Need to Split it later.
+*/
+ ret
[PATCH v9 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 43 +++--
1 file changed, 37 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index 20c060b..4e5a02d 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -199,6 +199,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(irte-low, irte_modified-low);
set_64bit(irte-high, irte_modified-high);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, index);
+#endif
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -260,6 +265,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu-ir_table-bitmap, index,
irq_iommu-irte_mask);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, -1);
+#endif
+
return qi_flush_iec(iommu, index, irq_iommu-irte_mask);
}
@@ -662,11 +672,20 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
- /*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
-*/
- iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /* Do notdisable irq and then re-enable again. */
+ } else {
+#endif
+ /*
+* Disable intr remapping and queued invalidation,
+* if already enabled prior to OS handover.
+*/
+ iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+ }
+#endif
dmar_disable_qi(iommu);
}
@@ -702,7 +721,19 @@ static int __init intel_enable_irq_remapping(void)
* Setup Interrupt-remapping for all the DRHD's now.
*/
for_each_iommu(iommu, drhd) {
- iommu_set_irq_remapping(iommu, eim);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu-reg + DMAR_IRTA_REG);
+ iommu-ir_table-base_old_phys = q VTD_PAGE_MASK;
+ iommu-ir_table-base_old_virt = ioremap_cache(
+ iommu-ir_table-base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+#endif
+ iommu_set_irq_remapping(iommu, eim);
setup = 1;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 62 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 66 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index 14de1ab..20c060b 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include linux/irq.h
#include linux/intel-iommu.h
#include linux/acpi.h
+#include linux/crash_dump.h
#include asm/io_apic.h
#include asm/smp.h
#include asm/cpu.h
@@ -17,6 +18,11 @@
#include irq_remapping.h
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1302,3 +1308,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ memcpy(iommu-ir_table-base,
+ iommu-ir_table-base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu-ir_table-base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ if (index -1 || index = INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu-ir_table-base_old_virt;
+ from = iommu-ir_table-base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 02/10] iommu/vt-d: Items required for kdump
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Remove the structure dve which is not used in new version.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 56 +
1 file changed, 56 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 76674a1..577d5de 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include linux/pci-ats.h
#include linux/memblock.h
#include linux/dma-contiguous.h
+#include linux/crash_dump.h
#include asm/irq_remapping.h
#include asm/cacheflush.h
#include asm/iommu.h
@@ -211,6 +212,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root-val VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -231,6 +238,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context-lo 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c-lo 1) 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c-lo 2) 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c-lo VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c-hi 0) 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c-hi 8) 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context-lo |= 1;
@@ -316,6 +349,29 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *in the old kernel to the new ones.
+ *
+ */
+
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 102
include/linux/intel-iommu.h | 9
2 files changed, 111 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f7dbe70..7f3484a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -374,6 +374,17 @@ static struct context_entry
*device_to_existing_context_entry(
u8 bus, u8 devfn);
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4822,4 +4833,95 @@ static struct context_entry
*device_to_existing_context_entry(
return ret;
}
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from VTD_PAGE_SHIFT;
+ offset = from (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to VTD_PAGE_SHIFT;
+ offset = to (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, __iommu_remapped_mem, list) {
+ iounmap(mem_entry-mem);
+ list_del(mem_entry-list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(__iommu_mem_list_lock);
+ return 0;
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
#include linux/iova.h
#include linux/io.h
#include linux/dma_remapping.h
+#include linux/crash_dump.h
#include asm/cacheflush.h
#include asm/iommu.h
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
extern const struct
[PATCH v9 03/10] iommu/vt-d: Function to get old context entry
Interface for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 27 +++
1 file changed, 27 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 577d5de..f7dbe70 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -369,6 +369,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
*
*/
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn);
+
#endif /* CONFIG_CRASH_DUMP */
@@ -4796,3 +4800,26 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n,
vtisochctrl);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static struct context_entry *device_to_existing_context_entry(
+ struct intel_iommu *iommu,
+ u8 bus, u8 devfn)
+{
+ struct root_entry *root;
+ struct context_entry *context;
+ struct context_entry *ret;
+ unsigned long flags;
+
+ ret = NULL;
+ spin_lock_irqsave(iommu-lock, flags);
+ root = iommu-root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context context_present(context+devfn))
+ ret = context[devfn];
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return ret;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v9 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu-root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 52 +
include/linux/intel-iommu.h | 5 +
2 files changed, 57 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 7f3484a..1cb9780 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -373,6 +373,9 @@ static struct context_entry
*device_to_existing_context_entry(
struct intel_iommu *iommu,
u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
/*
* A structure used to store the address allocated by ioremap();
@@ -4924,4 +4927,53 @@ int __iommu_free_mapped_mem(void)
return 0;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+ memcpy(iommu-root_entry, iommu-root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu-root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+
+ if (index -1 || index = ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu-root_entry_old_virt;
+ from = iommu-root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 62 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 66 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index a55b207..d37fd62 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -17,6 +18,11 @@
#include "irq_remapping.h"
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1296,3 +1302,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ memcpy(iommu->ir_table->base,
+ iommu->ir_table->base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu->ir_table->base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu->ir_table->base_old_virt;
+ from = iommu->ir_table->base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 42 -
1 file changed, 37 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index d37fd62..58356cb 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -198,6 +198,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(>low, irte_modified->low);
set_64bit(>high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, index);
+#endif
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -259,6 +264,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu->ir_table->bitmap, index,
irq_iommu->irte_mask);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, -1);
+#endif
+
return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
}
@@ -640,11 +650,20 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
- /*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
-*/
- iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /* Do notdisable irq and then re-enable again. */
+ } else {
+#endif
+ /*
+* Disable intr remapping and queued invalidation,
+* if already enabled prior to OS handover.
+*/
+ iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+ }
+#endif
dmar_disable_qi(iommu);
}
@@ -687,7 +706,20 @@ static int __init intel_enable_irq_remapping(void)
if (intel_setup_irq_remapping(iommu))
goto error;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+ iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+ iommu->ir_table->base_old_virt = ioremap_cache(
+ iommu->ir_table->base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+#endif
iommu_set_irq_remapping(iommu, eim);
+
setup = 1;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a
device, we assign a new and empty page-table, thus removing all
mappings from the old kernel for the device.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 56 ++---
1 file changed, 48 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 324c504..ccbad3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
#include
#include
#include
+#include
#include "irq_remapping.h"
@@ -455,6 +456,8 @@ static int copy_root_entry_table(struct intel_iommu *iommu,
void *ppap);
static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
int g_num_of_iommus);
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -3196,14 +3199,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR "Domain context map for %s failed",
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unmap_device_dma(domain, dev);
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err("Allocating domain for %s failed",
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+#endif
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err("Domain context map for %s failed",
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5691,4 +5710,25 @@ static int intel_iommu_load_translation_tables(struct
dmar_drhd_unit *drhd,
return 0;
}
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+ struct intel_iommu *iommu;
+ struct context_entry *ce;
+ struct iova *iova;
+ u8 bus, devfn;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+
+ iommu = device_to_iommu(dev, , );
+ ce = device_to_context_entry(iommu, bus, devfn);
+ phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(>iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 53 +
include/linux/intel-iommu.h | 5 +
2 files changed, 58 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2335831..5f11f43 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,10 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
+
struct iommu_remapped_entry {
struct list_head list;
void __iomem *mem;
@@ -4986,4 +4990,53 @@ static int device_to_domain_id(struct intel_iommu
*iommu, u8 bus, u8 devfn)
return did;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+ memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+
+ if (index < -1 || index >= ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu->root_entry_old_virt;
+ from = iommu->root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
malloc new context table and copy old context table to the new one.
malloc new page table and copy old page table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Minor change:
Update the usage of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that copies root entry table.
Use new function to copy old context entry tables and page tables.
Use "unsigned long" for physical address.
Change incorrect aw_shift[4] and a few comments in copy_context_entry().
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 547
1 file changed, 547 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5f11f43..277b294 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1 /* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+ u32 first; /* flag: first-time */
+ u32 last; /* flag: last-time */
+ u32 bus;/* last bus number we saw */
+ u32 devfn; /* last devfn we saw */
+ u32 shift; /* last shift we saw */
+ u64 pte;/* Page Table Entry */
+ u64 next_addr; /* next-expected page_addr */
+
+ u64 page_addr; /* page_addr accumulating size */
+ u64 page_size; /* page_size accumulated */
+
+ struct domain_values_entry *dve;/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+ void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+ u32 bus, void *ppap,
+ unsigned long *context_new_p,
+ unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+ int g_num_of_iommus);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -5039,4 +5095,495 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+ u64 pte, struct domain_values_entry *dve,
+ void *parms)
+{
+ struct copy_page_addr_parms *ppap = parms;
+
+ u64 page_size = ((u64)1 << shift); /* page_size */
+ u64 pfn_lo;
[PATCH v8 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
device_to_domain_id
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
Minor change,
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 135 +++-
1 file changed, 120 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 277b294..324c504 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -907,6 +907,11 @@ static struct context_entry *
device_to_context_entry(struct intel_iommu *iommu,
set_root_value(root, phy_addr);
set_root_present(root);
__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_root_entry(iommu, bus);
+#endif
}
spin_unlock_irqrestore(>lock, flags);
return [devfn];
@@ -958,7 +963,8 @@ static void free_context_table(struct intel_iommu *iommu)
spin_lock_irqsave(>lock, flags);
if (!iommu->root_entry) {
- goto out;
+ spin_unlock_irqrestore(>lock, flags);
+ return;
}
for (i = 0; i < ROOT_ENTRY_NR; i++) {
root = >root_entry[i];
@@ -966,10 +972,23 @@ static void free_context_table(struct intel_iommu *iommu)
if (context)
free_pgtable_page(context);
}
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ iommu->root_entry_old_phys = 0;
+ root = iommu->root_entry_old_virt;
+ iommu->root_entry_old_virt = NULL;
+ }
+#endif
free_pgtable_page(iommu->root_entry);
iommu->root_entry = NULL;
-out:
+
spin_unlock_irqrestore(>lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ iounmap(root);
+#endif
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2381,6 +2400,9 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+ struct domain_values_entry *dve = NULL;
+#endif /* CONFIG_CRASH_DUMP */
domain = find_domain(dev);
if (domain)
@@ -2414,6 +2436,24 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ did = device_to_domain_id(iommu, bus, devfn);
+ if (did > 0 || (did == 0 && !cap_caching_mode(iommu->cap)))
+ dve = intel_iommu_did_to_domain_values_entry(did,
+ iommu);
+ if (dve)
+ gaw = dve->gaw;
+ else
+ did = -1;
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
domain->id = iommu_attach_domain(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
@@ -2425,6 +2465,18 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
return NULL;
}
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel() && dve) {
+
+ if (domain->pgd)
+ free_pgtable_page(domain->pgd);
+
+ domain->pgd = dve->pgd;
+
+ copy_reserved_iova(>iovad, >iovad);
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
/* register PCI DMA alias device */
if (dev_is_pci(dev)) {
tmp = dmar_insert_dev_info(iommu, PCI_BUS_NUM(dma_alias),
@@ -2948,14 +3000,35 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root & context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ pr_info("IOMMU Copying translate tables from panicked
kernel\n");
+ ret = intel_iommu_load_translation_tables(drhd,
+ g_
[PATCH v8 02/10] iommu/vt-d: Items required for kdump
Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 70 +
1 file changed, 70 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8d5c400..a71de3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context->lo & 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c->hi >> 8) & 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context->lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte & ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *portions of the iova address ranges that are different
+ *from the iova address ranges that were being used by the old kernel
+ *at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+ struct list_head link; /* link entries into a list */
+ struct iova_domain iovad; /* iova's that belong to this domain */
+ struct dma_pte *pgd; /* virtual address */
+ intdid; /* domain id */
+ intgaw; /* max guest address width */
+ intiommu_superpage; /* Level of superpages supported:
+ 0 == 4KiB (no superpages), 1 == 2MiB,
+ 2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 03/10] iommu/vt-d: Add domain-id functions
Interfaces for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 62 +
1 file changed, 62 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a71de3f..c594b2c 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -381,6 +381,13 @@ struct domain_values_entry {
2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
};
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu);
+
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4828,3 +4835,58 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n",
vtisochctrl);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Interfaces for when a new domain in the crashdump kernel needs some
+ * values from the panicked kernel's context entries
+ *
+ */
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ list_for_each_entry(dve, _values_list[iommu->seq_id], link)
+ if (dve->did == did)
+ return dve;
+ return NULL;
+}
+
+/* Mark domain-id's from old kernel as in-use on this iommu so that a new
+ * domain-id is allocated in the case where there is a device in the new kernel
+ * that was not in the old kernel -- and therefore a new domain-id is needed.
+ */
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ pr_info("IOMMU:%d Domain ids from panicked kernel:\n", iommu->seq_id);
+
+ list_for_each_entry(dve, _values_list[iommu->seq_id], link) {
+ set_bit(dve->did, iommu->domain_ids);
+ pr_info("DID did:%d(0x%4.4x)\n", dve->did, dve->did);
+ }
+
+ pr_info("\n");
+ return 0;
+}
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
+{
+ int did = -1; /* domain-id returned */
+ struct root_entry *root;
+ struct context_entry *context;
+ unsigned long flags;
+
+ spin_lock_irqsave(>lock, flags);
+ root = >root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context && context_present(context+devfn))
+ did = context_domain_id(context+devfn);
+ spin_unlock_irqrestore(>lock, flags);
+ return did;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 97 +
include/linux/intel-iommu.h | 9 +
2 files changed, 106 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c594b2c..2335831 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,13 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4839,6 +4846,96 @@ static void __init check_tylersburg_isoch(void)
#ifdef CONFIG_CRASH_DUMP
/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from >> VTD_PAGE_SHIFT;
+ offset = from & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to >> VTD_PAGE_SHIFT;
+ offset = to & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(&__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+ iounmap(mem_entry->mem);
+ list_del(_entry->list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(&__iommu_mem_list_lock);
+ return 0;
+}
+/*
* Interfaces for when a new domain in the crashdump kernel needs some
* values from the panicked kernel's context entries
*
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
#include
#include
#include
+#include
#include
#include
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
extern const struct attribute_group *intel_iommu_groups[];
+#ifdef CONFIG_CR
[PATCH v8 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
s, make them consistent with current context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out "#define DEBUG 1" to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(>iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use "unsigned long" as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He helps testing this patchset. Takao Indoh gives valuable suggestions. iommu/vt-d: Update iommu_attach_domain() and its callers iommu/vt-d: Items required for kdump iommu/vt-d: Add domain-id functions iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel Signed-off-by: Bill Sumner Signed-off-by: Li, Zhen-Hua Signed-off-by: Takao Indoh Tested-by: Baoquan He --- drivers/iommu/intel-iommu.c | 1054 +-- drivers/iommu/intel_irq_remapping.c | 104 +++- include/linux/intel-iommu.h | 18 + 3 files changed, 1134 insertions(+), 42 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 01/10] iommu/vt-d: Update iommu_attach_domain() and its callers
Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says "no did specified, allocate a new one".
This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.
Bill Sumner:
Original version.
Li, Zhenhua:
Minor change, add change to function __iommu_attach_domain.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 34 --
1 file changed, 20 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 40dfbc0..8d5c400 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
}
static int __iommu_attach_domain(struct dmar_domain *domain,
-struct intel_iommu *iommu)
+struct intel_iommu *iommu,
+int domain_number)
{
int num;
unsigned long ndomains;
ndomains = cap_ndoms(iommu->cap);
- num = find_first_zero_bit(iommu->domain_ids, ndomains);
- if (num < ndomains) {
- set_bit(num, iommu->domain_ids);
- iommu->domains[num] = domain;
- } else {
- num = -ENOSPC;
- }
+ if (domain_number < 0) {
+ num = find_first_zero_bit(iommu->domain_ids, ndomains);
+ if (num < ndomains) {
+ set_bit(num, iommu->domain_ids);
+ iommu->domains[num] = domain;
+ } else {
+ num = -ENOSPC;
+ }
+ } else
+ num = domain_number;
return num;
}
static int iommu_attach_domain(struct dmar_domain *domain,
- struct intel_iommu *iommu)
+ struct intel_iommu *iommu,
+ int domain_number)
{
int num;
unsigned long flags;
spin_lock_irqsave(>lock, flags);
- num = __iommu_attach_domain(domain, iommu);
+ num = __iommu_attach_domain(domain, iommu, domain_number);
spin_unlock_irqrestore(>lock, flags);
if (num < 0)
pr_err("IOMMU: no free domain ids\n");
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain
*domain,
if (iommu->domains[num] == domain)
return num;
- return __iommu_attach_domain(domain, iommu);
+ return __iommu_attach_domain(domain, iommu, -1);
}
static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to "no domain_id supplied" */
domain = find_domain(dev);
if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain->id = iommu_attach_domain(domain, iommu);
+ domain->id = iommu_attach_domain(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
return -EFAULT;
for_each_active_iommu(iommu, drhd) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret < 0) {
domain_exit(si_domain);
return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
iommu_enable_translation(iommu);
if (si_domain) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret < 0 || si_domain->id != ret)
goto disable_iommu;
domain_attach_iommu(si_domain, iommu);
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 62 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 66 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index a55b207..d37fd62 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include linux/irq.h
#include linux/intel-iommu.h
#include linux/acpi.h
+#include linux/crash_dump.h
#include asm/io_apic.h
#include asm/smp.h
#include asm/cpu.h
@@ -17,6 +18,11 @@
#include irq_remapping.h
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1296,3 +1302,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ memcpy(iommu-ir_table-base,
+ iommu-ir_table-base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu-ir_table-base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ if (index -1 || index = INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu-ir_table-base_old_virt;
+ from = iommu-ir_table-base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 42 -
1 file changed, 37 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index d37fd62..58356cb 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -198,6 +198,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(irte-low, irte_modified-low);
set_64bit(irte-high, irte_modified-high);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, index);
+#endif
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -259,6 +264,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu-ir_table-bitmap, index,
irq_iommu-irte_mask);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, -1);
+#endif
+
return qi_flush_iec(iommu, index, irq_iommu-irte_mask);
}
@@ -640,11 +650,20 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
- /*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
-*/
- iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /* Do notdisable irq and then re-enable again. */
+ } else {
+#endif
+ /*
+* Disable intr remapping and queued invalidation,
+* if already enabled prior to OS handover.
+*/
+ iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+ }
+#endif
dmar_disable_qi(iommu);
}
@@ -687,7 +706,20 @@ static int __init intel_enable_irq_remapping(void)
if (intel_setup_irq_remapping(iommu))
goto error;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu-reg + DMAR_IRTA_REG);
+ iommu-ir_table-base_old_phys = q VTD_PAGE_MASK;
+ iommu-ir_table-base_old_virt = ioremap_cache(
+ iommu-ir_table-base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+#endif
iommu_set_irq_remapping(iommu, eim);
+
setup = 1;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu-root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 53 +
include/linux/intel-iommu.h | 5 +
2 files changed, 58 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2335831..5f11f43 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,10 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
+
struct iommu_remapped_entry {
struct list_head list;
void __iomem *mem;
@@ -4986,4 +4990,53 @@ static int device_to_domain_id(struct intel_iommu
*iommu, u8 bus, u8 devfn)
return did;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+ memcpy(iommu-root_entry, iommu-root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu-root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+
+ if (index -1 || index = ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu-root_entry_old_virt;
+ from = iommu-root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
device_to_domain_id
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
Minor change,
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 135 +++-
1 file changed, 120 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 277b294..324c504 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -907,6 +907,11 @@ static struct context_entry *
device_to_context_entry(struct intel_iommu *iommu,
set_root_value(root, phy_addr);
set_root_present(root);
__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_root_entry(iommu, bus);
+#endif
}
spin_unlock_irqrestore(iommu-lock, flags);
return context[devfn];
@@ -958,7 +963,8 @@ static void free_context_table(struct intel_iommu *iommu)
spin_lock_irqsave(iommu-lock, flags);
if (!iommu-root_entry) {
- goto out;
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return;
}
for (i = 0; i ROOT_ENTRY_NR; i++) {
root = iommu-root_entry[i];
@@ -966,10 +972,23 @@ static void free_context_table(struct intel_iommu *iommu)
if (context)
free_pgtable_page(context);
}
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ iommu-root_entry_old_phys = 0;
+ root = iommu-root_entry_old_virt;
+ iommu-root_entry_old_virt = NULL;
+ }
+#endif
free_pgtable_page(iommu-root_entry);
iommu-root_entry = NULL;
-out:
+
spin_unlock_irqrestore(iommu-lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ iounmap(root);
+#endif
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2381,6 +2400,9 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to no domain_id supplied */
+#ifdef CONFIG_CRASH_DUMP
+ struct domain_values_entry *dve = NULL;
+#endif /* CONFIG_CRASH_DUMP */
domain = find_domain(dev);
if (domain)
@@ -2414,6 +2436,24 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ did = device_to_domain_id(iommu, bus, devfn);
+ if (did 0 || (did == 0 !cap_caching_mode(iommu-cap)))
+ dve = intel_iommu_did_to_domain_values_entry(did,
+ iommu);
+ if (dve)
+ gaw = dve-gaw;
+ else
+ did = -1;
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
domain-id = iommu_attach_domain(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
@@ -2425,6 +2465,18 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
return NULL;
}
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel() dve) {
+
+ if (domain-pgd)
+ free_pgtable_page(domain-pgd);
+
+ domain-pgd = dve-pgd;
+
+ copy_reserved_iova(dve-iovad, domain-iovad);
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
/* register PCI DMA alias device */
if (dev_is_pci(dev)) {
tmp = dmar_insert_dev_info(iommu, PCI_BUS_NUM(dma_alias),
@@ -2948,14 +3000,35 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ pr_info(IOMMU Copying translate tables from panicked
kernel\n);
+ ret = intel_iommu_load_translation_tables(drhd,
+ g_num_of_iommus);
+ if (ret
[PATCH v8 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a
device, we assign a new and empty page-table, thus removing all
mappings from the old kernel for the device.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 56 ++---
1 file changed, 48 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 324c504..ccbad3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
#include asm/irq_remapping.h
#include asm/cacheflush.h
#include asm/iommu.h
+#include linux/dma-mapping.h
#include irq_remapping.h
@@ -455,6 +456,8 @@ static int copy_root_entry_table(struct intel_iommu *iommu,
void *ppap);
static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
int g_num_of_iommus);
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -3196,14 +3199,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR Domain context map for %s failed,
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unmap_device_dma(domain, dev);
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err(Allocating domain for %s failed,
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+#endif
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err(Domain context map for %s failed,
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5691,4 +5710,25 @@ static int intel_iommu_load_translation_tables(struct
dmar_drhd_unit *drhd,
return 0;
}
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+ struct intel_iommu *iommu;
+ struct context_entry *ce;
+ struct iova *iova;
+ u8 bus, devfn;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+
+ iommu = device_to_iommu(dev, bus, devfn);
+ ce = device_to_context_entry(iommu, bus, devfn);
+ phys_addr = context_address_root(ce) VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(domain-iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
malloc new context table and copy old context table to the new one.
malloc new page table and copy old page table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Minor change:
Update the usage of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that copies root entry table.
Use new function to copy old context entry tables and page tables.
Use unsigned long for physical address.
Change incorrect aw_shift[4] and a few comments in copy_context_entry().
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 547
1 file changed, 547 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5f11f43..277b294 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1 /* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+ u32 first; /* flag: first-time */
+ u32 last; /* flag: last-time */
+ u32 bus;/* last bus number we saw */
+ u32 devfn; /* last devfn we saw */
+ u32 shift; /* last shift we saw */
+ u64 pte;/* Page Table Entry */
+ u64 next_addr; /* next-expected page_addr */
+
+ u64 page_addr; /* page_addr accumulating size */
+ u64 page_size; /* page_size accumulated */
+
+ struct domain_values_entry *dve;/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+ void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+ u32 bus, void *ppap,
+ unsigned long *context_new_p,
+ unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+ int g_num_of_iommus);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -5039,4 +5095,495 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+ u64 pte, struct domain_values_entry *dve,
+ void *parms)
+{
+ struct copy_page_addr_parms *ppap = parms;
+
+ u64 page_size = ((u64)1 shift); /* page_size */
+ u64 pfn_lo
[PATCH v8 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
context get/set functions. 4. Add change to function __iommu_attach_domain. Changelog[v3]: 1. Commented-out #define DEBUG 1 to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(domain-iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use unsigned long as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He b...@redhat.com helps testing this patchset. Takao Indoh indou.ta...@jp.fujitsu.com gives valuable suggestions. iommu/vt-d: Update iommu_attach_domain() and its callers iommu/vt-d: Items required for kdump iommu/vt-d: Add domain-id functions iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel Signed-off-by: Bill Sumner billsumnerli...@gmail.com Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com Tested-by: Baoquan He b...@redhat.com --- drivers/iommu/intel-iommu.c | 1054 +-- drivers/iommu/intel_irq_remapping.c | 104 +++- include/linux/intel-iommu.h | 18 + 3 files changed, 1134 insertions(+), 42 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 01/10] iommu/vt-d: Update iommu_attach_domain() and its callers
Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says no did specified, allocate a new one.
This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.
Bill Sumner:
Original version.
Li, Zhenhua:
Minor change, add change to function __iommu_attach_domain.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 34 --
1 file changed, 20 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 40dfbc0..8d5c400 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
}
static int __iommu_attach_domain(struct dmar_domain *domain,
-struct intel_iommu *iommu)
+struct intel_iommu *iommu,
+int domain_number)
{
int num;
unsigned long ndomains;
ndomains = cap_ndoms(iommu-cap);
- num = find_first_zero_bit(iommu-domain_ids, ndomains);
- if (num ndomains) {
- set_bit(num, iommu-domain_ids);
- iommu-domains[num] = domain;
- } else {
- num = -ENOSPC;
- }
+ if (domain_number 0) {
+ num = find_first_zero_bit(iommu-domain_ids, ndomains);
+ if (num ndomains) {
+ set_bit(num, iommu-domain_ids);
+ iommu-domains[num] = domain;
+ } else {
+ num = -ENOSPC;
+ }
+ } else
+ num = domain_number;
return num;
}
static int iommu_attach_domain(struct dmar_domain *domain,
- struct intel_iommu *iommu)
+ struct intel_iommu *iommu,
+ int domain_number)
{
int num;
unsigned long flags;
spin_lock_irqsave(iommu-lock, flags);
- num = __iommu_attach_domain(domain, iommu);
+ num = __iommu_attach_domain(domain, iommu, domain_number);
spin_unlock_irqrestore(iommu-lock, flags);
if (num 0)
pr_err(IOMMU: no free domain ids\n);
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain
*domain,
if (iommu-domains[num] == domain)
return num;
- return __iommu_attach_domain(domain, iommu);
+ return __iommu_attach_domain(domain, iommu, -1);
}
static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to no domain_id supplied */
domain = find_domain(dev);
if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain-id = iommu_attach_domain(domain, iommu);
+ domain-id = iommu_attach_domain(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
return -EFAULT;
for_each_active_iommu(iommu, drhd) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret 0) {
domain_exit(si_domain);
return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
iommu_enable_translation(iommu);
if (si_domain) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret 0 || si_domain-id != ret)
goto disable_iommu;
domain_attach_iommu(si_domain, iommu);
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 02/10] iommu/vt-d: Items required for kdump
Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 70 +
1 file changed, 70 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8d5c400..a71de3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include linux/pci-ats.h
#include linux/memblock.h
#include linux/dma-contiguous.h
+#include linux/crash_dump.h
#include asm/irq_remapping.h
#include asm/cacheflush.h
#include asm/iommu.h
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root-val VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context-lo 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c-lo 1) 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c-lo 2) 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c-lo VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c-hi 0) 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c-hi 8) 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context-lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *portions of the iova address ranges that are different
+ *from the iova address ranges that were being used by the old kernel
+ *at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+ struct list_head link; /* link entries into a list */
+ struct iova_domain iovad; /* iova's that belong to this domain */
+ struct dma_pte *pgd; /* virtual address */
+ intdid; /* domain id */
+ intgaw; /* max guest address width */
+ intiommu_superpage; /* Level of superpages supported:
+ 0 == 4KiB (no superpages), 1 == 2MiB,
+ 2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 03/10] iommu/vt-d: Add domain-id functions
Interfaces for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 62 +
1 file changed, 62 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a71de3f..c594b2c 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -381,6 +381,13 @@ struct domain_values_entry {
2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
};
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu);
+
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4828,3 +4835,58 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n,
vtisochctrl);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Interfaces for when a new domain in the crashdump kernel needs some
+ * values from the panicked kernel's context entries
+ *
+ */
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ list_for_each_entry(dve, domain_values_list[iommu-seq_id], link)
+ if (dve-did == did)
+ return dve;
+ return NULL;
+}
+
+/* Mark domain-id's from old kernel as in-use on this iommu so that a new
+ * domain-id is allocated in the case where there is a device in the new kernel
+ * that was not in the old kernel -- and therefore a new domain-id is needed.
+ */
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ pr_info(IOMMU:%d Domain ids from panicked kernel:\n, iommu-seq_id);
+
+ list_for_each_entry(dve, domain_values_list[iommu-seq_id], link) {
+ set_bit(dve-did, iommu-domain_ids);
+ pr_info(DID did:%d(0x%4.4x)\n, dve-did, dve-did);
+ }
+
+ pr_info(\n);
+ return 0;
+}
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
+{
+ int did = -1; /* domain-id returned */
+ struct root_entry *root;
+ struct context_entry *context;
+ unsigned long flags;
+
+ spin_lock_irqsave(iommu-lock, flags);
+ root = iommu-root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context context_present(context+devfn))
+ did = context_domain_id(context+devfn);
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return did;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v8 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 97 +
include/linux/intel-iommu.h | 9 +
2 files changed, 106 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c594b2c..2335831 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,13 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4839,6 +4846,96 @@ static void __init check_tylersburg_isoch(void)
#ifdef CONFIG_CRASH_DUMP
/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from VTD_PAGE_SHIFT;
+ offset = from (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to VTD_PAGE_SHIFT;
+ offset = to (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, __iommu_remapped_mem, list) {
+ iounmap(mem_entry-mem);
+ list_del(mem_entry-list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(__iommu_mem_list_lock);
+ return 0;
+}
+/*
* Interfaces for when a new domain in the crashdump kernel needs some
* values from the panicked kernel's context entries
*
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
#include linux/iova.h
#include linux/io.h
#include linux/dma_remapping.h
+#include linux/crash_dump.h
#include asm/cacheflush.h
#include asm/iommu.h
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
extern const struct attribute_group
RE: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In your log, it seems something incorrect while copying pages. Your last DMAR fault is: DMAR:[fault reason 01] Present bit in root entry is clear But this time, it is: DMAR:[fault reason 05] PTE Write access is not set So I think this line I added to this version , it works. function intel_iommu_load_translation_tables, line: __iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE); I checked the code, found I missed one flush in function copy_page_table. How do you think we add one flush after this lines: ret = copy_page_table(_pte_next, (p->val & VTD_PAGE_MASK), shift-9, page_addr | (u << shift), iommu, bus, devfn, dve, ppap); + __iommu_flush_cache(iommu, phys_to_virt(dma_pte_next), + VTD_PAGE_SIZE); If this does not work, I have no ideas currently, need to dig the code more. Regards Zhenhua -Original Message- From: Takao Indoh [mailto:indou.ta...@jp.fujitsu.com] Sent: Thursday, January 08, 2015 9:00 AM To: Li, Zhen-Hua; b...@redhat.com Cc: dw...@infradead.org; j...@8bytes.org; vgo...@redhat.com; dyo...@redhat.com; io...@lists.linux-foundation.org; linux-kernel@vger.kernel.org; linux-...@vger.kernel.org; ke...@lists.infradead.org; alex.william...@redhat.com; ddut...@redhat.com; ishii.hiron...@jp.fujitsu.com; bhelg...@google.com; Hatch, Douglas B (HPS Linux PM); Hoemann, Jerry; Vaden, Tom (HP Server OS Architecture); Zhang, Li (Zoe@HPservers-Core-OE-PSC); Mitchell, Lisa (MCLinux in Fort Collins); billsumnerli...@gmail.com; Wright, Randy (HP Servers Linux) Subject: Re: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel On 2015/01/07 17:52, Li, ZhenHua wrote: > Well, that's quite good news. > Looking forward Takao's testing on his system. Unfortunately DMAR fault still occurs with this patch... I attach console log. Thanks, Takao Indoh > > Regards > Zhenhua > On 01/07/2015 04:28 PM, Baoquan He wrote: >> On 01/07/15 at 01:25pm, Li, ZhenHua wrote: >>> It is same as the last one I send to you yesterday. >>> >>> The continuous memory that needed for data in this patchset: >>> RE: PAGE_SIZE, 4096 Bytes; >>> IRTE: 65536 * 16 ; 1M Bytes; >>> >>> It should use same memory as the old versions of this patchset. The >>> changes for the last version do not need more memory. >> >> Hi Zhenhua, >> >> It was my mistake because I didn't strip the debug info of modules, >> then initramfs is bloated very big. Just now I tested the latest >> version, it works well and dump is successful. No dmar fault and >> intr-remap fault seen any more, good job! >> >> Thanks >> Baoquan >> >> >>> >>> Regards >>> Zhenhua >>> >>> On 01/07/2015 01:02 PM, Baoquan He wrote: >>>> On 01/07/15 at 12:11pm, Li, ZhenHua wrote: >>>>> Many thanks to Takao Indoh and Baoquan He, for your testing on >>>>> more different systems. >>>>> >>>>> The calling of flush functions are added to this version. >>>>> >>>>> The usage of __iommu_flush_cache function : >>>>> 1. Fixes a dump on Takao's system. >>>>> 2. Reduces the count of faults on Baoquan's system. >>>> >>>> I am testing the version you sent to me yesterday afternoon. Is >>>> that different with this patchset? I found your patchset man >>>> reserve a big contiguous memory region under 896M, this will cause >>>> the crashkernel reservation failed when I set crashkernel=320M. The >>>> reason I increase the crashkerenl reservation to 320M is 256M is >>>> not enough and cause OOM when that patchset is tested. >>>> >>>> I am checking what happened. >>>> >>>> >>>> Thanks >>>> Baoquan >>>> >>>>> >>>>> Regards >>>>> Zhenhua >>>>> >>>>> On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote: >>>>>> This patchset is an update of Bill Sumner's patchset, implements a fix >>>>>> for: >>>>>> If a kernel boots with intel_iommu=on on a system that supports >>>>>> intel vt-d, when a panic happens, the kdump kernel will boot with these >>>>>> faults: >>>>>> >>>>>> dmar: DRHD: handling fault status reg 102 >>>>>> dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff8 >>>>>> DMAR:[fault reason 01] Pres
RE: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In your log, it seems something incorrect while copying pages. Your last DMAR fault is: DMAR:[fault reason 01] Present bit in root entry is clear But this time, it is: DMAR:[fault reason 05] PTE Write access is not set So I think this line I added to this version , it works. function intel_iommu_load_translation_tables, line: __iommu_flush_cache(iommu, iommu-root_entry, PAGE_SIZE); I checked the code, found I missed one flush in function copy_page_table. How do you think we add one flush after this lines: ret = copy_page_table(dma_pte_next, (p-val VTD_PAGE_MASK), shift-9, page_addr | (u shift), iommu, bus, devfn, dve, ppap); + __iommu_flush_cache(iommu, phys_to_virt(dma_pte_next), + VTD_PAGE_SIZE); If this does not work, I have no ideas currently, need to dig the code more. Regards Zhenhua -Original Message- From: Takao Indoh [mailto:indou.ta...@jp.fujitsu.com] Sent: Thursday, January 08, 2015 9:00 AM To: Li, Zhen-Hua; b...@redhat.com Cc: dw...@infradead.org; j...@8bytes.org; vgo...@redhat.com; dyo...@redhat.com; io...@lists.linux-foundation.org; linux-kernel@vger.kernel.org; linux-...@vger.kernel.org; ke...@lists.infradead.org; alex.william...@redhat.com; ddut...@redhat.com; ishii.hiron...@jp.fujitsu.com; bhelg...@google.com; Hatch, Douglas B (HPS Linux PM); Hoemann, Jerry; Vaden, Tom (HP Server OS Architecture); Zhang, Li (Zoe@HPservers-Core-OE-PSC); Mitchell, Lisa (MCLinux in Fort Collins); billsumnerli...@gmail.com; Wright, Randy (HP Servers Linux) Subject: Re: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel On 2015/01/07 17:52, Li, ZhenHua wrote: Well, that's quite good news. Looking forward Takao's testing on his system. Unfortunately DMAR fault still occurs with this patch... I attach console log. Thanks, Takao Indoh Regards Zhenhua On 01/07/2015 04:28 PM, Baoquan He wrote: On 01/07/15 at 01:25pm, Li, ZhenHua wrote: It is same as the last one I send to you yesterday. The continuous memory that needed for data in this patchset: RE: PAGE_SIZE, 4096 Bytes; IRTE: 65536 * 16 ; 1M Bytes; It should use same memory as the old versions of this patchset. The changes for the last version do not need more memory. Hi Zhenhua, It was my mistake because I didn't strip the debug info of modules, then initramfs is bloated very big. Just now I tested the latest version, it works well and dump is successful. No dmar fault and intr-remap fault seen any more, good job! Thanks Baoquan Regards Zhenhua On 01/07/2015 01:02 PM, Baoquan He wrote: On 01/07/15 at 12:11pm, Li, ZhenHua wrote: Many thanks to Takao Indoh and Baoquan He, for your testing on more different systems. The calling of flush functions are added to this version. The usage of __iommu_flush_cache function : 1. Fixes a dump on Takao's system. 2. Reduces the count of faults on Baoquan's system. I am testing the version you sent to me yesterday afternoon. Is that different with this patchset? I found your patchset man reserve a big contiguous memory region under 896M, this will cause the crashkernel reservation failed when I set crashkernel=320M. The reason I increase the crashkerenl reservation to 320M is 256M is not enough and cause OOM when that patchset is tested. I am checking what happened. Thanks Baoquan Regards Zhenhua On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote: This patchset is an update of Bill Sumner's patchset, implements a fix for: If a kernel boots with intel_iommu=on on a system that supports intel vt-d, when a panic happens, the kdump kernel will boot with these faults: dmar: DRHD: handling fault status reg 102 dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff8 DMAR:[fault reason 01] Present bit in root entry is clear dmar: DRHD: handling fault status reg 2 dmar: INTR-REMAP: Request device [[61:00.0] fault index 42 INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear On some system, the interrupt remapping fault will also happen even if the intel_iommu is not set to on, because the interrupt remapping will be enabled when x2apic is needed by the system. The cause of the DMA fault is described in Bill's original version, and the INTR-Remap fault is caused by a similar reason. In short, the initialization of vt-d drivers causes the in-flight DMA and interrupt requests get wrong response. To fix this problem, we modifies the behaviors of the intel vt-d in the crashdump kernel: For DMA Remapping: 1. To accept the vt-d hardware in an active state, 2. Do not disable and re-enable the translation, keep it enabled. 3. Use the old root entry table, do not rewrite the RTA register. 4. Malloc and use new context entry table and page
[PATCH v7 02/10] iommu/vt-d: Items required for kdump
Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 70 +
1 file changed, 70 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2dc6250..5ce2850 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context->lo & 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c->hi >> 8) & 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context->lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte & ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *portions of the iova address ranges that are different
+ *from the iova address ranges that were being used by the old kernel
+ *at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+ struct list_head link; /* link entries into a list */
+ struct iova_domain iovad; /* iova's that belong to this domain */
+ struct dma_pte *pgd; /* virtual address */
+ intdid; /* domain id */
+ intgaw; /* max guest address width */
+ intiommu_superpage; /* Level of superpages supported:
+ 0 == 4KiB (no superpages), 1 == 2MiB,
+ 2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 42 -
1 file changed, 37 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index d37fd62..58356cb 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -198,6 +198,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(>low, irte_modified->low);
set_64bit(>high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, index);
+#endif
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -259,6 +264,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu->ir_table->bitmap, index,
irq_iommu->irte_mask);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, -1);
+#endif
+
return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
}
@@ -640,11 +650,20 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
- /*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
-*/
- iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /* Do notdisable irq and then re-enable again. */
+ } else {
+#endif
+ /*
+* Disable intr remapping and queued invalidation,
+* if already enabled prior to OS handover.
+*/
+ iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+ }
+#endif
dmar_disable_qi(iommu);
}
@@ -687,7 +706,20 @@ static int __init intel_enable_irq_remapping(void)
if (intel_setup_irq_remapping(iommu))
goto error;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+ iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+ iommu->ir_table->base_old_virt = ioremap_cache(
+ iommu->ir_table->base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+#endif
iommu_set_irq_remapping(iommu, eim);
+
setup = 1;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
device_to_domain_id
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
Minor change,
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 135 +++-
1 file changed, 120 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d2c19a0..8807710 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -907,6 +907,11 @@ static struct context_entry *
device_to_context_entry(struct intel_iommu *iommu,
set_root_value(root, phy_addr);
set_root_present(root);
__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_root_entry(iommu, bus);
+#endif
}
spin_unlock_irqrestore(>lock, flags);
return [devfn];
@@ -958,7 +963,8 @@ static void free_context_table(struct intel_iommu *iommu)
spin_lock_irqsave(>lock, flags);
if (!iommu->root_entry) {
- goto out;
+ spin_unlock_irqrestore(>lock, flags);
+ return;
}
for (i = 0; i < ROOT_ENTRY_NR; i++) {
root = >root_entry[i];
@@ -966,10 +972,23 @@ static void free_context_table(struct intel_iommu *iommu)
if (context)
free_pgtable_page(context);
}
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ iommu->root_entry_old_phys = 0;
+ root = iommu->root_entry_old_virt;
+ iommu->root_entry_old_virt = NULL;
+ }
+#endif
free_pgtable_page(iommu->root_entry);
iommu->root_entry = NULL;
-out:
+
spin_unlock_irqrestore(>lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ iounmap(root);
+#endif
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2381,6 +2400,9 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+ struct domain_values_entry *dve = NULL;
+#endif /* CONFIG_CRASH_DUMP */
domain = find_domain(dev);
if (domain)
@@ -2414,6 +2436,24 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ did = device_to_domain_id(iommu, bus, devfn);
+ if (did > 0 || (did == 0 && !cap_caching_mode(iommu->cap)))
+ dve = intel_iommu_did_to_domain_values_entry(did,
+ iommu);
+ if (dve)
+ gaw = dve->gaw;
+ else
+ did = -1;
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
domain->id = iommu_attach_domain(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
@@ -2425,6 +2465,18 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
return NULL;
}
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel() && dve) {
+
+ if (domain->pgd)
+ free_pgtable_page(domain->pgd);
+
+ domain->pgd = dve->pgd;
+
+ copy_reserved_iova(>iovad, >iovad);
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
/* register PCI DMA alias device */
if (dev_is_pci(dev)) {
tmp = dmar_insert_dev_info(iommu, PCI_BUS_NUM(dma_alias),
@@ -2948,14 +3000,35 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root & context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ pr_info("IOMMU Copying translate tables from panicked
kernel\n");
+ ret = intel_iommu_load_translation_tables(drhd,
+ g_
[PATCH v7 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a
device, we assign a new and empty page-table, thus removing all
mappings from the old kernel for the device.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 56 ++---
1 file changed, 48 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8807710..57ae08b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
#include
#include
#include
+#include
#include "irq_remapping.h"
@@ -455,6 +456,8 @@ static int copy_root_entry_table(struct intel_iommu *iommu,
void *ppap);
static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
int g_num_of_iommus);
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -3196,14 +3199,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR "Domain context map for %s failed",
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unmap_device_dma(domain, dev);
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err("Allocating domain for %s failed",
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+#endif
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err("Domain context map for %s failed",
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5691,4 +5710,25 @@ static int intel_iommu_load_translation_tables(struct
dmar_drhd_unit *drhd,
return 0;
}
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+ struct intel_iommu *iommu;
+ struct context_entry *ce;
+ struct iova *iova;
+ u8 bus, devfn;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+
+ iommu = device_to_iommu(dev, , );
+ ce = device_to_context_entry(iommu, bus, devfn);
+ phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(>iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 62 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 66 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index a55b207..d37fd62 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -17,6 +18,11 @@
#include "irq_remapping.h"
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1296,3 +1302,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ memcpy(iommu->ir_table->base,
+ iommu->ir_table->base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu->ir_table->base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu->ir_table->base_old_virt;
+ from = iommu->ir_table->base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 97 +
include/linux/intel-iommu.h | 9 +
2 files changed, 106 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c0bebd6..8a7ad72 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,13 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4843,6 +4850,96 @@ static void __init check_tylersburg_isoch(void)
#ifdef CONFIG_CRASH_DUMP
/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from >> VTD_PAGE_SHIFT;
+ offset = from & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to >> VTD_PAGE_SHIFT;
+ offset = to & (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(&__iommu_mem_list_lock);
+ mapped->mem = virt_mem;
+ list_add_tail(>list, &__iommu_remapped_mem);
+ mutex_unlock(&__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(&__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+ iounmap(mem_entry->mem);
+ list_del(_entry->list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(&__iommu_mem_list_lock);
+ return 0;
+}
+/*
* Interfaces for when a new domain in the crashdump kernel needs some
* values from the panicked kernel's context entries
*
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
#include
#include
#include
+#include
#include
#include
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
extern const struct attribute_group *intel_iommu_groups[];
+#ifdef CONFIG_CR
[PATCH v7 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
malloc new context table and copy old context table to the new one.
malloc new page table and copy old page table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Minor change:
Update the usage of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that copies root entry table.
Use new function to copy old context entry tables and page tables.
Use "unsigned long" for physical address.
Change incorrect aw_shift[4] and a few comments in copy_context_entry().
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 543
1 file changed, 543 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3059b8..d2c19a0 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1 /* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+ u32 first; /* flag: first-time */
+ u32 last; /* flag: last-time */
+ u32 bus;/* last bus number we saw */
+ u32 devfn; /* last devfn we saw */
+ u32 shift; /* last shift we saw */
+ u64 pte;/* Page Table Entry */
+ u64 next_addr; /* next-expected page_addr */
+
+ u64 page_addr; /* page_addr accumulating size */
+ u64 page_size; /* page_size accumulated */
+
+ struct domain_values_entry *dve;/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+ void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+ u32 bus, void *ppap,
+ unsigned long *context_new_p,
+ unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+ int g_num_of_iommus);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -5043,4 +5099,491 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+ u64 pte, struct domain_values_entry *dve,
+ void *parms)
+{
+ struct copy_page_addr_parms *ppap = parms;
+
+ u64 page_size = ((u64)1 << shift); /* page_size */
+ u64 pfn_lo;
[PATCH v7 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua
Signed-off-by: Takao Indoh
---
drivers/iommu/intel-iommu.c | 53 +
include/linux/intel-iommu.h | 5 +
2 files changed, 58 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8a7ad72..f3059b8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,10 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
+
struct iommu_remapped_entry {
struct list_head list;
void __iomem *mem;
@@ -4990,4 +4994,53 @@ static int device_to_domain_id(struct intel_iommu
*iommu, u8 bus, u8 devfn)
return did;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+ memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->root_entry)
+ || (!iommu->root_entry_old_virt)
+ || (!iommu->root_entry_old_phys))
+ return;
+
+ if (index < -1 || index >= ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu->root_entry_old_virt;
+ from = iommu->root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 01/10] iommu/vt-d: Update iommu_attach_domain() and its callers
Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says "no did specified, allocate a new one".
This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.
Bill Sumner:
Original version.
Li, Zhenhua:
Minor change, add change to function __iommu_attach_domain.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 34 --
1 file changed, 20 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1232336..2dc6250 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
}
static int __iommu_attach_domain(struct dmar_domain *domain,
-struct intel_iommu *iommu)
+struct intel_iommu *iommu,
+int domain_number)
{
int num;
unsigned long ndomains;
ndomains = cap_ndoms(iommu->cap);
- num = find_first_zero_bit(iommu->domain_ids, ndomains);
- if (num < ndomains) {
- set_bit(num, iommu->domain_ids);
- iommu->domains[num] = domain;
- } else {
- num = -ENOSPC;
- }
+ if (domain_number < 0) {
+ num = find_first_zero_bit(iommu->domain_ids, ndomains);
+ if (num < ndomains) {
+ set_bit(num, iommu->domain_ids);
+ iommu->domains[num] = domain;
+ } else {
+ num = -ENOSPC;
+ }
+ } else
+ num = domain_number;
return num;
}
static int iommu_attach_domain(struct dmar_domain *domain,
- struct intel_iommu *iommu)
+ struct intel_iommu *iommu,
+ int domain_number)
{
int num;
unsigned long flags;
spin_lock_irqsave(>lock, flags);
- num = __iommu_attach_domain(domain, iommu);
+ num = __iommu_attach_domain(domain, iommu, domain_number);
spin_unlock_irqrestore(>lock, flags);
if (num < 0)
pr_err("IOMMU: no free domain ids\n");
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain
*domain,
if (iommu->domains[num] == domain)
return num;
- return __iommu_attach_domain(domain, iommu);
+ return __iommu_attach_domain(domain, iommu, -1);
}
static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to "no domain_id supplied" */
domain = find_domain(dev);
if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain->id = iommu_attach_domain(domain, iommu);
+ domain->id = iommu_attach_domain(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
return -EFAULT;
for_each_active_iommu(iommu, drhd) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret < 0) {
domain_exit(si_domain);
return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
iommu_enable_translation(iommu);
if (si_domain) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret < 0 || si_domain->id != ret)
goto disable_iommu;
domain_attach_iommu(si_domain, iommu);
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
]: 1. Commented-out "#define DEBUG 1" to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(>iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use "unsigned long" as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He helps testing this patchset. iommu/vt-d: Update iommu_attach_domain() and its callers iommu/vt-d: Items required for kdump iommu/vt-d: Add domain-id functions iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel Signed-off-by: Bill Sumner Signed-off-by: Li, Zhen-Hua Signed-off-by: Takao Indoh Tested-by: Baoquan He --- drivers/iommu/intel-iommu.c | 1050 +-- drivers/iommu/intel_irq_remapping.c | 104 +++- include/linux/intel-iommu.h | 18 + 3 files changed, 1130 insertions(+), 42 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 03/10] iommu/vt-d: Add domain-id functions
Interfaces for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 62 +
1 file changed, 62 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5ce2850..c0bebd6 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -381,6 +381,13 @@ struct domain_values_entry {
2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
};
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu);
+
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4832,3 +4839,58 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n",
vtisochctrl);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Interfaces for when a new domain in the crashdump kernel needs some
+ * values from the panicked kernel's context entries
+ *
+ */
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ list_for_each_entry(dve, _values_list[iommu->seq_id], link)
+ if (dve->did == did)
+ return dve;
+ return NULL;
+}
+
+/* Mark domain-id's from old kernel as in-use on this iommu so that a new
+ * domain-id is allocated in the case where there is a device in the new kernel
+ * that was not in the old kernel -- and therefore a new domain-id is needed.
+ */
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ pr_info("IOMMU:%d Domain ids from panicked kernel:\n", iommu->seq_id);
+
+ list_for_each_entry(dve, _values_list[iommu->seq_id], link) {
+ set_bit(dve->did, iommu->domain_ids);
+ pr_info("DID did:%d(0x%4.4x)\n", dve->did, dve->did);
+ }
+
+ pr_info("\n");
+ return 0;
+}
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
+{
+ int did = -1; /* domain-id returned */
+ struct root_entry *root;
+ struct context_entry *context;
+ unsigned long flags;
+
+ spin_lock_irqsave(>lock, flags);
+ root = >root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context && context_present(context+devfn))
+ did = context_domain_id(context+devfn);
+ spin_unlock_irqrestore(>lock, flags);
+ return did;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 04/10] iommu/vt-d: functions to copy data from old mem
Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.
To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Check if pfn is ram:
if (page_is_ram(pfn))
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 97 +
include/linux/intel-iommu.h | 9 +
2 files changed, 106 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c0bebd6..8a7ad72 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,13 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+struct iommu_remapped_entry {
+ struct list_head list;
+ void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4843,6 +4850,96 @@ static void __init check_tylersburg_isoch(void)
#ifdef CONFIG_CRASH_DUMP
/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = from VTD_PAGE_SHIFT;
+ offset = from (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+ } else{
+
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)from, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(to, virt_mem, size);
+
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+ unsigned long pfn; /* Page Frame Number */
+ size_t csize = (size_t)size;/* Num(bytes to copy) */
+ unsigned long offset; /* Lower 12 bits of to */
+ void __iomem *virt_mem;
+ struct iommu_remapped_entry *mapped;
+
+ pfn = to VTD_PAGE_SHIFT;
+ offset = to (~VTD_PAGE_MASK);
+
+ if (page_is_ram(pfn)) {
+ memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+ } else{
+ mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+ GFP_KERNEL);
+ if (!mapped)
+ return -ENOMEM;
+
+ virt_mem = ioremap_cache((unsigned long)to, size);
+ if (!virt_mem) {
+ kfree(mapped);
+ return -ENOMEM;
+ }
+ memcpy(virt_mem, from, size);
+ mutex_lock(__iommu_mem_list_lock);
+ mapped-mem = virt_mem;
+ list_add_tail(mapped-list, __iommu_remapped_mem);
+ mutex_unlock(__iommu_mem_list_lock);
+ }
+ return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+ struct iommu_remapped_entry *mem_entry, *tmp;
+
+ mutex_lock(__iommu_mem_list_lock);
+ list_for_each_entry_safe(mem_entry, tmp, __iommu_remapped_mem, list) {
+ iounmap(mem_entry-mem);
+ list_del(mem_entry-list);
+ kfree(mem_entry);
+ }
+ mutex_unlock(__iommu_mem_list_lock);
+ return 0;
+}
+/*
* Interfaces for when a new domain in the crashdump kernel needs some
* values from the panicked kernel's context entries
*
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
#include linux/iova.h
#include linux/io.h
#include linux/dma_remapping.h
+#include linux/crash_dump.h
#include asm/cacheflush.h
#include asm/iommu.h
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
extern const struct attribute_group
[PATCH v7 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
malloc new context table and copy old context table to the new one.
malloc new page table and copy old page table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Minor change:
Update the usage of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that copies root entry table.
Use new function to copy old context entry tables and page tables.
Use unsigned long for physical address.
Change incorrect aw_shift[4] and a few comments in copy_context_entry().
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 543
1 file changed, 543 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3059b8..d2c19a0 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1 /* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+ u32 first; /* flag: first-time */
+ u32 last; /* flag: last-time */
+ u32 bus;/* last bus number we saw */
+ u32 devfn; /* last devfn we saw */
+ u32 shift; /* last shift we saw */
+ u64 pte;/* Page Table Entry */
+ u64 next_addr; /* next-expected page_addr */
+
+ u64 page_addr; /* page_addr accumulating size */
+ u64 page_size; /* page_size accumulated */
+
+ struct domain_values_entry *dve;/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+ void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+ u32 bus, void *ppap,
+ unsigned long *context_new_p,
+ unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+ int g_num_of_iommus);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -5043,4 +5099,491 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
__iommu_flush_cache(iommu, to + start, size);
}
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+ u64 pte, struct domain_values_entry *dve,
+ void *parms)
+{
+ struct copy_page_addr_parms *ppap = parms;
+
+ u64 page_size = ((u64)1 shift); /* page_size */
+ u64 pfn_lo
[PATCH v7 05/10] iommu/vt-d: Add functions to load and save old re
Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.
We use the old RTA in dump kernel, and when the iommu-root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.
Li, Zhen-hua:
The functions and logics.
Takao Indoh:
Add __iommu_flush_cache.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com
---
drivers/iommu/intel-iommu.c | 53 +
include/linux/intel-iommu.h | 5 +
2 files changed, 58 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8a7ad72..f3059b8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,10 @@ static int intel_iommu_get_dids_from_old_kernel(struct
intel_iommu *iommu);
static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int
index);
+
struct iommu_remapped_entry {
struct list_head list;
void __iomem *mem;
@@ -4990,4 +4994,53 @@ static int device_to_domain_id(struct intel_iommu
*iommu, u8 bus, u8 devfn)
return did;
}
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+ memcpy(iommu-root_entry, iommu-root_entry_old_virt, PAGE_SIZE);
+
+ __iommu_flush_cache(iommu, iommu-root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+ u8 start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-root_entry)
+ || (!iommu-root_entry_old_virt)
+ || (!iommu-root_entry_old_phys))
+ return;
+
+ if (index -1 || index = ROOT_ENTRY_NR)
+ return;
+
+ if (index == -1) {
+ start = 0;
+ size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+ } else {
+ start = index * sizeof(struct root_entry);
+ size = sizeof(struct root_entry);
+ }
+ to = iommu-root_entry_old_virt;
+ from = iommu-root_entry;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+}
+
#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
spinlock_t lock; /* protect context, domain ids */
struct root_entry *root_entry; /* virtual address */
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *root_entry_old_virt; /* mapped from old root entry */
+ unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
struct iommu_flush flush;
#endif
struct q_inval *qi;/* Queued invalidation info */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 01/10] iommu/vt-d: Update iommu_attach_domain() and its callers
Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says no did specified, allocate a new one.
This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.
Bill Sumner:
Original version.
Li, Zhenhua:
Minor change, add change to function __iommu_attach_domain.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 34 --
1 file changed, 20 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1232336..2dc6250 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
}
static int __iommu_attach_domain(struct dmar_domain *domain,
-struct intel_iommu *iommu)
+struct intel_iommu *iommu,
+int domain_number)
{
int num;
unsigned long ndomains;
ndomains = cap_ndoms(iommu-cap);
- num = find_first_zero_bit(iommu-domain_ids, ndomains);
- if (num ndomains) {
- set_bit(num, iommu-domain_ids);
- iommu-domains[num] = domain;
- } else {
- num = -ENOSPC;
- }
+ if (domain_number 0) {
+ num = find_first_zero_bit(iommu-domain_ids, ndomains);
+ if (num ndomains) {
+ set_bit(num, iommu-domain_ids);
+ iommu-domains[num] = domain;
+ } else {
+ num = -ENOSPC;
+ }
+ } else
+ num = domain_number;
return num;
}
static int iommu_attach_domain(struct dmar_domain *domain,
- struct intel_iommu *iommu)
+ struct intel_iommu *iommu,
+ int domain_number)
{
int num;
unsigned long flags;
spin_lock_irqsave(iommu-lock, flags);
- num = __iommu_attach_domain(domain, iommu);
+ num = __iommu_attach_domain(domain, iommu, domain_number);
spin_unlock_irqrestore(iommu-lock, flags);
if (num 0)
pr_err(IOMMU: no free domain ids\n);
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain
*domain,
if (iommu-domains[num] == domain)
return num;
- return __iommu_attach_domain(domain, iommu);
+ return __iommu_attach_domain(domain, iommu, -1);
}
static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to no domain_id supplied */
domain = find_domain(dev);
if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain-id = iommu_attach_domain(domain, iommu);
+ domain-id = iommu_attach_domain(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
return -EFAULT;
for_each_active_iommu(iommu, drhd) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret 0) {
domain_exit(si_domain);
return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
iommu_enable_translation(iommu);
if (si_domain) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret 0 || si_domain-id != ret)
goto disable_iommu;
domain_attach_iommu(si_domain, iommu);
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
#define DEBUG 1 to eliminate debug messages. 2. Updated the comments about changes in each version. 3. Fixed: one-line added to Copy-Translations patch to initialize the iovad struct as recommended by Baoquan He [b...@redhat.com] init_iova_domain(domain-iovad, DMA_32BIT_PFN); Changelog[v2]: The following series implements a fix for: A kdump problem about DMA that has been discussed for a long time. That is, when a kernel panics and boots into the kdump kernel, DMA started by the panicked kernel is not stopped before the kdump kernel is booted and the kdump kernel disables the IOMMU while this DMA continues. This causes the IOMMU to stop translating the DMA addresses as IOVAs and begin to treat them as physical memory addresses -- which causes the DMA to either: (1) generate DMAR errors or (2) generate PCI SERR errors or (3) transfer data to or from incorrect areas of memory. Often this causes the dump to fail. Changelog[v1]: The original version. Changed in this version: 1. Do not disable and re-enable traslation and interrupt remapping. 2. Use old root entry table. 3. Use old interrupt remapping table. 4. Use unsigned long as physical address. 5. Use intel_unmap to unmap the old dma; Baoquan He b...@redhat.com helps testing this patchset. iommu/vt-d: Update iommu_attach_domain() and its callers iommu/vt-d: Items required for kdump iommu/vt-d: Add domain-id functions iommu/vt-d: functions to copy data from old mem iommu/vt-d: Add functions to load and save old re iommu/vt-d: datatypes and functions used for kdump iommu/vt-d: enable kdump support in iommu module iommu/vt-d: assign new page table for dma_map iommu/vt-d: Copy functions for irte iommu/vt-d: Use old irte in kdump kernel Signed-off-by: Bill Sumner billsumnerli...@gmail.com Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com Signed-off-by: Takao Indoh indou.ta...@jp.fujitsu.com Tested-by: Baoquan He b...@redhat.com --- drivers/iommu/intel-iommu.c | 1050 +-- drivers/iommu/intel_irq_remapping.c | 104 +++- include/linux/intel-iommu.h | 18 + 3 files changed, 1130 insertions(+), 42 deletions(-) -- 2.0.0-rc0 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 03/10] iommu/vt-d: Add domain-id functions
Interfaces for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 62 +
1 file changed, 62 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5ce2850..c0bebd6 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -381,6 +381,13 @@ struct domain_values_entry {
2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
};
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu);
+
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -4832,3 +4839,58 @@ static void __init check_tylersburg_isoch(void)
printk(KERN_WARNING DMAR: Recommended TLB entries for ISOCH unit is
16; your BIOS set %d\n,
vtisochctrl);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Interfaces for when a new domain in the crashdump kernel needs some
+ * values from the panicked kernel's context entries
+ *
+ */
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+ int did, struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ list_for_each_entry(dve, domain_values_list[iommu-seq_id], link)
+ if (dve-did == did)
+ return dve;
+ return NULL;
+}
+
+/* Mark domain-id's from old kernel as in-use on this iommu so that a new
+ * domain-id is allocated in the case where there is a device in the new kernel
+ * that was not in the old kernel -- and therefore a new domain-id is needed.
+ */
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu)
+{
+ struct domain_values_entry *dve;/* iterator */
+
+ pr_info(IOMMU:%d Domain ids from panicked kernel:\n, iommu-seq_id);
+
+ list_for_each_entry(dve, domain_values_list[iommu-seq_id], link) {
+ set_bit(dve-did, iommu-domain_ids);
+ pr_info(DID did:%d(0x%4.4x)\n, dve-did, dve-did);
+ }
+
+ pr_info(\n);
+ return 0;
+}
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
+{
+ int did = -1; /* domain-id returned */
+ struct root_entry *root;
+ struct context_entry *context;
+ unsigned long flags;
+
+ spin_lock_irqsave(iommu-lock, flags);
+ root = iommu-root_entry[bus];
+ context = get_context_addr_from_root(root);
+ if (context context_present(context+devfn))
+ did = context_domain_id(context+devfn);
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return did;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 07/10] iommu/vt-d: enable kdump support in iommu module
Modify the operation of the following functions when called during crash dump:
device_to_domain_id
get_domain_for_dev
init_dmars
intel_iommu_init
Bill Sumner:
Original version.
Zhenhua:
Minor change,
The name of new calling functions.
Do not disable and re-enable TE in kdump kernel.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 135 +++-
1 file changed, 120 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d2c19a0..8807710 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -907,6 +907,11 @@ static struct context_entry *
device_to_context_entry(struct intel_iommu *iommu,
set_root_value(root, phy_addr);
set_root_present(root);
__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_root_entry(iommu, bus);
+#endif
}
spin_unlock_irqrestore(iommu-lock, flags);
return context[devfn];
@@ -958,7 +963,8 @@ static void free_context_table(struct intel_iommu *iommu)
spin_lock_irqsave(iommu-lock, flags);
if (!iommu-root_entry) {
- goto out;
+ spin_unlock_irqrestore(iommu-lock, flags);
+ return;
}
for (i = 0; i ROOT_ENTRY_NR; i++) {
root = iommu-root_entry[i];
@@ -966,10 +972,23 @@ static void free_context_table(struct intel_iommu *iommu)
if (context)
free_pgtable_page(context);
}
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ iommu-root_entry_old_phys = 0;
+ root = iommu-root_entry_old_virt;
+ iommu-root_entry_old_virt = NULL;
+ }
+#endif
free_pgtable_page(iommu-root_entry);
iommu-root_entry = NULL;
-out:
+
spin_unlock_irqrestore(iommu-lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ iounmap(root);
+#endif
}
static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2381,6 +2400,9 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
unsigned long flags;
u8 bus, devfn;
int did = -1; /* Default to no domain_id supplied */
+#ifdef CONFIG_CRASH_DUMP
+ struct domain_values_entry *dve = NULL;
+#endif /* CONFIG_CRASH_DUMP */
domain = find_domain(dev);
if (domain)
@@ -2414,6 +2436,24 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /*
+* if this device had a did in the old kernel
+* use its values instead of generating new ones
+*/
+ did = device_to_domain_id(iommu, bus, devfn);
+ if (did 0 || (did == 0 !cap_caching_mode(iommu-cap)))
+ dve = intel_iommu_did_to_domain_values_entry(did,
+ iommu);
+ if (dve)
+ gaw = dve-gaw;
+ else
+ did = -1;
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
domain-id = iommu_attach_domain(domain, iommu, did);
if (domain-id 0) {
free_domain_mem(domain);
@@ -2425,6 +2465,18 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
return NULL;
}
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel() dve) {
+
+ if (domain-pgd)
+ free_pgtable_page(domain-pgd);
+
+ domain-pgd = dve-pgd;
+
+ copy_reserved_iova(dve-iovad, domain-iovad);
+ }
+#endif /* CONFIG_CRASH_DUMP */
+
/* register PCI DMA alias device */
if (dev_is_pci(dev)) {
tmp = dmar_insert_dev_info(iommu, PCI_BUS_NUM(dma_alias),
@@ -2948,14 +3000,35 @@ static int __init init_dmars(void)
if (ret)
goto free_iommu;
- /*
-* TBD:
-* we could share the same root context tables
-* among all IOMMU's. Need to Split it later.
-*/
- ret = iommu_alloc_root_entry(iommu);
- if (ret)
- goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ pr_info(IOMMU Copying translate tables from panicked
kernel\n);
+ ret = intel_iommu_load_translation_tables(drhd,
+ g_num_of_iommus);
+ if (ret
[PATCH v7 02/10] iommu/vt-d: Items required for kdump
Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Signed-off-by: Bill Sumner billsumnerli...@gmail.com
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 70 +
1 file changed, 70 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2dc6250..5ce2850 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include linux/pci-ats.h
#include linux/memblock.h
#include linux/dma-contiguous.h
+#include linux/crash_dump.h
#include asm/irq_remapping.h
#include asm/cacheflush.h
#include asm/iommu.h
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root-val VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context-lo 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c-lo 1) 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c-lo 2) 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c-lo VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c-hi 0) 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c-hi 8) 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context-lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *portions of the iova address ranges that are different
+ *from the iova address ranges that were being used by the old kernel
+ *at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+ struct list_head link; /* link entries into a list */
+ struct iova_domain iovad; /* iova's that belong to this domain */
+ struct dma_pte *pgd; /* virtual address */
+ intdid; /* domain id */
+ intgaw; /* max guest address width */
+ intiommu_superpage; /* Level of superpages supported:
+ 0 == 4KiB (no superpages), 1 == 2MiB,
+ 2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 10/10] iommu/vt-d: Use old irte in kdump kernel
Fix the intr-remapping fault.
[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear
Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 42 -
1 file changed, 37 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index d37fd62..58356cb 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -198,6 +198,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
set_64bit(irte-low, irte_modified-low);
set_64bit(irte-high, irte_modified-high);
+
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, index);
+#endif
__iommu_flush_cache(iommu, irte, sizeof(*irte));
rc = qi_flush_iec(iommu, index, 0);
@@ -259,6 +264,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
bitmap_release_region(iommu-ir_table-bitmap, index,
irq_iommu-irte_mask);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel())
+ __iommu_update_old_irte(iommu, -1);
+#endif
+
return qi_flush_iec(iommu, index, irq_iommu-irte_mask);
}
@@ -640,11 +650,20 @@ static int __init intel_enable_irq_remapping(void)
*/
dmar_fault(-1, iommu);
- /*
-* Disable intr remapping and queued invalidation, if already
-* enabled prior to OS handover.
-*/
- iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ /* Do notdisable irq and then re-enable again. */
+ } else {
+#endif
+ /*
+* Disable intr remapping and queued invalidation,
+* if already enabled prior to OS handover.
+*/
+ iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+ }
+#endif
dmar_disable_qi(iommu);
}
@@ -687,7 +706,20 @@ static int __init intel_enable_irq_remapping(void)
if (intel_setup_irq_remapping(iommu))
goto error;
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unsigned long long q;
+
+ q = dmar_readq(iommu-reg + DMAR_IRTA_REG);
+ iommu-ir_table-base_old_phys = q VTD_PAGE_MASK;
+ iommu-ir_table-base_old_virt = ioremap_cache(
+ iommu-ir_table-base_old_phys,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+ __iommu_load_old_irte(iommu);
+ } else
+#endif
iommu_set_irq_remapping(iommu, eim);
+
setup = 1;
}
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 08/10] iommu/vt-d: assign new page table for dma_map
When a device driver issues the first dma_map command for a
device, we assign a new and empty page-table, thus removing all
mappings from the old kernel for the device.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel-iommu.c | 56 ++---
1 file changed, 48 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8807710..57ae08b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
#include asm/irq_remapping.h
#include asm/cacheflush.h
#include asm/iommu.h
+#include linux/dma-mapping.h
#include irq_remapping.h
@@ -455,6 +456,8 @@ static int copy_root_entry_table(struct intel_iommu *iommu,
void *ppap);
static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
int g_num_of_iommus);
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -3196,14 +3199,30 @@ static struct dmar_domain
*__get_valid_domain_for_dev(struct device *dev)
return NULL;
}
- /* make sure context mapping is ok */
- if (unlikely(!domain_context_mapped(dev))) {
- ret = domain_context_mapping(domain, dev,
CONTEXT_TT_MULTI_LEVEL);
- if (ret) {
- printk(KERN_ERR Domain context map for %s failed,
- dev_name(dev));
- return NULL;
- }
+ /* if in kdump kernel, we need to unmap the mapped dma pages,
+* detach this device first.
+*/
+ if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+ if (is_kdump_kernel()) {
+ unmap_device_dma(domain, dev);
+ domain = get_domain_for_dev(dev,
+ DEFAULT_DOMAIN_ADDRESS_WIDTH);
+ if (!domain) {
+ pr_err(Allocating domain for %s failed,
+ dev_name(dev));
+ return NULL;
+ }
+ } else
+#endif
+ return domain;
+ }
+
+ ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+ if (ret) {
+ pr_err(Domain context map for %s failed,
+ dev_name(dev));
+ return NULL;
}
return domain;
@@ -5691,4 +5710,25 @@ static int intel_iommu_load_translation_tables(struct
dmar_drhd_unit *drhd,
return 0;
}
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+ struct intel_iommu *iommu;
+ struct context_entry *ce;
+ struct iova *iova;
+ u8 bus, devfn;
+ phys_addr_t phys_addr;
+ dma_addr_t dev_addr;
+
+ iommu = device_to_iommu(dev, bus, devfn);
+ ce = device_to_context_entry(iommu, bus, devfn);
+ phys_addr = context_address_root(ce) VTD_PAGE_SHIFT;
+ dev_addr = phys_to_dma(dev, phys_addr);
+
+ iova = find_iova(domain-iovad, IOVA_PFN(dev_addr));
+ if (iova)
+ intel_unmap(dev, dev_addr);
+
+ domain_remove_one_dev_info(domain, dev);
+}
+
#endif /* CONFIG_CRASH_DUMP */
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH v7 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua zhen-h...@hp.com
---
drivers/iommu/intel_irq_remapping.c | 62 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 66 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index a55b207..d37fd62 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include linux/irq.h
#include linux/intel-iommu.h
#include linux/acpi.h
+#include linux/crash_dump.h
#include asm/io_apic.h
#include asm/smp.h
#include asm/cpu.h
@@ -17,6 +18,11 @@
#include irq_remapping.h
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1296,3 +1302,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ memcpy(iommu-ir_table-base,
+ iommu-ir_table-base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ __iommu_flush_cache(iommu, iommu-ir_table-base,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu-ir_table)
+ || (!iommu-ir_table-base)
+ || (!iommu-ir_table-base_old_phys)
+ || (!iommu-ir_table-base_old_virt))
+ return -1;
+
+ if (index -1 || index = INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu-ir_table-base_old_virt;
+ from = iommu-ir_table-base;
+ memcpy(to + start, from + start, size);
+
+ __iommu_flush_cache(iommu, to + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH 02/10] iommu/vt-d: Items required for kdump
Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.
Bill Sumner:
Original version;
Li, Zhenhua:
Changed the name of new functions, make them consistent with current
context get/set functions.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 70 +
1 file changed, 70 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2dc6250..5ce2850 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
NULL);
}
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+ return root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
/*
* low 64 bits:
* 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry
*context)
{
return (context->lo & 1);
}
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+ return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+ return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+ return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+ return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+ return((c->hi >> 8) & 0x);
+}
+
static inline void context_set_present(struct context_entry *context)
{
context->lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
return !((unsigned long)pte & ~VTD_PAGE_MASK);
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *portions of the iova address ranges that are different
+ *from the iova address ranges that were being used by the old kernel
+ *at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+ struct list_head link; /* link entries into a list */
+ struct iova_domain iovad; /* iova's that belong to this domain */
+ struct dma_pte *pgd; /* virtual address */
+ intdid; /* domain id */
+ intgaw; /* max guest address width */
+ intiommu_superpage; /* Level of superpages supported:
+ 0 == 4KiB (no superpages), 1 == 2MiB,
+ 2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
/*
* This domain is a statically identity mapping domain.
* 1. This domain creats a static 1:1 mapping to all usable memory.
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH 01/10] iommu/vt-d: Update iommu_attach_domain() and its callers
Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says "no did specified, allocate a new one".
This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.
Bill Sumner:
Original version.
Li, Zhenhua:
Minor change, add change to function __iommu_attach_domain.
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 34 --
1 file changed, 20 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1232336..2dc6250 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
}
static int __iommu_attach_domain(struct dmar_domain *domain,
-struct intel_iommu *iommu)
+struct intel_iommu *iommu,
+int domain_number)
{
int num;
unsigned long ndomains;
ndomains = cap_ndoms(iommu->cap);
- num = find_first_zero_bit(iommu->domain_ids, ndomains);
- if (num < ndomains) {
- set_bit(num, iommu->domain_ids);
- iommu->domains[num] = domain;
- } else {
- num = -ENOSPC;
- }
+ if (domain_number < 0) {
+ num = find_first_zero_bit(iommu->domain_ids, ndomains);
+ if (num < ndomains) {
+ set_bit(num, iommu->domain_ids);
+ iommu->domains[num] = domain;
+ } else {
+ num = -ENOSPC;
+ }
+ } else
+ num = domain_number;
return num;
}
static int iommu_attach_domain(struct dmar_domain *domain,
- struct intel_iommu *iommu)
+ struct intel_iommu *iommu,
+ int domain_number)
{
int num;
unsigned long flags;
spin_lock_irqsave(>lock, flags);
- num = __iommu_attach_domain(domain, iommu);
+ num = __iommu_attach_domain(domain, iommu, domain_number);
spin_unlock_irqrestore(>lock, flags);
if (num < 0)
pr_err("IOMMU: no free domain ids\n");
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain
*domain,
if (iommu->domains[num] == domain)
return num;
- return __iommu_attach_domain(domain, iommu);
+ return __iommu_attach_domain(domain, iommu, -1);
}
static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
u16 dma_alias;
unsigned long flags;
u8 bus, devfn;
+ int did = -1; /* Default to "no domain_id supplied" */
domain = find_domain(dev);
if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct
device *dev, int gaw)
domain = alloc_domain(0);
if (!domain)
return NULL;
- domain->id = iommu_attach_domain(domain, iommu);
+ domain->id = iommu_attach_domain(domain, iommu, did);
if (domain->id < 0) {
free_domain_mem(domain);
return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
return -EFAULT;
for_each_active_iommu(iommu, drhd) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret < 0) {
domain_exit(si_domain);
return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
iommu_enable_translation(iommu);
if (si_domain) {
- ret = iommu_attach_domain(si_domain, iommu);
+ ret = iommu_attach_domain(si_domain, iommu, -1);
if (ret < 0 || si_domain->id != ret)
goto disable_iommu;
domain_attach_iommu(si_domain, iommu);
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH 06/10] iommu/vt-d: datatypes and functions used for kdump
Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.
Functions:
malloc new context table and copy old context table to the new one.
malloc new page table and copy old page table to the new one.
Bill Sumner:
Original version, the creation of the data types and functions.
Li, Zhenhua:
Minor change:
Update the usage of context_get_* and context_put*, use context_*
and context_set_* for replacement.
Update the name of the function that copies root entry table.
Use new function to copy old context entry tables and page tables.
Use "unsigned long" for physical address.
Change incorrect aw_shift[4] and a few comments in copy_context_entry().
Signed-off-by: Bill Sumner
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel-iommu.c | 540
1 file changed, 540 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 126294db..f9849cb 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
static LIST_HEAD(__iommu_remapped_mem);
static DEFINE_MUTEX(__iommu_mem_list_lock);
+/*
+ * Copy iommu translation tables from old kernel into new kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ *
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1 /* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+ u32 first; /* flag: first-time */
+ u32 last; /* flag: last-time */
+ u32 bus;/* last bus number we saw */
+ u32 devfn; /* last devfn we saw */
+ u32 shift; /* last shift we saw */
+ u64 pte;/* Page Table Entry */
+ u64 next_addr; /* next-expected page_addr */
+
+ u64 page_addr; /* page_addr accumulating size */
+ u64 page_size; /* page_size accumulated */
+
+ struct domain_values_entry *dve;/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+ void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+ u32 bus, void *ppap,
+ unsigned long *context_new_p,
+ unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+ int g_num_of_iommus);
+
#endif /* CONFIG_CRASH_DUMP */
/*
@@ -5039,4 +5095,488 @@ static void __iommu_update_old_root_entry(struct
intel_iommu *iommu, int index)
memcpy(to + start, from + start, size);
}
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+ u64 pte, struct domain_values_entry *dve,
+ void *parms)
+{
+ struct copy_page_addr_parms *ppap = parms;
+
+ u64 page_size = ((u64)1 << shift); /* page_size */
+ u64 pfn_lo; /* Fo
[PATCH 09/10] iommu/vt-d: Copy functions for irte
Functions to copy the irte data from the old kernel into the kdump kernel.
Signed-off-by: Li, Zhen-Hua
---
drivers/iommu/intel_irq_remapping.c | 57 +
include/linux/intel-iommu.h | 4 +++
2 files changed, 61 insertions(+)
diff --git a/drivers/iommu/intel_irq_remapping.c
b/drivers/iommu/intel_irq_remapping.c
index a55b207..13f2034 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -17,6 +18,11 @@
#include "irq_remapping.h"
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
struct ioapic_scope {
struct intel_iommu *iommu;
unsigned int id;
@@ -1296,3 +1302,54 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool
insert)
return ret;
}
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ memcpy(iommu->ir_table->base,
+ iommu->ir_table->base_old_virt,
+ INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+ return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+ int start;
+ unsigned long size;
+ void __iomem *to;
+ void *from;
+
+ if ((!iommu)
+ || (!iommu->ir_table)
+ || (!iommu->ir_table->base)
+ || (!iommu->ir_table->base_old_phys)
+ || (!iommu->ir_table->base_old_virt))
+ return -1;
+
+ if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+ return -1;
+
+ if (index == -1) {
+ start = 0;
+ size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+ } else {
+ start = index * sizeof(struct irte);
+ size = sizeof(struct irte);
+ }
+
+ to = iommu->ir_table->base_old_virt;
+ from = iommu->ir_table->base;
+ memcpy(to + start, from + start, size);
+
+ return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
struct ir_table {
struct irte *base;
unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+ void __iomem *base_old_virt;
+ unsigned long base_old_phys;
+#endif
};
#endif
--
2.0.0-rc0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
