Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-05 Thread Yisheng Xie 


On 2017/11/3 17:37, Jean-Philippe Brucker wrote:
> On 03/11/17 05:45, Yisheng Xie wrote:
>> Hi Jean,
>>
>> On 2017/11/3 1:02, Shameerali Kolothum Thodi wrote:
>>>
>>>
>>>> -Original Message-
>>>> From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com]
>>>> Sent: Thursday, November 02, 2017 3:52 PM
>>>> To: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>
>>>> Cc: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; linux-
>>>> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
>>>> foundation.org; Mark Rutland <mark.rutl...@arm.com>; xieyisheng (A)
>>>> <xieyishe...@huawei.com>; Gabriele Paoloni
>>>> <gabriele.paol...@huawei.com>; Catalin Marinas
>>>> <catalin.mari...@arm.com>; Will Deacon <will.dea...@arm.com>;
>>>> ok...@codeaurora.org; yi.l@intel.com; Lorenzo Pieralisi
>>>> <lorenzo.pieral...@arm.com>; ashok@intel.com; t...@semihalf.com;
>>>> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
>>>> jacob.jun@linux.intel.com; alex.william...@redhat.com;
>>>> robh...@kernel.org; Leizhen (ThunderTown) <thunder.leiz...@huawei.com>;
>>>> bhelg...@google.com; dw...@infradead.org; liubo (CU)
>>>> <liub...@huawei.com>; r...@rjwysocki.net; robdcl...@gmail.com;
>>>> hanjun@linaro.org; Sudeep Holla <sudeep.ho...@arm.com>; Robin
>>>> Murphy <robin.mur...@arm.com>; nwatt...@codeaurora.org; Linuxarm
>>>> <linux...@huawei.com>
>>>> Subject: Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
>>>> Substream IDs
>>>>
>>>> Hi Shameer,
>>>>
>>>> On Thu, Nov 02, 2017 at 12:49:32PM +, Shameerali Kolothum Thodi wrote:
>>>>> We had a go with this series on HiSIlicon D05 platform which doesn't have
>>>>> support for ssids/ATS/PRI, to make sure it generally works.
>>>>>
>>>>> But observed the below crash on boot,
>>>>>
>>>>> [   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883
>>>> __alloc_pages_nodemask+0x19c/0xc48
>>>>> [   16.026797] Modules linked in:
>>>>> [   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 4.14.0-rc1-
>>>> 159539-ge42aca3 #236
>>>>> [...]
>>>>> [   16.068206] Workqueue: events deferred_probe_work_func
>>>>> [   16.078557] task: 8017d38a task.stack: 0b198000
>>>>> [   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
>>>>> [   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
>>>>> [   16.469220] [] __alloc_pages_nodemask+0x19c/0xc48
>>>>> [   16.481854] [] alloc_pages_current+0x80/0xcc
>>>>> [   16.493607] [] __get_free_pages+0xc/0x38
>>>>> [   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
>>>>> [   16.517117] [] __dma_alloc+0x110/0x204
>>>>> [   16.527820] [] dmam_alloc_coherent+0x88/0xf0
>>>>> [   16.539575] []
>>>> arm_smmu_domain_finalise_s1+0x60/0x248
>>>>> [   16.552909] [] arm_smmu_attach_dev+0x264/0x300
>>>>> [   16.565013] [] __iommu_attach_device+0x48/0x5c
>>>>> [   16.577117] [] iommu_group_add_device+0x144/0x3a4
>>>>> [   16.589746] [] iommu_group_get_for_dev+0x70/0xf8
>>>>> [   16.602201] [] arm_smmu_add_device+0x1a4/0x418
>>>>> [   16.614308] [] iort_iommu_configure+0xf0/0x16c
>>>>> [   16.626416] [] acpi_dma_configure+0x30/0x70
>>>>> [   16.637994] [] dma_configure+0xa8/0xd4
>>>>> [   16.648695] [] driver_probe_device+0x1a4/0x2dc
>>>>> [   16.673081] [] bus_for_each_drv+0x54/0x94
>>>>> [   16.684307] [] __device_attach+0xc4/0x12c
>>>>> [   16.695533] [] device_initial_probe+0x10/0x18
>>>>> [   16.707462] [] bus_probe_device+0x90/0x98
>>>>>
>>>>> After a bit of debug it looks like on platforms where ssid is not 
>>>>> supported,
>>>>> s1_cfg.num_contexts is set to zero and it eventually results in this crash
>>>>> in,
>>>>> arm_smmu_domain_finalise_s1() -->arm_smmu_alloc_cd_tables()-->
>>>>> arm_smmu_alloc_cd_leaf_table() as num_leaf_entries is zero.
>>>>>
>>>>> With the below fix, it works on D05 now,
>>>>>
>>>>> diff --git a/drivers/i

RE: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-03 Thread Shameerali Kolothum Thodi 


> -Original Message-
> From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com]
> Sent: Friday, November 03, 2017 9:37 AM
> To: xieyisheng (A) <xieyishe...@huawei.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.th...@huawei.com>
> Cc: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; linux-
> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
> foundation.org; Mark Rutland <mark.rutl...@arm.com>; Gabriele Paoloni
> <gabriele.paol...@huawei.com>; Catalin Marinas
> <catalin.mari...@arm.com>; Will Deacon <will.dea...@arm.com>;
> ok...@codeaurora.org; yi.l@intel.com; Lorenzo Pieralisi
> <lorenzo.pieral...@arm.com>; ashok@intel.com; t...@semihalf.com;
> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
> jacob.jun@linux.intel.com; alex.william...@redhat.com;
> robh...@kernel.org; Leizhen (ThunderTown) <thunder.leiz...@huawei.com>;
> bhelg...@google.com; dw...@infradead.org; liubo (CU)
> <liub...@huawei.com>; r...@rjwysocki.net; robdcl...@gmail.com;
> hanjun@linaro.org; Sudeep Holla <sudeep.ho...@arm.com>; Robin
> Murphy <robin.mur...@arm.com>; nwatt...@codeaurora.org; Linuxarm
> <linux...@huawei.com>
> Subject: Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
> Substream IDs
> 
> On 03/11/17 05:45, Yisheng Xie wrote:
> > Hi Jean,
> >
> > On 2017/11/3 1:02, Shameerali Kolothum Thodi wrote:
> >>
> >>
> >>> -Original Message-
> >>> From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com]
> >>> Sent: Thursday, November 02, 2017 3:52 PM
> >>> To: Shameerali Kolothum Thodi
> <shameerali.kolothum.th...@huawei.com>
> >>> Cc: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; 
> >>> linux-
> >>> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
> >>> foundation.org; Mark Rutland <mark.rutl...@arm.com>; xieyisheng (A)
> >>> <xieyishe...@huawei.com>; Gabriele Paoloni
> >>> <gabriele.paol...@huawei.com>; Catalin Marinas
> >>> <catalin.mari...@arm.com>; Will Deacon <will.dea...@arm.com>;
> >>> ok...@codeaurora.org; yi.l@intel.com; Lorenzo Pieralisi
> >>> <lorenzo.pieral...@arm.com>; ashok@intel.com; t...@semihalf.com;
> >>> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
> >>> jacob.jun@linux.intel.com; alex.william...@redhat.com;
> >>> robh...@kernel.org; Leizhen (ThunderTown)
> <thunder.leiz...@huawei.com>;
> >>> bhelg...@google.com; dw...@infradead.org; liubo (CU)
> >>> <liub...@huawei.com>; r...@rjwysocki.net; robdcl...@gmail.com;
> >>> hanjun@linaro.org; Sudeep Holla <sudeep.ho...@arm.com>; Robin
> >>> Murphy <robin.mur...@arm.com>; nwatt...@codeaurora.org; Linuxarm
> >>> <linux...@huawei.com>
> >>> Subject: Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
> >>> Substream IDs
> >>>
> >>> Hi Shameer,
> >>>
> >>> On Thu, Nov 02, 2017 at 12:49:32PM +, Shameerali Kolothum Thodi
> wrote:
> >>>> We had a go with this series on HiSIlicon D05 platform which doesn't have
> >>>> support for ssids/ATS/PRI, to make sure it generally works.
> >>>>
> >>>> But observed the below crash on boot,
> >>>>
> >>>> [   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883
> >>> __alloc_pages_nodemask+0x19c/0xc48
> >>>> [   16.026797] Modules linked in:
> >>>> [   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 4.14.0-
> rc1-
> >>> 159539-ge42aca3 #236
> >>>> [...]
> >>>> [   16.068206] Workqueue: events deferred_probe_work_func
> >>>> [   16.078557] task: 8017d38a task.stack: 0b198000
> >>>> [   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
> >>>> [   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
> >>>> [   16.469220] []
> __alloc_pages_nodemask+0x19c/0xc48
> >>>> [   16.481854] [] alloc_pages_current+0x80/0xcc
> >>>> [   16.493607] [] __get_free_pages+0xc/0x38
> >>>> [   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
> >>>> [   16.517117] [] __dma_alloc+0x110/0x204
> >>>> [   16.527820] [] dmam_alloc_coherent+0x88/0xf0
> >>>> [   16.539575] []
> >>> arm_smmu_domain_finalise_s1

Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-03 Thread Jean-Philippe Brucker 
On 03/11/17 05:45, Yisheng Xie wrote:
> Hi Jean,
> 
> On 2017/11/3 1:02, Shameerali Kolothum Thodi wrote:
>>
>>
>>> -Original Message-
>>> From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com]
>>> Sent: Thursday, November 02, 2017 3:52 PM
>>> To: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>
>>> Cc: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; linux-
>>> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
>>> foundation.org; Mark Rutland <mark.rutl...@arm.com>; xieyisheng (A)
>>> <xieyishe...@huawei.com>; Gabriele Paoloni
>>> <gabriele.paol...@huawei.com>; Catalin Marinas
>>> <catalin.mari...@arm.com>; Will Deacon <will.dea...@arm.com>;
>>> ok...@codeaurora.org; yi.l@intel.com; Lorenzo Pieralisi
>>> <lorenzo.pieral...@arm.com>; ashok@intel.com; t...@semihalf.com;
>>> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
>>> jacob.jun@linux.intel.com; alex.william...@redhat.com;
>>> robh...@kernel.org; Leizhen (ThunderTown) <thunder.leiz...@huawei.com>;
>>> bhelg...@google.com; dw...@infradead.org; liubo (CU)
>>> <liub...@huawei.com>; r...@rjwysocki.net; robdcl...@gmail.com;
>>> hanjun....@linaro.org; Sudeep Holla <sudeep.ho...@arm.com>; Robin
>>> Murphy <robin.mur...@arm.com>; nwatt...@codeaurora.org; Linuxarm
>>> <linux...@huawei.com>
>>> Subject: Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
>>> Substream IDs
>>>
>>> Hi Shameer,
>>>
>>> On Thu, Nov 02, 2017 at 12:49:32PM +, Shameerali Kolothum Thodi wrote:
>>>> We had a go with this series on HiSIlicon D05 platform which doesn't have
>>>> support for ssids/ATS/PRI, to make sure it generally works.
>>>>
>>>> But observed the below crash on boot,
>>>>
>>>> [   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883
>>> __alloc_pages_nodemask+0x19c/0xc48
>>>> [   16.026797] Modules linked in:
>>>> [   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 4.14.0-rc1-
>>> 159539-ge42aca3 #236
>>>> [...]
>>>> [   16.068206] Workqueue: events deferred_probe_work_func
>>>> [   16.078557] task: 8017d38a task.stack: 0b198000
>>>> [   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
>>>> [   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
>>>> [   16.469220] [] __alloc_pages_nodemask+0x19c/0xc48
>>>> [   16.481854] [] alloc_pages_current+0x80/0xcc
>>>> [   16.493607] [] __get_free_pages+0xc/0x38
>>>> [   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
>>>> [   16.517117] [] __dma_alloc+0x110/0x204
>>>> [   16.527820] [] dmam_alloc_coherent+0x88/0xf0
>>>> [   16.539575] []
>>> arm_smmu_domain_finalise_s1+0x60/0x248
>>>> [   16.552909] [] arm_smmu_attach_dev+0x264/0x300
>>>> [   16.565013] [] __iommu_attach_device+0x48/0x5c
>>>> [   16.577117] [] iommu_group_add_device+0x144/0x3a4
>>>> [   16.589746] [] iommu_group_get_for_dev+0x70/0xf8
>>>> [   16.602201] [] arm_smmu_add_device+0x1a4/0x418
>>>> [   16.614308] [] iort_iommu_configure+0xf0/0x16c
>>>> [   16.626416] [] acpi_dma_configure+0x30/0x70
>>>> [   16.637994] [] dma_configure+0xa8/0xd4
>>>> [   16.648695] [] driver_probe_device+0x1a4/0x2dc
>>>> [   16.673081] [] bus_for_each_drv+0x54/0x94
>>>> [   16.684307] [] __device_attach+0xc4/0x12c
>>>> [   16.695533] [] device_initial_probe+0x10/0x18
>>>> [   16.707462] [] bus_probe_device+0x90/0x98
>>>>
>>>> After a bit of debug it looks like on platforms where ssid is not 
>>>> supported,
>>>> s1_cfg.num_contexts is set to zero and it eventually results in this crash
>>>> in,
>>>> arm_smmu_domain_finalise_s1() -->arm_smmu_alloc_cd_tables()-->
>>>> arm_smmu_alloc_cd_leaf_table() as num_leaf_entries is zero.
>>>>
>>>> With the below fix, it works on D05 now,
>>>>
>>>> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
>>>> index 8ad90e2..51f5821 100644
>>>> --- a/drivers/iommu/arm-smmu-v3.c
>>>> +++ b/drivers/iommu/arm-smmu-v3.c
>>>> @@ -2433,7 +2433,10 @@ static int arm_smmu_domain_finalise(struct
>>> iommu_domain *domain,
>>>> domain->min_pa

Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-02 Thread Yisheng Xie 
Hi Jean,

On 2017/11/3 1:02, Shameerali Kolothum Thodi wrote:
> 
> 
>> -Original Message-
>> From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com]
>> Sent: Thursday, November 02, 2017 3:52 PM
>> To: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>
>> Cc: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; linux-
>> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
>> foundation.org; Mark Rutland <mark.rutl...@arm.com>; xieyisheng (A)
>> <xieyishe...@huawei.com>; Gabriele Paoloni
>> <gabriele.paol...@huawei.com>; Catalin Marinas
>> <catalin.mari...@arm.com>; Will Deacon <will.dea...@arm.com>;
>> ok...@codeaurora.org; yi.l@intel.com; Lorenzo Pieralisi
>> <lorenzo.pieral...@arm.com>; ashok@intel.com; t...@semihalf.com;
>> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
>> jacob.jun@linux.intel.com; alex.william...@redhat.com;
>> robh...@kernel.org; Leizhen (ThunderTown) <thunder.leiz...@huawei.com>;
>> bhelg...@google.com; dw...@infradead.org; liubo (CU)
>> <liub...@huawei.com>; r...@rjwysocki.net; robdcl...@gmail.com;
>> hanjun....@linaro.org; Sudeep Holla <sudeep.ho...@arm.com>; Robin
>> Murphy <robin.mur...@arm.com>; nwatt...@codeaurora.org; Linuxarm
>> <linux...@huawei.com>
>> Subject: Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
>> Substream IDs
>>
>> Hi Shameer,
>>
>> On Thu, Nov 02, 2017 at 12:49:32PM +, Shameerali Kolothum Thodi wrote:
>>> We had a go with this series on HiSIlicon D05 platform which doesn't have
>>> support for ssids/ATS/PRI, to make sure it generally works.
>>>
>>> But observed the below crash on boot,
>>>
>>> [   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883
>> __alloc_pages_nodemask+0x19c/0xc48
>>> [   16.026797] Modules linked in:
>>> [   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 4.14.0-rc1-
>> 159539-ge42aca3 #236
>>> [...]
>>> [   16.068206] Workqueue: events deferred_probe_work_func
>>> [   16.078557] task: 8017d38a task.stack: 0b198000
>>> [   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
>>> [   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
>>> [   16.469220] [] __alloc_pages_nodemask+0x19c/0xc48
>>> [   16.481854] [] alloc_pages_current+0x80/0xcc
>>> [   16.493607] [] __get_free_pages+0xc/0x38
>>> [   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
>>> [   16.517117] [] __dma_alloc+0x110/0x204
>>> [   16.527820] [] dmam_alloc_coherent+0x88/0xf0
>>> [   16.539575] []
>> arm_smmu_domain_finalise_s1+0x60/0x248
>>> [   16.552909] [] arm_smmu_attach_dev+0x264/0x300
>>> [   16.565013] [] __iommu_attach_device+0x48/0x5c
>>> [   16.577117] [] iommu_group_add_device+0x144/0x3a4
>>> [   16.589746] [] iommu_group_get_for_dev+0x70/0xf8
>>> [   16.602201] [] arm_smmu_add_device+0x1a4/0x418
>>> [   16.614308] [] iort_iommu_configure+0xf0/0x16c
>>> [   16.626416] [] acpi_dma_configure+0x30/0x70
>>> [   16.637994] [] dma_configure+0xa8/0xd4
>>> [   16.648695] [] driver_probe_device+0x1a4/0x2dc
>>> [   16.673081] [] bus_for_each_drv+0x54/0x94
>>> [   16.684307] [] __device_attach+0xc4/0x12c
>>> [   16.695533] [] device_initial_probe+0x10/0x18
>>> [   16.707462] [] bus_probe_device+0x90/0x98
>>>
>>> After a bit of debug it looks like on platforms where ssid is not supported,
>>> s1_cfg.num_contexts is set to zero and it eventually results in this crash
>>> in,
>>> arm_smmu_domain_finalise_s1() -->arm_smmu_alloc_cd_tables()-->
>>> arm_smmu_alloc_cd_leaf_table() as num_leaf_entries is zero.
>>>
>>> With the below fix, it works on D05 now,
>>>
>>> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
>>> index 8ad90e2..51f5821 100644
>>> --- a/drivers/iommu/arm-smmu-v3.c
>>> +++ b/drivers/iommu/arm-smmu-v3.c
>>> @@ -2433,7 +2433,10 @@ static int arm_smmu_domain_finalise(struct
>> iommu_domain *domain,
>>> domain->min_pasid = 1;
>>> domain->max_pasid = master->num_ssids - 1;
>>> smmu_domain->s1_cfg.num_contexts = 
>>> master->num_ssids;
>>> +   } else {
>>> +   smmu_domain->s1_cfg.num_contexts = 1;
>>> }
>>> +
>>>

RE: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-02 Thread Shameerali Kolothum Thodi 


> -Original Message-
> From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com]
> Sent: Thursday, November 02, 2017 3:52 PM
> To: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>
> Cc: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; linux-
> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
> foundation.org; Mark Rutland <mark.rutl...@arm.com>; xieyisheng (A)
> <xieyishe...@huawei.com>; Gabriele Paoloni
> <gabriele.paol...@huawei.com>; Catalin Marinas
> <catalin.mari...@arm.com>; Will Deacon <will.dea...@arm.com>;
> ok...@codeaurora.org; yi.l@intel.com; Lorenzo Pieralisi
> <lorenzo.pieral...@arm.com>; ashok@intel.com; t...@semihalf.com;
> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
> jacob.jun@linux.intel.com; alex.william...@redhat.com;
> robh...@kernel.org; Leizhen (ThunderTown) <thunder.leiz...@huawei.com>;
> bhelg...@google.com; dw...@infradead.org; liubo (CU)
> <liub...@huawei.com>; r...@rjwysocki.net; robdcl...@gmail.com;
> hanjun@linaro.org; Sudeep Holla <sudeep.ho...@arm.com>; Robin
> Murphy <robin.mur...@arm.com>; nwatt...@codeaurora.org; Linuxarm
> <linux...@huawei.com>
> Subject: Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
> Substream IDs
> 
> Hi Shameer,
> 
> On Thu, Nov 02, 2017 at 12:49:32PM +, Shameerali Kolothum Thodi wrote:
> > We had a go with this series on HiSIlicon D05 platform which doesn't have
> > support for ssids/ATS/PRI, to make sure it generally works.
> >
> > But observed the below crash on boot,
> >
> > [   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883
> __alloc_pages_nodemask+0x19c/0xc48
> > [   16.026797] Modules linked in:
> > [   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 4.14.0-rc1-
> 159539-ge42aca3 #236
> > [...]
> > [   16.068206] Workqueue: events deferred_probe_work_func
> > [   16.078557] task: 8017d38a task.stack: 0b198000
> > [   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
> > [   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
> > [   16.469220] [] __alloc_pages_nodemask+0x19c/0xc48
> > [   16.481854] [] alloc_pages_current+0x80/0xcc
> > [   16.493607] [] __get_free_pages+0xc/0x38
> > [   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
> > [   16.517117] [] __dma_alloc+0x110/0x204
> > [   16.527820] [] dmam_alloc_coherent+0x88/0xf0
> > [   16.539575] []
> arm_smmu_domain_finalise_s1+0x60/0x248
> > [   16.552909] [] arm_smmu_attach_dev+0x264/0x300
> > [   16.565013] [] __iommu_attach_device+0x48/0x5c
> > [   16.577117] [] iommu_group_add_device+0x144/0x3a4
> > [   16.589746] [] iommu_group_get_for_dev+0x70/0xf8
> > [   16.602201] [] arm_smmu_add_device+0x1a4/0x418
> > [   16.614308] [] iort_iommu_configure+0xf0/0x16c
> > [   16.626416] [] acpi_dma_configure+0x30/0x70
> > [   16.637994] [] dma_configure+0xa8/0xd4
> > [   16.648695] [] driver_probe_device+0x1a4/0x2dc
> > [   16.673081] [] bus_for_each_drv+0x54/0x94
> > [   16.684307] [] __device_attach+0xc4/0x12c
> > [   16.695533] [] device_initial_probe+0x10/0x18
> > [   16.707462] [] bus_probe_device+0x90/0x98
> >
> > After a bit of debug it looks like on platforms where ssid is not supported,
> > s1_cfg.num_contexts is set to zero and it eventually results in this crash
> > in,
> > arm_smmu_domain_finalise_s1() -->arm_smmu_alloc_cd_tables()-->
> > arm_smmu_alloc_cd_leaf_table() as num_leaf_entries is zero.
> >
> > With the below fix, it works on D05 now,
> >
> > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
> > index 8ad90e2..51f5821 100644
> > --- a/drivers/iommu/arm-smmu-v3.c
> > +++ b/drivers/iommu/arm-smmu-v3.c
> > @@ -2433,7 +2433,10 @@ static int arm_smmu_domain_finalise(struct
> iommu_domain *domain,
> > domain->min_pasid = 1;
> > domain->max_pasid = master->num_ssids - 1;
> > smmu_domain->s1_cfg.num_contexts = 
> > master->num_ssids;
> > +   } else {
> > +   smmu_domain->s1_cfg.num_contexts = 1;
> > }
> > +
> > smmu_domain->s1_cfg.can_stall = master->ste.can_stall;
> > break;
> > case ARM_SMMU_DOMAIN_NESTED:
> >
> >
> > I am not sure this is right place do this. Please take a look.
> 
> Thanks for testing the series and reporting the bug. I added the
> following patch to branch svm/curre

Re: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-02 Thread Jean-Philippe Brucker 
Hi Shameer,

On Thu, Nov 02, 2017 at 12:49:32PM +, Shameerali Kolothum Thodi wrote:
> We had a go with this series on HiSIlicon D05 platform which doesn't have
> support for ssids/ATS/PRI, to make sure it generally works.
> 
> But observed the below crash on boot,
> 
> [   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883 
> __alloc_pages_nodemask+0x19c/0xc48
> [   16.026797] Modules linked in:
> [   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 
> 4.14.0-rc1-159539-ge42aca3 #236
> [...]
> [   16.068206] Workqueue: events deferred_probe_work_func
> [   16.078557] task: 8017d38a task.stack: 0b198000
> [   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
> [   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
> [   16.469220] [] __alloc_pages_nodemask+0x19c/0xc48
> [   16.481854] [] alloc_pages_current+0x80/0xcc
> [   16.493607] [] __get_free_pages+0xc/0x38
> [   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
> [   16.517117] [] __dma_alloc+0x110/0x204
> [   16.527820] [] dmam_alloc_coherent+0x88/0xf0
> [   16.539575] [] arm_smmu_domain_finalise_s1+0x60/0x248
> [   16.552909] [] arm_smmu_attach_dev+0x264/0x300
> [   16.565013] [] __iommu_attach_device+0x48/0x5c
> [   16.577117] [] iommu_group_add_device+0x144/0x3a4
> [   16.589746] [] iommu_group_get_for_dev+0x70/0xf8
> [   16.602201] [] arm_smmu_add_device+0x1a4/0x418
> [   16.614308] [] iort_iommu_configure+0xf0/0x16c
> [   16.626416] [] acpi_dma_configure+0x30/0x70
> [   16.637994] [] dma_configure+0xa8/0xd4
> [   16.648695] [] driver_probe_device+0x1a4/0x2dc
> [   16.673081] [] bus_for_each_drv+0x54/0x94
> [   16.684307] [] __device_attach+0xc4/0x12c
> [   16.695533] [] device_initial_probe+0x10/0x18
> [   16.707462] [] bus_probe_device+0x90/0x98
> 
> After a bit of debug it looks like on platforms where ssid is not supported,
> s1_cfg.num_contexts is set to zero and it eventually results in this crash 
> in,
> arm_smmu_domain_finalise_s1() -->arm_smmu_alloc_cd_tables()-->
> arm_smmu_alloc_cd_leaf_table() as num_leaf_entries is zero.
> 
> With the below fix, it works on D05 now,
> 
> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
> index 8ad90e2..51f5821 100644
> --- a/drivers/iommu/arm-smmu-v3.c
> +++ b/drivers/iommu/arm-smmu-v3.c
> @@ -2433,7 +2433,10 @@ static int arm_smmu_domain_finalise(struct 
> iommu_domain *domain,
> domain->min_pasid = 1;
> domain->max_pasid = master->num_ssids - 1;
> smmu_domain->s1_cfg.num_contexts = master->num_ssids;
> +   } else {
> +   smmu_domain->s1_cfg.num_contexts = 1;
> }
> +
> smmu_domain->s1_cfg.can_stall = master->ste.can_stall;
> break;
> case ARM_SMMU_DOMAIN_NESTED:
> 
> 
> I am not sure this is right place do this. Please take a look.

Thanks for testing the series and reporting the bug. I added the
following patch to branch svm/current, does it work for you?

Thanks,
Jean

diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 42c8378624ed..edda466adc81 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -3169,9 +3169,7 @@ static int arm_smmu_add_device(struct device *dev)
}
}

-   if (smmu->ssid_bits)
-   master->num_ssids = 1 << min(smmu->ssid_bits,
-fwspec->num_pasid_bits);
+   master->num_ssids = 1 << min(smmu->ssid_bits, fwspec->num_pasid_bits);

if (fwspec->can_stall && smmu->features & ARM_SMMU_FEAT_STALLS) {
master->can_fault = true;

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for Substream IDs

2017-11-02 Thread Shameerali Kolothum Thodi 
Hi Jean,

> -Original Message-
> From: linux-arm-kernel [mailto:linux-arm-kernel-boun...@lists.infradead.org]
> On Behalf Of Jean-Philippe Brucker
> Sent: Friday, October 06, 2017 2:32 PM
> To: linux-arm-ker...@lists.infradead.org; linux-...@vger.kernel.org; linux-
> a...@vger.kernel.org; devicet...@vger.kernel.org; iommu@lists.linux-
> foundation.org
> Cc: mark.rutl...@arm.com; xieyisheng (A) ;
> Gabriele Paoloni ; catalin.mari...@arm.com;
> will.dea...@arm.com; ok...@codeaurora.org; yi.l@intel.com;
> lorenzo.pieral...@arm.com; ashok@intel.com; t...@semihalf.com;
> j...@8bytes.org; rfr...@cavium.com; l...@kernel.org;
> jacob.jun@linux.intel.com; alex.william...@redhat.com;
> robh...@kernel.org; Leizhen (ThunderTown) ;
> bhelg...@google.com; dw...@infradead.org; liubo (CU)
> ; r...@rjwysocki.net; robdcl...@gmail.com;
> hanjun@linaro.org; sudeep.ho...@arm.com; robin.mur...@arm.com;
> nwatt...@codeaurora.org
> Subject: [RFCv2 PATCH 14/36] iommu/arm-smmu-v3: Add support for
> Substream IDs
> 
> At the moment, the SMMUv3 driver offers only one stage-1 or stage-2
> address space to each device. SMMUv3 allows to associate multiple address
> spaces per device. In addition to the Stream ID (SID), that identifies a
> device, we can now have Substream IDs (SSID) identifying an address space.
> In PCIe lingo, SID is called Requester ID (RID) and SSID is called Process
> Address-Space ID (PASID).

We had a go with this series on HiSIlicon D05 platform which doesn't have
support for ssids/ATS/PRI, to make sure it generally works.

But observed the below crash on boot,

[   16.009084] WARNING: CPU: 59 PID: 391 at mm/page_alloc.c:3883 
__alloc_pages_nodemask+0x19c/0xc48
[   16.026797] Modules linked in:
[   16.032944] CPU: 59 PID: 391 Comm: kworker/59:1 Not tainted 
4.14.0-rc1-159539-ge42aca3 #236
[...]
[   16.068206] Workqueue: events deferred_probe_work_func
[   16.078557] task: 8017d38a task.stack: 0b198000
[   16.090486] PC is at __alloc_pages_nodemask+0x19c/0xc48
[   16.101013] LR is at __alloc_pages_nodemask+0xe0/0xc48
[   16.469220] [] __alloc_pages_nodemask+0x19c/0xc48
[   16.481854] [] alloc_pages_current+0x80/0xcc
[   16.493607] [] __get_free_pages+0xc/0x38
[   16.504661] [] swiotlb_alloc_coherent+0x64/0x190
[   16.517117] [] __dma_alloc+0x110/0x204
[   16.527820] [] dmam_alloc_coherent+0x88/0xf0
[   16.539575] [] arm_smmu_domain_finalise_s1+0x60/0x248
[   16.552909] [] arm_smmu_attach_dev+0x264/0x300
[   16.565013] [] __iommu_attach_device+0x48/0x5c
[   16.577117] [] iommu_group_add_device+0x144/0x3a4
[   16.589746] [] iommu_group_get_for_dev+0x70/0xf8
[   16.602201] [] arm_smmu_add_device+0x1a4/0x418
[   16.614308] [] iort_iommu_configure+0xf0/0x16c
[   16.626416] [] acpi_dma_configure+0x30/0x70
[   16.637994] [] dma_configure+0xa8/0xd4
[   16.648695] [] driver_probe_device+0x1a4/0x2dc
[   16.673081] [] bus_for_each_drv+0x54/0x94
[   16.684307] [] __device_attach+0xc4/0x12c
[   16.695533] [] device_initial_probe+0x10/0x18
[   16.707462] [] bus_probe_device+0x90/0x98

After a bit of debug it looks like on platforms where ssid is not supported,
s1_cfg.num_contexts is set to zero and it eventually results in this crash 
in,
arm_smmu_domain_finalise_s1() -->arm_smmu_alloc_cd_tables()-->
arm_smmu_alloc_cd_leaf_table() as num_leaf_entries is zero.

With the below fix, it works on D05 now,

diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 8ad90e2..51f5821 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2433,7 +2433,10 @@ static int arm_smmu_domain_finalise(struct iommu_domain 
*domain,
domain->min_pasid = 1;
domain->max_pasid = master->num_ssids - 1;
smmu_domain->s1_cfg.num_contexts = master->num_ssids;
+   } else {
+   smmu_domain->s1_cfg.num_contexts = 1;
}
+
smmu_domain->s1_cfg.can_stall = master->ste.can_stall;
break;
case ARM_SMMU_DOMAIN_NESTED:


I am not sure this is right place do this. Please take a look.

Thanks,
Shameer

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu