Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Michael Ellerman]

Joerg Roedel  writes:

> Hi Michael,
>
> On Thu, Aug 24, 2017 at 12:04:13PM +1000, Michael Ellerman wrote:
>> Joerg Roedel  writes:
>> > Can you please try the attached patch?
>> 
>> Thanks, that works.
>
> Great, thanks for testing it. I'll queue it on-top of the original
> patch-set.

Thanks.

cheers

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Michael Ellerman]

Joerg Roedel  writes:

> Hi Michael,
>
> On Thu, Aug 24, 2017 at 12:04:13PM +1000, Michael Ellerman wrote:
>> Joerg Roedel  writes:
>> > Can you please try the attached patch?
>> 
>> Thanks, that works.
>
> Great, thanks for testing it. I'll queue it on-top of the original
> patch-set.

Thanks.

cheers

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Joerg Roedel]

Hi Michael,

On Thu, Aug 24, 2017 at 12:04:13PM +1000, Michael Ellerman wrote:
> Joerg Roedel  writes:
> > Can you please try the attached patch?
> 
> Thanks, that works.

Great, thanks for testing it. I'll queue it on-top of the original
patch-set.

> 
> It boots happily, much later in boot I see these messages:
> 
> [2.085616] iommu: Adding device ffe301000.jr to group 9
> [2.091101] iommu: Adding device ffe302000.jr to group 10
> [2.096633] iommu: Adding device ffe303000.jr to group 11
> [2.102161] iommu: Adding device ffe304000.jr to group 12

Looks good.

> 
> In /sys I see:
> 
> root@p5020ds:~# ls -l /sys/class/iommu/iommu0
> lrwxrwxrwx 1 root root 0 Aug 24 12:01 /sys/class/iommu/iommu0 -> 
> ../../devices/virtual/iommu/iommu0
> 
> And:
> 
> root@p5020ds:~# ls -l /sys/devices/virtual/iommu/iommu0/devices/
> total 0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 :00:00.0 -> 
> ../../../../platform/ffe20.pcie/pci:00/:00:00.0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:00:00.0 -> 
> ../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.0 -> 
> ../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.1 -> 
> ../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.1
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe100300.dma -> 
> ../../../../platform/ffe00.soc/ffe100300.dma
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe101300.dma -> 
> ../../../../platform/ffe00.soc/ffe101300.dma
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe114000.sdhc -> 
> ../../../../platform/ffe00.soc/ffe114000.sdhc
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe20.pcie -> 
> ../../../../platform/ffe20.pcie
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe202000.pcie -> 
> ../../../../platform/ffe202000.pcie
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe21.usb -> 
> ../../../../platform/ffe00.soc/ffe21.usb
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe211000.usb -> 
> ../../../../platform/ffe00.soc/ffe211000.usb
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe22.sata -> 
> ../../../../platform/ffe00.soc/ffe22.sata
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe221000.sata -> 
> ../../../../platform/ffe00.soc/ffe221000.sata
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe301000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe301000.jr
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe302000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe302000.jr
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe303000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe303000.jr
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe304000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe304000.jr
> 
> 
> Which seems like it's working?

Exactly, that is what I'd expect to see with the patches.


Thanks again and cheers,


   Joerg

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Joerg Roedel]

Hi Michael,

On Thu, Aug 24, 2017 at 12:04:13PM +1000, Michael Ellerman wrote:
> Joerg Roedel  writes:
> > Can you please try the attached patch?
> 
> Thanks, that works.

Great, thanks for testing it. I'll queue it on-top of the original
patch-set.

> 
> It boots happily, much later in boot I see these messages:
> 
> [2.085616] iommu: Adding device ffe301000.jr to group 9
> [2.091101] iommu: Adding device ffe302000.jr to group 10
> [2.096633] iommu: Adding device ffe303000.jr to group 11
> [2.102161] iommu: Adding device ffe304000.jr to group 12

Looks good.

> 
> In /sys I see:
> 
> root@p5020ds:~# ls -l /sys/class/iommu/iommu0
> lrwxrwxrwx 1 root root 0 Aug 24 12:01 /sys/class/iommu/iommu0 -> 
> ../../devices/virtual/iommu/iommu0
> 
> And:
> 
> root@p5020ds:~# ls -l /sys/devices/virtual/iommu/iommu0/devices/
> total 0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 :00:00.0 -> 
> ../../../../platform/ffe20.pcie/pci:00/:00:00.0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:00:00.0 -> 
> ../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.0 -> 
> ../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.0
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.1 -> 
> ../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.1
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe100300.dma -> 
> ../../../../platform/ffe00.soc/ffe100300.dma
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe101300.dma -> 
> ../../../../platform/ffe00.soc/ffe101300.dma
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe114000.sdhc -> 
> ../../../../platform/ffe00.soc/ffe114000.sdhc
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe20.pcie -> 
> ../../../../platform/ffe20.pcie
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe202000.pcie -> 
> ../../../../platform/ffe202000.pcie
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe21.usb -> 
> ../../../../platform/ffe00.soc/ffe21.usb
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe211000.usb -> 
> ../../../../platform/ffe00.soc/ffe211000.usb
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe22.sata -> 
> ../../../../platform/ffe00.soc/ffe22.sata
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe221000.sata -> 
> ../../../../platform/ffe00.soc/ffe221000.sata
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe301000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe301000.jr
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe302000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe302000.jr
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe303000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe303000.jr
> lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe304000.jr -> 
> ../../../../platform/ffe00.soc/ffe30.crypto/ffe304000.jr
> 
> 
> Which seems like it's working?

Exactly, that is what I'd expect to see with the patches.


Thanks again and cheers,


   Joerg

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Michael Ellerman]

Joerg Roedel  writes:

> Hello Michael,
>
> On Wed, Aug 23, 2017 at 10:17:39PM +1000, Michael Ellerman wrote:
>
>> [0.358192] Call Trace:
>> [0.360624] [c000f7173680] [c000f7173710] 0xc000f7173710 
>> (unreliable)
>> [0.368088] [c000f7173710] [c02abf7c] 
>> .kernfs_find_and_get_ns+0x4c/0x7c
>> [0.375729] [c000f71737a0] [c02b13c8] 
>> .sysfs_add_link_to_group+0x44/0x9c
>> [0.383456] [c000f7173840] [c0591660] 
>> .iommu_device_link+0x70/0x144
>> [0.390744] [c000f71738e0] [c05942dc] 
>> .fsl_pamu_add_device+0x4c/0x80
>> [0.398121] [c000f7173960] [c058ce8c] 
>> .add_iommu_group+0x5c/0x9c
>> [0.405153] [c000f71739e0] [c059d6b8] 
>> .bus_for_each_dev+0x98/0xfc
>> [0.412269] [c000f7173a80] [c058f1a0] 
>> .bus_set_iommu+0xd8/0x11c
>> [0.419218] [c000f7173b20] [c0d86998] 
>> .pamu_domain_init+0xb0/0xe0
>> [0.426331] [c000f7173ba0] [c0d86864] 
>> .fsl_pamu_init+0xec/0x170
>> [0.433276] [c000f7173c30] [c0001934] 
>> .do_one_initcall+0x68/0x1b8
>> [0.440395] [c000f7173d00] [c0d440e4] 
>> .kernel_init_freeable+0x24c/0x324
>> [0.448031] [c000f7173db0] [c0001b90] .kernel_init+0x20/0x140
>> [0.454801] [c000f7173e30] [c9bc] 
>> .ret_from_kernel_thread+0x58/0x9c
>> [0.462438] Instruction dump:
>> [0.465390] 7c0802a6 fb81ffe0 f8010010 fba1ffe8 fbc1fff0 fbe1fff8 
>> f821ff71 7c7e1b78 
>> [0.473114] 7cbd2b78 7c9c2378 6000 6000  311d 
>> ebfe0048 552906b4 
>> [0.481017] ---[ end trace 5d11d3e6c29380bd ]---
>
> Thanks for the report, this looks like an initialization ordering
> problem.

Yes I suspect so.

> Can you please try the attached patch?

Thanks, that works.

It boots happily, much later in boot I see these messages:

[2.085616] iommu: Adding device ffe301000.jr to group 9
[2.091101] iommu: Adding device ffe302000.jr to group 10
[2.096633] iommu: Adding device ffe303000.jr to group 11
[2.102161] iommu: Adding device ffe304000.jr to group 12

In /sys I see:

root@p5020ds:~# ls -l /sys/class/iommu/iommu0
lrwxrwxrwx 1 root root 0 Aug 24 12:01 /sys/class/iommu/iommu0 -> 
../../devices/virtual/iommu/iommu0

And:

root@p5020ds:~# ls -l /sys/devices/virtual/iommu/iommu0/devices/
total 0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 :00:00.0 -> 
../../../../platform/ffe20.pcie/pci:00/:00:00.0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:00:00.0 -> 
../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.0 -> 
../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.1 -> 
../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.1
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe100300.dma -> 
../../../../platform/ffe00.soc/ffe100300.dma
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe101300.dma -> 
../../../../platform/ffe00.soc/ffe101300.dma
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe114000.sdhc -> 
../../../../platform/ffe00.soc/ffe114000.sdhc
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe20.pcie -> 
../../../../platform/ffe20.pcie
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe202000.pcie -> 
../../../../platform/ffe202000.pcie
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe21.usb -> 
../../../../platform/ffe00.soc/ffe21.usb
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe211000.usb -> 
../../../../platform/ffe00.soc/ffe211000.usb
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe22.sata -> 
../../../../platform/ffe00.soc/ffe22.sata
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe221000.sata -> 
../../../../platform/ffe00.soc/ffe221000.sata
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe301000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe301000.jr
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe302000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe302000.jr
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe303000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe303000.jr
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe304000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe304000.jr


Which seems like it's working?

cheers

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Michael Ellerman]

Joerg Roedel  writes:

> Hello Michael,
>
> On Wed, Aug 23, 2017 at 10:17:39PM +1000, Michael Ellerman wrote:
>
>> [0.358192] Call Trace:
>> [0.360624] [c000f7173680] [c000f7173710] 0xc000f7173710 
>> (unreliable)
>> [0.368088] [c000f7173710] [c02abf7c] 
>> .kernfs_find_and_get_ns+0x4c/0x7c
>> [0.375729] [c000f71737a0] [c02b13c8] 
>> .sysfs_add_link_to_group+0x44/0x9c
>> [0.383456] [c000f7173840] [c0591660] 
>> .iommu_device_link+0x70/0x144
>> [0.390744] [c000f71738e0] [c05942dc] 
>> .fsl_pamu_add_device+0x4c/0x80
>> [0.398121] [c000f7173960] [c058ce8c] 
>> .add_iommu_group+0x5c/0x9c
>> [0.405153] [c000f71739e0] [c059d6b8] 
>> .bus_for_each_dev+0x98/0xfc
>> [0.412269] [c000f7173a80] [c058f1a0] 
>> .bus_set_iommu+0xd8/0x11c
>> [0.419218] [c000f7173b20] [c0d86998] 
>> .pamu_domain_init+0xb0/0xe0
>> [0.426331] [c000f7173ba0] [c0d86864] 
>> .fsl_pamu_init+0xec/0x170
>> [0.433276] [c000f7173c30] [c0001934] 
>> .do_one_initcall+0x68/0x1b8
>> [0.440395] [c000f7173d00] [c0d440e4] 
>> .kernel_init_freeable+0x24c/0x324
>> [0.448031] [c000f7173db0] [c0001b90] .kernel_init+0x20/0x140
>> [0.454801] [c000f7173e30] [c9bc] 
>> .ret_from_kernel_thread+0x58/0x9c
>> [0.462438] Instruction dump:
>> [0.465390] 7c0802a6 fb81ffe0 f8010010 fba1ffe8 fbc1fff0 fbe1fff8 
>> f821ff71 7c7e1b78 
>> [0.473114] 7cbd2b78 7c9c2378 6000 6000  311d 
>> ebfe0048 552906b4 
>> [0.481017] ---[ end trace 5d11d3e6c29380bd ]---
>
> Thanks for the report, this looks like an initialization ordering
> problem.

Yes I suspect so.

> Can you please try the attached patch?

Thanks, that works.

It boots happily, much later in boot I see these messages:

[2.085616] iommu: Adding device ffe301000.jr to group 9
[2.091101] iommu: Adding device ffe302000.jr to group 10
[2.096633] iommu: Adding device ffe303000.jr to group 11
[2.102161] iommu: Adding device ffe304000.jr to group 12

In /sys I see:

root@p5020ds:~# ls -l /sys/class/iommu/iommu0
lrwxrwxrwx 1 root root 0 Aug 24 12:01 /sys/class/iommu/iommu0 -> 
../../devices/virtual/iommu/iommu0

And:

root@p5020ds:~# ls -l /sys/devices/virtual/iommu/iommu0/devices/
total 0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 :00:00.0 -> 
../../../../platform/ffe20.pcie/pci:00/:00:00.0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:00:00.0 -> 
../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.0 -> 
../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.0
lrwxrwxrwx 1 root root 0 Aug 24 12:02 2000:01:00.1 -> 
../../../../platform/ffe202000.pcie/pci2000:00/2000:00:00.0/2000:01:00.1
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe100300.dma -> 
../../../../platform/ffe00.soc/ffe100300.dma
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe101300.dma -> 
../../../../platform/ffe00.soc/ffe101300.dma
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe114000.sdhc -> 
../../../../platform/ffe00.soc/ffe114000.sdhc
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe20.pcie -> 
../../../../platform/ffe20.pcie
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe202000.pcie -> 
../../../../platform/ffe202000.pcie
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe21.usb -> 
../../../../platform/ffe00.soc/ffe21.usb
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe211000.usb -> 
../../../../platform/ffe00.soc/ffe211000.usb
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe22.sata -> 
../../../../platform/ffe00.soc/ffe22.sata
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe221000.sata -> 
../../../../platform/ffe00.soc/ffe221000.sata
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe301000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe301000.jr
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe302000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe302000.jr
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe303000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe303000.jr
lrwxrwxrwx 1 root root 0 Aug 24 12:02 ffe304000.jr -> 
../../../../platform/ffe00.soc/ffe30.crypto/ffe304000.jr


Which seems like it's working?

cheers

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Joerg Roedel]

Hello Michael,

On Wed, Aug 23, 2017 at 10:17:39PM +1000, Michael Ellerman wrote:

> [0.358192] Call Trace:
> [0.360624] [c000f7173680] [c000f7173710] 0xc000f7173710 
> (unreliable)
> [0.368088] [c000f7173710] [c02abf7c] 
> .kernfs_find_and_get_ns+0x4c/0x7c
> [0.375729] [c000f71737a0] [c02b13c8] 
> .sysfs_add_link_to_group+0x44/0x9c
> [0.383456] [c000f7173840] [c0591660] 
> .iommu_device_link+0x70/0x144
> [0.390744] [c000f71738e0] [c05942dc] 
> .fsl_pamu_add_device+0x4c/0x80
> [0.398121] [c000f7173960] [c058ce8c] 
> .add_iommu_group+0x5c/0x9c
> [0.405153] [c000f71739e0] [c059d6b8] 
> .bus_for_each_dev+0x98/0xfc
> [0.412269] [c000f7173a80] [c058f1a0] .bus_set_iommu+0xd8/0x11c
> [0.419218] [c000f7173b20] [c0d86998] 
> .pamu_domain_init+0xb0/0xe0
> [0.426331] [c000f7173ba0] [c0d86864] .fsl_pamu_init+0xec/0x170
> [0.433276] [c000f7173c30] [c0001934] 
> .do_one_initcall+0x68/0x1b8
> [0.440395] [c000f7173d00] [c0d440e4] 
> .kernel_init_freeable+0x24c/0x324
> [0.448031] [c000f7173db0] [c0001b90] .kernel_init+0x20/0x140
> [0.454801] [c000f7173e30] [c9bc] 
> .ret_from_kernel_thread+0x58/0x9c
> [0.462438] Instruction dump:
> [0.465390] 7c0802a6 fb81ffe0 f8010010 fba1ffe8 fbc1fff0 fbe1fff8 f821ff71 
> 7c7e1b78 
> [0.473114] 7cbd2b78 7c9c2378 6000 6000  311d 
> ebfe0048 552906b4 
> [0.481017] ---[ end trace 5d11d3e6c29380bd ]---

Thanks for the report, this looks like an initialization ordering
problem. Can you please try the attached patch?

diff --git a/drivers/iommu/fsl_pamu.c b/drivers/iommu/fsl_pamu.c
index 9238a85de53e..9ee8e9e161f5 100644
--- a/drivers/iommu/fsl_pamu.c
+++ b/drivers/iommu/fsl_pamu.c
@@ -44,8 +44,6 @@ static struct paace *spaact;
 
 static bool probed;/* Has PAMU been probed? */
 
-struct iommu_device pamu_iommu;/* IOMMU core code handle */
-
 /*
  * Table for matching compatible strings, for device tree
  * guts node, for QorIQ SOCs.
@@ -1156,18 +1154,6 @@ static int fsl_pamu_probe(struct platform_device *pdev)
if (ret)
goto error_genpool;
 
-   ret = iommu_device_sysfs_add(_iommu, dev, NULL, "iommu0");
-   if (ret)
-   goto error_genpool;
-
-   iommu_device_set_ops(_iommu, _pamu_ops);
-
-   ret = iommu_device_register(_iommu);
-   if (ret) {
-   dev_err(dev, "Can't register iommu device\n");
-   goto error_sysfs;
-   }
-
pamubypenr = in_be32(_regs->pamubypenr);
 
for (pamu_reg_off = 0, pamu_counter = 0x8000; pamu_reg_off < size;
@@ -1195,9 +1181,6 @@ static int fsl_pamu_probe(struct platform_device *pdev)
 
return 0;
 
-error_sysfs:
-   iommu_device_sysfs_remove(_iommu);
-
 error_genpool:
gen_pool_destroy(spaace_pool);
 
diff --git a/drivers/iommu/fsl_pamu.h b/drivers/iommu/fsl_pamu.h
index fa48222f3421..c3434f29c967 100644
--- a/drivers/iommu/fsl_pamu.h
+++ b/drivers/iommu/fsl_pamu.h
@@ -391,9 +391,6 @@ struct ome {
 #define EOE_WWSAOL  0x1e/* Write with stash allocate only and lock */
 #define EOE_VALID   0x80
 
-extern const struct iommu_ops fsl_pamu_ops;
-extern struct iommu_device pamu_iommu; /* IOMMU core code handle */
-
 /* Function prototypes */
 int pamu_domain_init(void);
 int pamu_enable_liodn(int liodn);
diff --git a/drivers/iommu/fsl_pamu_domain.c b/drivers/iommu/fsl_pamu_domain.c
index 914953b87bf1..e0fcd079cca9 100644
--- a/drivers/iommu/fsl_pamu_domain.c
+++ b/drivers/iommu/fsl_pamu_domain.c
@@ -33,6 +33,8 @@ static struct kmem_cache *fsl_pamu_domain_cache;
 static struct kmem_cache *iommu_devinfo_cache;
 static DEFINE_SPINLOCK(device_domain_lock);
 
+struct iommu_device pamu_iommu;/* IOMMU core code handle */
+
 static struct fsl_dma_domain *to_fsl_dma_domain(struct iommu_domain *dom)
 {
return container_of(dom, struct fsl_dma_domain, iommu_domain);
@@ -1050,7 +1052,7 @@ static u32 fsl_pamu_get_windows(struct iommu_domain 
*domain)
return dma_domain->win_cnt;
 }
 
-const struct iommu_ops fsl_pamu_ops = {
+static const struct iommu_ops fsl_pamu_ops = {
.capable= fsl_pamu_capable,
.domain_alloc   = fsl_pamu_domain_alloc,
.domain_free= fsl_pamu_domain_free,
@@ -1076,6 +1078,19 @@ int __init pamu_domain_init(void)
if (ret)
return ret;
 
+   ret = iommu_device_sysfs_add(_iommu, NULL, NULL, "iommu0");
+   if (ret)
+   return ret;
+
+   iommu_device_set_ops(_iommu, _pamu_ops);
+
+   ret = iommu_device_register(_iommu);
+   if (ret) {
+   iommu_device_sysfs_remove(_iommu);
+   pr_err("Can't register iommu device\n");
+   return ret;
+   }
+
bus_set_iommu(_bus_type, _pamu_ops);

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Joerg Roedel]

Hello Michael,

On Wed, Aug 23, 2017 at 10:17:39PM +1000, Michael Ellerman wrote:

> [0.358192] Call Trace:
> [0.360624] [c000f7173680] [c000f7173710] 0xc000f7173710 
> (unreliable)
> [0.368088] [c000f7173710] [c02abf7c] 
> .kernfs_find_and_get_ns+0x4c/0x7c
> [0.375729] [c000f71737a0] [c02b13c8] 
> .sysfs_add_link_to_group+0x44/0x9c
> [0.383456] [c000f7173840] [c0591660] 
> .iommu_device_link+0x70/0x144
> [0.390744] [c000f71738e0] [c05942dc] 
> .fsl_pamu_add_device+0x4c/0x80
> [0.398121] [c000f7173960] [c058ce8c] 
> .add_iommu_group+0x5c/0x9c
> [0.405153] [c000f71739e0] [c059d6b8] 
> .bus_for_each_dev+0x98/0xfc
> [0.412269] [c000f7173a80] [c058f1a0] .bus_set_iommu+0xd8/0x11c
> [0.419218] [c000f7173b20] [c0d86998] 
> .pamu_domain_init+0xb0/0xe0
> [0.426331] [c000f7173ba0] [c0d86864] .fsl_pamu_init+0xec/0x170
> [0.433276] [c000f7173c30] [c0001934] 
> .do_one_initcall+0x68/0x1b8
> [0.440395] [c000f7173d00] [c0d440e4] 
> .kernel_init_freeable+0x24c/0x324
> [0.448031] [c000f7173db0] [c0001b90] .kernel_init+0x20/0x140
> [0.454801] [c000f7173e30] [c9bc] 
> .ret_from_kernel_thread+0x58/0x9c
> [0.462438] Instruction dump:
> [0.465390] 7c0802a6 fb81ffe0 f8010010 fba1ffe8 fbc1fff0 fbe1fff8 f821ff71 
> 7c7e1b78 
> [0.473114] 7cbd2b78 7c9c2378 6000 6000  311d 
> ebfe0048 552906b4 
> [0.481017] ---[ end trace 5d11d3e6c29380bd ]---

Thanks for the report, this looks like an initialization ordering
problem. Can you please try the attached patch?

diff --git a/drivers/iommu/fsl_pamu.c b/drivers/iommu/fsl_pamu.c
index 9238a85de53e..9ee8e9e161f5 100644
--- a/drivers/iommu/fsl_pamu.c
+++ b/drivers/iommu/fsl_pamu.c
@@ -44,8 +44,6 @@ static struct paace *spaact;
 
 static bool probed;/* Has PAMU been probed? */
 
-struct iommu_device pamu_iommu;/* IOMMU core code handle */
-
 /*
  * Table for matching compatible strings, for device tree
  * guts node, for QorIQ SOCs.
@@ -1156,18 +1154,6 @@ static int fsl_pamu_probe(struct platform_device *pdev)
if (ret)
goto error_genpool;
 
-   ret = iommu_device_sysfs_add(_iommu, dev, NULL, "iommu0");
-   if (ret)
-   goto error_genpool;
-
-   iommu_device_set_ops(_iommu, _pamu_ops);
-
-   ret = iommu_device_register(_iommu);
-   if (ret) {
-   dev_err(dev, "Can't register iommu device\n");
-   goto error_sysfs;
-   }
-
pamubypenr = in_be32(_regs->pamubypenr);
 
for (pamu_reg_off = 0, pamu_counter = 0x8000; pamu_reg_off < size;
@@ -1195,9 +1181,6 @@ static int fsl_pamu_probe(struct platform_device *pdev)
 
return 0;
 
-error_sysfs:
-   iommu_device_sysfs_remove(_iommu);
-
 error_genpool:
gen_pool_destroy(spaace_pool);
 
diff --git a/drivers/iommu/fsl_pamu.h b/drivers/iommu/fsl_pamu.h
index fa48222f3421..c3434f29c967 100644
--- a/drivers/iommu/fsl_pamu.h
+++ b/drivers/iommu/fsl_pamu.h
@@ -391,9 +391,6 @@ struct ome {
 #define EOE_WWSAOL  0x1e/* Write with stash allocate only and lock */
 #define EOE_VALID   0x80
 
-extern const struct iommu_ops fsl_pamu_ops;
-extern struct iommu_device pamu_iommu; /* IOMMU core code handle */
-
 /* Function prototypes */
 int pamu_domain_init(void);
 int pamu_enable_liodn(int liodn);
diff --git a/drivers/iommu/fsl_pamu_domain.c b/drivers/iommu/fsl_pamu_domain.c
index 914953b87bf1..e0fcd079cca9 100644
--- a/drivers/iommu/fsl_pamu_domain.c
+++ b/drivers/iommu/fsl_pamu_domain.c
@@ -33,6 +33,8 @@ static struct kmem_cache *fsl_pamu_domain_cache;
 static struct kmem_cache *iommu_devinfo_cache;
 static DEFINE_SPINLOCK(device_domain_lock);
 
+struct iommu_device pamu_iommu;/* IOMMU core code handle */
+
 static struct fsl_dma_domain *to_fsl_dma_domain(struct iommu_domain *dom)
 {
return container_of(dom, struct fsl_dma_domain, iommu_domain);
@@ -1050,7 +1052,7 @@ static u32 fsl_pamu_get_windows(struct iommu_domain 
*domain)
return dma_domain->win_cnt;
 }
 
-const struct iommu_ops fsl_pamu_ops = {
+static const struct iommu_ops fsl_pamu_ops = {
.capable= fsl_pamu_capable,
.domain_alloc   = fsl_pamu_domain_alloc,
.domain_free= fsl_pamu_domain_free,
@@ -1076,6 +1078,19 @@ int __init pamu_domain_init(void)
if (ret)
return ret;
 
+   ret = iommu_device_sysfs_add(_iommu, NULL, NULL, "iommu0");
+   if (ret)
+   return ret;
+
+   iommu_device_set_ops(_iommu, _pamu_ops);
+
+   ret = iommu_device_register(_iommu);
+   if (ret) {
+   iommu_device_sysfs_remove(_iommu);
+   pr_err("Can't register iommu device\n");
+   return ret;
+   }
+
bus_set_iommu(_bus_type, _pamu_ops);

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Michael Ellerman]

Joerg Roedel  writes:

> From: Joerg Roedel 
>
> This patch adds a global iommu-handle to the pamu driver and
> initializes it at probe time. Also link devices added to the
> iommu to this handle.
>
> Signed-off-by: Joerg Roedel 
> ---
>  drivers/iommu/fsl_pamu.c| 17 +
>  drivers/iommu/fsl_pamu.h|  3 +++
>  drivers/iommu/fsl_pamu_domain.c |  5 -
>  drivers/iommu/fsl_pamu_domain.h |  2 ++
>  4 files changed, 26 insertions(+), 1 deletion(-)

This seems to be causing my p5020ds to blow up with:

[0.096728] Machine: fsl,P5020DS
[0.096729] SoC family: QorIQ
[0.096730] SoC ID: svr:0x82280020, Revision: 2.0
[0.098143] Found FSL PCI host bridge at 0x000ffe20. Firmware bus 
number: 0->0
[0.098145] PCI host bridge /pcie@ffe20  ranges:
[0.098149]  MEM 0x000c..0x000c1fff -> 
0xe000 
[0.098151]   IO 0x000ff800..0x000ff800 -> 0x
[0.098161] /pcie@ffe20: PCICSRBAR @ 0xdf00
[0.098162] setup_pci_atmu: end of DRAM 1
[0.098167] /pcie@ffe20: Setup 64-bit PCI DMA window
[0.098168] /pcie@ffe20: DMA window size is 0xdf00
[0.098326] Found FSL PCI host bridge at 0x000ffe202000. Firmware bus 
number: 0->1
[0.098327] PCI host bridge /pcie@ffe202000  ranges:
[0.098330]  MEM 0x000c4000..0x000c5fff -> 
0xe000 
[0.098332]   IO 0x000ff802..0x000ff802 -> 0x
[0.098340] /pcie@ffe202000: PCICSRBAR @ 0xdf00
[0.098340] setup_pci_atmu: end of DRAM 1
[0.098345] /pcie@ffe202000: Setup 64-bit PCI DMA window
[0.098346] /pcie@ffe202000: DMA window size is 0xdf00
[0.204289] iommu: Adding device ffe100300.dma to group 0
[0.209606] Unable to handle kernel paging request for data at address 
0x0068
[0.217059] Faulting instruction address: 0xc02abe18
[0.222703] Oops: Kernel access of bad area, sig: 11 [#1]
[0.228078] SMP NR_CPUS=24 
[0.228080] CoreNet Generic
[0.233634] Modules linked in:
[0.236674] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 
4.13.0-rc5-gcc-6.3.1-4-g68a17f0be6fe #41
[0.245613] task: c000f7168000 task.stack: c000f717
[0.251515] NIP: c02abe18 LR: c02abf7c CTR: c0025dc8
[0.258545] REGS: c000f7173400 TRAP: 0300   Not tainted  
(4.13.0-rc5-gcc-6.3.1-4-g68a17f0be6fe)
[0.267919] MSR: 80029000 
[0.267924]   CR: 28adb242  XER: 2000
[0.276165] DEAR: 0068 ESR:  SOFTE: 1 
[0.276165] GPR00: c02abf7c c000f7173680 c0fbf600 
 
[0.276165] GPR04: c0c6dc88  c000f72e0190 
 
[0.276165] GPR08: c000f7168000   
0038 
[0.276165] GPR12: 22adb444 c0003fff5000 c0001b70 
 
[0.276165] GPR16:    
 
[0.276165] GPR20:    
 
[0.276165] GPR24:  c0ef4f58 c0d3b408 
c0c6dc88 
[0.276165] GPR28: c0c6dc88   
c0e3b658 
[0.346218] NIP [c02abe18] .kernfs_find_ns+0x30/0x148
[0.351943] LR [c02abf7c] .kernfs_find_and_get_ns+0x4c/0x7c
[0.358192] Call Trace:
[0.360624] [c000f7173680] [c000f7173710] 0xc000f7173710 
(unreliable)
[0.368088] [c000f7173710] [c02abf7c] 
.kernfs_find_and_get_ns+0x4c/0x7c
[0.375729] [c000f71737a0] [c02b13c8] 
.sysfs_add_link_to_group+0x44/0x9c
[0.383456] [c000f7173840] [c0591660] 
.iommu_device_link+0x70/0x144
[0.390744] [c000f71738e0] [c05942dc] 
.fsl_pamu_add_device+0x4c/0x80
[0.398121] [c000f7173960] [c058ce8c] .add_iommu_group+0x5c/0x9c
[0.405153] [c000f71739e0] [c059d6b8] .bus_for_each_dev+0x98/0xfc
[0.412269] [c000f7173a80] [c058f1a0] .bus_set_iommu+0xd8/0x11c
[0.419218] [c000f7173b20] [c0d86998] .pamu_domain_init+0xb0/0xe0
[0.426331] [c000f7173ba0] [c0d86864] .fsl_pamu_init+0xec/0x170
[0.433276] [c000f7173c30] [c0001934] .do_one_initcall+0x68/0x1b8
[0.440395] [c000f7173d00] [c0d440e4] 
.kernel_init_freeable+0x24c/0x324
[0.448031] [c000f7173db0] [c0001b90] .kernel_init+0x20/0x140
[0.454801] [c000f7173e30] [c9bc] 
.ret_from_kernel_thread+0x58/0x9c
[0.462438] Instruction dump:
[0.465390] 7c0802a6 fb81ffe0 f8010010 fba1ffe8 fbc1fff0 fbe1fff8 f821ff71 
7c7e1b78 
[0.473114] 7cbd2b78 7c9c2378 6000 6000  311d ebfe0048 
552906b4 
[0.481017] ---[ end trace 5d11d3e6c29380bd ]---

Re: [PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Michael Ellerman]

Joerg Roedel  writes:

> From: Joerg Roedel 
>
> This patch adds a global iommu-handle to the pamu driver and
> initializes it at probe time. Also link devices added to the
> iommu to this handle.
>
> Signed-off-by: Joerg Roedel 
> ---
>  drivers/iommu/fsl_pamu.c| 17 +
>  drivers/iommu/fsl_pamu.h|  3 +++
>  drivers/iommu/fsl_pamu_domain.c |  5 -
>  drivers/iommu/fsl_pamu_domain.h |  2 ++
>  4 files changed, 26 insertions(+), 1 deletion(-)

This seems to be causing my p5020ds to blow up with:

[0.096728] Machine: fsl,P5020DS
[0.096729] SoC family: QorIQ
[0.096730] SoC ID: svr:0x82280020, Revision: 2.0
[0.098143] Found FSL PCI host bridge at 0x000ffe20. Firmware bus 
number: 0->0
[0.098145] PCI host bridge /pcie@ffe20  ranges:
[0.098149]  MEM 0x000c..0x000c1fff -> 
0xe000 
[0.098151]   IO 0x000ff800..0x000ff800 -> 0x
[0.098161] /pcie@ffe20: PCICSRBAR @ 0xdf00
[0.098162] setup_pci_atmu: end of DRAM 1
[0.098167] /pcie@ffe20: Setup 64-bit PCI DMA window
[0.098168] /pcie@ffe20: DMA window size is 0xdf00
[0.098326] Found FSL PCI host bridge at 0x000ffe202000. Firmware bus 
number: 0->1
[0.098327] PCI host bridge /pcie@ffe202000  ranges:
[0.098330]  MEM 0x000c4000..0x000c5fff -> 
0xe000 
[0.098332]   IO 0x000ff802..0x000ff802 -> 0x
[0.098340] /pcie@ffe202000: PCICSRBAR @ 0xdf00
[0.098340] setup_pci_atmu: end of DRAM 1
[0.098345] /pcie@ffe202000: Setup 64-bit PCI DMA window
[0.098346] /pcie@ffe202000: DMA window size is 0xdf00
[0.204289] iommu: Adding device ffe100300.dma to group 0
[0.209606] Unable to handle kernel paging request for data at address 
0x0068
[0.217059] Faulting instruction address: 0xc02abe18
[0.222703] Oops: Kernel access of bad area, sig: 11 [#1]
[0.228078] SMP NR_CPUS=24 
[0.228080] CoreNet Generic
[0.233634] Modules linked in:
[0.236674] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 
4.13.0-rc5-gcc-6.3.1-4-g68a17f0be6fe #41
[0.245613] task: c000f7168000 task.stack: c000f717
[0.251515] NIP: c02abe18 LR: c02abf7c CTR: c0025dc8
[0.258545] REGS: c000f7173400 TRAP: 0300   Not tainted  
(4.13.0-rc5-gcc-6.3.1-4-g68a17f0be6fe)
[0.267919] MSR: 80029000 
[0.267924]   CR: 28adb242  XER: 2000
[0.276165] DEAR: 0068 ESR:  SOFTE: 1 
[0.276165] GPR00: c02abf7c c000f7173680 c0fbf600 
 
[0.276165] GPR04: c0c6dc88  c000f72e0190 
 
[0.276165] GPR08: c000f7168000   
0038 
[0.276165] GPR12: 22adb444 c0003fff5000 c0001b70 
 
[0.276165] GPR16:    
 
[0.276165] GPR20:    
 
[0.276165] GPR24:  c0ef4f58 c0d3b408 
c0c6dc88 
[0.276165] GPR28: c0c6dc88   
c0e3b658 
[0.346218] NIP [c02abe18] .kernfs_find_ns+0x30/0x148
[0.351943] LR [c02abf7c] .kernfs_find_and_get_ns+0x4c/0x7c
[0.358192] Call Trace:
[0.360624] [c000f7173680] [c000f7173710] 0xc000f7173710 
(unreliable)
[0.368088] [c000f7173710] [c02abf7c] 
.kernfs_find_and_get_ns+0x4c/0x7c
[0.375729] [c000f71737a0] [c02b13c8] 
.sysfs_add_link_to_group+0x44/0x9c
[0.383456] [c000f7173840] [c0591660] 
.iommu_device_link+0x70/0x144
[0.390744] [c000f71738e0] [c05942dc] 
.fsl_pamu_add_device+0x4c/0x80
[0.398121] [c000f7173960] [c058ce8c] .add_iommu_group+0x5c/0x9c
[0.405153] [c000f71739e0] [c059d6b8] .bus_for_each_dev+0x98/0xfc
[0.412269] [c000f7173a80] [c058f1a0] .bus_set_iommu+0xd8/0x11c
[0.419218] [c000f7173b20] [c0d86998] .pamu_domain_init+0xb0/0xe0
[0.426331] [c000f7173ba0] [c0d86864] .fsl_pamu_init+0xec/0x170
[0.433276] [c000f7173c30] [c0001934] .do_one_initcall+0x68/0x1b8
[0.440395] [c000f7173d00] [c0d440e4] 
.kernel_init_freeable+0x24c/0x324
[0.448031] [c000f7173db0] [c0001b90] .kernel_init+0x20/0x140
[0.454801] [c000f7173e30] [c9bc] 
.ret_from_kernel_thread+0x58/0x9c
[0.462438] Instruction dump:
[0.465390] 7c0802a6 fb81ffe0 f8010010 fba1ffe8 fbc1fff0 fbe1fff8 f821ff71 
7c7e1b78 
[0.473114] 7cbd2b78 7c9c2378 6000 6000  311d ebfe0048 
552906b4 
[0.481017] ---[ end trace 5d11d3e6c29380bd ]---


# first bad commit: 

[PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Joerg Roedel]

From: Joerg Roedel 

This patch adds a global iommu-handle to the pamu driver and
initializes it at probe time. Also link devices added to the
iommu to this handle.

Signed-off-by: Joerg Roedel 
---
 drivers/iommu/fsl_pamu.c| 17 +
 drivers/iommu/fsl_pamu.h|  3 +++
 drivers/iommu/fsl_pamu_domain.c |  5 -
 drivers/iommu/fsl_pamu_domain.h |  2 ++
 4 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/fsl_pamu.c b/drivers/iommu/fsl_pamu.c
index 9ee8e9e..9238a85 100644
--- a/drivers/iommu/fsl_pamu.c
+++ b/drivers/iommu/fsl_pamu.c
@@ -44,6 +44,8 @@ static struct paace *spaact;
 
 static bool probed;/* Has PAMU been probed? */
 
+struct iommu_device pamu_iommu;/* IOMMU core code handle */
+
 /*
  * Table for matching compatible strings, for device tree
  * guts node, for QorIQ SOCs.
@@ -1154,6 +1156,18 @@ static int fsl_pamu_probe(struct platform_device *pdev)
if (ret)
goto error_genpool;
 
+   ret = iommu_device_sysfs_add(_iommu, dev, NULL, "iommu0");
+   if (ret)
+   goto error_genpool;
+
+   iommu_device_set_ops(_iommu, _pamu_ops);
+
+   ret = iommu_device_register(_iommu);
+   if (ret) {
+   dev_err(dev, "Can't register iommu device\n");
+   goto error_sysfs;
+   }
+
pamubypenr = in_be32(_regs->pamubypenr);
 
for (pamu_reg_off = 0, pamu_counter = 0x8000; pamu_reg_off < size;
@@ -1181,6 +1195,9 @@ static int fsl_pamu_probe(struct platform_device *pdev)
 
return 0;
 
+error_sysfs:
+   iommu_device_sysfs_remove(_iommu);
+
 error_genpool:
gen_pool_destroy(spaace_pool);
 
diff --git a/drivers/iommu/fsl_pamu.h b/drivers/iommu/fsl_pamu.h
index c3434f2..fa48222 100644
--- a/drivers/iommu/fsl_pamu.h
+++ b/drivers/iommu/fsl_pamu.h
@@ -391,6 +391,9 @@ struct ome {
 #define EOE_WWSAOL  0x1e/* Write with stash allocate only and lock */
 #define EOE_VALID   0x80
 
+extern const struct iommu_ops fsl_pamu_ops;
+extern struct iommu_device pamu_iommu; /* IOMMU core code handle */
+
 /* Function prototypes */
 int pamu_domain_init(void);
 int pamu_enable_liodn(int liodn);
diff --git a/drivers/iommu/fsl_pamu_domain.c b/drivers/iommu/fsl_pamu_domain.c
index da0e1e3..914953b 100644
--- a/drivers/iommu/fsl_pamu_domain.c
+++ b/drivers/iommu/fsl_pamu_domain.c
@@ -983,11 +983,14 @@ static int fsl_pamu_add_device(struct device *dev)
 
iommu_group_put(group);
 
+   iommu_device_link(_iommu, dev);
+
return 0;
 }
 
 static void fsl_pamu_remove_device(struct device *dev)
 {
+   iommu_device_unlink(_iommu, dev);
iommu_group_remove_device(dev);
 }
 
@@ -1047,7 +1050,7 @@ static u32 fsl_pamu_get_windows(struct iommu_domain 
*domain)
return dma_domain->win_cnt;
 }
 
-static const struct iommu_ops fsl_pamu_ops = {
+const struct iommu_ops fsl_pamu_ops = {
.capable= fsl_pamu_capable,
.domain_alloc   = fsl_pamu_domain_alloc,
.domain_free= fsl_pamu_domain_free,
diff --git a/drivers/iommu/fsl_pamu_domain.h b/drivers/iommu/fsl_pamu_domain.h
index f2b0f74..6d8661e 100644
--- a/drivers/iommu/fsl_pamu_domain.h
+++ b/drivers/iommu/fsl_pamu_domain.h
@@ -21,6 +21,8 @@
 
 #include "fsl_pamu.h"
 
+const struct iommu_ops fsl_pamu_ops;
+
 struct dma_window {
phys_addr_t paddr;
u64 size;
-- 
2.7.4

[PATCH 4/4] iommu/pamu: Add support for generic iommu-device

[Joerg Roedel]

From: Joerg Roedel 

This patch adds a global iommu-handle to the pamu driver and
initializes it at probe time. Also link devices added to the
iommu to this handle.

Signed-off-by: Joerg Roedel 
---
 drivers/iommu/fsl_pamu.c| 17 +
 drivers/iommu/fsl_pamu.h|  3 +++
 drivers/iommu/fsl_pamu_domain.c |  5 -
 drivers/iommu/fsl_pamu_domain.h |  2 ++
 4 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/fsl_pamu.c b/drivers/iommu/fsl_pamu.c
index 9ee8e9e..9238a85 100644
--- a/drivers/iommu/fsl_pamu.c
+++ b/drivers/iommu/fsl_pamu.c
@@ -44,6 +44,8 @@ static struct paace *spaact;
 
 static bool probed;/* Has PAMU been probed? */
 
+struct iommu_device pamu_iommu;/* IOMMU core code handle */
+
 /*
  * Table for matching compatible strings, for device tree
  * guts node, for QorIQ SOCs.
@@ -1154,6 +1156,18 @@ static int fsl_pamu_probe(struct platform_device *pdev)
if (ret)
goto error_genpool;
 
+   ret = iommu_device_sysfs_add(_iommu, dev, NULL, "iommu0");
+   if (ret)
+   goto error_genpool;
+
+   iommu_device_set_ops(_iommu, _pamu_ops);
+
+   ret = iommu_device_register(_iommu);
+   if (ret) {
+   dev_err(dev, "Can't register iommu device\n");
+   goto error_sysfs;
+   }
+
pamubypenr = in_be32(_regs->pamubypenr);
 
for (pamu_reg_off = 0, pamu_counter = 0x8000; pamu_reg_off < size;
@@ -1181,6 +1195,9 @@ static int fsl_pamu_probe(struct platform_device *pdev)
 
return 0;
 
+error_sysfs:
+   iommu_device_sysfs_remove(_iommu);
+
 error_genpool:
gen_pool_destroy(spaace_pool);
 
diff --git a/drivers/iommu/fsl_pamu.h b/drivers/iommu/fsl_pamu.h
index c3434f2..fa48222 100644
--- a/drivers/iommu/fsl_pamu.h
+++ b/drivers/iommu/fsl_pamu.h
@@ -391,6 +391,9 @@ struct ome {
 #define EOE_WWSAOL  0x1e/* Write with stash allocate only and lock */
 #define EOE_VALID   0x80
 
+extern const struct iommu_ops fsl_pamu_ops;
+extern struct iommu_device pamu_iommu; /* IOMMU core code handle */
+
 /* Function prototypes */
 int pamu_domain_init(void);
 int pamu_enable_liodn(int liodn);
diff --git a/drivers/iommu/fsl_pamu_domain.c b/drivers/iommu/fsl_pamu_domain.c
index da0e1e3..914953b 100644
--- a/drivers/iommu/fsl_pamu_domain.c
+++ b/drivers/iommu/fsl_pamu_domain.c
@@ -983,11 +983,14 @@ static int fsl_pamu_add_device(struct device *dev)
 
iommu_group_put(group);
 
+   iommu_device_link(_iommu, dev);
+
return 0;
 }
 
 static void fsl_pamu_remove_device(struct device *dev)
 {
+   iommu_device_unlink(_iommu, dev);
iommu_group_remove_device(dev);
 }
 
@@ -1047,7 +1050,7 @@ static u32 fsl_pamu_get_windows(struct iommu_domain 
*domain)
return dma_domain->win_cnt;
 }
 
-static const struct iommu_ops fsl_pamu_ops = {
+const struct iommu_ops fsl_pamu_ops = {
.capable= fsl_pamu_capable,
.domain_alloc   = fsl_pamu_domain_alloc,
.domain_free= fsl_pamu_domain_free,
diff --git a/drivers/iommu/fsl_pamu_domain.h b/drivers/iommu/fsl_pamu_domain.h
index f2b0f74..6d8661e 100644
--- a/drivers/iommu/fsl_pamu_domain.h
+++ b/drivers/iommu/fsl_pamu_domain.h
@@ -21,6 +21,8 @@
 
 #include "fsl_pamu.h"
 
+const struct iommu_ops fsl_pamu_ops;
+
 struct dma_window {
phys_addr_t paddr;
u64 size;
-- 
2.7.4