Re: [patch] VFIO: platform: reset: fix a warning message condition

[Eric Auger]

Hi Dan,

thanks for pointing this out.

Reviewed-by: Eric Auger <eric.au...@linaro.org>

I add Thomas in CC since the bug also exists in the native driver I
think (drivers/net/ethernet/amd/xgbe/drivers/net/ethernet/amd/xgbe
/xgbe-dev.c, xgbe_exit function).

Best Regards

Eric

On 12/17/2015 01:27 PM, Dan Carpenter wrote:
> This loop ends with count set to -1 and not zero so the warning message
> isn't printed when it should be.  I've fixed this by change the postop
> to a preop.
> 
> Fixes: 0990822c9866 ('VFIO: platform: reset: AMD xgbe reset module')
> Signed-off-by: Dan Carpenter <dan.carpen...@oracle.com>
> 
> diff --git a/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c 
> b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
> index da5356f..d4030d0 100644
> --- a/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
> +++ b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
> @@ -110,7 +110,7 @@ int vfio_platform_amdxgbe_reset(struct 
> vfio_platform_device *vdev)
>   usleep_range(10, 15);
>  
>   count = 2000;
> - while (count-- && (ioread32(xgmac_regs->ioaddr + DMA_MR) & 1))
> + while (--count && (ioread32(xgmac_regs->ioaddr + DMA_MR) & 1))
>   usleep_range(500, 600);
>  
>   if (!count)
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 05/13] VFIO: platform: add vfio_platform_set_forwarded

[Eric Auger]

This function allows to change the forwarded mode and selects
the IRQ handler accordingly.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v3 -> v3:
- renamed vfio_platform_set_automasked into
  vfio_platform_set_forwarded
- do not change VFIO_IRQ_INFO_AUTOMASKED setting when turning
  forwarding on/off

v1 -> v2:
- set forwarded flag
---
 drivers/vfio/platform/vfio_platform_irq.c | 17 +
 1 file changed, 17 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c 
b/drivers/vfio/platform/vfio_platform_irq.c
index c58e0ad..257200b 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -186,6 +186,23 @@ static irqreturn_t vfio_handler(int irq, void *dev_id)
return ret;
 }
 
+static int vfio_platform_set_forwarded(struct vfio_platform_irq *irq,
+  bool forwarded)
+{
+   unsigned long flags;
+
+   spin_lock_irqsave(>lock, flags);
+   irq->forwarded = forwarded;
+
+   if (!forwarded && (irq->flags & VFIO_IRQ_INFO_AUTOMASKED))
+   irq->handler = vfio_automasked_irq_handler;
+   else
+   irq->handler = vfio_irq_handler;
+
+   spin_unlock_irqrestore(>lock, flags);
+   return 0;
+}
+
 static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
 {
 }
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 09/13] KVM: arm/arm64: vgic: support irqfd injection of a mapped IRQ

[Eric Auger]

Up to now irqfd injection was only possible for unmapped IRQ. This
patch adds support for injecting mapped interrupts.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 virt/kvm/arm/vgic.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index dba8eb6..e96f79e 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -2476,13 +2476,21 @@ int kvm_irq_map_chip_pin(struct kvm *kvm, unsigned 
irqchip, unsigned pin)
 int kvm_set_irq(struct kvm *kvm, int irq_source_id,
u32 irq, int level, bool line_status)
 {
+   struct vgic_dist *dist = >arch.vgic;
unsigned int spi = irq + VGIC_NR_PRIVATE_IRQS;
+   struct irq_phys_map *map;
+   int vcpu_id;
 
trace_kvm_set_irq(irq, level, irq_source_id);
 
BUG_ON(!vgic_initialized(kvm));
 
-   return kvm_vgic_inject_irq(kvm, 0, spi, level);
+   vcpu_id = dist->irq_spi_cpu[irq];
+   map = vgic_irq_map_search(kvm_get_vcpu(kvm, vcpu_id), spi);
+   if (!map)
+   return kvm_vgic_inject_irq(kvm, vcpu_id, spi, level);
+   else
+   return kvm_vgic_inject_mapped_irq(kvm, vcpu_id, map, level);
 }
 
 /* MSI not implemented yet */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] KVM: arm/arm64: vgic: leave the LR active state on GICD_ICENABLERn access

[Eric Auger]

Currently on clear-enable MMIO we retire the corresponding LR whatever
its state. More precisely we do not sync ACTIVE state but we erase the LR
state.

In case of a forwarded IRQ, the physical IRQ source is also erased meaning
the physical IRQ will never be deactivated. In case of a non forwarded
IRQ, the LR can be reused (since the state was reset) and the guest can
deactivate an IRQ that is not marked in the LR anymore.

This patch adds a parameter to vgic_retire_lr that makes possible to select
the type of the LR that must be retired.  unqueue will retire/sync all LR's
while disable will leave the active LR's.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 virt/kvm/arm/vgic.c | 45 +++--
 1 file changed, 23 insertions(+), 22 deletions(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 5335383..bc30d93 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -105,7 +105,7 @@
 #include "vgic.h"
 
 static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu);
-static void vgic_retire_lr(int lr_nr, struct kvm_vcpu *vcpu);
+static void vgic_retire_lr(int lr_nr, struct kvm_vcpu *vcpu, unsigned state);
 static struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr);
 static void vgic_set_lr(struct kvm_vcpu *vcpu, int lr, struct vgic_lr lr_desc);
 static u64 vgic_get_elrsr(struct kvm_vcpu *vcpu);
@@ -713,18 +713,10 @@ void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
add_sgi_source(vcpu, lr.irq, lr.source);
 
/*
-* If the LR holds an active (10) or a pending and active (11)
-* interrupt then move the active state to the
-* distributor tracking bit.
+* retire pending, active, active and pending LR's and
+* sync their state back to the distributor
 */
-   if (lr.state & LR_STATE_ACTIVE)
-   vgic_irq_set_active(vcpu, lr.irq);
-
-   /*
-* Reestablish the pending state on the distributor and the
-* CPU interface and mark the LR as free for other use.
-*/
-   vgic_retire_lr(i, vcpu);
+   vgic_retire_lr(i, vcpu, LR_STATE_ACTIVE | LR_STATE_PENDING);
 
/* Finally update the VGIC state. */
vgic_update_state(vcpu->kvm);
@@ -1077,22 +1069,25 @@ static inline void vgic_enable(struct kvm_vcpu *vcpu)
vgic_ops->enable(vcpu);
 }
 
-static void vgic_retire_lr(int lr_nr, struct kvm_vcpu *vcpu)
+static void vgic_retire_lr(int lr_nr, struct kvm_vcpu *vcpu, unsigned state)
 {
struct vgic_lr vlr = vgic_get_lr(vcpu, lr_nr);
 
-   vgic_irq_clear_queued(vcpu, vlr.irq);
+   if (vlr.state & LR_STATE_ACTIVE & state) {
+   vgic_irq_set_active(vcpu, vlr.irq);
+   vlr.state &= ~LR_STATE_ACTIVE;
+   }
 
-   /*
-* We must transfer the pending state back to the distributor before
-* retiring the LR, otherwise we may loose edge-triggered interrupts.
-*/
-   if (vlr.state & LR_STATE_PENDING) {
+   if (vlr.state & LR_STATE_PENDING & state) {
vgic_dist_irq_set_pending(vcpu, vlr.irq);
-   vlr.hwirq = 0;
+   vlr.state &= ~LR_STATE_PENDING;
}
 
-   vlr.state = 0;
+   if (!(vlr.state & LR_STATE_MASK)) {
+   vlr.hwirq = 0;
+   vlr.state = 0;
+   vgic_irq_clear_queued(vcpu, vlr.irq);
+   }
vgic_set_lr(vcpu, lr_nr, vlr);
 }
 
@@ -1114,8 +1109,14 @@ static void vgic_retire_disabled_irqs(struct kvm_vcpu 
*vcpu)
for_each_clear_bit(lr, elrsr_ptr, vgic->nr_lr) {
struct vgic_lr vlr = vgic_get_lr(vcpu, lr);
 
+   /*
+* retire pending only LR's and sync their state
+* back to the distributor. Active LR's cannot be
+* retired since the guest will attempt to deactivate
+* the IRQ.
+*/
if (!vgic_irq_is_enabled(vcpu, vlr.irq))
-   vgic_retire_lr(lr, vcpu);
+   vgic_retire_lr(lr, vcpu, LR_STATE_PENDING);
}
 }
 
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 07/13] VFIO: platform: add irq bypass producer management

[Eric Auger]

This patch populates the IRQ bypass callacks:
- stop/start producer simply consist in disabling/enabling the host irq
- add/del consumer: basically set the automasked flag to false/true

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v3 -> v4:
- use vfio_platform_set_forwarded in place of vfio_platform_set_automasked
- use vfio_platform_irq_is_active and its returned value
- reword comments related to auto/explicit VFIO masking

v2 -> v3:
- vfio_platform_irq_bypass_add_consumer now returns an error in case
  the IRQ is recognized as active
- active boolean not set anymore
- do not VFIO mask the IRQ anymore on unforward

v1 -> v2:
- device handle caching in vfio_platform_device is introduced in a
  separate patch
- use container_of
---
 drivers/vfio/platform/vfio_platform_irq.c | 28 +++-
 1 file changed, 27 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c 
b/drivers/vfio/platform/vfio_platform_irq.c
index cb7d5e9..e486c32 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -224,23 +224,49 @@ out:
 
 static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
 {
+   disable_irq(prod->irq);
 }
 
 static void vfio_platform_irq_bypass_start(struct irq_bypass_producer *prod)
 {
+   enable_irq(prod->irq);
 }
 
 static int vfio_platform_irq_bypass_add_consumer(
struct irq_bypass_producer *prod,
struct irq_bypass_consumer *cons)
 {
-   return 0;
+   struct vfio_platform_irq *irq =
+   container_of(prod, struct vfio_platform_irq, producer);
+   int ret;
+
+   /*
+* if the IRQ is active at irqchip level or masked by VFIO
+* this means the host IRQ may be under injection and
+* it is not safe to change the forwarding state at that time.
+* It is not possible to discriminate VFIO explicit masking
+* from VFIO auto-masking, leading to possible false detection of
+* IRQ activity. It is up to the user-space to avoid VFIO explicit
+* masking situation. In case of failure, no retry is attempted and
+* bypass setup simply fails.
+*/
+   ret = vfio_platform_irq_is_active(irq);
+   if (ret < 0)
+   return ret;
+   else if (ret == 1)
+   return -EAGAIN;
+
+   return vfio_platform_set_forwarded(irq, true);
 }
 
 static void vfio_platform_irq_bypass_del_consumer(
struct irq_bypass_producer *prod,
struct irq_bypass_consumer *cons)
 {
+   struct vfio_platform_irq *irq =
+   container_of(prod, struct vfio_platform_irq, producer);
+
+   vfio_platform_set_forwarded(irq, false);
 }
 
 static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 12/13] KVM: arm/arm64: vgic: implement clear pending for non shared mapped IRQ

[Eric Auger]

This patch implements the clear of a pending non shared mapped IRQ.
In case of an edge IRQ, we deactivate the physical IRQ that will never
be deactivated by the guest. In case of a level sensitive IRQ we check
the level of the input signal. If it is asserted we leave the virtual
IRQ pending. In the opposite, we remove the pending state and deactivate
the IRQ.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 virt/kvm/arm/vgic.c | 34 --
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 98ae15f..4be5972 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -550,20 +550,50 @@ bool vgic_handle_clear_pending_reg(struct kvm *kvm,
   phys_addr_t offset, int vcpu_id)
 {
u32 *level_active;
-   u32 *reg, orig;
+   u32 *reg, orig, cleared;
int mode = ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT;
+   unsigned int i;
struct vgic_dist *dist = >arch.vgic;
+   struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, vcpu_id);
 
reg = vgic_bitmap_get_reg(>irq_pending, vcpu_id, offset);
orig = *reg;
vgic_reg_access(mmio, reg, offset, mode);
if (mmio->is_write) {
-   /* Re-set level triggered level-active interrupts */
level_active = vgic_bitmap_get_reg(>irq_level,
  vcpu_id, offset);
+   cleared = orig ^ *reg;
+
+   /* Re-set level triggered level-active interrupts */
reg = vgic_bitmap_get_reg(>irq_pending, vcpu_id, offset);
*reg |= *level_active;
 
+   for (i = 0; i < 32; i++) {
+   struct irq_phys_map *map;
+   bool phys_pending;
+   unsigned int irq_num;
+
+   if (!(cleared && (1 << i)))
+   continue;
+   irq_num = (offset * 8) + i;
+   map = vgic_irq_map_search(vcpu, irq_num);
+   if (!map || (map && map->shared))
+   continue;
+   /* check whether the signal is asserted */
+   irq_get_irqchip_state(map->irq,
+ IRQCHIP_STATE_PENDING,
+ _pending);
+   if (!vgic_irq_is_edge(vcpu, irq_num) && phys_pending) {
+   vgic_dist_irq_set_pending(vcpu, irq_num);
+   continue;
+   }
+
+   vgic_dist_irq_clear_pending(vcpu, irq_num);
+   irq_set_irqchip_state(map->irq,
+ IRQCHIP_STATE_ACTIVE,
+ false);
+   }
+
/* Ignore writes to SGIs */
if (offset < 2) {
*reg &= ~0x;
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 13/13] KVM: arm/arm64: implement IRQ bypass consumer functions

[Eric Auger]

Implement IRQ bypass callbacks for arm/arm64 IRQ forwarding:
- kvm_arch_irq_bypass_add_producer: perform VGIC/irqchip
  settings for forwarding
- kvm_arch_irq_bypass_del_producer: same for inverse operation
- kvm_arch_irq_bypass_stop: halt guest execution
- kvm_arch_irq_bypass_start: resume guest execution

and set CONFIG_HAVE_KVM_IRQ_BYPASS for arm/arm64.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v3 -> v4:
- kvm_set/unset_forward now takes the virtual IRQ ID as a parameter

v2 -> v3:
- kvm_arch_irq_bypass_resume renamed into kvm_arch_irq_bypass_start
- kvm_vgic_unset_forward does not take the active bool param anymore
- kvm_arch_irq_bypass_add_producer now returns an error value
- remove kvm_arch_irq_bypass_update

v1 -> v2:
- struct kvm_kernel_irqfd is retrieved with container_of
- function names changed
---
 arch/arm/kvm/Kconfig   |  1 +
 arch/arm/kvm/arm.c | 37 +
 arch/arm64/kvm/Kconfig |  1 +
 3 files changed, 39 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 73d7201..4c9a95e 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -34,6 +34,7 @@ config KVM
select HAVE_KVM_EVENTFD
select HAVE_KVM_IRQFD
select IRQ_BYPASS_MANAGER
+   select HAVE_KVM_IRQ_BYPASS
depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
---help---
  Support hosting virtualized guest machines.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index eab83b2..99517bd 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -27,6 +27,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 #include 
 
 #define CREATE_TRACE_POINTS
@@ -1165,6 +1167,41 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, 
unsigned long mpidr)
return NULL;
 }
 
+int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
+ struct irq_bypass_producer *prod)
+{
+   struct kvm_kernel_irqfd *irqfd =
+   container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+   return kvm_vgic_set_forward(irqfd->kvm, prod->irq,
+   irqfd->gsi + VGIC_NR_PRIVATE_IRQS);
+}
+void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
+ struct irq_bypass_producer *prod)
+{
+   struct kvm_kernel_irqfd *irqfd =
+   container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+   kvm_vgic_unset_forward(irqfd->kvm, prod->irq,
+  irqfd->gsi + VGIC_NR_PRIVATE_IRQS);
+}
+
+void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
+{
+   struct kvm_kernel_irqfd *irqfd =
+   container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+   kvm_arm_halt_guest(irqfd->kvm);
+}
+
+void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
+{
+   struct kvm_kernel_irqfd *irqfd =
+   container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+   kvm_arm_resume_guest(irqfd->kvm);
+}
+
 /**
  * Initialize Hyp-mode and memory mappings on all CPUs.
  */
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 64f460d..f6829af 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -38,6 +38,7 @@ config KVM
select HAVE_KVM_IRQFD
select KVM_ARM_VGIC_V3
select IRQ_BYPASS_MANAGER
+   select HAVE_KVM_IRQ_BYPASS
---help---
  Support hosting virtualized guest machines.
  We don't support KVM with 16K page tables yet, due to the multiple
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 11/13] KVM: arm/arm64: vgic: implement clear active for non shared mapped IRQ

[Eric Auger]

When disabling a non shared mapped IRQs, we must manually deactivate
the corresponding physical IRQ on top of removing the active state
from the distributor.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 virt/kvm/arm/vgic.c | 22 +-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index bd500b4..98ae15f 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -604,14 +604,34 @@ bool vgic_handle_clear_active_reg(struct kvm *kvm,
  struct kvm_exit_mmio *mmio,
  phys_addr_t offset, int vcpu_id)
 {
-   u32 *reg;
+   u32 *reg, orig, cleared;
struct vgic_dist *dist = >arch.vgic;
+   struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, vcpu_id);
+   unsigned int i;
 
reg = vgic_bitmap_get_reg(>irq_active, vcpu_id, offset);
+   orig = *reg;
vgic_reg_access(mmio, reg, offset,
ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
 
if (mmio->is_write) {
+   cleared = orig ^ *reg;
+
+   for (i = 0; i < 32; i++) {
+   struct irq_phys_map *map;
+   unsigned int irq_num;
+
+   if (!(cleared && (1 << i)))
+   continue;
+   irq_num = (offset * 8) + i;
+   map = vgic_irq_map_search(vcpu, irq_num);
+   if (!map || (map && map->shared))
+   continue;
+   irq_set_irqchip_state(map->irq,
+ IRQCHIP_STATE_ACTIVE,
+ false);
+   }
+
vgic_update_state(kvm);
return true;
}
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 04/13] VFIO: platform: single handler using function pointer

[Eric Auger]

A single handler now is registered whatever the use case: automasked
or not. A function pointer is set according to the wished behavior
and the handler calls this function.

The irq lock is taken/released in the root handler. eventfd_signal can
be called in regions not allowed to sleep.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v4: creation
---
 drivers/vfio/platform/vfio_platform_irq.c | 21 +++--
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c 
b/drivers/vfio/platform/vfio_platform_irq.c
index d4462fd..c58e0ad 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -148,11 +148,8 @@ static int vfio_platform_set_irq_unmask(struct 
vfio_platform_device *vdev,
 static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
 {
struct vfio_platform_irq *irq_ctx = dev_id;
-   unsigned long flags;
int ret = IRQ_NONE;
 
-   spin_lock_irqsave(_ctx->lock, flags);
-
if (!irq_ctx->masked) {
ret = IRQ_HANDLED;
 
@@ -161,8 +158,6 @@ static irqreturn_t vfio_automasked_irq_handler(int irq, 
void *dev_id)
irq_ctx->masked = true;
}
 
-   spin_unlock_irqrestore(_ctx->lock, flags);
-
if (ret == IRQ_HANDLED)
eventfd_signal(irq_ctx->trigger, 1);
 
@@ -178,6 +173,19 @@ static irqreturn_t vfio_irq_handler(int irq, void *dev_id)
return IRQ_HANDLED;
 }
 
+static irqreturn_t vfio_handler(int irq, void *dev_id)
+{
+   struct vfio_platform_irq *irq_ctx = dev_id;
+   unsigned long flags;
+   irqreturn_t ret;
+
+   spin_lock_irqsave(_ctx->lock, flags);
+   ret = irq_ctx->handler(irq, dev_id);
+   spin_unlock_irqrestore(_ctx->lock, flags);
+
+   return ret;
+}
+
 static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
 {
 }
@@ -230,9 +238,10 @@ static int vfio_set_trigger(struct vfio_platform_device 
*vdev, int index,
}
 
irq->trigger = trigger;
+   irq->handler = handler;
 
irq_set_status_flags(irq->hwirq, IRQ_NOAUTOEN);
-   ret = request_irq(irq->hwirq, handler, 0, irq->name, irq);
+   ret = request_irq(irq->hwirq, vfio_handler, 0, irq->name, irq);
if (ret) {
kfree(irq->name);
eventfd_ctx_put(trigger);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 55e8043..8a8c050 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -40,6 +40,7 @@ struct vfio_platform_irq {
struct virqfd   *mask;
struct irq_bypass_producer producer;
boolforwarded;
+   irqreturn_t (*handler)(int irq, void *dev_id);
 };
 
 struct vfio_platform_region {
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 10/13] KVM: arm/arm64: vgic: forwarding control

[Eric Auger]

Implements kvm_vgic_[set|unset]_forward.

Handle low-level VGIC programming: physical IRQ/guest IRQ mapping,
list register cleanup, VGIC state machine. Also interacts with
the irqchip.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v3 -> v4:
- use the target vCPU in set/unset_forward
- rebase after removal of vgic_irq_lr_map
- clarify set/unset_forward comments,
- host_irq renamed into irq and virt_irq (SPI ID) now used in
  place of SPI index (former guest_irq)
- replaced BUG_ON by WARN_ON
- simplify unset_forward implementation by using vgic_unqueue_irqs
- handle edge mapped unshared IRQs

v2 -> v3:
- on unforward, we do not compute & output the active state anymore.
  This means if the unforward happens while the physical IRQ is
  active, we will not VFIO mask the IRQ while deactiving it. If a
  new physical IRQ hits, the corresponding virtual IRQ might not
  be injected (hence lost) due to VGIC state machine.

bypass rfc v2:
- use irq_set_vcpu_affinity API
- use irq_set_irqchip_state instead of chip->irq_eoi

bypass rfc:
- rename kvm_arch_{set|unset}_forward into
  kvm_vgic_{set|unset}_forward. Remove __KVM_HAVE_ARCH_HALT_GUEST.
  The function is bound to be called by ARM code only.

v4 -> v5:
- fix arm64 compilation issues, ie. also defines
  __KVM_HAVE_ARCH_HALT_GUEST for arm64

v3 -> v4:
- code originally located in kvm_vfio_arm.c
- kvm_arch_vfio_{set|unset}_forward renamed into
  kvm_arch_{set|unset}_forward
- split into 2 functions (set/unset) since unset does not fail anymore
- unset can be invoked at whatever time. Extra care is taken to handle
  transition in VGIC state machine, LR cleanup, ...

v2 -> v3:
- renaming of kvm_arch_set_fwd_state into kvm_arch_vfio_set_forward
- takes a bool arg instead of kvm_fwd_irq_action enum
- removal of KVM_VFIO_IRQ_CLEANUP
- platform device check now happens here
- more precise errors returned
- irq_eoi handled externally to this patch (VGIC)
- correct enable_irq bug done twice
- reword the commit message
- correct check of platform_bus_type
- use raw_spin_lock_irqsave and check the validity of the handler
---
 include/kvm/arm_vgic.h |   5 +++
 virt/kvm/arm/vgic.c| 118 +
 2 files changed, 123 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 9bf6a30..8090ab4 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -368,4 +368,9 @@ static inline int vgic_v3_probe(struct device_node 
*vgic_node,
 }
 #endif
 
+int kvm_vgic_set_forward(struct kvm *kvm, unsigned int irq,
+unsigned int virt_irq);
+void kvm_vgic_unset_forward(struct kvm *kvm, unsigned int irq,
+   unsigned int virt_irq);
+
 #endif
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index e96f79e..bd500b4 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -2500,3 +2500,121 @@ int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
 {
return 0;
 }
+
+/**
+ * kvm_vgic_set_forward - Set IRQ forwarding
+ *
+ * @kvm: handle to the VM
+ * @irq: the host linux IRQ
+ * @virt_irq: the guest SPI ID
+ *
+ * This function is supposed to be called only if the IRQ
+ * is not in progress: ie. not active at GIC level and not
+ * currently under injection in the guest. The physical IRQ must
+ * also be disabled and all vCPUs must have been exited and
+ * prevented from being re-entered.
+ */
+int kvm_vgic_set_forward(struct kvm *kvm, unsigned int irq,
+unsigned int virt_irq)
+{
+   struct vgic_dist *dist = >arch.vgic;
+   struct kvm_vcpu *vcpu;
+   struct irq_phys_map *map;
+   int cpu_id;
+
+   kvm_debug("%s irq=%d virt_irq=%d\n", __func__, irq, virt_irq);
+
+   cpu_id = dist->irq_spi_cpu[virt_irq - VGIC_NR_PRIVATE_IRQS];
+   vcpu = kvm_get_vcpu(kvm, cpu_id);
+   if (!vcpu)
+   return 0;
+   /*
+* let's tell the irqchip driver that after this function
+* returns, a new occurrence of that physical irq will be handled
+* as a forwarded IRQ, ie. the host will only perform priority
+* drop but will not deactivate the physical IRQ: guest will
+*/
+   irq_set_vcpu_affinity(irq, vcpu);
+
+   /*
+* let's program the vgic so that after this function returns
+* any subsequent virt_irq injection will be considered as
+* forwarded and LR will be programmed with HWbit set
+*/
+   map = kvm_vgic_map_phys_irq(vcpu, virt_irq, irq, false);
+
+   return !map;
+}
+
+/**
+ * kvm_vgic_unset_forward - Unset IRQ forwarding
+ *
+ * @kvm: handle to the VM
+ * @irq: host Linux IRQ number
+ * @virt_irq: virtual SPI ID
+ *
+ * This function must be called when the host irq is disabled
+ * and all vCPUs have been exited and prevented from being re-entered.
+ */
+void kvm_vgic_unset_forward(struct kvm *kvm,
+ unsigned int irq,
+  

[PATCH v4 08/13] KVM: arm/arm64: vgic: adapt state machine for non shared mapped interrupts

[Eric Auger]

From: Marc Zyngier <marc.zyng...@arm.com>

So far, the only user of the mapped interrupt facility was the timer:
the physical distributor active state needed to be context-switched
for each vcpu, as the device is shared across all vcpus.

This patch allows to indicate whether a mapped IRQ originates from a
device shared between several VMs (typically the architected timer) or
from a device assigned to a single VM.

A new "shared" flag is added to irq_phys_map and passed to the
mapping function.

the VGIC state machine is adapted to support the non shared mapped IRQs:
- only can be sampled when it is pending
- when queueing the IRQ (programming the LR), the pending state is
  removed as for edge sensitive IRQs
- queued state is not modelled. Level state is not modelled
- its injection with high level always is valid since steming from the HW.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyng...@arm.com>

---

v3 -> v4:
- reword the patch title
- rebase on [PATCH v2 0/8] Rework architected timer and forwarded IRQs
  handling
- renamed shared_hw into non_shared_mapped_irq;
- handle edge-sensitive unshared mapped IRQs
- vgic_validate_injection rejects mapped unshared edge with
  level == 0
---
 include/kvm/arm_vgic.h|  3 ++-
 virt/kvm/arm/arch_timer.c |  2 +-
 virt/kvm/arm/vgic.c   | 37 -
 3 files changed, 31 insertions(+), 11 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 9c747cb..9bf6a30 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -158,6 +158,7 @@ struct irq_phys_map {
u32 virt_irq;
u32 phys_irq;
u32 irq;
+   boolshared;
 };
 
 struct irq_phys_map_entry {
@@ -344,7 +345,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
 int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
 int kvm_vgic_vcpu_active_irq(struct kvm_vcpu *vcpu);
 struct irq_phys_map *kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu,
-  int virt_irq, int irq);
+  int virt_irq, int irq, bool shared);
 int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, struct irq_phys_map *map);
 
 #define irqchip_in_kernel(k)   (!!((k)->arch.vgic.in_kernel))
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 21a0ab2..9eea751 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -289,7 +289,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 * Tell the VGIC that the virtual interrupt is tied to a
 * physical interrupt. We do that once per VCPU.
 */
-   map = kvm_vgic_map_phys_irq(vcpu, irq->irq, host_vtimer_irq);
+   map = kvm_vgic_map_phys_irq(vcpu, irq->irq, host_vtimer_irq, true);
if (WARN_ON(IS_ERR(map)))
return PTR_ERR(map);
 
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index bc30d93..dba8eb6 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -410,7 +410,11 @@ void vgic_cpu_irq_clear(struct kvm_vcpu *vcpu, int irq)
 
 static bool vgic_can_sample_irq(struct kvm_vcpu *vcpu, int irq)
 {
-   return !vgic_irq_is_queued(vcpu, irq);
+   struct irq_phys_map *map = vgic_irq_map_search(vcpu, irq);
+   bool non_shared_mapped_irq = map && !map->shared;
+
+   return !vgic_irq_is_queued(vcpu, irq) ||
+   (non_shared_mapped_irq && vgic_dist_irq_is_pending(vcpu, irq));
 }
 
 /**
@@ -1205,11 +1209,14 @@ bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 
sgi_source_id, int irq)
 
 static bool vgic_queue_hwirq(struct kvm_vcpu *vcpu, int irq)
 {
+   struct irq_phys_map *map = vgic_irq_map_search(vcpu, irq);
+   bool non_shared_mapped_irq = map && !map->shared;
+
if (!vgic_can_sample_irq(vcpu, irq))
return true; /* level interrupt, already queued */
 
if (vgic_queue_irq(vcpu, 0, irq)) {
-   if (vgic_irq_is_edge(vcpu, irq)) {
+   if (vgic_irq_is_edge(vcpu, irq) || non_shared_mapped_irq) {
vgic_dist_irq_clear_pending(vcpu, irq);
vgic_cpu_irq_clear(vcpu, irq);
} else {
@@ -1292,6 +1299,8 @@ static int process_queued_irq(struct kvm_vcpu *vcpu,
   int lr, struct vgic_lr vlr)
 {
int pending = 0;
+   struct irq_phys_map *map = vgic_irq_map_search(vcpu, vlr.irq);
+   bool non_shared_mapped_irq = map && !map->shared;
 
/*
 * If the IRQ was EOIed (called from vgic_process_maintenance) or it
@@ -1312,8 +1321,7 @@ static int process_queued_irq(struct kvm_vcpu *vcpu,
vgic_irq_clear_queued(vcpu, vlr.irq);
 
/* Any additional pending interrupt? */
-   if (vgic_irq_is_edge(vcpu, vlr.irq)) {
-   BUG_ON(!(vlr.stat

[PATCH v4 06/13] VFIO: platform: add vfio_platform_irq_is_active

[Eric Auger]

This function returns whether the IRQ is active at irqchip level or
VFIO masked. If either is true, the IRQ is considered active.
Currently there is no way to differentiate userspace masked IRQ from
automasked IRQ. There might be false detection of activity. However
it is currently acceptable to have false detection.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v3 -> v4:
- rename vfio_platform_is_active into vfio_platform_irq_is_active
- remove BUG_ON and return the error if irq_get_irqchip_state fails
---
 drivers/vfio/platform/vfio_platform_irq.c | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c 
b/drivers/vfio/platform/vfio_platform_irq.c
index 257200b..cb7d5e9 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -203,6 +203,25 @@ static int vfio_platform_set_forwarded(struct 
vfio_platform_irq *irq,
return 0;
 }
 
+static int vfio_platform_irq_is_active(struct vfio_platform_irq *irq)
+{
+   unsigned long flags;
+   bool active;
+   int ret;
+
+   spin_lock_irqsave(>lock, flags);
+
+   ret = irq_get_irqchip_state(irq->hwirq, IRQCHIP_STATE_ACTIVE, );
+   if (ret)
+   goto out;
+
+   ret = active || irq->masked;
+
+out:
+   spin_unlock_irqrestore(>lock, flags);
+   return ret;
+}
+
 static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
 {
 }
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 01/13] KVM: arm/arm64: select IRQ_BYPASS_MANAGER

[Eric Auger]

Select IRQ_BYPASS_MANAGER when CONFIG_KVM is set

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 arch/arm/kvm/Kconfig   | 2 ++
 arch/arm64/kvm/Kconfig | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 95a0005..73d7201 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
bool "Virtualization"
@@ -32,6 +33,7 @@ config KVM
select KVM_VFIO
select HAVE_KVM_EVENTFD
select HAVE_KVM_IRQFD
+   select IRQ_BYPASS_MANAGER
depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
---help---
  Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index a5272c0..64f460d 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -3,6 +3,7 @@
 #
 
 source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
 
 menuconfig VIRTUALIZATION
bool "Virtualization"
@@ -36,6 +37,7 @@ config KVM
select HAVE_KVM_EVENTFD
select HAVE_KVM_IRQFD
select KVM_ARM_VGIC_V3
+   select IRQ_BYPASS_MANAGER
---help---
  Support hosting virtualized guest machines.
  We don't support KVM with 16K page tables yet, due to the multiple
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 02/13] VFIO: platform: registration of a dummy IRQ bypass producer

[Eric Auger]

Register a dummy producer with void callbacks

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v3 -> v3:
- replace WARN_ON by pr_info() in case irq_bypass_register_producer
  fails

v2 -> v3:
- rename vfio_platform_irq_bypass_resume into *_start
---
 drivers/vfio/platform/vfio_platform_irq.c | 34 +++
 drivers/vfio/platform/vfio_platform_private.h |  2 ++
 2 files changed, 36 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c 
b/drivers/vfio/platform/vfio_platform_irq.c
index 46d4750..ab7658a 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -20,6 +20,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "vfio_platform_private.h"
 
@@ -177,6 +178,27 @@ static irqreturn_t vfio_irq_handler(int irq, void *dev_id)
return IRQ_HANDLED;
 }
 
+static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
+{
+}
+
+static void vfio_platform_irq_bypass_start(struct irq_bypass_producer *prod)
+{
+}
+
+static int vfio_platform_irq_bypass_add_consumer(
+   struct irq_bypass_producer *prod,
+   struct irq_bypass_consumer *cons)
+{
+   return 0;
+}
+
+static void vfio_platform_irq_bypass_del_consumer(
+   struct irq_bypass_producer *prod,
+   struct irq_bypass_consumer *cons)
+{
+}
+
 static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
int fd, irq_handler_t handler)
 {
@@ -187,6 +209,7 @@ static int vfio_set_trigger(struct vfio_platform_device 
*vdev, int index,
if (irq->trigger) {
irq_clear_status_flags(irq->hwirq, IRQ_NOAUTOEN);
free_irq(irq->hwirq, irq);
+   irq_bypass_unregister_producer(>producer);
kfree(irq->name);
eventfd_ctx_put(irq->trigger);
irq->trigger = NULL;
@@ -217,6 +240,17 @@ static int vfio_set_trigger(struct vfio_platform_device 
*vdev, int index,
return ret;
}
 
+   irq->producer.token = (void *)trigger;
+   irq->producer.irq = irq->hwirq;
+   irq->producer.add_consumer = vfio_platform_irq_bypass_add_consumer;
+   irq->producer.del_consumer = vfio_platform_irq_bypass_del_consumer;
+   irq->producer.stop = vfio_platform_irq_bypass_stop;
+   irq->producer.start = vfio_platform_irq_bypass_start;
+   ret = irq_bypass_register_producer(>producer);
+   if (ret)
+   pr_info("irq bypass producer (token %p) registration fails: 
%d\n",
+   irq->producer.token, ret);
+
if (!irq->masked)
enable_irq(irq->hwirq);
 
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 42816dd..979c191 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -17,6 +17,7 @@
 
 #include 
 #include 
+#include 
 
 #define VFIO_PLATFORM_OFFSET_SHIFT   40
 #define VFIO_PLATFORM_OFFSET_MASK (((u64)(1) << VFIO_PLATFORM_OFFSET_SHIFT) - 
1)
@@ -37,6 +38,7 @@ struct vfio_platform_irq {
spinlock_t  lock;
struct virqfd   *unmask;
struct virqfd   *mask;
+   struct irq_bypass_producer producer;
 };
 
 struct vfio_platform_region {
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 03/13] VFIO: platform: test forwarded state when selecting the IRQ handler

[Eric Auger]

Add a new forwarded flag in vfio_platform_irq.  In case the IRQ
is forwarded, the VFIO platform IRQ handler does not need to
disable the IRQ anymore.

When setting the IRQ handler we now also test the forwarded state. In
case the IRQ is forwarded we select the vfio_irq_handler.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2:
- add a new forwarded flag and do not use irqd_irq_forwarded anymore
---
 drivers/vfio/platform/vfio_platform_irq.c | 3 ++-
 drivers/vfio/platform/vfio_platform_private.h | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c 
b/drivers/vfio/platform/vfio_platform_irq.c
index ab7658a..d4462fd 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -265,7 +265,8 @@ static int vfio_platform_set_irq_trigger(struct 
vfio_platform_device *vdev,
struct vfio_platform_irq *irq = >irqs[index];
irq_handler_t handler;
 
-   if (vdev->irqs[index].flags & VFIO_IRQ_INFO_AUTOMASKED)
+   if (vdev->irqs[index].flags & VFIO_IRQ_INFO_AUTOMASKED &&
+   !irq->forwarded)
handler = vfio_automasked_irq_handler;
else
handler = vfio_irq_handler;
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 979c191..55e8043 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -39,6 +39,7 @@ struct vfio_platform_irq {
struct virqfd   *unmask;
struct virqfd   *mask;
struct irq_bypass_producer producer;
+   boolforwarded;
 };
 
 struct vfio_platform_region {
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 00/13] ARM IRQ forward control based on IRQ bypass manager

[Eric Auger]

This series allows to optimize the deactivation of virtual interrupts
associated to a vfio platform device IRQ. Let's call this optimization:
ARM IRQ forwarding.

Without that optimization the deactivation of the physical IRQ is done
by the host and the deactivation of the virtual IRQ, by the guest,
is trapped by KVM with a maintenance IRQ. With ARM IRQ forwarding, the
physical IRQ is automatically deactivated by the GIC interrupt controller
when the guest deactivates the virtual IRQ.

The series enables the auto-negotiation of this optimization, on ARM.
The auto-negotiation is coordinated by the IRQ bypass manager. The VFIO
platform driver and KVM/irqfd cooperate to turn that feature on.

The series implements the setup of IRQ forwarding and adapt the existing
VGIC state machine so that it works.

When the IRQ is forwarded, the VFIO platform driver does not need to
mask the physical IRQ anymore before signaling the eventfd. Indeed
genirq lowers the running priority, enabling other physical IRQ to hit
except that one.

The injection still is based on irqfd triggering. The only impact on
irqfd process is the resamplefd is not triggered anymore for level
sensitive IRQs since deactivation is not trapped by KVM.

More backgroung on ARM IRQ forwarding in the text below and at
http://www.linux-kvm.org/images/a/a8/01x04-ARMdevice.pdf.

This was tested on:
- Calxeda Midway, assigning the xgmac main IRQ.
- AMD Seattle, where all XGBE interrupts are assigned:
  - 2 level sensitive and 4 DMA edge sensitive IRQs

applies on top of:
KVM: arm/arm64: leave the LR active state on GICD_ICENABLERn access

All those pieces can be found at:
https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.4-rc1-forward-v4

History:

v3 -> v4:
- dependencies now are fixed (IRQ bypass manager, guest synchronous
  halt/resume, VGIC architected timer)
  - only applies on top of:
  KVM: arm/arm64: leave the LR active state on GICD_ICENABLERn access
- Rebase on vgic series related to forwarded shared interrupts, mainly
  [PATCH v2 0/8] Rework architected timer and forwarded IRQs handling &
  [PATCH 0/2] arm/arm64: KVM: Fix architected timer issues
- took into account Alex's comment on VFIO part:
  - vfio_platform_set_forwarded replacing vfio_platform_set_automasked
  - automasked flag not updated
  - vfio_platform_is_active renamed into vfio_platform_irq_is_active
  - handle returned value for vfio_platform_irq_is_active
  - remove WARN_ON on irq_bypass_register_producer failure
- took into account Christoffer's comments on vgic part
  - clarify comments
  - clarify edge-sensitive handling and validate_injection of edge
mapped unshared IRQ with level == 0
  - use actual vcpu_id instead of VCPU 0
  - revisit unset_forward implement using unqueue
- moved KVM: arm/arm64: select IRQ_BYPASS_MANAGER in this series
- add support for clear active and clear pending

v2 (RFC) -> v3(PATCH):
- all dependencies now have a patch status
- we dropped the producer active boolean exchanged between the
  VFIO producer and irqfd arm consumer. As a consequence, on
  unforward, if the IRQ is active, this latter is deactivated
  without VFIO-masking it. So we do not exactly come back to the
  exact state where we would be in unforwarded state. A new
  physical IRQ can hit while the previous virtual IRQ is under
  treatment. Its injection in the guest may be rejected thanks
  to the VGIC state machine. This IRQ will be lost but I don't
  think this is a severe issue. In case no new IRQ hits, the
  guest deactivation of the virtual IRQ will trigger the resamplefd
  which will VFIO unmask the non-masked IRQ. This also has no
  consequence.
- VFIO platform driver consumer_add now can fail. It rejects the
  transition for forwarding state in case the IRQ is active
- the series is rebased on new irq_vcpu_affinity series
- no dependency anymore on "chip/vgic adaptations for forwarded irq"
  which was partially integrated into Marc's series. A fix is still
  needed through.
- Guest synchronous halt/resume patch re-integrated into this series
- integrate a new patch file coming mixing
  [PATCH v4 11/11] KVM: arm/arm64: vgic: Allow HW interrupts for
   non-shared devices &
  [RFC v2 2/4] KVM: arm: vgic: fix state machine for forwarded IRQ

v1 -> v2:
- irq bypass manager and irqfd consumer moved in a separate patch
- kvm_arm_[halt,resume]_guest moved in a separate patch
- remove VFIO external functions since we do not need them anymore
- apply container_of strategy advised by Paolo. Only active field
  remains and discussions will tell whether we get rid of it.
- renamed kvm_arch functions

- kvm-vfio v6 -> RFC v1 based on IRQ bypass manager
  see previous history in https://lkml.org/lkml/2015/4/13/353).

Best Regards

Eric


Eric Auger (12):
  KVM: arm/arm64: select IRQ_BYPASS_MANAGER
  VFIO: platform: registration of a dummy IRQ bypass producer
  VFIO: platform: test forwarded state when selecting th

[PATCH v6 7/8] vfio: platform: add dev_info on device reset

[Eric Auger]

It might be helpful for the end-user to check the device reset
function was found by the vfio platform reset framework.

Lets store a pointer to the struct device in vfio_platform_device
and trace when the reset function is called or not found.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v3: creation
---
 drivers/vfio/platform/vfio_platform_common.c  | 14 --
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index f74836a..376d289 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -144,8 +144,12 @@ static void vfio_platform_release(void *device_data)
mutex_lock(_lock);
 
if (!(--vdev->refcnt)) {
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
vfio_platform_regions_cleanup(vdev);
vfio_platform_irq_cleanup(vdev);
}
@@ -174,8 +178,12 @@ static int vfio_platform_open(void *device_data)
if (ret)
goto err_irq;
 
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
}
 
vdev->refcnt++;
@@ -551,6 +559,8 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return -EINVAL;
}
 
+   vdev->device = dev;
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index d1b0668..42816dd 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -59,6 +59,7 @@ struct vfio_platform_device {
struct module   *parent_module;
const char  *compat;
struct module   *reset_module;
+   struct device   *device;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 3/8] vfio: platform: introduce module_vfio_reset_handler macro

[Eric Auger]

The module_vfio_reset_handler macro
- define a module alias
- implement module init/exit function which respectively registers
  and unregisters the reset function.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---
v4 -> v5:
- add Arnd's R-b

v3 -> v4:
- pass reset to vfio_platform_unregister_reset

v2 -> v3:
- use vfio_platform_register_reset macro

v1 -> v2:
- remove vfio_platform_reset_private.h and move back the macro to
  vfio_platform_private.h header: removed reset_module_register &
  unregister (symbol_get)
- defines the module_vfio_reset_handler macro as suggested by Arnd
  (formerly in vfio_platform_reset_private.h)
---
 drivers/vfio/platform/vfio_platform_private.h | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index c563940..fd262be 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -110,4 +110,18 @@ static struct vfio_platform_reset_node __reset ## _node = 
{\
 }; \
 __vfio_platform_register_reset(&__reset ## _node)
 
+#define module_vfio_reset_handler(compat, reset)   \
+MODULE_ALIAS("vfio-reset:" compat);\
+static int __init reset ## _module_init(void)  \
+{  \
+   vfio_platform_register_reset(compat, reset);\
+   return 0;   \
+}; \
+static void __exit reset ## _module_exit(void) \
+{  \
+   vfio_platform_unregister_reset(compat, reset);  \
+}; \
+module_init(reset ## _module_init);\
+module_exit(reset ## _module_exit)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 6/8] vfio: platform: use list of registered reset function

[Eric Auger]

Remove the static lookup table and use the dynamic list of registered
reset functions instead. Also load the reset module through its alias.
The reset struct module pointer is stored in vfio_platform_device.

We also remove the useless struct device pointer parameter in
vfio_platform_get_reset.

This patch fixes the issue related to the usage of __symbol_get, which
besides from being moot, prevented compilation with CONFIG_MODULES
disabled.

Also usage of MODULE_ALIAS makes possible to add a new reset module
without needing to update the framework. This was suggested by Arnd.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reported-by: Arnd Bergmann <a...@arndb.de>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v3 -> v4:
- add Arnd R-b.
- Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) here

v2 -> v3:
- remove clear of vfio_platform_device reset_module and reset
  in vfio_platform_put_reset
- single unlock in vfio_platform_lookup_reset
- use driver_lock instead of reset_lock

v1 -> v2:
- use reset_lock in vfio_platform_lookup_reset
- remove vfio_platform_reset_combo declaration
- remove struct device *dev parameter in vfio_platform_get_reset
- set reset_module and reset to NULL in put function
---
 .../platform/reset/vfio_platform_calxedaxgmac.c|  1 -
 drivers/vfio/platform/vfio_platform_common.c   | 52 --
 drivers/vfio/platform/vfio_platform_private.h  |  7 +--
 3 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 80718f2..640f5d8 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -76,7 +76,6 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 
return 0;
 }
-EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
 
 module_vfio_reset_handler("calxeda,hb-xgmac", 
vfio_platform_calxedaxgmac_reset);
 
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index f2d41a0..f74836a 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -30,37 +30,43 @@
 static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
-static const struct vfio_platform_reset_combo reset_lookup_table[] = {
-   {
-   .compat = "calxeda,hb-xgmac",
-   .reset_function_name = "vfio_platform_calxedaxgmac_reset",
-   .module_name = "vfio-platform-calxedaxgmac",
-   },
-};
-
-static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
-   struct device *dev)
+static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat,
+   struct module **module)
 {
-   int (*reset)(struct vfio_platform_device *);
-   int i;
+   struct vfio_platform_reset_node *iter;
+   vfio_platform_reset_fn_t reset_fn = NULL;
 
-   for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
-   request_module(reset_lookup_table[i].module_name);
-   reset = __symbol_get(
-   reset_lookup_table[i].reset_function_name);
-   if (reset) {
-   vdev->reset = reset;
-   return;
-   }
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat) &&
+   try_module_get(iter->owner)) {
+   *module = iter->owner;
+   reset_fn = iter->reset;
+   break;
}
}
+   mutex_unlock(_lock);
+   return reset_fn;
+}
+
+static void vfio_platform_get_reset(struct vfio_platform_device *vdev)
+{
+   char modname[256];
+
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+   >reset_module);
+   if (!vdev->reset) {
+   snprintf(modname, 256, "vfio-reset:%s", vdev->compat);
+   request_module(modname);
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+>reset_module);
+   }
 }
 
 static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
 {
if (vdev->reset)
-   symbol_put_addr(vdev->reset);
+   module_put(vdev->reset_module);
 }
 
 static int vfio_platform_regions_init(struct vfio_platform_device *vdev)
@@ -557,7 +563,7 @@ int vfio_platform_probe_common(struct vfio_platf

[PATCH v6 1/8] vfio: platform: introduce vfio-platform-base module

[Eric Auger]

To prepare for vfio platform reset rework let's build
vfio_platform_common.c and vfio_platform_irq.c in a separate
module from vfio-platform and vfio-amba. This makes possible
to have separate module inits and works around a race between
platform driver init and vfio reset module init: that way we
make sure symbols exported by base are available when vfio-platform
driver gets probed.

The open/release being implemented in the base module, the ref
count is applied to the parent module instead.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Suggested-by: Arnd Bergmann <a...@arndb.de>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---
v3 -> v4:
- add Arnd R-b

v3: creation
---
 drivers/vfio/platform/Makefile|  6 --
 drivers/vfio/platform/vfio_amba.c |  1 +
 drivers/vfio/platform/vfio_platform.c |  1 +
 drivers/vfio/platform/vfio_platform_common.c  | 13 +++--
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 5 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/platform/Makefile b/drivers/vfio/platform/Makefile
index 9ce8afe..41a6224 100644
--- a/drivers/vfio/platform/Makefile
+++ b/drivers/vfio/platform/Makefile
@@ -1,10 +1,12 @@
-
-vfio-platform-y := vfio_platform.o vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-base-y := vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-y := vfio_platform.o
 
 obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform.o
+obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_PLATFORM) += reset/
 
 vfio-amba-y := vfio_amba.o
 
 obj-$(CONFIG_VFIO_AMBA) += vfio-amba.o
+obj-$(CONFIG_VFIO_AMBA) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_AMBA) += reset/
diff --git a/drivers/vfio/platform/vfio_amba.c 
b/drivers/vfio/platform/vfio_amba.c
index ff0331f..a66479b 100644
--- a/drivers/vfio/platform/vfio_amba.c
+++ b/drivers/vfio/platform/vfio_amba.c
@@ -67,6 +67,7 @@ static int vfio_amba_probe(struct amba_device *adev, const 
struct amba_id *id)
vdev->flags = VFIO_DEVICE_FLAGS_AMBA;
vdev->get_resource = get_amba_resource;
vdev->get_irq = get_amba_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret) {
diff --git a/drivers/vfio/platform/vfio_platform.c 
b/drivers/vfio/platform/vfio_platform.c
index cef645c..f1625dc 100644
--- a/drivers/vfio/platform/vfio_platform.c
+++ b/drivers/vfio/platform/vfio_platform.c
@@ -65,6 +65,7 @@ static int vfio_platform_probe(struct platform_device *pdev)
vdev->flags = VFIO_DEVICE_FLAGS_PLATFORM;
vdev->get_resource = get_platform_resource;
vdev->get_irq = get_platform_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret)
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e43efb5..184e9d2 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -23,6 +23,10 @@
 
 #include "vfio_platform_private.h"
 
+#define DRIVER_VERSION  "0.10"
+#define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
+#define DRIVER_DESC "VFIO platform base module"
+
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -146,7 +150,7 @@ static void vfio_platform_release(void *device_data)
 
mutex_unlock(_lock);
 
-   module_put(THIS_MODULE);
+   module_put(vdev->parent_module);
 }
 
 static int vfio_platform_open(void *device_data)
@@ -154,7 +158,7 @@ static int vfio_platform_open(void *device_data)
struct vfio_platform_device *vdev = device_data;
int ret;
 
-   if (!try_module_get(THIS_MODULE))
+   if (!try_module_get(vdev->parent_module))
return -ENODEV;
 
mutex_lock(_lock);
@@ -573,3 +577,8 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
return vdev;
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
+
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR(DRIVER_AUTHOR);
+MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1c9b3d5..7128690 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -56,6 +56,7 @@ struct vfio_platform_device {
u32 num_irqs;
int refcnt;
struct mutexigate;
+   struct module   *parent_module;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 5/8] vfio: platform: add compat in vfio_platform_device

[Eric Auger]

Let's retrieve the compatibility string on probe and store it
in the vfio_platform_device struct

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2 -> v3:
- populate compat after vdev check
---
 drivers/vfio/platform/vfio_platform_common.c  | 15 ---
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 3b7e52c..f2d41a0 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -41,16 +41,11 @@ static const struct vfio_platform_reset_combo 
reset_lookup_table[] = {
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   const char *compat;
int (*reset)(struct vfio_platform_device *);
-   int ret, i;
-
-   ret = device_property_read_string(dev, "compatible", );
-   if (ret)
-   return;
+   int i;
 
for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, compat)) {
+   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
request_module(reset_lookup_table[i].module_name);
reset = __symbol_get(
reset_lookup_table[i].reset_function_name);
@@ -544,6 +539,12 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
if (!vdev)
return -EINVAL;
 
+   ret = device_property_read_string(dev, "compatible", >compat);
+   if (ret) {
+   pr_err("VFIO: cannot retrieve compat for %s\n", vdev->name);
+   return -EINVAL;
+   }
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index fd262be..415310f 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -57,6 +57,7 @@ struct vfio_platform_device {
int refcnt;
struct mutexigate;
struct module   *parent_module;
+   const char  *compat;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 2/8] vfio: platform: add capability to register a reset function

[Eric Auger]

In preparation for subsequent changes in reset function lookup,
lets introduce a dynamic list of reset combos (compat string,
reset module, reset function). The list can be populated/voided with
vfio_platform_register/unregister_reset. Those are not yet used in
this patch.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v4 -> v5:
- add Arnd's R-b

v3 -> v4:
- __vfio_platform_register_reset does not return any value anymore
- vfio_platform_unregister_reset also takes the reset function pointer
  as parameter

v2 -> v3:
- use goto out to have a single mutex_unlock
- implement vfio_platform_register_reset as a macro (suggested by Arnd)
- move reset_node struct declaration back to vfio_platform_private.h
- vfio_platform_unregister_reset does not return any value anymore

v1 -> v2:
- reset_list becomes static
- vfio_platform_register/unregister_reset take a const char * as compat
- fix node leak
- add reset_lock to protect the reset list manipulation
- move vfio_platform_reset_node declaration in vfio_platform_common.c
---
 drivers/vfio/platform/vfio_platform_common.c  | 27 +++
 drivers/vfio/platform/vfio_platform_private.h | 20 
 2 files changed, 47 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 184e9d2..3b7e52c 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -27,6 +27,7 @@
 #define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
 #define DRIVER_DESC "VFIO platform base module"
 
+static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -578,6 +579,32 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
 
+void __vfio_platform_register_reset(struct vfio_platform_reset_node *node)
+{
+   mutex_lock(_lock);
+   list_add(>link, _list);
+   mutex_unlock(_lock);
+}
+EXPORT_SYMBOL_GPL(__vfio_platform_register_reset);
+
+void vfio_platform_unregister_reset(const char *compat,
+   vfio_platform_reset_fn_t fn)
+{
+   struct vfio_platform_reset_node *iter, *temp;
+
+   mutex_lock(_lock);
+   list_for_each_entry_safe(iter, temp, _list, link) {
+   if (!strcmp(iter->compat, compat) && (iter->reset == fn)) {
+   list_del(>link);
+   break;
+   }
+   }
+
+   mutex_unlock(_lock);
+
+}
+EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 7128690..c563940 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -71,6 +71,15 @@ struct vfio_platform_device {
int (*reset)(struct vfio_platform_device *vdev);
 };
 
+typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
+
+struct vfio_platform_reset_node {
+   struct list_head link;
+   char *compat;
+   struct module *owner;
+   vfio_platform_reset_fn_t reset;
+};
+
 struct vfio_platform_reset_combo {
const char *compat;
const char *reset_function_name;
@@ -90,4 +99,15 @@ extern int vfio_platform_set_irqs_ioctl(struct 
vfio_platform_device *vdev,
unsigned start, unsigned count,
void *data);
 
+extern void __vfio_platform_register_reset(struct vfio_platform_reset_node *n);
+extern void vfio_platform_unregister_reset(const char *compat,
+  vfio_platform_reset_fn_t fn);
+#define vfio_platform_register_reset(__compat, __reset)\
+static struct vfio_platform_reset_node __reset ## _node = {\
+   .owner = THIS_MODULE,   \
+   .compat = __compat, \
+   .reset = __reset,   \
+}; \
+__vfio_platform_register_reset(&__reset ## _node)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 0/8] VFIO platform reset module rework

[Eric Auger]

This series fixes the current implementation by getting rid of the
usage of __symbol_get which caused a compilation issue with
CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
possible to add a new reset module without being obliged to update the
framework. The new implementation relies on the reset module registering
its reset function to the vfio-platform driver.

The series is available at

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rework-v6

Best Regards

Eric

v5 -> v6:
- add "vfio: platform: reset: calxedaxgmac: fix ioaddr leak"

v4 -> v5:
- no code change
- only added Arnd's new R-b

v3 -> v4:
- Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) later
  in [6/7], to keep the functionality working all along the series
- Add Arnd R-b (I dared to keep them despite the above change)
- vfio_platform_unregister_reset gets the reset function to do a double
  check on the compat and the function pointer too
- __vfio_platform_register_reset turned to 'void'

v2 -> v3:
- use driver_mutex instead of reset_mutex
- style fixes: single mutex_unlock
- use static nodes; vfio_platform_register_reset now is a macro
- vfio_platform_reset_private.h removed since reset_module_(un)register
  disappear. No use of symbol_get anymore.
- new patch introducing vfio-platform-base
- reset look-up moved back at vfio-platform probe time
- new patch featuring dev_info/dev_warn

v1 -> v2:
* in vfio_platform_common.c:
  - move reset lookup at load time and put reset at release: this is to
prevent a race between the 2 load module loads
  - reset_list becomes static
  - vfio_platform_register/unregister_reset take a const char * as compat
  - fix node link
  - remove old combo struct and cleanup proto of vfio_platform_get_reset
  - add mutex to protect the reset list
* in calxeda xgmac reset module
  - introduce vfio_platform_reset_private.h
  - use module_vfio_reset_handler macro
  - do not export vfio_platform_calxedaxgmac_reset symbol anymore
  - add a pr_info to show the device is reset by vfio reset module



Eric Auger (8):
  vfio: platform: introduce vfio-platform-base module
  vfio: platform: add capability to register a reset function
  vfio: platform: introduce module_vfio_reset_handler macro
  vfio: platform: reset: calxedaxgmac: add reset function registration
  vfio: platform: add compat in vfio_platform_device
  vfio: platform: use list of registered reset function
  vfio: platform: add dev_info on device reset
  vfio: platform: reset: calxedaxgmac: fix ioaddr leak

 drivers/vfio/platform/Makefile |   6 +-
 .../platform/reset/vfio_platform_calxedaxgmac.c|  19 ++--
 drivers/vfio/platform/vfio_amba.c  |   1 +
 drivers/vfio/platform/vfio_platform.c  |   1 +
 drivers/vfio/platform/vfio_platform_common.c   | 119 +++--
 drivers/vfio/platform/vfio_platform_private.h  |  40 ++-
 6 files changed, 137 insertions(+), 49 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5] VFIO: platform: reset: AMD xgbe reset module

[Eric Auger]

This patch introduces a module that registers and implements a low-level
reset function for the AMD XGBE device.

it performs the following actions:
- reset the PHY
- disable auto-negotiation
- disable & clear auto-negotiation IRQ
- soft-reset the MAC

Those tiny pieces of code are inherited from the native xgbe driver.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

Applies on top of [PATCH v6 0/8] VFIO platform reset module rework

v4 -> v5:
- fix ioaddr leak
  iounmap is done in vfio platform driver (vfio_platform_regions_cleanup)
  after reset (vfio_platform_release)

v3 -> v4:
- add Arnd's R-b

v2 -> v3:
- in Kconfig, add empty line between the 2 options
- remove DRIVER_VERSION, DRIVER_AUTHOR and DRIVER_DESC and put
  strings directly in MODULE macros

v1 -> v2:
- uses module_vfio_reset_handler macro
---
 drivers/vfio/platform/reset/Kconfig|   8 ++
 drivers/vfio/platform/reset/Makefile   |   2 +
 .../vfio/platform/reset/vfio_platform_amdxgbe.c| 127 +
 3 files changed, 137 insertions(+)
 create mode 100644 drivers/vfio/platform/reset/vfio_platform_amdxgbe.c

diff --git a/drivers/vfio/platform/reset/Kconfig 
b/drivers/vfio/platform/reset/Kconfig
index 746b96b..705 100644
--- a/drivers/vfio/platform/reset/Kconfig
+++ b/drivers/vfio/platform/reset/Kconfig
@@ -5,3 +5,11 @@ config VFIO_PLATFORM_CALXEDAXGMAC_RESET
  Enables the VFIO platform driver to handle reset for Calxeda xgmac
 
  If you don't know what to do here, say N.
+
+config VFIO_PLATFORM_AMDXGBE_RESET
+   tristate "VFIO support for AMD XGBE reset"
+   depends on VFIO_PLATFORM
+   help
+ Enables the VFIO platform driver to handle reset for AMD XGBE
+
+ If you don't know what to do here, say N.
diff --git a/drivers/vfio/platform/reset/Makefile 
b/drivers/vfio/platform/reset/Makefile
index 2a486af..93f4e23 100644
--- a/drivers/vfio/platform/reset/Makefile
+++ b/drivers/vfio/platform/reset/Makefile
@@ -1,5 +1,7 @@
 vfio-platform-calxedaxgmac-y := vfio_platform_calxedaxgmac.o
+vfio-platform-amdxgbe-y := vfio_platform_amdxgbe.o
 
 ccflags-y += -Idrivers/vfio/platform
 
 obj-$(CONFIG_VFIO_PLATFORM_CALXEDAXGMAC_RESET) += vfio-platform-calxedaxgmac.o
+obj-$(CONFIG_VFIO_PLATFORM_AMDXGBE_RESET) += vfio-platform-amdxgbe.o
diff --git a/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c 
b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
new file mode 100644
index 000..da5356f
--- /dev/null
+++ b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
@@ -0,0 +1,127 @@
+/*
+ * VFIO platform driver specialized for AMD xgbe reset
+ * reset code is inherited from AMD xgbe native driver
+ *
+ * Copyright (c) 2015 Linaro Ltd.
+ *  www.linaro.org
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vfio_platform_private.h"
+
+#define DMA_MR 0x3000
+#define MAC_VR 0x0110
+#define DMA_ISR0x3008
+#define MAC_ISR0x00b0
+#define PCS_MMD_SELECT 0xff
+#define MDIO_AN_INT0x8002
+#define MDIO_AN_INTMASK0x8001
+
+static unsigned int xmdio_read(void *ioaddr, unsigned int mmd,
+  unsigned int reg)
+{
+   unsigned int mmd_address, value;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   value = ioread32(ioaddr + ((mmd_address & 0xff) << 2));
+   return value;
+}
+
+static void xmdio_write(void *ioaddr, unsigned int mmd,
+   unsigned int reg, unsigned int value)
+{
+   unsigned int mmd_address;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   iowrite32(value, ioaddr + ((mmd_address & 0xff) << 2));
+}
+
+int vfio_platform_amdxgbe_reset(struct vfio_platform_device *vdev)
+{
+   struct vfio_platform_region *xgmac_regs = >regions[0];
+   struct vfio_platform_region *xpcs_regs = >regions[1];
+   u32 dma_mr_value, pcs_value, value;
+   unsigned int count;
+
+   if (!xgmac_regs->ioaddr) {
+   xg

Re: [PATCH] vfio/platform: store mapped memory in region, instead of an on-stack copy

[Eric Auger]

Hi,
On 10/30/2015 09:51 AM, Baptiste Reynal wrote:
> Hi James,
> 
> Thanks for this fix.
> 
> Acked-by: Baptiste Reynal 
> Tested-by: Baptiste Reynal 
> 
> On Thu, Oct 29, 2015 at 5:50 PM, James Morse  wrote:
>> vfio_platform_{read,write}_mmio() call ioremap_nocache() to map
>> a region of io memory, which they store in struct vfio_platform_region to
>> be eventually re-used, or unmapped by vfio_platform_regions_cleanup().
>>
>> These functions receive a copy of their struct vfio_platform_region
>> argument on the stack - so these mapped areas are always allocated, and
>> always leaked.
I just noticed I have a leak in reset modules too. I am going to correct
this.

Thanks

Eric
>>
>> Pass this argument as a pointer instead.
>>
>> Fixes: 6e3f26456009 "vfio/platform: read and write support for the device fd"
>> Signed-off-by: James Morse 
>> ---
>>  drivers/vfio/platform/vfio_platform_common.c | 36 
>> ++--
>>  1 file changed, 18 insertions(+), 18 deletions(-)
>>
>> diff --git a/drivers/vfio/platform/vfio_platform_common.c 
>> b/drivers/vfio/platform/vfio_platform_common.c
>> index f3b6299..ccf5da5 100644
>> --- a/drivers/vfio/platform/vfio_platform_common.c
>> +++ b/drivers/vfio/platform/vfio_platform_common.c
>> @@ -308,17 +308,17 @@ static long vfio_platform_ioctl(void *device_data,
>> return -ENOTTY;
>>  }
>>
>> -static ssize_t vfio_platform_read_mmio(struct vfio_platform_region reg,
>> +static ssize_t vfio_platform_read_mmio(struct vfio_platform_region *reg,
>>char __user *buf, size_t count,
>>loff_t off)
>>  {
>> unsigned int done = 0;
>>
>> -   if (!reg.ioaddr) {
>> -   reg.ioaddr =
>> -   ioremap_nocache(reg.addr, reg.size);
>> +   if (!reg->ioaddr) {
>> +   reg->ioaddr =
>> +   ioremap_nocache(reg->addr, reg->size);
>>
>> -   if (!reg.ioaddr)
>> +   if (!reg->ioaddr)
>> return -ENOMEM;
>> }
>>
>> @@ -328,7 +328,7 @@ static ssize_t vfio_platform_read_mmio(struct 
>> vfio_platform_region reg,
>> if (count >= 4 && !(off % 4)) {
>> u32 val;
>>
>> -   val = ioread32(reg.ioaddr + off);
>> +   val = ioread32(reg->ioaddr + off);
>> if (copy_to_user(buf, , 4))
>> goto err;
>>
>> @@ -336,7 +336,7 @@ static ssize_t vfio_platform_read_mmio(struct 
>> vfio_platform_region reg,
>> } else if (count >= 2 && !(off % 2)) {
>> u16 val;
>>
>> -   val = ioread16(reg.ioaddr + off);
>> +   val = ioread16(reg->ioaddr + off);
>> if (copy_to_user(buf, , 2))
>> goto err;
>>
>> @@ -344,7 +344,7 @@ static ssize_t vfio_platform_read_mmio(struct 
>> vfio_platform_region reg,
>> } else {
>> u8 val;
>>
>> -   val = ioread8(reg.ioaddr + off);
>> +   val = ioread8(reg->ioaddr + off);
>> if (copy_to_user(buf, , 1))
>> goto err;
>>
>> @@ -377,7 +377,7 @@ static ssize_t vfio_platform_read(void *device_data, 
>> char __user *buf,
>> return -EINVAL;
>>
>> if (vdev->regions[index].type & VFIO_PLATFORM_REGION_TYPE_MMIO)
>> -   return vfio_platform_read_mmio(vdev->regions[index],
>> +   return vfio_platform_read_mmio(>regions[index],
>> buf, count, off);
>> else if (vdev->regions[index].type & VFIO_PLATFORM_REGION_TYPE_PIO)
>> return -EINVAL; /* not implemented */
>> @@ -385,17 +385,17 @@ static ssize_t vfio_platform_read(void *device_data, 
>> char __user *buf,
>> return -EINVAL;
>>  }
>>
>> -static ssize_t vfio_platform_write_mmio(struct vfio_platform_region reg,
>> +static ssize_t vfio_platform_write_mmio(struct vfio_platform_region *reg,
>> const char __user *buf, size_t count,
>> loff_t off)
>>  {
>> unsigned int done = 0;
>>
>> -   if (!reg.ioaddr) {
>> -   reg.ioaddr =
>> -   ioremap_nocache(reg.addr, reg.size);
>> +   if (!reg->ioaddr) {
>> +   reg->ioaddr =
>> +   ioremap_nocache(reg->addr, reg->size);
>>
>> -   if (!reg.ioaddr)
>> +   if (!reg->ioaddr)
>> return -ENOMEM;
>> }
>>
>> @@ -407,7 +407,7 @@ static ssize_t vfio_platform_write_mmio(struct 
>> vfio_platform_region reg,
>>
>> if (copy_from_user(, buf, 4))

[PATCH] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

Current vfio_pgsize_bitmap code hides the supported IOMMU page
sizes smaller than PAGE_SIZE. As a result, in case the IOMMU
does not support PAGE_SIZE page, the alignment check on map/unmap
is done with larger page sizes, if any. This can fail although
mapping could be done with pages smaller than PAGE_SIZE.

This patch modifies vfio_pgsize_bitmap implementation so that,
in case the IOMMU supports page sizes smaller than PAGE_HOST
we pretend PAGE_HOST is supported and hide sub-PAGE_HOST sizes.
That way the user will be able to map/unmap buffers whose size/
start address is aligned with PAGE_HOST. Pinning code uses that
granularity while iommu driver can use the sub-PAGE_HOST size
to map the buffer.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Signed-off-by: Alex Williamson <alex.william...@redhat.com>

---

This was tested on AMD Seattle with 64kB page host. ARM MMU 401
currently expose 4kB, 2MB and 1GB page support. With a 64kB page host,
the map/unmap check is done against 2MB. Some alignment check fail
so VFIO_IOMMU_MAP_DMA fail while we could map using 4kB IOMMU page
size.

RFC -> PATCH v1:
- move all modifications in vfio_pgsize_bitmap following Alex'
  suggestion to expose a fake PAGE_HOST support
- restore WARN_ON's
---
 drivers/vfio/vfio_iommu_type1.c | 15 ++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 57d8c37..cee504a 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -403,13 +403,26 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
struct vfio_dma *dma)
 static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
 {
struct vfio_domain *domain;
-   unsigned long bitmap = PAGE_MASK;
+   unsigned long bitmap = ULONG_MAX;
 
mutex_lock(>lock);
list_for_each_entry(domain, >domain_list, next)
bitmap &= domain->domain->ops->pgsize_bitmap;
mutex_unlock(>lock);
 
+   /*
+* In case the IOMMU supports page sizes smaller than PAGE_HOST
+* we pretend PAGE_HOST is supported and hide sub-PAGE_HOST sizes.
+* That way the user will be able to map/unmap buffers whose size/
+* start address is aligned with PAGE_HOST. Pinning code uses that
+* granularity while iommu driver can use the sub-PAGE_HOST size
+* to map the buffer.
+*/
+   if (bitmap & ~PAGE_MASK) {
+   bitmap &= PAGE_MASK;
+   bitmap |= PAGE_SIZE;
+   }
+
return bitmap;
 }
 
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

Current vfio_pgsize_bitmap code hides the supported IOMMU page
sizes smaller than PAGE_SIZE. As a result, in case the IOMMU
does not support PAGE_SIZE page, the alignment check on map/unmap
is done with larger page sizes, if any. This can fail although
mapping could be done with pages smaller than PAGE_SIZE.

This patch modifies vfio_pgsize_bitmap implementation so that,
in case the IOMMU supports page sizes smaller than PAGE_SIZE
we pretend PAGE_SIZE is supported and hide sub-PAGE_SIZE sizes.
That way the user will be able to map/unmap buffers whose size/
start address is aligned with PAGE_SIZE. Pinning code uses that
granularity while iommu driver can use the sub-PAGE_SIZE size
to map the buffer.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Signed-off-by: Alex Williamson <alex.william...@redhat.com>
Acked-by: Will Deacon <will.dea...@arm.com>

---

This was tested on AMD Seattle with 64kB page host. ARM MMU 401
currently expose 4kB, 2MB and 1GB page support. With a 64kB page host,
the map/unmap check is done against 2MB. Some alignment check fail
so VFIO_IOMMU_MAP_DMA fail while we could map using 4kB IOMMU page
size.

v1 -> v2:
- correct PAGE_HOST type in comment and commit msg
- Add Will's R-b

RFC -> PATCH v1:
- move all modifications in vfio_pgsize_bitmap following Alex'
  suggestion to expose a fake PAGE_SIZE support
- restore WARN_ON's
---
 drivers/vfio/vfio_iommu_type1.c | 15 ++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 57d8c37..59d47cb 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -403,13 +403,26 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
struct vfio_dma *dma)
 static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
 {
struct vfio_domain *domain;
-   unsigned long bitmap = PAGE_MASK;
+   unsigned long bitmap = ULONG_MAX;
 
mutex_lock(>lock);
list_for_each_entry(domain, >domain_list, next)
bitmap &= domain->domain->ops->pgsize_bitmap;
mutex_unlock(>lock);
 
+   /*
+* In case the IOMMU supports page sizes smaller than PAGE_SIZE
+* we pretend PAGE_SIZE is supported and hide sub-PAGE_SIZE sizes.
+* That way the user will be able to map/unmap buffers whose size/
+* start address is aligned with PAGE_SIZE. Pinning code uses that
+* granularity while iommu driver can use the sub-PAGE_SIZE size
+* to map the buffer.
+*/
+   if (bitmap & ~PAGE_MASK) {
+   bitmap &= PAGE_MASK;
+   bitmap |= PAGE_SIZE;
+   }
+
return bitmap;
 }
 
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 4/7] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

This patch adds the reset function registration/unregistration.
This is handled through the module_vfio_reset_handler macro. This
latter also defines a MODULE_ALIAS which simplifies the load from
vfio-platform.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v3 -> v4:
- I restored the EXPORT_SYMBOL which will be removed when switching the
  lookup method
- Add Arnd R-b.

v2 -> v3:
- do not include vfio_platform_reset_private.h anymore (removed)
- remove pr_info
- rework commit message

v1 -> v2:
- uses the module_vfio_reset_handler macro
- add pr_info on vfio reset
- do not export vfio_platform_calxedaxgmac_reset symbol anymore
---
 drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 619dc7d..80718f2 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -30,8 +30,6 @@
 #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
 #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform device"
 
-#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
-
 /* XGMAC Register definitions */
 #define XGMAC_CONTROL   0x  /* MAC Configuration */
 
@@ -80,6 +78,8 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
 
+module_vfio_reset_handler("calxeda,hb-xgmac", 
vfio_platform_calxedaxgmac_reset);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4] VFIO: platform: reset: AMD xgbe reset module

[Eric Auger]

This patch introduces a module that registers and implements a low-level
reset function for the AMD XGBE device.

it performs the following actions:
- reset the PHY
- disable auto-negotiation
- disable & clear auto-negotiation IRQ
- soft-reset the MAC

Those tiny pieces of code are inherited from the native xgbe driver.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

Applies on top of [PATCH v5 0/7] VFIO platform reset module rework

v3 -> v4:
- add Arnd's R-b

v2 -> v3:
- in Kconfig, add empty line between the 2 options
- remove DRIVER_VERSION, DRIVER_AUTHOR and DRIVER_DESC and put
  strings directly in MODULE macros

v1 -> v2:
- uses module_vfio_reset_handler macro
---
 drivers/vfio/platform/reset/Kconfig|   8 ++
 drivers/vfio/platform/reset/Makefile   |   2 +
 .../vfio/platform/reset/vfio_platform_amdxgbe.c| 127 +
 3 files changed, 137 insertions(+)
 create mode 100644 drivers/vfio/platform/reset/vfio_platform_amdxgbe.c

diff --git a/drivers/vfio/platform/reset/Kconfig 
b/drivers/vfio/platform/reset/Kconfig
index 746b96b..705 100644
--- a/drivers/vfio/platform/reset/Kconfig
+++ b/drivers/vfio/platform/reset/Kconfig
@@ -5,3 +5,11 @@ config VFIO_PLATFORM_CALXEDAXGMAC_RESET
  Enables the VFIO platform driver to handle reset for Calxeda xgmac
 
  If you don't know what to do here, say N.
+
+config VFIO_PLATFORM_AMDXGBE_RESET
+   tristate "VFIO support for AMD XGBE reset"
+   depends on VFIO_PLATFORM
+   help
+ Enables the VFIO platform driver to handle reset for AMD XGBE
+
+ If you don't know what to do here, say N.
diff --git a/drivers/vfio/platform/reset/Makefile 
b/drivers/vfio/platform/reset/Makefile
index 2a486af..93f4e23 100644
--- a/drivers/vfio/platform/reset/Makefile
+++ b/drivers/vfio/platform/reset/Makefile
@@ -1,5 +1,7 @@
 vfio-platform-calxedaxgmac-y := vfio_platform_calxedaxgmac.o
+vfio-platform-amdxgbe-y := vfio_platform_amdxgbe.o
 
 ccflags-y += -Idrivers/vfio/platform
 
 obj-$(CONFIG_VFIO_PLATFORM_CALXEDAXGMAC_RESET) += vfio-platform-calxedaxgmac.o
+obj-$(CONFIG_VFIO_PLATFORM_AMDXGBE_RESET) += vfio-platform-amdxgbe.o
diff --git a/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c 
b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
new file mode 100644
index 000..1636e22
--- /dev/null
+++ b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
@@ -0,0 +1,127 @@
+/*
+ * VFIO platform driver specialized for AMD xgbe reset
+ * reset code is inherited from AMD xgbe native driver
+ *
+ * Copyright (c) 2015 Linaro Ltd.
+ *  www.linaro.org
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vfio_platform_private.h"
+
+#define DMA_MR 0x3000
+#define MAC_VR 0x0110
+#define DMA_ISR0x3008
+#define MAC_ISR0x00b0
+#define PCS_MMD_SELECT 0xff
+#define MDIO_AN_INT0x8002
+#define MDIO_AN_INTMASK0x8001
+
+static unsigned int xmdio_read(void *ioaddr, unsigned int mmd,
+  unsigned int reg)
+{
+   unsigned int mmd_address, value;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   value = ioread32(ioaddr + ((mmd_address & 0xff) << 2));
+   return value;
+}
+
+static void xmdio_write(void *ioaddr, unsigned int mmd,
+   unsigned int reg, unsigned int value)
+{
+   unsigned int mmd_address;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   iowrite32(value, ioaddr + ((mmd_address & 0xff) << 2));
+}
+
+int vfio_platform_amdxgbe_reset(struct vfio_platform_device *vdev)
+{
+   struct vfio_platform_region xgmac_regs = vdev->regions[0];
+   struct vfio_platform_region xpcs_regs = vdev->regions[1];
+   u32 dma_mr_value, pcs_value, value;
+   unsigned int count;
+
+   if (!xgmac_regs.ioaddr) {
+   xgmac_regs.ioaddr =
+   ioremap_nocache(xgmac_regs.addr, xgmac_regs.size);
+   if (!xgmac_regs.io

[PATCH v5 2/7] vfio: platform: add capability to register a reset function

[Eric Auger]

In preparation for subsequent changes in reset function lookup,
lets introduce a dynamic list of reset combos (compat string,
reset module, reset function). The list can be populated/voided with
vfio_platform_register/unregister_reset. Those are not yet used in
this patch.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v4 -> v5:
- add Arnd's R-b

v3 -> v4:
- __vfio_platform_register_reset does not return any value anymore
- vfio_platform_unregister_reset also takes the reset function pointer
  as parameter

v2 -> v3:
- use goto out to have a single mutex_unlock
- implement vfio_platform_register_reset as a macro (suggested by Arnd)
- move reset_node struct declaration back to vfio_platform_private.h
- vfio_platform_unregister_reset does not return any value anymore

v1 -> v2:
- reset_list becomes static
- vfio_platform_register/unregister_reset take a const char * as compat
- fix node leak
- add reset_lock to protect the reset list manipulation
- move vfio_platform_reset_node declaration in vfio_platform_common.c
---
 drivers/vfio/platform/vfio_platform_common.c  | 27 +++
 drivers/vfio/platform/vfio_platform_private.h | 20 
 2 files changed, 47 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 184e9d2..3b7e52c 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -27,6 +27,7 @@
 #define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
 #define DRIVER_DESC "VFIO platform base module"
 
+static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -578,6 +579,32 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
 
+void __vfio_platform_register_reset(struct vfio_platform_reset_node *node)
+{
+   mutex_lock(_lock);
+   list_add(>link, _list);
+   mutex_unlock(_lock);
+}
+EXPORT_SYMBOL_GPL(__vfio_platform_register_reset);
+
+void vfio_platform_unregister_reset(const char *compat,
+   vfio_platform_reset_fn_t fn)
+{
+   struct vfio_platform_reset_node *iter, *temp;
+
+   mutex_lock(_lock);
+   list_for_each_entry_safe(iter, temp, _list, link) {
+   if (!strcmp(iter->compat, compat) && (iter->reset == fn)) {
+   list_del(>link);
+   break;
+   }
+   }
+
+   mutex_unlock(_lock);
+
+}
+EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 7128690..c563940 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -71,6 +71,15 @@ struct vfio_platform_device {
int (*reset)(struct vfio_platform_device *vdev);
 };
 
+typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
+
+struct vfio_platform_reset_node {
+   struct list_head link;
+   char *compat;
+   struct module *owner;
+   vfio_platform_reset_fn_t reset;
+};
+
 struct vfio_platform_reset_combo {
const char *compat;
const char *reset_function_name;
@@ -90,4 +99,15 @@ extern int vfio_platform_set_irqs_ioctl(struct 
vfio_platform_device *vdev,
unsigned start, unsigned count,
void *data);
 
+extern void __vfio_platform_register_reset(struct vfio_platform_reset_node *n);
+extern void vfio_platform_unregister_reset(const char *compat,
+  vfio_platform_reset_fn_t fn);
+#define vfio_platform_register_reset(__compat, __reset)\
+static struct vfio_platform_reset_node __reset ## _node = {\
+   .owner = THIS_MODULE,   \
+   .compat = __compat, \
+   .reset = __reset,   \
+}; \
+__vfio_platform_register_reset(&__reset ## _node)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 3/7] vfio: platform: introduce module_vfio_reset_handler macro

[Eric Auger]

The module_vfio_reset_handler macro
- define a module alias
- implement module init/exit function which respectively registers
  and unregisters the reset function.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---
v4 -> v5:
- add Arnd's R-b

v3 -> v4:
- pass reset to vfio_platform_unregister_reset

v2 -> v3:
- use vfio_platform_register_reset macro

v1 -> v2:
- remove vfio_platform_reset_private.h and move back the macro to
  vfio_platform_private.h header: removed reset_module_register &
  unregister (symbol_get)
- defines the module_vfio_reset_handler macro as suggested by Arnd
  (formerly in vfio_platform_reset_private.h)
---
 drivers/vfio/platform/vfio_platform_private.h | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index c563940..fd262be 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -110,4 +110,18 @@ static struct vfio_platform_reset_node __reset ## _node = 
{\
 }; \
 __vfio_platform_register_reset(&__reset ## _node)
 
+#define module_vfio_reset_handler(compat, reset)   \
+MODULE_ALIAS("vfio-reset:" compat);\
+static int __init reset ## _module_init(void)  \
+{  \
+   vfio_platform_register_reset(compat, reset);\
+   return 0;   \
+}; \
+static void __exit reset ## _module_exit(void) \
+{  \
+   vfio_platform_unregister_reset(compat, reset);  \
+}; \
+module_init(reset ## _module_init);\
+module_exit(reset ## _module_exit)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 1/7] vfio: platform: introduce vfio-platform-base module

[Eric Auger]

To prepare for vfio platform reset rework let's build
vfio_platform_common.c and vfio_platform_irq.c in a separate
module from vfio-platform and vfio-amba. This makes possible
to have separate module inits and works around a race between
platform driver init and vfio reset module init: that way we
make sure symbols exported by base are available when vfio-platform
driver gets probed.

The open/release being implemented in the base module, the ref
count is applied to the parent module instead.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Suggested-by: Arnd Bergmann <a...@arndb.de>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---
v3 -> v4:
- add Arnd R-b

v3: creation
---
 drivers/vfio/platform/Makefile|  6 --
 drivers/vfio/platform/vfio_amba.c |  1 +
 drivers/vfio/platform/vfio_platform.c |  1 +
 drivers/vfio/platform/vfio_platform_common.c  | 13 +++--
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 5 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/platform/Makefile b/drivers/vfio/platform/Makefile
index 9ce8afe..41a6224 100644
--- a/drivers/vfio/platform/Makefile
+++ b/drivers/vfio/platform/Makefile
@@ -1,10 +1,12 @@
-
-vfio-platform-y := vfio_platform.o vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-base-y := vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-y := vfio_platform.o
 
 obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform.o
+obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_PLATFORM) += reset/
 
 vfio-amba-y := vfio_amba.o
 
 obj-$(CONFIG_VFIO_AMBA) += vfio-amba.o
+obj-$(CONFIG_VFIO_AMBA) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_AMBA) += reset/
diff --git a/drivers/vfio/platform/vfio_amba.c 
b/drivers/vfio/platform/vfio_amba.c
index ff0331f..a66479b 100644
--- a/drivers/vfio/platform/vfio_amba.c
+++ b/drivers/vfio/platform/vfio_amba.c
@@ -67,6 +67,7 @@ static int vfio_amba_probe(struct amba_device *adev, const 
struct amba_id *id)
vdev->flags = VFIO_DEVICE_FLAGS_AMBA;
vdev->get_resource = get_amba_resource;
vdev->get_irq = get_amba_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret) {
diff --git a/drivers/vfio/platform/vfio_platform.c 
b/drivers/vfio/platform/vfio_platform.c
index cef645c..f1625dc 100644
--- a/drivers/vfio/platform/vfio_platform.c
+++ b/drivers/vfio/platform/vfio_platform.c
@@ -65,6 +65,7 @@ static int vfio_platform_probe(struct platform_device *pdev)
vdev->flags = VFIO_DEVICE_FLAGS_PLATFORM;
vdev->get_resource = get_platform_resource;
vdev->get_irq = get_platform_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret)
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e43efb5..184e9d2 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -23,6 +23,10 @@
 
 #include "vfio_platform_private.h"
 
+#define DRIVER_VERSION  "0.10"
+#define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
+#define DRIVER_DESC "VFIO platform base module"
+
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -146,7 +150,7 @@ static void vfio_platform_release(void *device_data)
 
mutex_unlock(_lock);
 
-   module_put(THIS_MODULE);
+   module_put(vdev->parent_module);
 }
 
 static int vfio_platform_open(void *device_data)
@@ -154,7 +158,7 @@ static int vfio_platform_open(void *device_data)
struct vfio_platform_device *vdev = device_data;
int ret;
 
-   if (!try_module_get(THIS_MODULE))
+   if (!try_module_get(vdev->parent_module))
return -ENODEV;
 
mutex_lock(_lock);
@@ -573,3 +577,8 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
return vdev;
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
+
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR(DRIVER_AUTHOR);
+MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1c9b3d5..7128690 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -56,6 +56,7 @@ struct vfio_platform_device {
u32 num_irqs;
int refcnt;
struct mutexigate;
+   struct module   *parent_module;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 0/7] VFIO platform reset module rework

[Eric Auger]

This series fixes the current implementation by getting rid of the
usage of __symbol_get which caused a compilation issue with
CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
possible to add a new reset module without being obliged to update the
framework. The new implementation relies on the reset module registering
its reset function to the vfio-platform driver.

The series is available at

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc7-rework-v5

Best Regards

Eric

v4 -> v5:
- no code change
- only added Arnd's new R-b

v3 -> v4:
- Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) later
  in [6/7], to keep the functionality working all along the series
- Add Arnd R-b (I dared to keep them despite the above change)
- vfio_platform_unregister_reset gets the reset function to do a double
  check on the compat and the function pointer too
- __vfio_platform_register_reset turned to 'void'

v2 -> v3:
- use driver_mutex instead of reset_mutex
- style fixes: single mutex_unlock
- use static nodes; vfio_platform_register_reset now is a macro
- vfio_platform_reset_private.h removed since reset_module_(un)register
  disappear. No use of symbol_get anymore.
- new patch introducing vfio-platform-base
- reset look-up moved back at vfio-platform probe time
- new patch featuring dev_info/dev_warn

v1 -> v2:
* in vfio_platform_common.c:
  - move reset lookup at load time and put reset at release: this is to
prevent a race between the 2 load module loads
  - reset_list becomes static
  - vfio_platform_register/unregister_reset take a const char * as compat
  - fix node link
  - remove old combo struct and cleanup proto of vfio_platform_get_reset
  - add mutex to protect the reset list
* in calxeda xgmac reset module
  - introduce vfio_platform_reset_private.h
  - use module_vfio_reset_handler macro
  - do not export vfio_platform_calxedaxgmac_reset symbol anymore
  - add a pr_info to show the device is reset by vfio reset module


Eric Auger (7):
  vfio: platform: introduce vfio-platform-base module
  vfio: platform: add capability to register a reset function
  vfio: platform: introduce module_vfio_reset_handler macro
  vfio: platform: reset: calxedaxgmac: add reset function registration
  vfio: platform: add compat in vfio_platform_device
  vfio: platform: use list of registered reset function
  vfio: platform: add dev_info on device reset

 drivers/vfio/platform/Makefile |   6 +-
 .../platform/reset/vfio_platform_calxedaxgmac.c|   5 +-
 drivers/vfio/platform/vfio_amba.c  |   1 +
 drivers/vfio/platform/vfio_platform.c  |   1 +
 drivers/vfio/platform/vfio_platform_common.c   | 119 +++--
 drivers/vfio/platform/vfio_platform_private.h  |  40 ++-
 6 files changed, 130 insertions(+), 42 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

Current vfio_pgsize_bitmap code hides the supported IOMMU page
sizes smaller than PAGE_SIZE. As a result, in case the IOMMU
does not support PAGE_SIZE page, the alignment check on map/unmap
is done with larger page sizes, if any. This can fail although
mapping could be done with pages smaller than PAGE_SIZE.

vfio_pgsize_bitmap is modified to expose the IOMMU page sizes,
supported by all domains, even those smaller than PAGE_SIZE. The
alignment check on map is performed against PAGE_SIZE if the minimum
IOMMU size is less than PAGE_SIZE or against the min page size greater
than PAGE_SIZE.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

This was tested on AMD Seattle with 64kB page host. ARM MMU 401
currently expose 4kB, 2MB and 1GB page support. With a 64kB page host,
the map/unmap check is done against 2MB. Some alignment check fail
so VFIO_IOMMU_MAP_DMA fail while we could map using 4kB IOMMU page
size.
---
 drivers/vfio/vfio_iommu_type1.c | 25 +++--
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 57d8c37..13fb974 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -403,7 +403,7 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
struct vfio_dma *dma)
 static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
 {
struct vfio_domain *domain;
-   unsigned long bitmap = PAGE_MASK;
+   unsigned long bitmap = ULONG_MAX;
 
mutex_lock(>lock);
list_for_each_entry(domain, >domain_list, next)
@@ -416,20 +416,18 @@ static unsigned long vfio_pgsize_bitmap(struct vfio_iommu 
*iommu)
 static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
 struct vfio_iommu_type1_dma_unmap *unmap)
 {
-   uint64_t mask;
struct vfio_dma *dma;
size_t unmapped = 0;
int ret = 0;
+   unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
+   unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
+   PAGE_SIZE : min_pagesz;
 
-   mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
-
-   if (unmap->iova & mask)
+   if (!IS_ALIGNED(unmap->iova, requested_alignment))
return -EINVAL;
-   if (!unmap->size || unmap->size & mask)
+   if (!unmap->size || !IS_ALIGNED(unmap->size, requested_alignment))
return -EINVAL;
 
-   WARN_ON(mask & PAGE_MASK);
-
mutex_lock(>lock);
 
/*
@@ -553,25 +551,24 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu,
size_t size = map->size;
long npage;
int ret = 0, prot = 0;
-   uint64_t mask;
struct vfio_dma *dma;
unsigned long pfn;
+   unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
+   unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
+   PAGE_SIZE : min_pagesz;
 
/* Verify that none of our __u64 fields overflow */
if (map->size != size || map->vaddr != vaddr || map->iova != iova)
return -EINVAL;
 
-   mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
-
-   WARN_ON(mask & PAGE_MASK);
-
/* READ/WRITE from device perspective */
if (map->flags & VFIO_DMA_MAP_FLAG_WRITE)
prot |= IOMMU_WRITE;
if (map->flags & VFIO_DMA_MAP_FLAG_READ)
prot |= IOMMU_READ;
 
-   if (!prot || !size || (size | iova | vaddr) & mask)
+   if (!prot || !size ||
+   !IS_ALIGNED(size | iova | vaddr, requested_alignment))
return -EINVAL;
 
/* Don't allow IOVA or virtual address wrap */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 5/7] vfio: platform: add compat in vfio_platform_device

[Eric Auger]

Let's retrieve the compatibility string on probe and store it
in the vfio_platform_device struct

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2 -> v3:
- populate compat after vdev check
---
 drivers/vfio/platform/vfio_platform_common.c  | 15 ---
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 3b7e52c..f2d41a0 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -41,16 +41,11 @@ static const struct vfio_platform_reset_combo 
reset_lookup_table[] = {
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   const char *compat;
int (*reset)(struct vfio_platform_device *);
-   int ret, i;
-
-   ret = device_property_read_string(dev, "compatible", );
-   if (ret)
-   return;
+   int i;
 
for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, compat)) {
+   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
request_module(reset_lookup_table[i].module_name);
reset = __symbol_get(
reset_lookup_table[i].reset_function_name);
@@ -544,6 +539,12 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
if (!vdev)
return -EINVAL;
 
+   ret = device_property_read_string(dev, "compatible", >compat);
+   if (ret) {
+   pr_err("VFIO: cannot retrieve compat for %s\n", vdev->name);
+   return -EINVAL;
+   }
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index fd262be..415310f 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -57,6 +57,7 @@ struct vfio_platform_device {
int refcnt;
struct mutexigate;
struct module   *parent_module;
+   const char  *compat;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 7/7] vfio: platform: add dev_info on device reset

[Eric Auger]

It might be helpful for the end-user to check the device reset
function was found by the vfio platform reset framework.

Lets store a pointer to the struct device in vfio_platform_device
and trace when the reset function is called or not found.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v3: creation
---
 drivers/vfio/platform/vfio_platform_common.c  | 14 --
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index f74836a..376d289 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -144,8 +144,12 @@ static void vfio_platform_release(void *device_data)
mutex_lock(_lock);
 
if (!(--vdev->refcnt)) {
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
vfio_platform_regions_cleanup(vdev);
vfio_platform_irq_cleanup(vdev);
}
@@ -174,8 +178,12 @@ static int vfio_platform_open(void *device_data)
if (ret)
goto err_irq;
 
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
}
 
vdev->refcnt++;
@@ -551,6 +559,8 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return -EINVAL;
}
 
+   vdev->device = dev;
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index d1b0668..42816dd 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -59,6 +59,7 @@ struct vfio_platform_device {
struct module   *parent_module;
const char  *compat;
struct module   *reset_module;
+   struct device   *device;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 6/7] vfio: platform: use list of registered reset function

[Eric Auger]

Remove the static lookup table and use the dynamic list of registered
reset functions instead. Also load the reset module through its alias.
The reset struct module pointer is stored in vfio_platform_device.

We also remove the useless struct device pointer parameter in
vfio_platform_get_reset.

This patch fixes the issue related to the usage of __symbol_get, which
besides from being moot, prevented compilation with CONFIG_MODULES
disabled.

Also usage of MODULE_ALIAS makes possible to add a new reset module
without needing to update the framework. This was suggested by Arnd.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reported-by: Arnd Bergmann <a...@arndb.de>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v3 -> v4:
- add Arnd R-b.
- Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) here

v2 -> v3:
- remove clear of vfio_platform_device reset_module and reset
  in vfio_platform_put_reset
- single unlock in vfio_platform_lookup_reset
- use driver_lock instead of reset_lock

v1 -> v2:
- use reset_lock in vfio_platform_lookup_reset
- remove vfio_platform_reset_combo declaration
- remove struct device *dev parameter in vfio_platform_get_reset
- set reset_module and reset to NULL in put function
---
 .../platform/reset/vfio_platform_calxedaxgmac.c|  1 -
 drivers/vfio/platform/vfio_platform_common.c   | 52 --
 drivers/vfio/platform/vfio_platform_private.h  |  7 +--
 3 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 80718f2..640f5d8 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -76,7 +76,6 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 
return 0;
 }
-EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
 
 module_vfio_reset_handler("calxeda,hb-xgmac", 
vfio_platform_calxedaxgmac_reset);
 
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index f2d41a0..f74836a 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -30,37 +30,43 @@
 static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
-static const struct vfio_platform_reset_combo reset_lookup_table[] = {
-   {
-   .compat = "calxeda,hb-xgmac",
-   .reset_function_name = "vfio_platform_calxedaxgmac_reset",
-   .module_name = "vfio-platform-calxedaxgmac",
-   },
-};
-
-static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
-   struct device *dev)
+static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat,
+   struct module **module)
 {
-   int (*reset)(struct vfio_platform_device *);
-   int i;
+   struct vfio_platform_reset_node *iter;
+   vfio_platform_reset_fn_t reset_fn = NULL;
 
-   for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
-   request_module(reset_lookup_table[i].module_name);
-   reset = __symbol_get(
-   reset_lookup_table[i].reset_function_name);
-   if (reset) {
-   vdev->reset = reset;
-   return;
-   }
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat) &&
+   try_module_get(iter->owner)) {
+   *module = iter->owner;
+   reset_fn = iter->reset;
+   break;
}
}
+   mutex_unlock(_lock);
+   return reset_fn;
+}
+
+static void vfio_platform_get_reset(struct vfio_platform_device *vdev)
+{
+   char modname[256];
+
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+   >reset_module);
+   if (!vdev->reset) {
+   snprintf(modname, 256, "vfio-reset:%s", vdev->compat);
+   request_module(modname);
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+>reset_module);
+   }
 }
 
 static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
 {
if (vdev->reset)
-   symbol_put_addr(vdev->reset);
+   module_put(vdev->reset_module);
 }
 
 static int vfio_platform_regions_init(struct vfio_platform_device *vdev)
@@ -557,7 +563,7 @@ int vfio_platform_probe_common(struct vfio_platf

Re: [RFC] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

Hi Alex,
On 10/28/2015 05:27 PM, Alex Williamson wrote:
> On Wed, 2015-10-28 at 13:12 +0000, Eric Auger wrote:
>> Current vfio_pgsize_bitmap code hides the supported IOMMU page
>> sizes smaller than PAGE_SIZE. As a result, in case the IOMMU
>> does not support PAGE_SIZE page, the alignment check on map/unmap
>> is done with larger page sizes, if any. This can fail although
>> mapping could be done with pages smaller than PAGE_SIZE.
>>
>> vfio_pgsize_bitmap is modified to expose the IOMMU page sizes,
>> supported by all domains, even those smaller than PAGE_SIZE. The
>> alignment check on map is performed against PAGE_SIZE if the minimum
>> IOMMU size is less than PAGE_SIZE or against the min page size greater
>> than PAGE_SIZE.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>>
>> ---
>>
>> This was tested on AMD Seattle with 64kB page host. ARM MMU 401
>> currently expose 4kB, 2MB and 1GB page support. With a 64kB page host,
>> the map/unmap check is done against 2MB. Some alignment check fail
>> so VFIO_IOMMU_MAP_DMA fail while we could map using 4kB IOMMU page
>> size.
>> ---
>>  drivers/vfio/vfio_iommu_type1.c | 25 +++--
>>  1 file changed, 11 insertions(+), 14 deletions(-)
>>
>> diff --git a/drivers/vfio/vfio_iommu_type1.c 
>> b/drivers/vfio/vfio_iommu_type1.c
>> index 57d8c37..13fb974 100644
>> --- a/drivers/vfio/vfio_iommu_type1.c
>> +++ b/drivers/vfio/vfio_iommu_type1.c
>> @@ -403,7 +403,7 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
>> struct vfio_dma *dma)
>>  static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
>>  {
>>  struct vfio_domain *domain;
>> -unsigned long bitmap = PAGE_MASK;
>> +unsigned long bitmap = ULONG_MAX;
> 
> Isn't this and removing the WARN_ON()s the only real change in this
> patch?  The rest looks like conversion to use IS_ALIGNED and the
> following test, that I don't really understand...
Yes basically you're right.
> 
>>  
>>  mutex_lock(>lock);
>>  list_for_each_entry(domain, >domain_list, next)
>> @@ -416,20 +416,18 @@ static unsigned long vfio_pgsize_bitmap(struct 
>> vfio_iommu *iommu)
>>  static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
>>   struct vfio_iommu_type1_dma_unmap *unmap)
>>  {
>> -uint64_t mask;
>>  struct vfio_dma *dma;
>>  size_t unmapped = 0;
>>  int ret = 0;
>> +unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>> +unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
>> +PAGE_SIZE : min_pagesz;
> 
> This one.  If we're going to support sub-PAGE_SIZE mappings, why do we
> care to cap alignment at PAGE_SIZE?
My intent in this patch isn't to allow the user-space to map/unmap
sub-PAGE_SIZE buffers. The new test makes sure the mapped area is bigger
or equal than a host page whatever the supported page sizes.

I noticed that chunk construction, pinning and other many things are
based on PAGE_SIZE and far be it from me to change that code! I want to
keep that minimal granularity for all those computation.

However on iommu side, I would like to rely on the fact the iommu driver
is clever enough to choose the right page size and even to choose a size
that is smaller than PAGE_SIZE if this latter is not supported.
> 
>> -mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
>> -
>> -if (unmap->iova & mask)
>> +if (!IS_ALIGNED(unmap->iova, requested_alignment))
>>  return -EINVAL;
>> -if (!unmap->size || unmap->size & mask)
>> +if (!unmap->size || !IS_ALIGNED(unmap->size, requested_alignment))
>>  return -EINVAL;
>>  
>> -WARN_ON(mask & PAGE_MASK);
>> -
>>  mutex_lock(>lock);
>>  
>>  /*
>> @@ -553,25 +551,24 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu,
>>  size_t size = map->size;
>>  long npage;
>>  int ret = 0, prot = 0;
>> -uint64_t mask;
>>  struct vfio_dma *dma;
>>  unsigned long pfn;
>> +unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>> +unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
>> +PAGE_SIZE : min_pagesz;
>>  
>>  /* Verify that none of our __u64 fields overflow */
>>  if (map->size != size || map->vaddr != vaddr || map->iova != iova)
>>  return -EINVAL;
>>  
>> -mask = ((uint6

Re: [RFC] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

On 10/28/2015 06:37 PM, Alex Williamson wrote:
> On Wed, 2015-10-28 at 18:10 +0100, Eric Auger wrote:
>> Hi Alex,
>> On 10/28/2015 05:27 PM, Alex Williamson wrote:
>>> On Wed, 2015-10-28 at 13:12 +, Eric Auger wrote:
>>>> Current vfio_pgsize_bitmap code hides the supported IOMMU page
>>>> sizes smaller than PAGE_SIZE. As a result, in case the IOMMU
>>>> does not support PAGE_SIZE page, the alignment check on map/unmap
>>>> is done with larger page sizes, if any. This can fail although
>>>> mapping could be done with pages smaller than PAGE_SIZE.
>>>>
>>>> vfio_pgsize_bitmap is modified to expose the IOMMU page sizes,
>>>> supported by all domains, even those smaller than PAGE_SIZE. The
>>>> alignment check on map is performed against PAGE_SIZE if the minimum
>>>> IOMMU size is less than PAGE_SIZE or against the min page size greater
>>>> than PAGE_SIZE.
>>>>
>>>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>>>>
>>>> ---
>>>>
>>>> This was tested on AMD Seattle with 64kB page host. ARM MMU 401
>>>> currently expose 4kB, 2MB and 1GB page support. With a 64kB page host,
>>>> the map/unmap check is done against 2MB. Some alignment check fail
>>>> so VFIO_IOMMU_MAP_DMA fail while we could map using 4kB IOMMU page
>>>> size.
>>>> ---
>>>>  drivers/vfio/vfio_iommu_type1.c | 25 +++--
>>>>  1 file changed, 11 insertions(+), 14 deletions(-)
>>>>
>>>> diff --git a/drivers/vfio/vfio_iommu_type1.c 
>>>> b/drivers/vfio/vfio_iommu_type1.c
>>>> index 57d8c37..13fb974 100644
>>>> --- a/drivers/vfio/vfio_iommu_type1.c
>>>> +++ b/drivers/vfio/vfio_iommu_type1.c
>>>> @@ -403,7 +403,7 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
>>>> struct vfio_dma *dma)
>>>>  static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
>>>>  {
>>>>struct vfio_domain *domain;
>>>> -  unsigned long bitmap = PAGE_MASK;
>>>> +  unsigned long bitmap = ULONG_MAX;
>>>
>>> Isn't this and removing the WARN_ON()s the only real change in this
>>> patch?  The rest looks like conversion to use IS_ALIGNED and the
>>> following test, that I don't really understand...
>> Yes basically you're right.
> 
> 
> Ok, so with hopefully correcting my understand of what this does, isn't
> this effectively the same:
> 
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 57d8c37..7db4f5a 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -403,13 +403,19 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
> stru
>  static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
>  {
> struct vfio_domain *domain;
> -   unsigned long bitmap = PAGE_MASK;
> +   unsigned long bitmap = ULONG_MAX;
>  
> mutex_lock(>lock);
> list_for_each_entry(domain, >domain_list, next)
> bitmap &= domain->domain->ops->pgsize_bitmap;
> mutex_unlock(>lock);
>  
> +   /* Some comment about how the IOMMU API splits requests */
> +   if (bitmap & ~PAGE_MASK) {
> +   bitmap &= PAGE_MASK;
> +   bitmap |= PAGE_SIZE;
> +   }
> +
> return bitmap;
>  }
Yes, to me it is indeed the same
>  
> This would also expose to the user that we're accepting PAGE_SIZE, which
> we weren't before, so it was not quite right to just let them do it
> anyway.  I don't think we even need to get rid of the WARN_ONs, do we?
> Thanks,

The end-user might be afraid of those latter. Personally I would get rid
of them but that's definitively up to you.

Just let me know and I will respin.

Best Regards

Eric

> 
> Alex
> 
>>>
>>>>  
>>>>mutex_lock(>lock);
>>>>list_for_each_entry(domain, >domain_list, next)
>>>> @@ -416,20 +416,18 @@ static unsigned long vfio_pgsize_bitmap(struct 
>>>> vfio_iommu *iommu)
>>>>  static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
>>>> struct vfio_iommu_type1_dma_unmap *unmap)
>>>>  {
>>>> -  uint64_t mask;
>>>>struct vfio_dma *dma;
>>>>size_t unmapped = 0;
>>>>int ret = 0;
>>>> +  unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>>>> +  unsigned int requested_alignment 

Re: [RFC] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

Hi Will,
On 10/28/2015 06:14 PM, Will Deacon wrote:
> On Wed, Oct 28, 2015 at 10:27:28AM -0600, Alex Williamson wrote:
>> On Wed, 2015-10-28 at 13:12 +0000, Eric Auger wrote:
>>> diff --git a/drivers/vfio/vfio_iommu_type1.c 
>>> b/drivers/vfio/vfio_iommu_type1.c
>>> index 57d8c37..13fb974 100644
>>> --- a/drivers/vfio/vfio_iommu_type1.c
>>> +++ b/drivers/vfio/vfio_iommu_type1.c
>>> @@ -403,7 +403,7 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
>>> struct vfio_dma *dma)
>>>  static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
>>>  {
>>> struct vfio_domain *domain;
>>> -   unsigned long bitmap = PAGE_MASK;
>>> +   unsigned long bitmap = ULONG_MAX;
>>
>> Isn't this and removing the WARN_ON()s the only real change in this
>> patch?  The rest looks like conversion to use IS_ALIGNED and the
>> following test, that I don't really understand...
>>
>>>  
>>> mutex_lock(>lock);
>>> list_for_each_entry(domain, >domain_list, next)
>>> @@ -416,20 +416,18 @@ static unsigned long vfio_pgsize_bitmap(struct 
>>> vfio_iommu *iommu)
>>>  static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
>>>  struct vfio_iommu_type1_dma_unmap *unmap)
>>>  {
>>> -   uint64_t mask;
>>> struct vfio_dma *dma;
>>> size_t unmapped = 0;
>>> int ret = 0;
>>> +   unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>>> +   unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
>>> +   PAGE_SIZE : min_pagesz;
>>
>> This one.  If we're going to support sub-PAGE_SIZE mappings, why do we
>> care to cap alignment at PAGE_SIZE?
> 
> Eric can clarify, but I think the intention here is to have VFIO continue
> doing things in PAGE_SIZE chunks precisely so that we don't have to rework
> all of the pinning code etc.
That's my intention indeed ;-)

Thanks

Eric
 The IOMMU API can then deal with the smaller
> page size.

> 
>>> -   mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
>>> -
>>> -   if (unmap->iova & mask)
>>> +   if (!IS_ALIGNED(unmap->iova, requested_alignment))
>>> return -EINVAL;
>>> -   if (!unmap->size || unmap->size & mask)
>>> +   if (!unmap->size || !IS_ALIGNED(unmap->size, requested_alignment))
>>> return -EINVAL;
>>>  
>>> -   WARN_ON(mask & PAGE_MASK);
>>> -
>>> mutex_lock(>lock);
>>>  
>>> /*
>>> @@ -553,25 +551,24 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu,
>>> size_t size = map->size;
>>> long npage;
>>> int ret = 0, prot = 0;
>>> -   uint64_t mask;
>>> struct vfio_dma *dma;
>>> unsigned long pfn;
>>> +   unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>>> +   unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
>>> +   PAGE_SIZE : min_pagesz;
>>>  
>>> /* Verify that none of our __u64 fields overflow */
>>> if (map->size != size || map->vaddr != vaddr || map->iova != iova)
>>> return -EINVAL;
>>>  
>>> -   mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
>>> -
>>> -   WARN_ON(mask & PAGE_MASK);
>>> -
>>> /* READ/WRITE from device perspective */
>>> if (map->flags & VFIO_DMA_MAP_FLAG_WRITE)
>>> prot |= IOMMU_WRITE;
>>> if (map->flags & VFIO_DMA_MAP_FLAG_READ)
>>> prot |= IOMMU_READ;
>>>  
>>> -   if (!prot || !size || (size | iova | vaddr) & mask)
>>> +   if (!prot || !size ||
>>> +   !IS_ALIGNED(size | iova | vaddr, requested_alignment))
>>> return -EINVAL;
>>>  
>>> /* Don't allow IOVA or virtual address wrap */
>>
>> This is mostly ignoring the problems with sub-PAGE_SIZE mappings.  For
>> instance, we can only pin on PAGE_SIZE and therefore we only do
>> accounting on PAGE_SIZE, so if the user does 4K mappings across your 64K
>> page, that page gets pinned and accounted 16 times.  Are we going to
>> tell users that their locked memory limit needs to be 16x now?  The rest
>> of the code would need an audit as well to see what other sub-page bugs
>> might be hiding.  Thanks,
> 
> I don't see that. The pinning all happens the same in VFIO, which can
> then happily pass a 64k region to iommu_map. iommu_map will then call
> ->map in 4k chunks on the IOMMU driver ops.
> 
> Will
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

On 10/28/2015 06:55 PM, Will Deacon wrote:
> On Wed, Oct 28, 2015 at 06:48:41PM +0100, Eric Auger wrote:
>> On 10/28/2015 06:37 PM, Alex Williamson wrote:
>>> Ok, so with hopefully correcting my understand of what this does, isn't
>>> this effectively the same:
>>>
>>> diff --git a/drivers/vfio/vfio_iommu_type1.c 
>>> b/drivers/vfio/vfio_iommu_type1.c
>>> index 57d8c37..7db4f5a 100644
>>> --- a/drivers/vfio/vfio_iommu_type1.c
>>> +++ b/drivers/vfio/vfio_iommu_type1.c
>>> @@ -403,13 +403,19 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
>>> stru
>>>  static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
>>>  {
>>> struct vfio_domain *domain;
>>> -   unsigned long bitmap = PAGE_MASK;
>>> +   unsigned long bitmap = ULONG_MAX;
>>>  
>>> mutex_lock(>lock);
>>> list_for_each_entry(domain, >domain_list, next)
>>> bitmap &= domain->domain->ops->pgsize_bitmap;
>>> mutex_unlock(>lock);
>>>  
>>> +   /* Some comment about how the IOMMU API splits requests */
>>> +   if (bitmap & ~PAGE_MASK) {
>>> +   bitmap &= PAGE_MASK;
>>> +   bitmap |= PAGE_SIZE;
>>> +   }
>>> +
>>> return bitmap;
>>>  }
>> Yes, to me it is indeed the same
>>>  
>>> This would also expose to the user that we're accepting PAGE_SIZE, which
>>> we weren't before, so it was not quite right to just let them do it
>>> anyway.  I don't think we even need to get rid of the WARN_ONs, do we?
>>> Thanks,
>>
>> The end-user might be afraid of those latter. Personally I would get rid
>> of them but that's definitively up to you.
> 
> I think Alex's point is that the WARN_ON's won't trigger with this patch,
> because he clears those lower bits in the bitmap.
ah yes sure!

Thanks

Eric
> 
> Will
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] vfio/type1: handle case where IOMMU does not support PAGE_SIZE size

[Eric Auger]

Alex,
On 10/28/2015 06:28 PM, Alex Williamson wrote:
> On Wed, 2015-10-28 at 17:14 +, Will Deacon wrote:
>> On Wed, Oct 28, 2015 at 10:27:28AM -0600, Alex Williamson wrote:
>>> On Wed, 2015-10-28 at 13:12 +, Eric Auger wrote:
>>>> diff --git a/drivers/vfio/vfio_iommu_type1.c 
>>>> b/drivers/vfio/vfio_iommu_type1.c
>>>> index 57d8c37..13fb974 100644
>>>> --- a/drivers/vfio/vfio_iommu_type1.c
>>>> +++ b/drivers/vfio/vfio_iommu_type1.c
>>>> @@ -403,7 +403,7 @@ static void vfio_remove_dma(struct vfio_iommu *iommu, 
>>>> struct vfio_dma *dma)
>>>>  static unsigned long vfio_pgsize_bitmap(struct vfio_iommu *iommu)
>>>>  {
>>>>struct vfio_domain *domain;
>>>> -  unsigned long bitmap = PAGE_MASK;
>>>> +  unsigned long bitmap = ULONG_MAX;
>>>
>>> Isn't this and removing the WARN_ON()s the only real change in this
>>> patch?  The rest looks like conversion to use IS_ALIGNED and the
>>> following test, that I don't really understand...
>>>
>>>>  
>>>>mutex_lock(>lock);
>>>>list_for_each_entry(domain, >domain_list, next)
>>>> @@ -416,20 +416,18 @@ static unsigned long vfio_pgsize_bitmap(struct 
>>>> vfio_iommu *iommu)
>>>>  static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
>>>> struct vfio_iommu_type1_dma_unmap *unmap)
>>>>  {
>>>> -  uint64_t mask;
>>>>struct vfio_dma *dma;
>>>>size_t unmapped = 0;
>>>>int ret = 0;
>>>> +  unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>>>> +  unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
>>>> +  PAGE_SIZE : min_pagesz;
>>>
>>> This one.  If we're going to support sub-PAGE_SIZE mappings, why do we
>>> care to cap alignment at PAGE_SIZE?
>>
>> Eric can clarify, but I think the intention here is to have VFIO continue
>> doing things in PAGE_SIZE chunks precisely so that we don't have to rework
>> all of the pinning code etc. The IOMMU API can then deal with the smaller
>> page size.
> 
> Gak, I read this wrong.  So really we're just artificially adding
> PAGE_SIZE as a supported IOMMU size so long as the IOMMU support
> something smaller than PAGE_SIZE, where PAGE_SIZE is obviously a
> multiple of that smaller size.  Ok, but should we just do this once in
> vfio_pgsize_bitmap()?  This is exactly why VT-d just reports ~(4k - 1)
> for the iommu bitmap.
Yes I can do this in vfio_pgsize_bitmap if you prefer.

Thanks

Eric

> 
>>>> -  mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
>>>> -
>>>> -  if (unmap->iova & mask)
>>>> +  if (!IS_ALIGNED(unmap->iova, requested_alignment))
>>>>return -EINVAL;
>>>> -  if (!unmap->size || unmap->size & mask)
>>>> +  if (!unmap->size || !IS_ALIGNED(unmap->size, requested_alignment))
>>>>return -EINVAL;
>>>>  
>>>> -  WARN_ON(mask & PAGE_MASK);
>>>> -
>>>>mutex_lock(>lock);
>>>>  
>>>>/*
>>>> @@ -553,25 +551,24 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu,
>>>>size_t size = map->size;
>>>>long npage;
>>>>int ret = 0, prot = 0;
>>>> -  uint64_t mask;
>>>>struct vfio_dma *dma;
>>>>unsigned long pfn;
>>>> +  unsigned int min_pagesz = __ffs(vfio_pgsize_bitmap(iommu));
>>>> +  unsigned int requested_alignment = (min_pagesz < PAGE_SIZE) ?
>>>> +  PAGE_SIZE : min_pagesz;
>>>>  
>>>>/* Verify that none of our __u64 fields overflow */
>>>>if (map->size != size || map->vaddr != vaddr || map->iova != iova)
>>>>return -EINVAL;
>>>>  
>>>> -  mask = ((uint64_t)1 << __ffs(vfio_pgsize_bitmap(iommu))) - 1;
>>>> -
>>>> -  WARN_ON(mask & PAGE_MASK);
>>>> -
>>>>/* READ/WRITE from device perspective */
>>>>if (map->flags & VFIO_DMA_MAP_FLAG_WRITE)
>>>>prot |= IOMMU_WRITE;
>>>>if (map->flags & VFIO_DMA_MAP_FLAG_READ)
>>>>prot |= IOMMU_READ;
>>>>  
>>>> -  if (!prot || !size || (size | iova | vaddr) & mask)
>>>> +  if (!prot || !size ||
>>>&

Re: [PATCH v3] VFIO: platform: reset: AMD xgbe reset module

[Eric Auger]

Dear all,

I think the problem comes from the fact this patch applies on top of
"VFIO platform reset module rework",
https://lkml.org/lkml/2015/10/23/385. This is where
module_vfio_reset_handler macro is introduced. Maybe I should have
reverted the patch into RFC after decision to rework the framework.

Best Regards

Eric


On 10/24/2015 10:41 AM, kbuild test robot wrote:
> Hi Eric,
> 
> [auto build test ERROR on asm-generic/master -- if it's inappropriate base, 
> please suggest rules for selecting the more suitable base]
> 
> url:
> https://github.com/0day-ci/linux/commits/Eric-Auger/VFIO-platform-reset-AMD-xgbe-reset-module/20151024-000245
> config: arm-allyesconfig (attached as .config)
> reproduce:
> wget 
> https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross
>  -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # save the attached .config to linux build tree
> make.cross ARCH=arm 
> 
> All errors (new ones prefixed by >>):
> 
>>> drivers/vfio/platform/reset/vfio_platform_amdxgbe.c:122:27: error: expected 
>>> declaration specifiers or '...' before string constant
> module_vfio_reset_handler("amd,xgbe-seattle-v1a", 
> vfio_platform_amdxgbe_reset);
>   ^
>>> drivers/vfio/platform/reset/vfio_platform_amdxgbe.c:122:51: error: expected 
>>> declaration specifiers or '...' before 'vfio_platform_amdxgbe_reset'
> module_vfio_reset_handler("amd,xgbe-seattle-v1a", 
> vfio_platform_amdxgbe_reset);
>   ^
> --
>>> /kbuild/src/defs/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c:122:27:
>>>  error: expected declaration specifiers or '...' before string constant
> module_vfio_reset_handler("amd,xgbe-seattle-v1a", 
> vfio_platform_amdxgbe_reset);
>   ^
>>> /kbuild/src/defs/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c:122:51:
>>>  error: expected declaration specifiers or '...' before 
>>> 'vfio_platform_amdxgbe_reset'
> module_vfio_reset_handler("amd,xgbe-seattle-v1a", 
> vfio_platform_amdxgbe_reset);
>   ^
> 
> vim +122 drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
> 
>116if (!count)
>117pr_warn("%s MAC SW reset failed\n", __func__);
>118
>119return 0;
>120}
>121
>  > 122module_vfio_reset_handler("amd,xgbe-seattle-v1a", 
> vfio_platform_amdxgbe_reset);
>123
>124MODULE_VERSION("0.1");
>125MODULE_LICENSE("GPL v2");
> 
> ---
> 0-DAY kernel test infrastructureOpen Source Technology Center
> https://lists.01.org/pipermail/kbuild-all   Intel Corporation
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 2/7] vfio: platform: add capability to register a reset function

[Eric Auger]

In preparation for subsequent changes in reset function lookup,
lets introduce a dynamic list of reset combos (compat string,
reset module, reset function). The list can be populated/voided with
vfio_platform_register/unregister_reset. Those are not yet used in
this patch.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v3 -> v4:
- __vfio_platform_register_reset does not return any value anymore
- vfio_platform_unregister_reset also takes the reset function pointer
  as parameter

v2 -> v3:
- use goto out to have a single mutex_unlock
- implement vfio_platform_register_reset as a macro (suggested by Arnd)
- move reset_node struct declaration back to vfio_platform_private.h
- vfio_platform_unregister_reset does not return any value anymore

v1 -> v2:
- reset_list becomes static
- vfio_platform_register/unregister_reset take a const char * as compat
- fix node leak
- add reset_lock to protect the reset list manipulation
- move vfio_platform_reset_node declaration in vfio_platform_common.c
---
 drivers/vfio/platform/vfio_platform_common.c  | 27 +++
 drivers/vfio/platform/vfio_platform_private.h | 20 
 2 files changed, 47 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 184e9d2..3b7e52c 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -27,6 +27,7 @@
 #define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
 #define DRIVER_DESC "VFIO platform base module"
 
+static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -578,6 +579,32 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
 
+void __vfio_platform_register_reset(struct vfio_platform_reset_node *node)
+{
+   mutex_lock(_lock);
+   list_add(>link, _list);
+   mutex_unlock(_lock);
+}
+EXPORT_SYMBOL_GPL(__vfio_platform_register_reset);
+
+void vfio_platform_unregister_reset(const char *compat,
+   vfio_platform_reset_fn_t fn)
+{
+   struct vfio_platform_reset_node *iter, *temp;
+
+   mutex_lock(_lock);
+   list_for_each_entry_safe(iter, temp, _list, link) {
+   if (!strcmp(iter->compat, compat) && (iter->reset == fn)) {
+   list_del(>link);
+   break;
+   }
+   }
+
+   mutex_unlock(_lock);
+
+}
+EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 7128690..c563940 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -71,6 +71,15 @@ struct vfio_platform_device {
int (*reset)(struct vfio_platform_device *vdev);
 };
 
+typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
+
+struct vfio_platform_reset_node {
+   struct list_head link;
+   char *compat;
+   struct module *owner;
+   vfio_platform_reset_fn_t reset;
+};
+
 struct vfio_platform_reset_combo {
const char *compat;
const char *reset_function_name;
@@ -90,4 +99,15 @@ extern int vfio_platform_set_irqs_ioctl(struct 
vfio_platform_device *vdev,
unsigned start, unsigned count,
void *data);
 
+extern void __vfio_platform_register_reset(struct vfio_platform_reset_node *n);
+extern void vfio_platform_unregister_reset(const char *compat,
+  vfio_platform_reset_fn_t fn);
+#define vfio_platform_register_reset(__compat, __reset)\
+static struct vfio_platform_reset_node __reset ## _node = {\
+   .owner = THIS_MODULE,   \
+   .compat = __compat, \
+   .reset = __reset,   \
+}; \
+__vfio_platform_register_reset(&__reset ## _node)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 4/7] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

This patch adds the reset function registration/unregistration.
This is handled through the module_vfio_reset_handler macro. This
latter also defines a MODULE_ALIAS which simplifies the load from
vfio-platform.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v3 -> v4:
- I restored the EXPORT_SYMBOL which will be removed when switching the
  lookup method
- Add Arnd R-b.

v2 -> v3:
- do not include vfio_platform_reset_private.h anymore (removed)
- remove pr_info
- rework commit message

v1 -> v2:
- uses the module_vfio_reset_handler macro
- add pr_info on vfio reset
- do not export vfio_platform_calxedaxgmac_reset symbol anymore
---
 drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 619dc7d..80718f2 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -30,8 +30,6 @@
 #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
 #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform device"
 
-#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
-
 /* XGMAC Register definitions */
 #define XGMAC_CONTROL   0x  /* MAC Configuration */
 
@@ -80,6 +78,8 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
 
+module_vfio_reset_handler("calxeda,hb-xgmac", 
vfio_platform_calxedaxgmac_reset);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 0/7] VFIO platform reset module rework

[Eric Auger]

This series fixes the current implementation by getting rid of the
usage of __symbol_get which caused a compilation issue with
CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
possible to add a new reset module without being obliged to update the
framework. The new implementation relies on the reset module registering
its reset function to the vfio-platform driver.

The series is available at

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc6-rework-v3

Best Regards

Eric

v3 -> v4:
- Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) later
  in [6/7], to keep the functionality working all along the series
- Add Arnd R-b (I dared to keep them despite the above change)
- vfio_platform_unregister_reset gets the reset function to do a double
  check on the compat and the function pointer too
- __vfio_platform_register_reset turned to 'void'

v2 -> v3:
- use driver_mutex instead of reset_mutex
- style fixes: single mutex_unlock
- use static nodes; vfio_platform_register_reset now is a macro
- vfio_platform_reset_private.h removed since reset_module_(un)register
  disappear. No use of symbol_get anymore.
- new patch introducing vfio-platform-base
- reset look-up moved back at vfio-platform probe time
- new patch featuring dev_info/dev_warn

v1 -> v2:
* in vfio_platform_common.c:
  - move reset lookup at load time and put reset at release: this is to
prevent a race between the 2 load module loads
  - reset_list becomes static
  - vfio_platform_register/unregister_reset take a const char * as compat
  - fix node link
  - remove old combo struct and cleanup proto of vfio_platform_get_reset
  - add mutex to protect the reset list
* in calxeda xgmac reset module
  - introduce vfio_platform_reset_private.h
  - use module_vfio_reset_handler macro
  - do not export vfio_platform_calxedaxgmac_reset symbol anymore
  - add a pr_info to show the device is reset by vfio reset module



Eric Auger (7):
  vfio: platform: introduce vfio-platform-base module
  vfio: platform: add capability to register a reset function
  vfio: platform: introduce module_vfio_reset_handler macro
  vfio: platform: reset: calxedaxgmac: add reset function registration
  vfio: platform: add compat in vfio_platform_device
  vfio: platform: use list of registered reset function
  vfio: platform: add dev_info on device reset

 drivers/vfio/platform/Makefile |   6 +-
 .../platform/reset/vfio_platform_calxedaxgmac.c|   5 +-
 drivers/vfio/platform/vfio_amba.c  |   1 +
 drivers/vfio/platform/vfio_platform.c  |   1 +
 drivers/vfio/platform/vfio_platform_common.c   | 119 +++--
 drivers/vfio/platform/vfio_platform_private.h  |  40 ++-
 6 files changed, 130 insertions(+), 42 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 3/7] vfio: platform: introduce module_vfio_reset_handler macro

[Eric Auger]

The module_vfio_reset_handler macro
- define a module alias
- implement module init/exit function which respectively registers
  and unregisters the reset function.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v3 -> v4:
- pass reset to vfio_platform_unregister_reset

v2 -> v3:
- use vfio_platform_register_reset macro

v1 -> v2:
- remove vfio_platform_reset_private.h and move back the macro to
  vfio_platform_private.h header: removed reset_module_register &
  unregister (symbol_get)
- defines the module_vfio_reset_handler macro as suggested by Arnd
  (formerly in vfio_platform_reset_private.h)
---
 drivers/vfio/platform/vfio_platform_private.h | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index c563940..fd262be 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -110,4 +110,18 @@ static struct vfio_platform_reset_node __reset ## _node = 
{\
 }; \
 __vfio_platform_register_reset(&__reset ## _node)
 
+#define module_vfio_reset_handler(compat, reset)   \
+MODULE_ALIAS("vfio-reset:" compat);\
+static int __init reset ## _module_init(void)  \
+{  \
+   vfio_platform_register_reset(compat, reset);\
+   return 0;   \
+}; \
+static void __exit reset ## _module_exit(void) \
+{  \
+   vfio_platform_unregister_reset(compat, reset);  \
+}; \
+module_init(reset ## _module_init);\
+module_exit(reset ## _module_exit)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3] VFIO: platform: reset: AMD xgbe reset module

[Eric Auger]

This patch introduces a module that registers and implements a low-level
reset function for the AMD XGBE device.

it performs the following actions:
- reset the PHY
- disable auto-negotiation
- disable & clear auto-negotiation IRQ
- soft-reset the MAC

Those tiny pieces of code are inherited from the native xgbe driver.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

Applies on top of [PATCH v3 0/7] VFIO platform reset module rework

v2 -> v3:
- in Kconfig, add empty line between the 2 options
- remove DRIVER_VERSION, DRIVER_AUTHOR and DRIVER_DESC and put
  strings directly in MODULE macros

v1 -> v2:
- uses module_vfio_reset_handler macro
---
 drivers/vfio/platform/reset/Kconfig|   8 ++
 drivers/vfio/platform/reset/Makefile   |   2 +
 .../vfio/platform/reset/vfio_platform_amdxgbe.c| 127 +
 3 files changed, 137 insertions(+)
 create mode 100644 drivers/vfio/platform/reset/vfio_platform_amdxgbe.c

diff --git a/drivers/vfio/platform/reset/Kconfig 
b/drivers/vfio/platform/reset/Kconfig
index 746b96b..705 100644
--- a/drivers/vfio/platform/reset/Kconfig
+++ b/drivers/vfio/platform/reset/Kconfig
@@ -5,3 +5,11 @@ config VFIO_PLATFORM_CALXEDAXGMAC_RESET
  Enables the VFIO platform driver to handle reset for Calxeda xgmac
 
  If you don't know what to do here, say N.
+
+config VFIO_PLATFORM_AMDXGBE_RESET
+   tristate "VFIO support for AMD XGBE reset"
+   depends on VFIO_PLATFORM
+   help
+ Enables the VFIO platform driver to handle reset for AMD XGBE
+
+ If you don't know what to do here, say N.
diff --git a/drivers/vfio/platform/reset/Makefile 
b/drivers/vfio/platform/reset/Makefile
index 2a486af..93f4e23 100644
--- a/drivers/vfio/platform/reset/Makefile
+++ b/drivers/vfio/platform/reset/Makefile
@@ -1,5 +1,7 @@
 vfio-platform-calxedaxgmac-y := vfio_platform_calxedaxgmac.o
+vfio-platform-amdxgbe-y := vfio_platform_amdxgbe.o
 
 ccflags-y += -Idrivers/vfio/platform
 
 obj-$(CONFIG_VFIO_PLATFORM_CALXEDAXGMAC_RESET) += vfio-platform-calxedaxgmac.o
+obj-$(CONFIG_VFIO_PLATFORM_AMDXGBE_RESET) += vfio-platform-amdxgbe.o
diff --git a/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c 
b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
new file mode 100644
index 000..1636e22
--- /dev/null
+++ b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
@@ -0,0 +1,127 @@
+/*
+ * VFIO platform driver specialized for AMD xgbe reset
+ * reset code is inherited from AMD xgbe native driver
+ *
+ * Copyright (c) 2015 Linaro Ltd.
+ *  www.linaro.org
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vfio_platform_private.h"
+
+#define DMA_MR 0x3000
+#define MAC_VR 0x0110
+#define DMA_ISR0x3008
+#define MAC_ISR0x00b0
+#define PCS_MMD_SELECT 0xff
+#define MDIO_AN_INT0x8002
+#define MDIO_AN_INTMASK0x8001
+
+static unsigned int xmdio_read(void *ioaddr, unsigned int mmd,
+  unsigned int reg)
+{
+   unsigned int mmd_address, value;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   value = ioread32(ioaddr + ((mmd_address & 0xff) << 2));
+   return value;
+}
+
+static void xmdio_write(void *ioaddr, unsigned int mmd,
+   unsigned int reg, unsigned int value)
+{
+   unsigned int mmd_address;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   iowrite32(value, ioaddr + ((mmd_address & 0xff) << 2));
+}
+
+int vfio_platform_amdxgbe_reset(struct vfio_platform_device *vdev)
+{
+   struct vfio_platform_region xgmac_regs = vdev->regions[0];
+   struct vfio_platform_region xpcs_regs = vdev->regions[1];
+   u32 dma_mr_value, pcs_value, value;
+   unsigned int count;
+
+   if (!xgmac_regs.ioaddr) {
+   xgmac_regs.ioaddr =
+   ioremap_nocache(xgmac_regs.addr, xgmac_regs.size);
+   if (!xgmac_regs.ioaddr)
+   return -ENOMEM;
+   }
+   if (!xpcs_regs.ioa

[PATCH v4 1/7] vfio: platform: introduce vfio-platform-base module

[Eric Auger]

To prepare for vfio platform reset rework let's build
vfio_platform_common.c and vfio_platform_irq.c in a separate
module from vfio-platform and vfio-amba. This makes possible
to have separate module inits and works around a race between
platform driver init and vfio reset module init: that way we
make sure symbols exported by base are available when vfio-platform
driver gets probed.

The open/release being implemented in the base module, the ref
count is applied to the parent module instead.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Suggested-by: Arnd Bergmann <a...@arndb.de>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---
v3 -> v4:
- add Arnd R-b

v3: creation
---
 drivers/vfio/platform/Makefile|  6 --
 drivers/vfio/platform/vfio_amba.c |  1 +
 drivers/vfio/platform/vfio_platform.c |  1 +
 drivers/vfio/platform/vfio_platform_common.c  | 13 +++--
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 5 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/platform/Makefile b/drivers/vfio/platform/Makefile
index 9ce8afe..41a6224 100644
--- a/drivers/vfio/platform/Makefile
+++ b/drivers/vfio/platform/Makefile
@@ -1,10 +1,12 @@
-
-vfio-platform-y := vfio_platform.o vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-base-y := vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-y := vfio_platform.o
 
 obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform.o
+obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_PLATFORM) += reset/
 
 vfio-amba-y := vfio_amba.o
 
 obj-$(CONFIG_VFIO_AMBA) += vfio-amba.o
+obj-$(CONFIG_VFIO_AMBA) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_AMBA) += reset/
diff --git a/drivers/vfio/platform/vfio_amba.c 
b/drivers/vfio/platform/vfio_amba.c
index ff0331f..a66479b 100644
--- a/drivers/vfio/platform/vfio_amba.c
+++ b/drivers/vfio/platform/vfio_amba.c
@@ -67,6 +67,7 @@ static int vfio_amba_probe(struct amba_device *adev, const 
struct amba_id *id)
vdev->flags = VFIO_DEVICE_FLAGS_AMBA;
vdev->get_resource = get_amba_resource;
vdev->get_irq = get_amba_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret) {
diff --git a/drivers/vfio/platform/vfio_platform.c 
b/drivers/vfio/platform/vfio_platform.c
index cef645c..f1625dc 100644
--- a/drivers/vfio/platform/vfio_platform.c
+++ b/drivers/vfio/platform/vfio_platform.c
@@ -65,6 +65,7 @@ static int vfio_platform_probe(struct platform_device *pdev)
vdev->flags = VFIO_DEVICE_FLAGS_PLATFORM;
vdev->get_resource = get_platform_resource;
vdev->get_irq = get_platform_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret)
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e43efb5..184e9d2 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -23,6 +23,10 @@
 
 #include "vfio_platform_private.h"
 
+#define DRIVER_VERSION  "0.10"
+#define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
+#define DRIVER_DESC "VFIO platform base module"
+
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -146,7 +150,7 @@ static void vfio_platform_release(void *device_data)
 
mutex_unlock(_lock);
 
-   module_put(THIS_MODULE);
+   module_put(vdev->parent_module);
 }
 
 static int vfio_platform_open(void *device_data)
@@ -154,7 +158,7 @@ static int vfio_platform_open(void *device_data)
struct vfio_platform_device *vdev = device_data;
int ret;
 
-   if (!try_module_get(THIS_MODULE))
+   if (!try_module_get(vdev->parent_module))
return -ENODEV;
 
mutex_lock(_lock);
@@ -573,3 +577,8 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
return vdev;
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
+
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR(DRIVER_AUTHOR);
+MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1c9b3d5..7128690 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -56,6 +56,7 @@ struct vfio_platform_device {
u32 num_irqs;
int refcnt;
struct mutexigate;
+   struct module   *parent_module;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 0/7] VFIO platform reset module rework

[Eric Auger]

On 10/23/2015 05:47 PM, Eric Auger wrote:
> This series fixes the current implementation by getting rid of the
> usage of __symbol_get which caused a compilation issue with
> CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
> possible to add a new reset module without being obliged to update the
> framework. The new implementation relies on the reset module registering
> its reset function to the vfio-platform driver.
> 
> The series is available at
> 
> https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc6-rework-v3
argh,

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc6-rework-v4

But scatterbrained as I am, I am pretty sure I will do a new respin.

Thanks for your patience :-(

Eric
> 
> Best Regards
> 
> Eric
> 
> v3 -> v4:
> - Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) later
>   in [6/7], to keep the functionality working all along the series
> - Add Arnd R-b (I dared to keep them despite the above change)
> - vfio_platform_unregister_reset gets the reset function to do a double
>   check on the compat and the function pointer too
> - __vfio_platform_register_reset turned to 'void'
> 
> v2 -> v3:
> - use driver_mutex instead of reset_mutex
> - style fixes: single mutex_unlock
> - use static nodes; vfio_platform_register_reset now is a macro
> - vfio_platform_reset_private.h removed since reset_module_(un)register
>   disappear. No use of symbol_get anymore.
> - new patch introducing vfio-platform-base
> - reset look-up moved back at vfio-platform probe time
> - new patch featuring dev_info/dev_warn
> 
> v1 -> v2:
> * in vfio_platform_common.c:
>   - move reset lookup at load time and put reset at release: this is to
> prevent a race between the 2 load module loads
>   - reset_list becomes static
>   - vfio_platform_register/unregister_reset take a const char * as compat
>   - fix node link
>   - remove old combo struct and cleanup proto of vfio_platform_get_reset
>   - add mutex to protect the reset list
> * in calxeda xgmac reset module
>   - introduce vfio_platform_reset_private.h
>   - use module_vfio_reset_handler macro
>   - do not export vfio_platform_calxedaxgmac_reset symbol anymore
>   - add a pr_info to show the device is reset by vfio reset module
> 
> 
> 
> Eric Auger (7):
>   vfio: platform: introduce vfio-platform-base module
>   vfio: platform: add capability to register a reset function
>   vfio: platform: introduce module_vfio_reset_handler macro
>   vfio: platform: reset: calxedaxgmac: add reset function registration
>   vfio: platform: add compat in vfio_platform_device
>   vfio: platform: use list of registered reset function
>   vfio: platform: add dev_info on device reset
> 
>  drivers/vfio/platform/Makefile |   6 +-
>  .../platform/reset/vfio_platform_calxedaxgmac.c|   5 +-
>  drivers/vfio/platform/vfio_amba.c  |   1 +
>  drivers/vfio/platform/vfio_platform.c  |   1 +
>  drivers/vfio/platform/vfio_platform_common.c   | 119 
> +++--
>  drivers/vfio/platform/vfio_platform_private.h  |  40 ++-
>  6 files changed, 130 insertions(+), 42 deletions(-)
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 5/7] vfio: platform: add compat in vfio_platform_device

[Eric Auger]

Let's retrieve the compatibility string on probe and store it
in the vfio_platform_device struct

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2 -> v3:
- populate compat after vdev check
---
 drivers/vfio/platform/vfio_platform_common.c  | 15 ---
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 3b7e52c..f2d41a0 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -41,16 +41,11 @@ static const struct vfio_platform_reset_combo 
reset_lookup_table[] = {
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   const char *compat;
int (*reset)(struct vfio_platform_device *);
-   int ret, i;
-
-   ret = device_property_read_string(dev, "compatible", );
-   if (ret)
-   return;
+   int i;
 
for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, compat)) {
+   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
request_module(reset_lookup_table[i].module_name);
reset = __symbol_get(
reset_lookup_table[i].reset_function_name);
@@ -544,6 +539,12 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
if (!vdev)
return -EINVAL;
 
+   ret = device_property_read_string(dev, "compatible", >compat);
+   if (ret) {
+   pr_err("VFIO: cannot retrieve compat for %s\n", vdev->name);
+   return -EINVAL;
+   }
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index fd262be..415310f 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -57,6 +57,7 @@ struct vfio_platform_device {
int refcnt;
struct mutexigate;
struct module   *parent_module;
+   const char  *compat;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 6/7] vfio: platform: use list of registered reset function

[Eric Auger]

Remove the static lookup table and use the dynamic list of registered
reset functions instead. Also load the reset module through its alias.
The reset struct module pointer is stored in vfio_platform_device.

We also remove the useless struct device pointer parameter in
vfio_platform_get_reset.

This patch fixes the issue related to the usage of __symbol_get, which
besides from being moot, prevented compilation with CONFIG_MODULES
disabled.

Also usage of MODULE_ALIAS makes possible to add a new reset module
without needing to update the framework. This was suggested by Arnd.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reported-by: Arnd Bergmann <a...@arndb.de>
Reviewed-by: Arnd Bergmann <a...@arndb.de>

---

v3 -> v4:
- add Arnd R-b.
- Remove the EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset) here

v2 -> v3:
- remove clear of vfio_platform_device reset_module and reset
  in vfio_platform_put_reset
- single unlock in vfio_platform_lookup_reset
- use driver_lock instead of reset_lock

v1 -> v2:
- use reset_lock in vfio_platform_lookup_reset
- remove vfio_platform_reset_combo declaration
- remove struct device *dev parameter in vfio_platform_get_reset
- set reset_module and reset to NULL in put function
---
 .../platform/reset/vfio_platform_calxedaxgmac.c|  1 -
 drivers/vfio/platform/vfio_platform_common.c   | 52 --
 drivers/vfio/platform/vfio_platform_private.h  |  7 +--
 3 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 80718f2..640f5d8 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -76,7 +76,6 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 
return 0;
 }
-EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
 
 module_vfio_reset_handler("calxeda,hb-xgmac", 
vfio_platform_calxedaxgmac_reset);
 
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index f2d41a0..f74836a 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -30,37 +30,43 @@
 static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
-static const struct vfio_platform_reset_combo reset_lookup_table[] = {
-   {
-   .compat = "calxeda,hb-xgmac",
-   .reset_function_name = "vfio_platform_calxedaxgmac_reset",
-   .module_name = "vfio-platform-calxedaxgmac",
-   },
-};
-
-static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
-   struct device *dev)
+static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat,
+   struct module **module)
 {
-   int (*reset)(struct vfio_platform_device *);
-   int i;
+   struct vfio_platform_reset_node *iter;
+   vfio_platform_reset_fn_t reset_fn = NULL;
 
-   for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
-   request_module(reset_lookup_table[i].module_name);
-   reset = __symbol_get(
-   reset_lookup_table[i].reset_function_name);
-   if (reset) {
-   vdev->reset = reset;
-   return;
-   }
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat) &&
+   try_module_get(iter->owner)) {
+   *module = iter->owner;
+   reset_fn = iter->reset;
+   break;
}
}
+   mutex_unlock(_lock);
+   return reset_fn;
+}
+
+static void vfio_platform_get_reset(struct vfio_platform_device *vdev)
+{
+   char modname[256];
+
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+   >reset_module);
+   if (!vdev->reset) {
+   snprintf(modname, 256, "vfio-reset:%s", vdev->compat);
+   request_module(modname);
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+>reset_module);
+   }
 }
 
 static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
 {
if (vdev->reset)
-   symbol_put_addr(vdev->reset);
+   module_put(vdev->reset_module);
 }
 
 static int vfio_platform_regions_init(struct vfio_platform_device *vdev)
@@ -557,7 +563,7 @@ int vfio_platform_probe_common(struct vfio_platf

[PATCH v4 7/7] vfio: platform: add dev_info on device reset

[Eric Auger]

It might be helpful for the end-user to check the device reset
function was found by the vfio platform reset framework.

Lets store a pointer to the struct device in vfio_platform_device
and trace when the reset function is called or not found.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v3: creation
---
 drivers/vfio/platform/vfio_platform_common.c  | 14 --
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index f74836a..376d289 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -144,8 +144,12 @@ static void vfio_platform_release(void *device_data)
mutex_lock(_lock);
 
if (!(--vdev->refcnt)) {
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
vfio_platform_regions_cleanup(vdev);
vfio_platform_irq_cleanup(vdev);
}
@@ -174,8 +178,12 @@ static int vfio_platform_open(void *device_data)
if (ret)
goto err_irq;
 
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
}
 
vdev->refcnt++;
@@ -551,6 +559,8 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return -EINVAL;
}
 
+   vdev->device = dev;
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index d1b0668..42816dd 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -59,6 +59,7 @@ struct vfio_platform_device {
struct module   *parent_module;
const char  *compat;
struct module   *reset_module;
+   struct device   *device;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 7/7] vfio: platform: add dev_info on device reset

[Eric Auger]

It might be helpful for the end-user to check the device reset
function was found by the vfio platform reset framework.

Lets store a pointer to the struct device in vfio_platform_device
and trace when the reset function is called or not found.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v3: creation
---
 drivers/vfio/platform/vfio_platform_common.c  | 14 --
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index de3fb33..207fab6 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -144,8 +144,12 @@ static void vfio_platform_release(void *device_data)
mutex_lock(_lock);
 
if (!(--vdev->refcnt)) {
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
vfio_platform_regions_cleanup(vdev);
vfio_platform_irq_cleanup(vdev);
}
@@ -174,8 +178,12 @@ static int vfio_platform_open(void *device_data)
if (ret)
goto err_irq;
 
-   if (vdev->reset)
+   if (vdev->reset) {
+   dev_info(vdev->device, "reset\n");
vdev->reset(vdev);
+   } else {
+   dev_warn(vdev->device, "no reset function found!\n");
+   }
}
 
vdev->refcnt++;
@@ -551,6 +559,8 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return -EINVAL;
}
 
+   vdev->device = dev;
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index e505c15..ccb99b4 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -59,6 +59,7 @@ struct vfio_platform_device {
struct module   *parent_module;
const char  *compat;
struct module   *reset_module;
+   struct device   *device;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 5/7] vfio: platform: add compat in vfio_platform_device

[Eric Auger]

Let's retrieve the compatibility string on probe and store it
in the vfio_platform_device struct

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2 -> v3:
- populate compat after vdev check
---
 drivers/vfio/platform/vfio_platform_common.c  | 15 ---
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 8eccd30..50a388b 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -41,16 +41,11 @@ static const struct vfio_platform_reset_combo 
reset_lookup_table[] = {
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   const char *compat;
int (*reset)(struct vfio_platform_device *);
-   int ret, i;
-
-   ret = device_property_read_string(dev, "compatible", );
-   if (ret)
-   return;
+   int i;
 
for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, compat)) {
+   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
request_module(reset_lookup_table[i].module_name);
reset = __symbol_get(
reset_lookup_table[i].reset_function_name);
@@ -544,6 +539,12 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
if (!vdev)
return -EINVAL;
 
+   ret = device_property_read_string(dev, "compatible", >compat);
+   if (ret) {
+   pr_err("VFIO: cannot retrieve compat for %s\n", vdev->name);
+   return -EINVAL;
+   }
+
group = iommu_group_get(dev);
if (!group) {
pr_err("VFIO: No IOMMU group for device %s\n", vdev->name);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 5a1e8e6..f8072d8 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -57,6 +57,7 @@ struct vfio_platform_device {
int refcnt;
struct mutexigate;
struct module   *parent_module;
+   const char  *compat;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 6/7] vfio: platform: use list of registered reset function

[Eric Auger]

Hi Arnd,
On 10/23/2015 03:12 PM, Arnd Bergmann wrote:
> On Friday 23 October 2015 14:37:14 Eric Auger wrote:
>> Remove the static lookup table and use the dynamic list of registered
>> reset functions instead. Also load the reset module through its alias.
>> The reset struct module pointer is stored in vfio_platform_device.
>>
>> We also remove the useless struct device pointer parameter in
>> vfio_platform_get_reset.
>>
>> This patch fixes the issue related to the usage of __symbol_get, which
>> besides from being moot, prevented compilation with CONFIG_MODULES
>> disabled.
>>
>> Also usage of MODULE_ALIAS makes possible to add a new reset module
>> without needing to update the framework. This was suggested by Arnd.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>> Reported-by: Arnd Bergmann <a...@arndb.de>
>>
>>
> Reviewed-by: Arnd Bergmann <a...@arndb.de>
> 
> but doesn't this need to come before patch 4/7?
Well I don't think so. In [4] we introduce the dynamic registration
method but until this patch we still use the old lookup method in the
static table. I tested and the reset lookup still works in [4].
If we put this one before the registration, the functionality will be
lost here.

Eric

>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 3/7] vfio: platform: introduce module_vfio_reset_handler macro

[Eric Auger]

Hi Arnd,
On 10/23/2015 03:09 PM, Arnd Bergmann wrote:
> On Friday 23 October 2015 14:37:11 Eric Auger wrote:
>> +static int __init reset ## _module_init(void)  \
>> +{  \
>> +   vfio_platform_register_reset(compat, reset);\
>> +   return 0;   \
>> +}; \
>>
> 
> I would make this 'return vfio_platform_register_reset(...)', so loading
> the driver fails if the handler cannot get registered. Alternatively,
> change the return type of vfio_platform_register_reset to 'void'
> so it can never fail.
I will turn __vfio_platform_register_reset to 'void' then since it has
no reason to fail now.

Thanks

Eric

> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 2/7] vfio: platform: add capability to register a reset function

[Eric Auger]

On 10/23/2015 03:07 PM, Arnd Bergmann wrote:
> On Friday 23 October 2015 14:37:10 Eric Auger wrote:
>> +
>> +void vfio_platform_unregister_reset(const char *compat)
>> +{
>> +   struct vfio_platform_reset_node *iter, *temp;
>> +
>> +   mutex_lock(_lock);
>> +   list_for_each_entry_safe(iter, temp, _list, link) {
>> +   if (!strcmp(iter->compat, compat)) {
>> +   list_del(>link);
>> +   break;
>> +   }
>> +   }
>> +
>> +   mutex_unlock(_lock);
>> +}
>> +EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
>>
> 
> This is slightly unsafe in case you ever get two drivers that register
> with the same compat string. If we care about that, we could pass
> and compare both the string and the function pointer, or the
> vfio_platform_reset_node.
OK

Thanks

Eric
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 1/7] vfio: platform: introduce vfio-platform-base module

[Eric Auger]

To prepare for vfio platform reset rework let's build
vfio_platform_common.c and vfio_platform_irq.c in a separate
module from vfio-platform and vfio-amba. This makes possible
to have separate module inits and works around a race between
platform driver init and vfio reset module init: that way we
make sure symbols exported by base are available when vfio-platform
driver gets probed.

The open/release being implemented in the base module, the ref
count is applied to the parent module instead.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Suggested-by: Arnd Bergmann <a...@arndb.de>

---

v3: creation
---
 drivers/vfio/platform/Makefile|  6 --
 drivers/vfio/platform/vfio_amba.c |  1 +
 drivers/vfio/platform/vfio_platform.c |  1 +
 drivers/vfio/platform/vfio_platform_common.c  | 13 +++--
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 5 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/platform/Makefile b/drivers/vfio/platform/Makefile
index 9ce8afe..41a6224 100644
--- a/drivers/vfio/platform/Makefile
+++ b/drivers/vfio/platform/Makefile
@@ -1,10 +1,12 @@
-
-vfio-platform-y := vfio_platform.o vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-base-y := vfio_platform_common.o vfio_platform_irq.o
+vfio-platform-y := vfio_platform.o
 
 obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform.o
+obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_PLATFORM) += reset/
 
 vfio-amba-y := vfio_amba.o
 
 obj-$(CONFIG_VFIO_AMBA) += vfio-amba.o
+obj-$(CONFIG_VFIO_AMBA) += vfio-platform-base.o
 obj-$(CONFIG_VFIO_AMBA) += reset/
diff --git a/drivers/vfio/platform/vfio_amba.c 
b/drivers/vfio/platform/vfio_amba.c
index ff0331f..a66479b 100644
--- a/drivers/vfio/platform/vfio_amba.c
+++ b/drivers/vfio/platform/vfio_amba.c
@@ -67,6 +67,7 @@ static int vfio_amba_probe(struct amba_device *adev, const 
struct amba_id *id)
vdev->flags = VFIO_DEVICE_FLAGS_AMBA;
vdev->get_resource = get_amba_resource;
vdev->get_irq = get_amba_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret) {
diff --git a/drivers/vfio/platform/vfio_platform.c 
b/drivers/vfio/platform/vfio_platform.c
index cef645c..f1625dc 100644
--- a/drivers/vfio/platform/vfio_platform.c
+++ b/drivers/vfio/platform/vfio_platform.c
@@ -65,6 +65,7 @@ static int vfio_platform_probe(struct platform_device *pdev)
vdev->flags = VFIO_DEVICE_FLAGS_PLATFORM;
vdev->get_resource = get_platform_resource;
vdev->get_irq = get_platform_irq;
+   vdev->parent_module = THIS_MODULE;
 
ret = vfio_platform_probe_common(vdev, >dev);
if (ret)
diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e43efb5..184e9d2 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -23,6 +23,10 @@
 
 #include "vfio_platform_private.h"
 
+#define DRIVER_VERSION  "0.10"
+#define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
+#define DRIVER_DESC "VFIO platform base module"
+
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -146,7 +150,7 @@ static void vfio_platform_release(void *device_data)
 
mutex_unlock(_lock);
 
-   module_put(THIS_MODULE);
+   module_put(vdev->parent_module);
 }
 
 static int vfio_platform_open(void *device_data)
@@ -154,7 +158,7 @@ static int vfio_platform_open(void *device_data)
struct vfio_platform_device *vdev = device_data;
int ret;
 
-   if (!try_module_get(THIS_MODULE))
+   if (!try_module_get(vdev->parent_module))
return -ENODEV;
 
mutex_lock(_lock);
@@ -573,3 +577,8 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
return vdev;
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
+
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR(DRIVER_AUTHOR);
+MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1c9b3d5..7128690 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -56,6 +56,7 @@ struct vfio_platform_device {
u32 num_irqs;
int refcnt;
struct mutexigate;
+   struct module   *parent_module;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2] VFIO: platform: reset: AMD xgbe reset module

[Eric Auger]

This patch introduces a module that registers and implements a low-level
reset function for the AMD XGBE device.

it performs the following actions:
- reset the PHY
- disable auto-negotiation
- disable & clear auto-negotiation IRQ
- soft-reset the MAC

Those tiny pieces of code are inherited from the native xgbe driver.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

Applies on top of [PATCH v3 0/7] VFIO platform reset module rework

v1 -> v2:
- uses module_vfio_reset_handler macro
---
 drivers/vfio/platform/reset/Kconfig|   7 ++
 drivers/vfio/platform/reset/Makefile   |   2 +
 .../vfio/platform/reset/vfio_platform_amdxgbe.c| 131 +
 3 files changed, 140 insertions(+)
 create mode 100644 drivers/vfio/platform/reset/vfio_platform_amdxgbe.c

diff --git a/drivers/vfio/platform/reset/Kconfig 
b/drivers/vfio/platform/reset/Kconfig
index 746b96b..ed9bb28 100644
--- a/drivers/vfio/platform/reset/Kconfig
+++ b/drivers/vfio/platform/reset/Kconfig
@@ -5,3 +5,10 @@ config VFIO_PLATFORM_CALXEDAXGMAC_RESET
  Enables the VFIO platform driver to handle reset for Calxeda xgmac
 
  If you don't know what to do here, say N.
+config VFIO_PLATFORM_AMDXGBE_RESET
+   tristate "VFIO support for AMD XGBE reset"
+   depends on VFIO_PLATFORM
+   help
+ Enables the VFIO platform driver to handle reset for AMD XGBE
+
+ If you don't know what to do here, say N.
diff --git a/drivers/vfio/platform/reset/Makefile 
b/drivers/vfio/platform/reset/Makefile
index 2a486af..93f4e23 100644
--- a/drivers/vfio/platform/reset/Makefile
+++ b/drivers/vfio/platform/reset/Makefile
@@ -1,5 +1,7 @@
 vfio-platform-calxedaxgmac-y := vfio_platform_calxedaxgmac.o
+vfio-platform-amdxgbe-y := vfio_platform_amdxgbe.o
 
 ccflags-y += -Idrivers/vfio/platform
 
 obj-$(CONFIG_VFIO_PLATFORM_CALXEDAXGMAC_RESET) += vfio-platform-calxedaxgmac.o
+obj-$(CONFIG_VFIO_PLATFORM_AMDXGBE_RESET) += vfio-platform-amdxgbe.o
diff --git a/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c 
b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
new file mode 100644
index 000..20b530a
--- /dev/null
+++ b/drivers/vfio/platform/reset/vfio_platform_amdxgbe.c
@@ -0,0 +1,131 @@
+/*
+ * VFIO platform driver specialized for AMD xgbe reset
+ * reset code is inherited from AMD xgbe native driver
+ *
+ * Copyright (c) 2015 Linaro Ltd.
+ *  www.linaro.org
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vfio_platform_private.h"
+
+#define DRIVER_VERSION  "0.1"
+#define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
+#define DRIVER_DESC "Reset support for AMD xgbe vfio platform device"
+
+#define DMA_MR 0x3000
+#define MAC_VR 0x0110
+#define DMA_ISR0x3008
+#define MAC_ISR0x00b0
+#define PCS_MMD_SELECT 0xff
+#define MDIO_AN_INT0x8002
+#define MDIO_AN_INTMASK0x8001
+
+static unsigned int xmdio_read(void *ioaddr, unsigned int mmd,
+  unsigned int reg)
+{
+   unsigned int mmd_address, value;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   value = ioread32(ioaddr + ((mmd_address & 0xff) << 2));
+   return value;
+}
+
+static void xmdio_write(void *ioaddr, unsigned int mmd,
+   unsigned int reg, unsigned int value)
+{
+   unsigned int mmd_address;
+
+   mmd_address = (mmd << 16) | ((reg) & 0x);
+   iowrite32(mmd_address >> 8, ioaddr + (PCS_MMD_SELECT << 2));
+   iowrite32(value, ioaddr + ((mmd_address & 0xff) << 2));
+}
+
+int vfio_platform_amdxgbe_reset(struct vfio_platform_device *vdev)
+{
+   struct vfio_platform_region xgmac_regs = vdev->regions[0];
+   struct vfio_platform_region xpcs_regs = vdev->regions[1];
+   u32 dma_mr_value, pcs_value, value;
+   unsigned int count;
+
+   if (!xgmac_regs.ioaddr) {
+   xgmac_regs.ioaddr =
+   ioremap_nocache(xgmac_regs.addr, xgmac_regs.size);
+   if (!xgmac_regs.ioaddr)
+   return -ENOMEM;
+  

[PATCH v3 3/7] vfio: platform: introduce module_vfio_reset_handler macro

[Eric Auger]

The module_vfio_reset_handler macro
- define a module alias
- implement module init/exit function which respectively registers
  and unregisters the reset function.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v2 -> v3:
- use vfio_platform_register_reset macro

v1 -> v2:
- remove vfio_platform_reset_private.h and move back the macro to
  vfio_platform_private.h header: removed reset_module_register &
  unregister (symbol_get)
- defines the module_vfio_reset_handler macro as suggested by Arnd
  (formerly in vfio_platform_reset_private.h)
---
 drivers/vfio/platform/vfio_platform_private.h | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 277521a..5a1e8e6 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -110,4 +110,18 @@ static struct vfio_platform_reset_node __reset ## _node = 
{\
 }; \
 __vfio_platform_register_reset(&__reset ## _node)
 
+#define module_vfio_reset_handler(compat, reset)   \
+MODULE_ALIAS("vfio-reset:" compat);\
+static int __init reset ## _module_init(void)  \
+{  \
+   vfio_platform_register_reset(compat, reset);\
+   return 0;   \
+}; \
+static void __exit reset ## _module_exit(void) \
+{  \
+   vfio_platform_unregister_reset(compat); \
+}; \
+module_init(reset ## _module_init);\
+module_exit(reset ## _module_exit)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 4/7] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

This patch adds the reset function registration/unregistration.
This is handled through the module_vfio_reset_handler macro. This
latter also defines a MODULE_ALIAS which simplifies the load from
vfio-platform.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---
v2 -> v3:
- do not include vfio_platform_reset_private.h anymore (removed)
- remove pr_info
- rework commit message

v1 -> v2:
- uses the module_vfio_reset_handler macro
- add pr_info on vfio reset
- do not export vfio_platform_calxedaxgmac_reset symbol anymore
---
 drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 619dc7d..640f5d8 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -30,8 +30,6 @@
 #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
 #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform device"
 
-#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
-
 /* XGMAC Register definitions */
 #define XGMAC_CONTROL   0x  /* MAC Configuration */
 
@@ -78,7 +76,8 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 
return 0;
 }
-EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
+
+module_vfio_reset_handler("calxeda,hb-xgmac", 
vfio_platform_calxedaxgmac_reset);
 
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 6/7] vfio: platform: use list of registered reset function

[Eric Auger]

Remove the static lookup table and use the dynamic list of registered
reset functions instead. Also load the reset module through its alias.
The reset struct module pointer is stored in vfio_platform_device.

We also remove the useless struct device pointer parameter in
vfio_platform_get_reset.

This patch fixes the issue related to the usage of __symbol_get, which
besides from being moot, prevented compilation with CONFIG_MODULES
disabled.

Also usage of MODULE_ALIAS makes possible to add a new reset module
without needing to update the framework. This was suggested by Arnd.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reported-by: Arnd Bergmann <a...@arndb.de>

---

v2 -> v3:
- remove clear of vfio_platform_device reset_module and reset
  in vfio_platform_put_reset
- single unlock in vfio_platform_lookup_reset
- use driver_lock instead of reset_lock

v1 -> v2:
- use reset_lock in vfio_platform_lookup_reset
- remove vfio_platform_reset_combo declaration
- remove struct device *dev parameter in vfio_platform_get_reset
- set reset_module and reset to NULL in put function
---
 drivers/vfio/platform/vfio_platform_common.c  | 52 +++
 drivers/vfio/platform/vfio_platform_private.h |  7 +---
 2 files changed, 30 insertions(+), 29 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 50a388b..de3fb33 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -30,37 +30,43 @@
 static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
-static const struct vfio_platform_reset_combo reset_lookup_table[] = {
-   {
-   .compat = "calxeda,hb-xgmac",
-   .reset_function_name = "vfio_platform_calxedaxgmac_reset",
-   .module_name = "vfio-platform-calxedaxgmac",
-   },
-};
-
-static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
-   struct device *dev)
+static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat,
+   struct module **module)
 {
-   int (*reset)(struct vfio_platform_device *);
-   int i;
+   struct vfio_platform_reset_node *iter;
+   vfio_platform_reset_fn_t reset_fn = NULL;
 
-   for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
-   request_module(reset_lookup_table[i].module_name);
-   reset = __symbol_get(
-   reset_lookup_table[i].reset_function_name);
-   if (reset) {
-   vdev->reset = reset;
-   return;
-   }
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat) &&
+   try_module_get(iter->owner)) {
+   *module = iter->owner;
+   reset_fn = iter->reset;
+   break;
}
}
+   mutex_unlock(_lock);
+   return reset_fn;
+}
+
+static void vfio_platform_get_reset(struct vfio_platform_device *vdev)
+{
+   char modname[256];
+
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+   >reset_module);
+   if (!vdev->reset) {
+   snprintf(modname, 256, "vfio-reset:%s", vdev->compat);
+   request_module(modname);
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+>reset_module);
+   }
 }
 
 static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
 {
if (vdev->reset)
-   symbol_put_addr(vdev->reset);
+   module_put(vdev->reset_module);
 }
 
 static int vfio_platform_regions_init(struct vfio_platform_device *vdev)
@@ -557,7 +563,7 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return ret;
}
 
-   vfio_platform_get_reset(vdev, dev);
+   vfio_platform_get_reset(vdev);
 
mutex_init(>igate);
 
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index f8072d8..e505c15 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -58,6 +58,7 @@ struct vfio_platform_device {
struct mutexigate;
struct module   *parent_module;
const char  *compat;
+   struct module   *reset_module;
 
/*
 * These fields should be filled by the bus specific binder
@@ -81,12 +82,6 @@ struct

[PATCH v3 2/7] vfio: platform: add capability to register a reset function

[Eric Auger]

In preparation for subsequent changes in reset function lookup,
lets introduce a dynamic list of reset combos (compat string,
reset module, reset function). The list can be populated/voided with
two new functions, vfio_platform_register/unregister_reset. Those are
not yet used in this patch.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2 -> v3:
- use goto out to have a single mutex_unlock
- implement vfio_platform_register_reset as a macro (suggested by Arnd)
- move reset_node struct declaration back to vfio_platform_private.h
- vfio_platform_unregister_reset does not return any value anymore

v1 -> v2:
- reset_list becomes static
- vfio_platform_register/unregister_reset take a const char * as compat
- fix node leak
- add reset_lock to protect the reset list manipulation
- move vfio_platform_reset_node declaration in vfio_platform_common.c
---
 drivers/vfio/platform/vfio_platform_common.c  | 26 ++
 drivers/vfio/platform/vfio_platform_private.h | 20 
 2 files changed, 46 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 184e9d2..8eccd30 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -27,6 +27,7 @@
 #define DRIVER_AUTHOR   "Antonios Motakis <a.mota...@virtualopensystems.com>"
 #define DRIVER_DESC "VFIO platform base module"
 
+static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -578,6 +579,31 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
 
+int __vfio_platform_register_reset(struct vfio_platform_reset_node *node)
+{
+   mutex_lock(_lock);
+   list_add(>link, _list);
+   mutex_unlock(_lock);
+   return 0;
+}
+EXPORT_SYMBOL_GPL(__vfio_platform_register_reset);
+
+void vfio_platform_unregister_reset(const char *compat)
+{
+   struct vfio_platform_reset_node *iter, *temp;
+
+   mutex_lock(_lock);
+   list_for_each_entry_safe(iter, temp, _list, link) {
+   if (!strcmp(iter->compat, compat)) {
+   list_del(>link);
+   break;
+   }
+   }
+
+   mutex_unlock(_lock);
+}
+EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 7128690..277521a 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -71,6 +71,15 @@ struct vfio_platform_device {
int (*reset)(struct vfio_platform_device *vdev);
 };
 
+typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
+
+struct vfio_platform_reset_node {
+   struct list_head link;
+   char *compat;
+   struct module *owner;
+   vfio_platform_reset_fn_t reset;
+};
+
 struct vfio_platform_reset_combo {
const char *compat;
const char *reset_function_name;
@@ -90,4 +99,15 @@ extern int vfio_platform_set_irqs_ioctl(struct 
vfio_platform_device *vdev,
unsigned start, unsigned count,
void *data);
 
+extern int __vfio_platform_register_reset(struct vfio_platform_reset_node *n);
+extern void vfio_platform_unregister_reset(const char *compat);
+
+#define vfio_platform_register_reset(__compat, __reset)\
+static struct vfio_platform_reset_node __reset ## _node = {\
+   .owner = THIS_MODULE,   \
+   .compat = __compat, \
+   .reset = __reset,   \
+}; \
+__vfio_platform_register_reset(&__reset ## _node)
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/7] VFIO platform reset module rework

[Eric Auger]

This series fixes the current implementation by getting rid of the
usage of __symbol_get which caused a compilation issue with
CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
possible to add a new reset module without being obliged to update the
framework. The new implementation relies on the reset module registering
its reset function to the vfio-platform driver.

The series is available at

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc6-rework-v3

Best Regards

Eric

v2 -> v3:
- use driver_mutex instead of reset_mutex
- style fixes: single mutex_unlock
- use static nodes; vfio_platform_register_reset now is a macro
- vfio_platform_reset_private.h removed since reset_module_(un)register
  disappear. No use of symbol_get anymore
- new patch introducing vfio-platform-base
- reset look-up moved back at vfio-platform probe time
- new patch featuring dev_info/dev_warn

v1 -> v2:
* in vfio_platform_common.c:
  - move reset lookup at load time and put reset at release: this is to
prevent a race between the 2 load module loads
  - reset_list becomes static
  - vfio_platform_register/unregister_reset take a const char * as compat
  - fix node link
  - remove old combo struct and cleanup proto of vfio_platform_get_reset
  - add mutex to protect the reset list
* in calxeda xgmac reset module
  - introduce vfio_platform_reset_private.h
  - use module_vfio_reset_handler macro
  - do not export vfio_platform_calxedaxgmac_reset symbol anymore
  - add a pr_info to show the device is reset by vfio reset module


Eric Auger (7):
  vfio: platform: introduce vfio-platform-base module
  vfio: platform: add capability to register a reset function
  vfio: platform: introduce module_vfio_reset_handler macro
  vfio: platform: reset: calxedaxgmac: add reset function registration
  vfio: platform: add compat in vfio_platform_device
  vfio: platform: use list of registered reset function
  vfio: platform: add dev_info on device reset

 drivers/vfio/platform/Makefile |   6 +-
 .../platform/reset/vfio_platform_calxedaxgmac.c|   5 +-
 drivers/vfio/platform/vfio_amba.c  |   1 +
 drivers/vfio/platform/vfio_platform.c  |   1 +
 drivers/vfio/platform/vfio_platform_common.c   | 118 +++--
 drivers/vfio/platform/vfio_platform_private.h  |  40 ++-
 6 files changed, 129 insertions(+), 42 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 6/7] vfio: platform: use list of registered reset function

[Eric Auger]

On 10/23/2015 04:23 PM, Arnd Bergmann wrote:
> On Friday 23 October 2015 16:11:08 Eric Auger wrote:
>> Hi Arnd,
>> On 10/23/2015 03:12 PM, Arnd Bergmann wrote:
>>> On Friday 23 October 2015 14:37:14 Eric Auger wrote:
>>>> Remove the static lookup table and use the dynamic list of registered
>>>> reset functions instead. Also load the reset module through its alias.
>>>> The reset struct module pointer is stored in vfio_platform_device.
>>>>
>>>> We also remove the useless struct device pointer parameter in
>>>> vfio_platform_get_reset.
>>>>
>>>> This patch fixes the issue related to the usage of __symbol_get, which
>>>> besides from being moot, prevented compilation with CONFIG_MODULES
>>>> disabled.
>>>>
>>>> Also usage of MODULE_ALIAS makes possible to add a new reset module
>>>> without needing to update the framework. This was suggested by Arnd.
>>>>
>>>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>>>> Reported-by: Arnd Bergmann <a...@arndb.de>
>>>>
>>>>
>>> Reviewed-by: Arnd Bergmann <a...@arndb.de>
>>>
>>> but doesn't this need to come before patch 4/7?
>> Well I don't think so. In [4] we introduce the dynamic registration
>> method but until this patch we still use the old lookup method in the
>> static table. I tested and the reset lookup still works in [4].
>> If we put this one before the registration, the functionality will be
>> lost here.
>>
> 
> Ok, I see. I was getting confused by the removal of the EXPORT_SYMBOL
> statement there and thought it would break the __get_symbol call.

Hum no actually you're right. I checked the reset module was loaded but
effectly the _get_symbol fails. So if I want to keep the functionality
all along the series I need to remove the EXPORT_SYMBOL when I swap the
lookup method.

Thanks

Eric
> 
>   Arnd
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 3/6] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

On 10/22/2015 12:13 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 11:41:59 Eric Auger wrote:
>> This patch adds the reset function registration/unregistration.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
> 
> Looks good, except one thing:
>> @@ -70,6 +69,8 @@ int vfio_platform_calxedaxgmac_reset(struct 
>> vfio_platform_device *vdev)
>>  return -ENOMEM;
>>  }
>>  
>> +pr_info("VFIO reset of %s\n", vdev->name);
>> +
>>  /* disable IRQ */
>>  writel(0, reg.ioaddr + XGMAC_DMA_INTR_ENA);
>>  
> 
> This probably slipped in from debugging, please remove it.
Well actually this is not an oversight but some unappropriate tracing
attempt I am afraid. I wanted to add a trace useful for the end-user to
make sure the VFIO reset function was called. Do you forbid that or do
recommend to use another tracing mechanism/level? In the past I tried
dynamic tracing but with module loading mechanism I found it not that handy.

Eric
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 5/6] vfio: platform: use list of registered reset function

[Eric Auger]

On 10/22/2015 12:19 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 11:42:01 Eric Auger wrote:
>> Remove the static lookup table and use the dynamic list of registered
>> reset functions instead. Also load the reset module through its alias.
>> The reset struct module pointer is stored in vfio_platform_device.
>>
>> We also remove the useless struct device pointer parameter in
>> vfio_platform_get_reset.
>>
>> This patch fixes the issue related to the usage of __symbol_get, which
>> besides from being moot, prevented compilation with CONFIG_MODULES
>> disabled.
>>
>> Also usage of MODULE_ALIAS makes possible to add a new reset module
>> without needing to update the framework. This was suggested by Arnd.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>> Reported-by: Arnd Bergmann <a...@arndb.de>
> 
> Looks good, just small style issues:
> 
>>
>> -}
>> +mutex_lock(_lock);
>> +list_for_each_entry(iter, _list, link) {
>> +if (!strcmp(iter->compat, compat) &&
>> +try_module_get(iter->owner)) {
>> +*module = iter->owner;
>> +mutex_unlock(_lock);
>> +return iter->reset;
>>  }
>>  }
>> +
>> +mutex_unlock(_lock);
>> +return NULL;
> 
> Better use 'break' to have a single mutex_unlock and return  statement.
OK
> 
>> +
>>  
>>  static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
>>  {
>> -if (vdev->reset)
>> -symbol_put_addr(vdev->reset);
>> +if (vdev->reset) {
>> +module_put(vdev->reset_module);
>> +vdev->reset_module = NULL;
>> +vdev->reset = NULL;
>> +}
>>  }
> 
> This gets called from the remove callback. No need to clear those
> struct members immediately before the kfree().
I should have added those assignments in next patch actually. This
latter moves the vfio_platform_put_reset call in the release function.

Best Regards

Eric
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 6/6] vfio: platform: move get/put reset at open/release

[Eric Auger]

Hi Arnd,
On 10/22/2015 12:29 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 11:42:02 Eric Auger wrote:
>> Currently reset lookup is done on probe. This introduces a
>> race with new registration mechanism in the case where the
>> vfio-platform driver is bound to the device before its module
>> is loaded: on the load, the probe happens which triggers the
>> reset module load which itself attempts to get the symbol for
>> the registration function (vfio_platform_register_reset). The
>> symbol is not yet available hence the lookup fails. In case we
>> do the lookup in the first open we are sure the vfio-platform
>> module is loaded and vfio_platform_register_reset is available.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
> 
> I don't understand the explanation. I would expect the request_module()
> call to block until the module is actually loaded. Is this not
> what happens?

Again many thanks for this new review.

My understanding is the following
1) vfio-platform is attached to the device through the override mechanism
2) vfio-platform get loaded through modprobe:
2-1) the platform driver init function eventually calls the
vfio-platform probe function.
2-2) request_module of vfio-platform-calxedaxgmac gets called.
3) The init of  vfio-platform-calxedaxgmac looks for
vfio_platform_register_reset. Unfortunately at that stage the init of
vfio-platform is not completed so the symbol is not available
3-1) in case symbol_get is used in vfio_platform_calxedaxgmac init, as
of today, this latter simply returns -EINVAL. Reset registration failed
but no stall.
3-2) in case symbol_get is *not* used, I think the module loader
attempts to load vfio-platform, which is already under load and this
causes a stall.

Let me know if you think this understanding is not correct.

Best Regards

Eric
> 
>> mutex_unlock(_lock);
>> @@ -181,6 +182,8 @@ static int vfio_platform_open(void *device_data)
>> if (ret)
>> goto err_irq;
>>  
>> +   vfio_platform_get_reset(vdev);
>> +
>> if (vdev->reset)
>> vdev->reset(vdev);
>>
> 
> This needs some error handling to ensure that the open() fails
> if there is no reset handler.
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 5/6] vfio: platform: use list of registered reset function

[Eric Auger]

Remove the static lookup table and use the dynamic list of registered
reset functions instead. Also load the reset module through its alias.
The reset struct module pointer is stored in vfio_platform_device.

We also remove the useless struct device pointer parameter in
vfio_platform_get_reset.

This patch fixes the issue related to the usage of __symbol_get, which
besides from being moot, prevented compilation with CONFIG_MODULES
disabled.

Also usage of MODULE_ALIAS makes possible to add a new reset module
without needing to update the framework. This was suggested by Arnd.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reported-by: Arnd Bergmann <a...@arndb.de>

---

v1 -> v2:
- use reset_lock in vfio_platform_lookup_reset
- remove vfio_platform_reset_combo declaration
- remove struct device *dev parameter in vfio_platform_get_reset
- set reset_module and reset to NULL in put function
---
 drivers/vfio/platform/vfio_platform_common.c  | 57 ---
 drivers/vfio/platform/vfio_platform_private.h |  7 +---
 2 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e7d615f..95b8294 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -34,37 +34,46 @@ static LIST_HEAD(reset_list);
 static DEFINE_MUTEX(reset_lock);
 static DEFINE_MUTEX(driver_lock);
 
-static const struct vfio_platform_reset_combo reset_lookup_table[] = {
-   {
-   .compat = "calxeda,hb-xgmac",
-   .reset_function_name = "vfio_platform_calxedaxgmac_reset",
-   .module_name = "vfio-platform-calxedaxgmac",
-   },
-};
-
-static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
-   struct device *dev)
+static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat,
+   struct module **module)
 {
-   int (*reset)(struct vfio_platform_device *);
-   int i;
+   struct vfio_platform_reset_node *iter;
 
-   for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
-   request_module(reset_lookup_table[i].module_name);
-   reset = __symbol_get(
-   reset_lookup_table[i].reset_function_name);
-   if (reset) {
-   vdev->reset = reset;
-   return;
-   }
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat) &&
+   try_module_get(iter->owner)) {
+   *module = iter->owner;
+   mutex_unlock(_lock);
+   return iter->reset;
}
}
+
+   mutex_unlock(_lock);
+   return NULL;
+}
+
+static void vfio_platform_get_reset(struct vfio_platform_device *vdev)
+{
+   char modname[256];
+
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+   >reset_module);
+   if (!vdev->reset) {
+   snprintf(modname, 256, "vfio-reset:%s", vdev->compat);
+   request_module(modname);
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+>reset_module);
+   }
 }
 
 static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
 {
-   if (vdev->reset)
-   symbol_put_addr(vdev->reset);
+   if (vdev->reset) {
+   module_put(vdev->reset_module);
+   vdev->reset_module = NULL;
+   vdev->reset = NULL;
+   }
 }
 
 static int vfio_platform_regions_init(struct vfio_platform_device *vdev)
@@ -561,7 +570,7 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return ret;
}
 
-   vfio_platform_get_reset(vdev, dev);
+   vfio_platform_get_reset(vdev);
 
mutex_init(>igate);
 
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 620ee40..e968454 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -57,6 +57,7 @@ struct vfio_platform_device {
int refcnt;
struct mutexigate;
const char  *compat;
+   struct module   *reset_module;
 
/*
 * These fields should be filled by the bus specific binder
@@ -71,12 +72,6 @@ struct vfio_platform_device {
int (*reset)(struct vfio_platform_device *vdev);

[PATCH v2 4/6] vfio: platform: add compat in vfio_platform_device

[Eric Auger]

Let's retrieve the compatibility string on probe and store it
in the vfio_platform_device struct

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 drivers/vfio/platform/vfio_platform_common.c  | 15 ---
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 52a4c7b..e7d615f 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -45,16 +45,11 @@ static const struct vfio_platform_reset_combo 
reset_lookup_table[] = {
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   const char *compat;
int (*reset)(struct vfio_platform_device *);
-   int ret, i;
-
-   ret = device_property_read_string(dev, "compatible", );
-   if (ret)
-   return;
+   int i;
 
for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, compat)) {
+   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
request_module(reset_lookup_table[i].module_name);
reset = __symbol_get(
reset_lookup_table[i].reset_function_name);
@@ -545,6 +540,12 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
struct iommu_group *group;
int ret;
 
+   ret = device_property_read_string(dev, "compatible", >compat);
+   if (ret) {
+   pr_err("VFIO: cannot retrieve compat for %s\n", vdev->name);
+   return -EINVAL;
+   }
+
if (!vdev)
return -EINVAL;
 
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1fb1410..620ee40 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -56,6 +56,7 @@ struct vfio_platform_device {
u32 num_irqs;
int refcnt;
struct mutexigate;
+   const char  *compat;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 2/6] vfio: platform: reset: add vfio_platform_reset_private.h

[Eric Auger]

This header is to be included in all vfio reset modules. It
defines the module_vfio_reset_handler macro whose role is
- to define a module alias
- implement module init/exit function which respectively registers
  and unregisters the reset function.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v2: creation
- this defines the module_vfio_reset_handler macro as suggested by Arnd

Although Arnd suggested me to remove the vfio_platform_register_reset
symbol_get (since the module manager can handle the case where the
vfio-platform driver is not loaded), I prefered to keep it while
introducing the macro. The rationale is, when using symbol_get/put
we are able to release the hold from the reset module on vfio-platform
as soon as the registration is complete and I think this makes sense.
---
 drivers/vfio/platform/reset/Makefile   |  2 +-
 .../platform/reset/vfio_platform_reset_private.h   | 66 ++
 2 files changed, 67 insertions(+), 1 deletion(-)
 create mode 100644 drivers/vfio/platform/reset/vfio_platform_reset_private.h

diff --git a/drivers/vfio/platform/reset/Makefile 
b/drivers/vfio/platform/reset/Makefile
index 2a486af..154a7d5 100644
--- a/drivers/vfio/platform/reset/Makefile
+++ b/drivers/vfio/platform/reset/Makefile
@@ -1,5 +1,5 @@
 vfio-platform-calxedaxgmac-y := vfio_platform_calxedaxgmac.o
 
-ccflags-y += -Idrivers/vfio/platform
+ccflags-y += -Idrivers/vfio/platform -Idrivers/vfio/platform/reset
 
 obj-$(CONFIG_VFIO_PLATFORM_CALXEDAXGMAC_RESET) += vfio-platform-calxedaxgmac.o
diff --git a/drivers/vfio/platform/reset/vfio_platform_reset_private.h 
b/drivers/vfio/platform/reset/vfio_platform_reset_private.h
new file mode 100644
index 000..f212a61
--- /dev/null
+++ b/drivers/vfio/platform/reset/vfio_platform_reset_private.h
@@ -0,0 +1,66 @@
+/*
+ * Interface used by VFIO platform reset modules to register/unregister
+ * their reset function
+ *
+ * Copyright (c) 2015 Linaro Ltd.
+ *  www.linaro.org
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef VFIO_PLATFORM_RESET_PRIVATE_H
+#define VFIO_PLATFORM_RESET_PRIVATE_H
+
+#include 
+#include "vfio_platform_private.h"
+
+static int reset_module_register(struct module *module,
+   const char *compat,
+   vfio_platform_reset_fn_t reset)
+{
+   int (*register_reset)(struct module *, const char*,
+   vfio_platform_reset_fn_t);
+   int ret;
+
+   register_reset = symbol_get(vfio_platform_register_reset);
+   if (!register_reset)
+   return -EINVAL;
+   ret = register_reset(module, compat, reset);
+   symbol_put(vfio_platform_register_reset);
+   return ret;
+}
+
+static void reset_module_unregister(const char *compat)
+{
+   int (*unregister_reset)(const char *);
+
+   unregister_reset = symbol_get(vfio_platform_unregister_reset);
+   if (!unregister_reset)
+   return;
+
+   unregister_reset(compat);
+
+   symbol_put(vfio_platform_unregister_reset);
+}
+
+#define module_vfio_reset_handler(compat, reset)   \
+MODULE_ALIAS("vfio-reset:" compat);\
+static int __init reset ## _module_init(void)  \
+{  \
+   return reset_module_register(THIS_MODULE, compat, );  \
+}; \
+static void __exit reset ## _module_exit(void) \
+{   \
+   reset_module_unregister(compat);\
+}; \
+module_init(reset ## _module_init);\
+module_exit(reset ## _module_exit)
+
+#endif /* VFIO_PLATFORM_RESET_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 1/6] vfio: platform: add capability to register a reset function

[Eric Auger]

In preparation for subsequent changes in reset function lookup,
lets introduce a dynamic list of reset combos (compat string,
reset module, reset function). The list can be populated/voided with
two new functions, vfio_platform_register/unregister_reset. Those are
not yet used in this patch.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v1 -> v2:
- reset_list becomes static
- vfio_platform_register/unregister_reset take a const char * as compat
- fix node leak
- add reset_lock to protect the reset list manipulation
- move vfio_platform_reset_node declaration in vfio_platform_common.c
---
 drivers/vfio/platform/vfio_platform_common.c  | 77 +++
 drivers/vfio/platform/vfio_platform_private.h |  7 +++
 2 files changed, 84 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e43efb5..52a4c7b 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -23,6 +23,15 @@
 
 #include "vfio_platform_private.h"
 
+struct vfio_platform_reset_node {
+   struct list_head link;
+   char *compat;
+   struct module *owner;
+   vfio_platform_reset_fn_t reset;
+};
+
+static LIST_HEAD(reset_list);
+static DEFINE_MUTEX(reset_lock);
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -573,3 +582,71 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
return vdev;
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
+
+int vfio_platform_register_reset(struct module *reset_owner,
+const char *compat,
+vfio_platform_reset_fn_t reset)
+{
+   struct vfio_platform_reset_node *node, *iter;
+   bool found = false;
+
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat)) {
+   found = true;
+   break;
+   }
+   }
+   if (found) {
+   mutex_unlock(_lock);
+   return -EINVAL;
+   }
+
+   node = kmalloc(sizeof(*node), GFP_KERNEL);
+   if (!node) {
+   mutex_unlock(_lock);
+   return -ENOMEM;
+   }
+
+   node->compat = kstrdup(compat, GFP_KERNEL);
+   if (!node->compat) {
+   kfree(node);
+   mutex_unlock(_lock);
+   return -ENOMEM;
+   }
+
+   node->owner = reset_owner;
+   node->reset = reset;
+
+   list_add(>link, _list);
+   mutex_unlock(_lock);
+   return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_platform_register_reset);
+
+int vfio_platform_unregister_reset(const char *compat)
+{
+   struct vfio_platform_reset_node *iter;
+   bool found = false;
+
+   mutex_lock(_lock);
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat)) {
+   found = true;
+   break;
+   }
+   }
+   if (!found) {
+   mutex_unlock(_lock);
+   return -EINVAL;
+   }
+
+   list_del(>link);
+   kfree(iter->compat);
+   kfree(iter);
+   mutex_unlock(_lock);
+
+   return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
+
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1c9b3d5..1fb1410 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -89,4 +89,11 @@ extern int vfio_platform_set_irqs_ioctl(struct 
vfio_platform_device *vdev,
unsigned start, unsigned count,
void *data);
 
+typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
+
+extern int vfio_platform_register_reset(struct module *owner,
+   const char *compat,
+   vfio_platform_reset_fn_t reset);
+extern int vfio_platform_unregister_reset(const char *compat);
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 0/6] VFIO platform reset module rework

[Eric Auger]

This series fixes the current implementation by getting rid of the
usage of __symbol_get which caused a compilation issue with
CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
possible to add a new reset module without being obliged to update the
framework. The new implementation relies on the reset module registering
its reset function to the vfio-platform driver.

The series is available at

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc6-rework-v2

Best Regards

Eric

v1 -> v2:
* in vfio_platform_common.c:
  - move reset lookup at load time and put reset at release: this is to
prevent a race between the 2 load module loads
  - reset_list becomes static
  - vfio_platform_register/unregister_reset take a const char * as compat
  - fix node link
  - remove old combo struct and cleanup proto of vfio_platform_get_reset
  - add mutex to protect the reset list
* in calxeda xgmac reset module
  - introduce vfio_platform_reset_private.h
  - use module_vfio_reset_handler macro
  - do not export vfio_platform_calxedaxgmac_reset symbol anymore
  - add a pr_info to show the device is reset by vfio reset module


Eric Auger (6):
  vfio: platform: add capability to register a reset function
  vfio: platform: reset: add vfio_platform_reset_private.h
  vfio: platform: reset: calxedaxgmac: add reset function registration
  vfio: platform: add compat in vfio_platform_device
  vfio: platform: use list of registered reset function
  vfio: platform: move get/put reset at open/release

 drivers/vfio/platform/reset/Makefile   |   2 +-
 .../platform/reset/vfio_platform_calxedaxgmac.c|   9 +-
 .../platform/reset/vfio_platform_reset_private.h   |  66 +
 drivers/vfio/platform/vfio_platform_common.c   | 151 -
 drivers/vfio/platform/vfio_platform_private.h  |  15 +-
 5 files changed, 201 insertions(+), 42 deletions(-)
 create mode 100644 drivers/vfio/platform/reset/vfio_platform_reset_private.h

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 3/6] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

This patch adds the reset function registration/unregistration.

Signed-off-by: Eric Auger <eric.au...@linaro.org>

---

v1 -> v2:
- uses the module_vfio_reset_handler macro
- add pr_info on vfio reset
- do not export vfio_platform_calxedaxgmac_reset symbol anymore
---
 drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 619dc7d..8b2 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -25,13 +25,12 @@
 #include 
 
 #include "vfio_platform_private.h"
+#include "vfio_platform_reset_private.h"
 
 #define DRIVER_VERSION  "0.1"
 #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
 #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform device"
 
-#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
-
 /* XGMAC Register definitions */
 #define XGMAC_CONTROL   0x  /* MAC Configuration */
 
@@ -70,6 +69,8 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
return -ENOMEM;
}
 
+   pr_info("VFIO reset of %s\n", vdev->name);
+
/* disable IRQ */
writel(0, reg.ioaddr + XGMAC_DMA_INTR_ENA);
 
@@ -78,7 +79,9 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 
return 0;
 }
-EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
+
+module_vfio_reset_handler("calxeda,hb-xgmac",
+ vfio_platform_calxedaxgmac_reset);
 
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 6/6] vfio: platform: move get/put reset at open/release

[Eric Auger]

Currently reset lookup is done on probe. This introduces a
race with new registration mechanism in the case where the
vfio-platform driver is bound to the device before its module
is loaded: on the load, the probe happens which triggers the
reset module load which itself attempts to get the symbol for
the registration function (vfio_platform_register_reset). The
symbol is not yet available hence the lookup fails. In case we
do the lookup in the first open we are sure the vfio-platform
module is loaded and vfio_platform_register_reset is available.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 drivers/vfio/platform/vfio_platform_common.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 95b8294..5ebae8c 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -155,6 +155,7 @@ static void vfio_platform_release(void *device_data)
vdev->reset(vdev);
vfio_platform_regions_cleanup(vdev);
vfio_platform_irq_cleanup(vdev);
+   vfio_platform_put_reset(vdev);
}
 
mutex_unlock(_lock);
@@ -181,6 +182,8 @@ static int vfio_platform_open(void *device_data)
if (ret)
goto err_irq;
 
+   vfio_platform_get_reset(vdev);
+
if (vdev->reset)
vdev->reset(vdev);
}
@@ -570,8 +573,6 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
return ret;
}
 
-   vfio_platform_get_reset(vdev);
-
mutex_init(>igate);
 
return 0;
@@ -585,7 +586,6 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
vdev = vfio_del_group_dev(dev);
 
if (vdev) {
-   vfio_platform_put_reset(vdev);
iommu_group_put(dev->iommu_group);
}
 
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 6/6] vfio: platform: move get/put reset at open/release

[Eric Auger]

On 10/22/2015 04:10 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 15:26:55 Eric Auger wrote:
>>>> @@ -181,6 +182,8 @@ static int vfio_platform_open(void *device_data)
>>>> if (ret)
>>>> goto err_irq;
>>>>  
>>>> +   vfio_platform_get_reset(vdev);
>>>> +
>>>> if (vdev->reset)
>>>> vdev->reset(vdev);
>>>>
>>>
>>> This needs some error handling to ensure that the open() fails
>>> if there is no reset handler.
>>
>> Is that really what we want? The code was meant to allow the use case
>> where the VFIO platform driver would be used without such reset module.
>>
>> I think the imperious need for a reset module depends on the device and
>> more importantly depends on the IOMMU mapping. With QEMU VFIO
>> integration this is needed because the whole VM memory is IOMMU mapped
>> but in a simpler user-space driver context, we might live without.
>>
>> Any thought?
> 
> I would think we need a reset driver for any device that can start DMA,
> otherwise things can go wrong as soon as you attach it to a different domain
> while there is ongoing DMA.
> 
> Maybe we could just allow devices to be attached without a reset handler,
> but then disallow DMA on them?

Well I am tempted to think that most assigned devices will perform DMA
accesses so to me this somehow comes to the same result, ie disallowing
functional passthrough for devices not properly/fully integrated.

Alex/Baptiste, any opinion on this?

Thanks

Eric


> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 6/6] vfio: platform: move get/put reset at open/release

[Eric Auger]

On 10/22/2015 02:05 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 13:40:16 Eric Auger wrote:
>> On 10/22/2015 12:29 PM, Arnd Bergmann wrote:
>>> On Thursday 22 October 2015 11:42:02 Eric Auger wrote:
>>>> Currently reset lookup is done on probe. This introduces a
>>>> race with new registration mechanism in the case where the
>>>> vfio-platform driver is bound to the device before its module
>>>> is loaded: on the load, the probe happens which triggers the
>>>> reset module load which itself attempts to get the symbol for
>>>> the registration function (vfio_platform_register_reset). The
>>>> symbol is not yet available hence the lookup fails. In case we
>>>> do the lookup in the first open we are sure the vfio-platform
>>>> module is loaded and vfio_platform_register_reset is available.
>>>>
>>>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>>>
>>> I don't understand the explanation. I would expect the request_module()
>>> call to block until the module is actually loaded. Is this not
>>> what happens?
>>
>> Again many thanks for this new review.
>>
>> My understanding is the following
>> 1) vfio-platform is attached to the device through the override mechanism
>> 2) vfio-platform get loaded through modprobe:
>> 2-1) the platform driver init function eventually calls the
>> vfio-platform probe function.
>> 2-2) request_module of vfio-platform-calxedaxgmac gets called.
>> 3) The init of  vfio-platform-calxedaxgmac looks for
>> vfio_platform_register_reset. Unfortunately at that stage the init of
>> vfio-platform is not completed so the symbol is not available
>> 3-1) in case symbol_get is used in vfio_platform_calxedaxgmac init, as
>> of today, this latter simply returns -EINVAL. Reset registration failed
>> but no stall.
>> 3-2) in case symbol_get is *not* used, I think the module loader
>> attempts to load vfio-platform, which is already under load and this
>> causes a stall.
>>
>> Let me know if you think this understanding is not correct.
> 
> I was expecting the vfio_platform_register_reset() function to be
> available by the time of step 2-2.
> 
> What I think is going on here is that we are still inside of the
> module_init() function of the vfio-platform driver at the time that
> we call the request_module(), and the symbols are not made visible
> to other modules until the module_init function returns with success.
> This is a side-effect of the probe function for the platform driver
> getting called from deep inside of the platform_driver_register()
> function for all devices that are already present.
yes we share the same understanding, this is what I meant.
> 
> I think it already works for the AMBA case, which uses separate modules,
> but we need to restructure the platform_device case slightly to do
> the same:

OK I will test the new module structure below and eventually remove the
symbol_get (I got the link issue). Thanks for the hint!

Eric
> 
> diff --git a/drivers/vfio/platform/Makefile b/drivers/vfio/platform/Makefile
> index 9ce8afe28450..a00a44814255 100644
> --- a/drivers/vfio/platform/Makefile
> +++ b/drivers/vfio/platform/Makefile
> @@ -1,10 +1,12 @@
>  
> -vfio-platform-y := vfio_platform.o vfio_platform_common.o vfio_platform_irq.o
> +vfio-platform-base-y := vfio_platform_common.o vfio_platform_irq.o
>  
> -obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform.o
> +obj-$(CONFIG_VFIO_PLATFORM) += vfio_platform.o
> +obj-$(CONFIG_VFIO_PLATFORM) += vfio-platform-base.o
>  obj-$(CONFIG_VFIO_PLATFORM) += reset/
>  
>  vfio-amba-y := vfio_amba.o
>  
>  obj-$(CONFIG_VFIO_AMBA) += vfio-amba.o
> +obj-$(CONFIG_VFIO_AMBA) += vfio-platform-base.o
>  obj-$(CONFIG_VFIO_AMBA) += reset/
> 
> 
>   Arnd
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 3/6] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

On 10/22/2015 02:09 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 13:54:42 Eric Auger wrote:
>> On 10/22/2015 12:13 PM, Arnd Bergmann wrote:
>>> On Thursday 22 October 2015 11:41:59 Eric Auger wrote:
>>>> This patch adds the reset function registration/unregistration.
>>>>
>>>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>>>
>>> Looks good, except one thing:
>>>> @@ -70,6 +69,8 @@ int vfio_platform_calxedaxgmac_reset(struct 
>>>> vfio_platform_device *vdev)
>>>>  return -ENOMEM;
>>>>  }
>>>>  
>>>> +pr_info("VFIO reset of %s\n", vdev->name);
>>>> +
>>>>  /* disable IRQ */
>>>>  writel(0, reg.ioaddr + XGMAC_DMA_INTR_ENA);
>>>>  
>>>
>>> This probably slipped in from debugging, please remove it.
>> Well actually this is not an oversight but some unappropriate tracing
>> attempt I am afraid. I wanted to add a trace useful for the end-user to
>> make sure the VFIO reset function was called. Do you forbid that or do
>> recommend to use another tracing mechanism/level? In the past I tried
>> dynamic tracing but with module loading mechanism I found it not that handy.
> 
> If you think it's useful to have in the long run, it should be a separate
> patch with a description what it's good for, rather than a side-effect of
> an unrelated patch. It just looked to me like it's something you do
> while debugging the reset code, rather than while using it.
> 
> Implementation-wise, you should use dev_info() instead of pr_info() where
> possible, and it probably makes sense to put this into the vfio_platform
> driver before calling the reset function, for consistency between the
> drivers.
OK thanks

Best Regards

Eric
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 6/6] vfio: platform: move get/put reset at open/release

[Eric Auger]

On 10/22/2015 12:29 PM, Arnd Bergmann wrote:
> On Thursday 22 October 2015 11:42:02 Eric Auger wrote:
>> Currently reset lookup is done on probe. This introduces a
>> race with new registration mechanism in the case where the
>> vfio-platform driver is bound to the device before its module
>> is loaded: on the load, the probe happens which triggers the
>> reset module load which itself attempts to get the symbol for
>> the registration function (vfio_platform_register_reset). The
>> symbol is not yet available hence the lookup fails. In case we
>> do the lookup in the first open we are sure the vfio-platform
>> module is loaded and vfio_platform_register_reset is available.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
> 
> I don't understand the explanation. I would expect the request_module()
> call to block until the module is actually loaded. Is this not
> what happens?
> 
>> mutex_unlock(_lock);
>> @@ -181,6 +182,8 @@ static int vfio_platform_open(void *device_data)
>> if (ret)
>> goto err_irq;
>>  
>> +   vfio_platform_get_reset(vdev);
>> +
>> if (vdev->reset)
>> vdev->reset(vdev);
>>
> 
> This needs some error handling to ensure that the open() fails
> if there is no reset handler.

Is that really what we want? The code was meant to allow the use case
where the VFIO platform driver would be used without such reset module.

I think the imperious need for a reset module depends on the device and
more importantly depends on the IOMMU mapping. With QEMU VFIO
integration this is needed because the whole VM memory is IOMMU mapped
but in a simpler user-space driver context, we might live without.

Any thought?

Eric
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] arm/arm64: KVM: Fix disabled distributor operation

[Eric Auger]

Hi Christoffer,
On 10/17/2015 10:30 PM, Christoffer Dall wrote:
> We currently do a single update of the vgic state when the distrbutor
distributor
> enable/disable control register is accessed and then bypass updating the
> state for as long as the distributor remains disabled.
> 
> This is incorrect, because updating the state does not consider the
> distributor enable bit, and this you can end up in a situation where an
> interrupt is marked as pending on the CPU interface, but not pending on
> the distributor, which is an impossible state to be in, and triggers a
> warning.  Consider for example the following sequence of events:
> 
> 1. An interrupt is marked as pending on the distributor
>- the interrupt is also forwarded to the CPU interface
> 2. The guest turns off the distributor (it's about to do a reboot)
>- we stop updating the CPU interface state from now on
> 3. The guest disables the pending interrupt
>- we remove the pending state from the distributor, but don't touch
>  the CPU interface, see point 2.
> 
> Since the distributor disable bit really means that no interrupts should
> be forwarded to the CPU interface, we modify the code to keep updating
> the internal VGIC state, but always set the CPU interface pending bits
> to zero when the distributor is disabled.
> 
> Signed-off-by: Christoffer Dall 
> ---
>  virt/kvm/arm/vgic.c | 11 ++-
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 58b1256..66c6616 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -1012,6 +1012,12 @@ static int compute_pending_for_cpu(struct kvm_vcpu 
> *vcpu)
>   pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
>   pend_shared = vcpu->arch.vgic_cpu.pending_shared;
>  
> + if (!dist->enabled) {
> + bitmap_zero(pend_percpu, VGIC_NR_PRIVATE_IRQS);
> + bitmap_zero(pend_shared, nr_shared);
> + return 0;
> + }
> +
>   pending = vgic_bitmap_get_cpu_map(>irq_pending, vcpu_id);
>   enabled = vgic_bitmap_get_cpu_map(>irq_enabled, vcpu_id);
>   bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
> @@ -1039,11 +1045,6 @@ void vgic_update_state(struct kvm *kvm)
>   struct kvm_vcpu *vcpu;
>   int c;
>  
> - if (!dist->enabled) {
> - set_bit(0, dist->irq_pending_on_cpu);
> - return;
I am confused. Don't you want to clear the whole bitmap?

Shouldn't we also handle interrupts programmed in the LR. Spec says any
ack should return a spurious ID. Is it what is going to happen with the
current implementation?

Eric
> - }
> -
>   kvm_for_each_vcpu(c, vcpu, kvm) {
>   if (compute_pending_for_cpu(vcpu))
>   set_bit(c, dist->irq_pending_on_cpu);
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] arm/arm64: KVM: Fix disabled distributor operation

[Eric Auger]

On 10/20/2015 11:44 AM, Christoffer Dall wrote:
> On Tue, Oct 20, 2015 at 11:08:44AM +0200, Eric Auger wrote:
>> Hi Christoffer,
>> On 10/17/2015 10:30 PM, Christoffer Dall wrote:
>>> We currently do a single update of the vgic state when the distrbutor
>> distributor
>>> enable/disable control register is accessed and then bypass updating the
>>> state for as long as the distributor remains disabled.
>>>
>>> This is incorrect, because updating the state does not consider the
>>> distributor enable bit, and this you can end up in a situation where an
>>> interrupt is marked as pending on the CPU interface, but not pending on
>>> the distributor, which is an impossible state to be in, and triggers a
>>> warning.  Consider for example the following sequence of events:
>>>
>>> 1. An interrupt is marked as pending on the distributor
>>>- the interrupt is also forwarded to the CPU interface
>>> 2. The guest turns off the distributor (it's about to do a reboot)
>>>- we stop updating the CPU interface state from now on
>>> 3. The guest disables the pending interrupt
>>>- we remove the pending state from the distributor, but don't touch
>>>  the CPU interface, see point 2.
>>>
>>> Since the distributor disable bit really means that no interrupts should
>>> be forwarded to the CPU interface, we modify the code to keep updating
>>> the internal VGIC state, but always set the CPU interface pending bits
>>> to zero when the distributor is disabled.
>>>
>>> Signed-off-by: Christoffer Dall <christoffer.d...@linaro.org>
>>> ---
>>>  virt/kvm/arm/vgic.c | 11 ++-
>>>  1 file changed, 6 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>>> index 58b1256..66c6616 100644
>>> --- a/virt/kvm/arm/vgic.c
>>> +++ b/virt/kvm/arm/vgic.c
>>> @@ -1012,6 +1012,12 @@ static int compute_pending_for_cpu(struct kvm_vcpu 
>>> *vcpu)
>>> pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
>>> pend_shared = vcpu->arch.vgic_cpu.pending_shared;
>>>  
>>> +   if (!dist->enabled) {
>>> +   bitmap_zero(pend_percpu, VGIC_NR_PRIVATE_IRQS);
>>> +   bitmap_zero(pend_shared, nr_shared);
>>> +   return 0;
>>> +   }
>>> +
>>> pending = vgic_bitmap_get_cpu_map(>irq_pending, vcpu_id);
>>> enabled = vgic_bitmap_get_cpu_map(>irq_enabled, vcpu_id);
>>> bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
>>> @@ -1039,11 +1045,6 @@ void vgic_update_state(struct kvm *kvm)
>>> struct kvm_vcpu *vcpu;
>>> int c;
>>>  
>>> -   if (!dist->enabled) {
>>> -   set_bit(0, dist->irq_pending_on_cpu);
>>> -   return;
>> I am confused. Don't you want to clear the whole bitmap?
> 
> So the first line used to set irq_pending_on_cpu for VCPU0 when the
> distributor was disabled, which I think basically worked around the
> guest kernel expecting to see a timer interrupt during boot when doing a
> WFI.  Now when that's fixed, we don't need this (gigantuous hack) anymore.
> 
> The return statement was also weird and buggy, because it never
> prevented anything from going to the CPU interface, it just stopped
> updating things.

Yeah sorry I read it as an addition so I didn't understand that code but
I definitively understand you remove it. Sorry - tired.

Best Regards

Eric
> 
> 
>>
>> Shouldn't we also handle interrupts programmed in the LR. Spec says any
>> ack should return a spurious ID. Is it what is going to happen with the
>> current implementation?
>>
> 
> yes, we really should.  We should unqueue them, but I haven't seen any
> bugs from this, and I feel like we're already changing a lot with short
> notice, so I'd rather not distrupt anything more right now.
> 
> Besides, when we get around to redesigning this whole thing, the concept
> of unqueueing goes away.
> 
> I know it sucks reviewing fixes that only fix a subset of a bad
> implementation, but I'm aiming for 'slightly better than current state'
> right now :)
> 
> -Christoffer
> 
> 
>>> -   }
>>> -
>>> kvm_for_each_vcpu(c, vcpu, kvm) {
>>> if (compute_pending_for_cpu(vcpu))
>>> set_bit(c, dist->irq_pending_on_cpu);
>>>
>>

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/4] vfio: platform: add capability to register a reset function

[Eric Auger]

Hi Alex,
On 10/19/2015 09:45 PM, Alex Williamson wrote:
> On Sun, 2015-10-18 at 18:00 +0200, Eric Auger wrote:
>> In preparation for subsequent changes in reset function lookup,
>> lets introduce a dynamic list of reset combos (compat string,
>> reset module, reset function). The list can be populated/voided with
>> two new functions, vfio_platform_register/unregister_reset. Those are
>> not yet used in this patch.
>>
>> Signed-off-by: Eric Auger <eric.au...@linaro.org>
>> ---
>>  drivers/vfio/platform/vfio_platform_common.c  | 55 
>> +++
>>  drivers/vfio/platform/vfio_platform_private.h | 14 +++
>>  2 files changed, 69 insertions(+)
>>
>> diff --git a/drivers/vfio/platform/vfio_platform_common.c 
>> b/drivers/vfio/platform/vfio_platform_common.c
>> index e43efb5..d36afc9 100644
>> --- a/drivers/vfio/platform/vfio_platform_common.c
>> +++ b/drivers/vfio/platform/vfio_platform_common.c
>> @@ -23,6 +23,8 @@
>>  
>>  #include "vfio_platform_private.h"
>>  
>> +struct list_head reset_list;
> 
> What's the purpose of this one above?
useless indeed
> 
>> +LIST_HEAD(reset_list);
> 
> static
> 
> I think you also need a mutex protecting this list, otherwise nothing
> prevents concurrent list changes and walks.  A rw lock probably fits the
> usage model best, but I don't expect much contention if you just want to
> use a mutex.
yes you're right. I am going to use a standard mutex I think.
> 
>>  static DEFINE_MUTEX(driver_lock);
>>  
>>  static const struct vfio_platform_reset_combo reset_lookup_table[] = {
>> @@ -573,3 +575,56 @@ struct vfio_platform_device 
>> *vfio_platform_remove_common(struct device *dev)
>>  return vdev;
>>  }
>>  EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
>> +
>> +int vfio_platform_register_reset(struct module *reset_owner, char *compat,
> 
> const char *
ok
> 
>> + vfio_platform_reset_fn_t reset)
>> +{
>> +struct vfio_platform_reset_node *node, *iter;
>> +bool found = false;
>> +
>> +list_for_each_entry(iter, _list, link) {
>> +if (!strcmp(iter->compat, compat)) {
>> +found = true;
>> +break;
>> +}
>> +}
>> +if (found)
>> +return -EINVAL;
>> +
>> +node = kmalloc(sizeof(*node), GFP_KERNEL);
>> +if (!node)
>> +return -ENOMEM;
>> +
>> +node->compat = kstrdup(compat, GFP_KERNEL);
>> +if (!node->compat)
>> +return -ENOMEM;
> 
> Leaks node
thanks
> 
>> +
>> +node->owner = reset_owner;
>> +node->reset = reset;
>> +
>> +list_add(>link, _list);
>> +return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(vfio_platform_register_reset);
>> +
>> +int vfio_platform_unregister_reset(char *compat)
> 
> const char *
ok

Thanks for your quick review!

Best Regards

Eric
> 
>> +{
>> +struct vfio_platform_reset_node *iter;
>> +bool found = false;
>> +
>> +list_for_each_entry(iter, _list, link) {
>> +if (!strcmp(iter->compat, compat)) {
>> +found = true;
>> +break;
>> +}
>> +}
>> +if (!found)
>> +return -EINVAL;
>> +
>> +list_del(>link);
>> +kfree(iter->compat);
>> +kfree(iter);
>> +return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
>> +
>> diff --git a/drivers/vfio/platform/vfio_platform_private.h 
>> b/drivers/vfio/platform/vfio_platform_private.h
>> index 1c9b3d5..17323f0 100644
>> --- a/drivers/vfio/platform/vfio_platform_private.h
>> +++ b/drivers/vfio/platform/vfio_platform_private.h
>> @@ -76,6 +76,15 @@ struct vfio_platform_reset_combo {
>>  const char *module_name;
>>  };
>>  
>> +typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
>> +
>> +struct vfio_platform_reset_node {
>> +struct list_head link;
>> +char *compat;
>> +struct module *owner;
>> +vfio_platform_reset_fn_t reset;
>> +};
>> +
>>  extern int vfio_platform_probe_common(struct vfio_platform_device *vdev,
>>struct device *dev);
>>  extern struct vfio_platform_device *vfio_platform_remove_common
>> @@ -89,4 +98,9 @@ extern int vfio_platform_set_irqs_ioctl(struct 
>> vfio_platform_device *vdev,
>>  unsigned start, unsigned count,
>>  void *data);
>>  
>> +extern int vfio_platform_register_reset(struct module *owner,
>> +char *compat,
>> +vfio_platform_reset_fn_t reset);
>> +extern int vfio_platform_unregister_reset(char *compat);
>> +
>>  #endif /* VFIO_PLATFORM_PRIVATE_H */
> 
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] arm/arm64: KVM: Fix arch timer behavior for disabled interrupts

[Eric Auger]

Hi Christoffer,
On 10/17/2015 10:30 PM, Christoffer Dall wrote:
> We have an interesting issue when the guest disables the timer interrupt
> on the VGIC, which happens when turning VCPUs off using PSCI, for
> example.
> 
> The problem is that because the guest disables the virtual interrupt at
> the VGIC level, we never inject interrupts to the guest and therefore
> never mark the interrupt as active on the physical distributor.  The
> host also never takes the timer interrupt (we only use the timer device
> to trigger a guest exit and everything else is done in software), so the
> interrupt does not become active through normal means.
> 
> The result is that we keep entering the guest with a programmed timer
> that will always fire as soon as we context switch the hardware timer
> state and run the guest, preventing forward progress for the VCPU.
> 
> Since the active state on the physical distributor is really part of the
> timer logic, it is the job of our virtual arch timer driver to manage
> this state.
> 
> The timer->map->active boolean field indicates whether we have signalled
> this interrupt to the vgic and if that interrupt is still pending or
> active.  As long as that is the case, the hardware doesn't have to
> generate physical interrupts and therefore we mark the interrupt as
> active on the physical distributor.
> 
> Cc: Marc Zyngier 
> Reported-by: Lorenzo Pieralisi 
> Signed-off-by: Christoffer Dall 
> ---
>  virt/kvm/arm/arch_timer.c | 19 +++
>  virt/kvm/arm/vgic.c   | 43 +++
>  2 files changed, 30 insertions(+), 32 deletions(-)
> 
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index 48c6e1a..b9d3a32 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -137,6 +137,8 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
>  void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
>  {
>   struct arch_timer_cpu *timer = >arch.timer_cpu;
> + bool phys_active;
> + int ret;
>  
>   /*
>* We're about to run this vcpu again, so there is no need to
> @@ -151,6 +153,23 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
>*/
>   if (kvm_timer_should_fire(vcpu))
>   kvm_timer_inject_irq(vcpu);
> +
> + /*
> +  * We keep track of whether the edge-triggered interrupt has been
> +  * signalled to the vgic/guest, and if so, we mask the interrupt and
> +  * the physical distributor to prevent the timer from raising a
> +  * physical interrupt whenever we run a guest, preventing forward
> +  * VCPU progress.
In practice don't you simply mark the IRQ as active at GIC physical
distributor level, hence preventing the same IRQ from hitting again
> +  */
> + if (kvm_vgic_get_phys_irq_active(timer->map))
> + phys_active = true;
> + else
> + phys_active = false;
> +
> + ret = irq_set_irqchip_state(timer->map->irq,
> + IRQCHIP_STATE_ACTIVE,
> + phys_active);

physical distributor state is set in arch timer flush. It relates to a
shared device behavior so I find it natural to do it there.

However the map->active is set in arch_timer IRQ injection and unset in
vgic sync. Why not doing the set in kvm_vgic_inject_mapped_irq?

> + WARN_ON(ret);
>  }
>  
>  /**
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 596455a..ea21bc2 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -1092,6 +1092,15 @@ static void vgic_retire_lr(int lr_nr, int irq, struct 
> kvm_vcpu *vcpu)
>   struct vgic_cpu *vgic_cpu = >arch.vgic_cpu;
>   struct vgic_lr vlr = vgic_get_lr(vcpu, lr_nr);
>  
> + /*
> +  * We must transfer the pending state back to the distributor before
> +  * retiring the LR, otherwise we may loose edge-triggered interrupts.
> +  */
> + if (vlr.state & LR_STATE_PENDING) {
> + vgic_dist_irq_set_pending(vcpu, irq);
> + vlr.hwirq = 0;
> + }
That fix applies to any edge-sensitive IRQ, ie. not especially the
timer's one? In the positive shouldn't you precise this in the commit
msg too?

Best Regards

Eric

> +
>   vlr.state = 0;
>   vgic_set_lr(vcpu, lr_nr, vlr);
>   clear_bit(lr_nr, vgic_cpu->lr_used);
> @@ -1241,7 +1250,7 @@ static void __kvm_vgic_flush_hwstate(struct kvm_vcpu 
> *vcpu)
>   struct vgic_cpu *vgic_cpu = >arch.vgic_cpu;
>   struct vgic_dist *dist = >kvm->arch.vgic;
>   unsigned long *pa_percpu, *pa_shared;
> - int i, vcpu_id, lr, ret;
> + int i, vcpu_id;
>   int overflow = 0;
>   int nr_shared = vgic_nr_shared_irqs(dist);
>  
> @@ -1296,31 +1305,6 @@ epilog:
>*/
>   clear_bit(vcpu_id, dist->irq_pending_on_cpu);
>   }
> -
> - for (lr = 0; lr < vgic->nr_lr; lr++) {
> - struct 

Re: [PATCH 2/4] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

On 10/19/2015 03:25 PM, Arnd Bergmann wrote:
> On Monday 19 October 2015 15:17:30 Eric Auger wrote:
>> Hi Arnd,
>> On 10/19/2015 03:04 PM, Arnd Bergmann wrote:
>>> On Sunday 18 October 2015 18:00:13 Eric Auger wrote:
>>>> diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
>>>> b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
>>>> index 619dc7d..4f76b17 100644
>>>> --- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
>>>> +++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
>>>> @@ -29,8 +29,7 @@
>>>>  #define DRIVER_VERSION  "0.1"
>>>>  #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
>>>>  #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform 
>>>> device"
>>>> -
>>>> -#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
>>>> +#define COMPAT"calxeda,hb-xgmac"
>>>
>>> Why the rename?
>> This define was not used. Since the code is meant to be duplicated from
>> one reset module to the other I thought it did not bring any have a
>> specialized name
> 
> I'd say it would be clearer to remove the macro then, and pass the
> string literal in the function call.
OK
> 
>>>>  /* XGMAC Register definitions */
>>>>  #define XGMAC_CONTROL   0x  /* MAC Configuration */
>>>> @@ -80,6 +79,43 @@ int vfio_platform_calxedaxgmac_reset(struct 
>>>> vfio_platform_device *vdev)
>>>>  }
>>>>  EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
>>>>  
>>>> +static int __init vfio_platform_calxedaxgmac_init(void)
>>>> +{
>>>> +  int (*register_reset)(struct module *, char*,
>>>> +  vfio_platform_reset_fn_t);
>>>> +  int ret;
>>>> +
>>>> +  register_reset = symbol_get(vfio_platform_register_reset);
>>>> +  if (!register_reset)
>>>> +  return -EINVAL;
>>>
>>> I don't understand what you do the symbol_get() here for.
>>> Why not just call that function directly
>> the function is implemented in a separate module. This is just to make
>> sure the vfio-platform module is loaded, in case the end-user attempts
>> to load the reset module without having this latter loaded.
> 
> The module loader does all this for you.
Ah OK

I will respin shortly taking into account all your comments

Thanks for your time!

Best Regards

Eric


> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/4] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

Hi Arnd,
On 10/19/2015 03:04 PM, Arnd Bergmann wrote:
> On Sunday 18 October 2015 18:00:13 Eric Auger wrote:
>> diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
>> b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
>> index 619dc7d..4f76b17 100644
>> --- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
>> +++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
>> @@ -29,8 +29,7 @@
>>  #define DRIVER_VERSION  "0.1"
>>  #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
>>  #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform 
>> device"
>> -
>> -#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
>> +#define COMPAT  "calxeda,hb-xgmac"
> 
> Why the rename?
This define was not used. Since the code is meant to be duplicated from
one reset module to the other I thought it did not bring any have a
specialized name
> 
>>  /* XGMAC Register definitions */
>>  #define XGMAC_CONTROL   0x  /* MAC Configuration */
>> @@ -80,6 +79,43 @@ int vfio_platform_calxedaxgmac_reset(struct 
>> vfio_platform_device *vdev)
>>  }
>>  EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
>>  
>> +static int __init vfio_platform_calxedaxgmac_init(void)
>> +{
>> +int (*register_reset)(struct module *, char*,
>> +vfio_platform_reset_fn_t);
>> +int ret;
>> +
>> +register_reset = symbol_get(vfio_platform_register_reset);
>> +if (!register_reset)
>> +return -EINVAL;
> 
> I don't understand what you do the symbol_get() here for.
> Why not just call that function directly
the function is implemented in a separate module. This is just to make
sure the vfio-platform module is loaded, in case the end-user attempts
to load the reset module without having this latter loaded.
> 
>> +ret = register_reset(THIS_MODULE, COMPAT,
>> +vfio_platform_calxedaxgmac_reset);
> 
> This is whitespace damaged, the second line needs to be indented
> to the opening braces.
ok
> 
> I would also suggest defining register_reset as a macro that
> implicitly passes the THIS_MODULE argument, as other subsystems
> do.
ok

Thanks

Best Regards

Eric
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] arm/arm64: KVM: Fix arch timer behavior for disabled interrupts

[Eric Auger]

On 10/19/2015 03:14 PM, Christoffer Dall wrote:
> On Mon, Oct 19, 2015 at 03:07:16PM +0200, Eric Auger wrote:
>> Hi Christoffer,
>> On 10/17/2015 10:30 PM, Christoffer Dall wrote:
>>> We have an interesting issue when the guest disables the timer interrupt
>>> on the VGIC, which happens when turning VCPUs off using PSCI, for
>>> example.
>>>
>>> The problem is that because the guest disables the virtual interrupt at
>>> the VGIC level, we never inject interrupts to the guest and therefore
>>> never mark the interrupt as active on the physical distributor.  The
>>> host also never takes the timer interrupt (we only use the timer device
>>> to trigger a guest exit and everything else is done in software), so the
>>> interrupt does not become active through normal means.
>>>
>>> The result is that we keep entering the guest with a programmed timer
>>> that will always fire as soon as we context switch the hardware timer
>>> state and run the guest, preventing forward progress for the VCPU.
>>>
>>> Since the active state on the physical distributor is really part of the
>>> timer logic, it is the job of our virtual arch timer driver to manage
>>> this state.
>>>
>>> The timer->map->active boolean field indicates whether we have signalled
>>> this interrupt to the vgic and if that interrupt is still pending or
>>> active.  As long as that is the case, the hardware doesn't have to
>>> generate physical interrupts and therefore we mark the interrupt as
>>> active on the physical distributor.
>>>
>>> Cc: Marc Zyngier <marc.zyng...@arm.com>
>>> Reported-by: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>
>>> Signed-off-by: Christoffer Dall <christoffer.d...@linaro.org>
>>> ---
>>>  virt/kvm/arm/arch_timer.c | 19 +++
>>>  virt/kvm/arm/vgic.c   | 43 +++
>>>  2 files changed, 30 insertions(+), 32 deletions(-)
>>>
>>> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
>>> index 48c6e1a..b9d3a32 100644
>>> --- a/virt/kvm/arm/arch_timer.c
>>> +++ b/virt/kvm/arm/arch_timer.c
>>> @@ -137,6 +137,8 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
>>>  void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
>>>  {
>>> struct arch_timer_cpu *timer = >arch.timer_cpu;
>>> +   bool phys_active;
>>> +   int ret;
>>>  
>>> /*
>>>  * We're about to run this vcpu again, so there is no need to
>>> @@ -151,6 +153,23 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
>>>  */
>>> if (kvm_timer_should_fire(vcpu))
>>> kvm_timer_inject_irq(vcpu);
>>> +
>>> +   /*
>>> +* We keep track of whether the edge-triggered interrupt has been
>>> +* signalled to the vgic/guest, and if so, we mask the interrupt and
>>> +* the physical distributor to prevent the timer from raising a
>>> +* physical interrupt whenever we run a guest, preventing forward
>>> +* VCPU progress.
>> In practice don't you simply mark the IRQ as active at GIC physical
>> distributor level, hence preventing the same IRQ from hitting again
> 
> yes, that's what I meant with my comment, I should reword to "...we mark
> the interrupt as active on the physical distributor..."
> 
>>> +*/
>>> +   if (kvm_vgic_get_phys_irq_active(timer->map))
>>> +   phys_active = true;
>>> +   else
>>> +   phys_active = false;
>>> +
>>> +   ret = irq_set_irqchip_state(timer->map->irq,
>>> +   IRQCHIP_STATE_ACTIVE,
>>> +   phys_active);
>>
>> physical distributor state is set in arch timer flush. It relates to a
>> shared device behavior so I find it natural to do it there.
>>
>> However the map->active is set in arch_timer IRQ injection and unset in
>> vgic sync. Why not doing the set in kvm_vgic_inject_mapped_irq?
> 
> Because you have to set it at every entry to the guest if you run
> multiple VCPUs/VMs on this CPU or migrate this VCPU to a different CPU.
I meant kvm_vgic_set_phys_irq_active(timer->map, true) call in
kvm_timer_inject_irq? Couldn' that been done in
kvm_vgic_inject_mapped_irq instead. Doesn't this apply to all mapped IRQs?

Eric
> 
>>
>>> +   WARN_ON(ret);
>>>  }
>>>  
>>>  /**
>>> diff --git a/virt/kv

Re: [PATCH 2/3] arm/arm64: KVM: Clear map->active on pend/active clear

[Eric Auger]

On 10/19/2015 05:39 PM, Christoffer Dall wrote:
> On Mon, Oct 19, 2015 at 05:32:42PM +0200, Eric Auger wrote:
>> Hi,
>> On 10/17/2015 10:30 PM, Christoffer Dall wrote:
>>> When a guest reboots or offlines/onlines CPUs, it is not uncommon for it
>>> to clear the pending and active states of an interrupt through the
>>> emulated VGIC distributor.  However, since we emulate an edge-triggered
>>> based on a level-triggered device,
>> I do not get this sentence.
> 
> Fair enough.  The guest expects a level-triggered timer, so it expects
> to be able to do:
> 
>   disable interrupt, set timer(*), clear pending state, enable interrupt
> 
> and this only works because the device is level-triggered, so it should
> hit again after clearing the pending state and the guest should see it
> after enabling interrupts.
> 
> but because we have virtual edge-triggered behavior, we loose the
> interrupt generated at (*) unless we inject it again, which we will
> after this patch, because the actual device is level triggered, so we'll
> sample its state in the KVM timer code contiuously and now  (after this
> patch) notice that we have to inject the edge-triggered IRQ again.

That helps ;-)

Eric
> 
> Yes, I know, we're lucky we can make it work like this.
> 
>>
>> Besides that
>> Reviewed-by: Eric Auger <eric.au...@linaro.org>
> 
> Thanks!
> 
> -Christoffer
> 
>>  the guest expects the timer interrupt
>>> to hit even after clearing the pending state.
>>>
>>> We currently do not signal the VGIC when the map->active field is true,
>>> because it indicates that the guest has already been signalled of the
>>> interrupt as required.  Normally this field is set to false when the
>>> guest deactivates the virtual interrupt through the sync path.
>>>
>>> We also need to catch the case where the guest deactivates the interrupt
>>> through the emulated distributor, again allowing guests to boot even if
>>> the original virtual timer signal hit before the guest's GIC
>>> initialization sequence is run.
>>>
>>> Signed-off-by: Christoffer Dall <christoffer.d...@linaro.org>
>>> ---
>>>  virt/kvm/arm/vgic.c | 32 +++-
>>>  1 file changed, 31 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>>> index ea21bc2..58b1256 100644
>>> --- a/virt/kvm/arm/vgic.c
>>> +++ b/virt/kvm/arm/vgic.c
>>> @@ -531,6 +531,34 @@ bool vgic_handle_set_pending_reg(struct kvm *kvm,
>>> return false;
>>>  }
>>>  
>>> +/*
>>> + * If a mapped interrupt's state has been modified by the guest such that 
>>> it
>>> + * is no longer active or pending, without it have gone through the sync 
>>> path,
>>> + * then the map->active field must be cleared so the interrupt can be taken
>>> + * again.
>>> + */
>>> +static void vgic_handle_clear_mapped_irq(struct kvm_vcpu *vcpu)
>>> +{
>>> +   struct vgic_cpu *vgic_cpu = >arch.vgic_cpu;
>>> +   struct list_head *root;
>>> +   struct irq_phys_map_entry *entry;
>>> +   struct irq_phys_map *map;
>>> +
>>> +   rcu_read_lock();
>>> +
>>> +   /* Check for PPIs */
>>> +   root = _cpu->irq_phys_map_list;
>>> +   list_for_each_entry_rcu(entry, root, entry) {
>>> +   map = >map;
>>> +
>>> +   if (!vgic_dist_irq_is_pending(vcpu, map->virt_irq) &&
>>> +   !vgic_irq_is_active(vcpu, map->virt_irq))
>>> +   map->active = false;
>>> +   }
>>> +
>>> +   rcu_read_unlock();
>>> +}
>>> +
>>>  bool vgic_handle_clear_pending_reg(struct kvm *kvm,
>>>struct kvm_exit_mmio *mmio,
>>>phys_addr_t offset, int vcpu_id)
>>> @@ -561,6 +589,7 @@ bool vgic_handle_clear_pending_reg(struct kvm *kvm,
>>>   vcpu_id, offset);
>>> vgic_reg_access(mmio, reg, offset, mode);
>>>  
>>> +   vgic_handle_clear_mapped_irq(kvm_get_vcpu(kvm, vcpu_id));
>>> vgic_update_state(kvm);
>>> return true;
>>> }
>>> @@ -598,6 +627,7 @@ bool vgic_handle_clear_active_reg(struct kvm *kvm,
>>> ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
>>>  
>>> if (mmio->is_write) {
>>> +   vgic_handle_clear_mapped_irq(kvm_get_vcpu(kvm, vcpu_id));
>>> vgic_update_state(kvm);
>>> return true;
>>> }
>>> @@ -1406,7 +1436,7 @@ static int vgic_sync_hwirq(struct kvm_vcpu *vcpu, 
>>> struct vgic_lr vlr)
>>> return 0;
>>>  
>>> map = vgic_irq_map_search(vcpu, vlr.irq);
>>> -   BUG_ON(!map || !map->active);
>>> +   BUG_ON(!map);
>>>  
>>> ret = irq_get_irqchip_state(map->irq,
>>> IRQCHIP_STATE_ACTIVE,
>>>
>>

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] arm/arm64: KVM: Clear map->active on pend/active clear

[Eric Auger]

Hi,
On 10/17/2015 10:30 PM, Christoffer Dall wrote:
> When a guest reboots or offlines/onlines CPUs, it is not uncommon for it
> to clear the pending and active states of an interrupt through the
> emulated VGIC distributor.  However, since we emulate an edge-triggered
> based on a level-triggered device,
I do not get this sentence.

Besides that
Reviewed-by: Eric Auger <eric.au...@linaro.org>

Best Regards

Eric
 the guest expects the timer interrupt
> to hit even after clearing the pending state.
> 
> We currently do not signal the VGIC when the map->active field is true,
> because it indicates that the guest has already been signalled of the
> interrupt as required.  Normally this field is set to false when the
> guest deactivates the virtual interrupt through the sync path.
> 
> We also need to catch the case where the guest deactivates the interrupt
> through the emulated distributor, again allowing guests to boot even if
> the original virtual timer signal hit before the guest's GIC
> initialization sequence is run.
> 
> Signed-off-by: Christoffer Dall <christoffer.d...@linaro.org>
> ---
>  virt/kvm/arm/vgic.c | 32 +++-
>  1 file changed, 31 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index ea21bc2..58b1256 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -531,6 +531,34 @@ bool vgic_handle_set_pending_reg(struct kvm *kvm,
>   return false;
>  }
>  
> +/*
> + * If a mapped interrupt's state has been modified by the guest such that it
> + * is no longer active or pending, without it have gone through the sync 
> path,
> + * then the map->active field must be cleared so the interrupt can be taken
> + * again.
> + */
> +static void vgic_handle_clear_mapped_irq(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_cpu *vgic_cpu = >arch.vgic_cpu;
> + struct list_head *root;
> + struct irq_phys_map_entry *entry;
> + struct irq_phys_map *map;
> +
> + rcu_read_lock();
> +
> + /* Check for PPIs */
> + root = _cpu->irq_phys_map_list;
> + list_for_each_entry_rcu(entry, root, entry) {
> + map = >map;
> +
> + if (!vgic_dist_irq_is_pending(vcpu, map->virt_irq) &&
> + !vgic_irq_is_active(vcpu, map->virt_irq))
> + map->active = false;
> + }
> +
> + rcu_read_unlock();
> +}
> +
>  bool vgic_handle_clear_pending_reg(struct kvm *kvm,
>  struct kvm_exit_mmio *mmio,
>  phys_addr_t offset, int vcpu_id)
> @@ -561,6 +589,7 @@ bool vgic_handle_clear_pending_reg(struct kvm *kvm,
> vcpu_id, offset);
>   vgic_reg_access(mmio, reg, offset, mode);
>  
> + vgic_handle_clear_mapped_irq(kvm_get_vcpu(kvm, vcpu_id));
>   vgic_update_state(kvm);
>   return true;
>   }
> @@ -598,6 +627,7 @@ bool vgic_handle_clear_active_reg(struct kvm *kvm,
>   ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
>  
>   if (mmio->is_write) {
> + vgic_handle_clear_mapped_irq(kvm_get_vcpu(kvm, vcpu_id));
>   vgic_update_state(kvm);
>   return true;
>   }
> @@ -1406,7 +1436,7 @@ static int vgic_sync_hwirq(struct kvm_vcpu *vcpu, 
> struct vgic_lr vlr)
>   return 0;
>  
>   map = vgic_irq_map_search(vcpu, vlr.irq);
> - BUG_ON(!map || !map->active);
> + BUG_ON(!map);
>  
>   ret = irq_get_irqchip_state(map->irq,
>   IRQCHIP_STATE_ACTIVE,
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/4] vfio: platform: add capability to register a reset function

[Eric Auger]

In preparation for subsequent changes in reset function lookup,
lets introduce a dynamic list of reset combos (compat string,
reset module, reset function). The list can be populated/voided with
two new functions, vfio_platform_register/unregister_reset. Those are
not yet used in this patch.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 drivers/vfio/platform/vfio_platform_common.c  | 55 +++
 drivers/vfio/platform/vfio_platform_private.h | 14 +++
 2 files changed, 69 insertions(+)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index e43efb5..d36afc9 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -23,6 +23,8 @@
 
 #include "vfio_platform_private.h"
 
+struct list_head reset_list;
+LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
 static const struct vfio_platform_reset_combo reset_lookup_table[] = {
@@ -573,3 +575,56 @@ struct vfio_platform_device 
*vfio_platform_remove_common(struct device *dev)
return vdev;
 }
 EXPORT_SYMBOL_GPL(vfio_platform_remove_common);
+
+int vfio_platform_register_reset(struct module *reset_owner, char *compat,
+vfio_platform_reset_fn_t reset)
+{
+   struct vfio_platform_reset_node *node, *iter;
+   bool found = false;
+
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat)) {
+   found = true;
+   break;
+   }
+   }
+   if (found)
+   return -EINVAL;
+
+   node = kmalloc(sizeof(*node), GFP_KERNEL);
+   if (!node)
+   return -ENOMEM;
+
+   node->compat = kstrdup(compat, GFP_KERNEL);
+   if (!node->compat)
+   return -ENOMEM;
+
+   node->owner = reset_owner;
+   node->reset = reset;
+
+   list_add(>link, _list);
+   return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_platform_register_reset);
+
+int vfio_platform_unregister_reset(char *compat)
+{
+   struct vfio_platform_reset_node *iter;
+   bool found = false;
+
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat)) {
+   found = true;
+   break;
+   }
+   }
+   if (!found)
+   return -EINVAL;
+
+   list_del(>link);
+   kfree(iter->compat);
+   kfree(iter);
+   return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_platform_unregister_reset);
+
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 1c9b3d5..17323f0 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -76,6 +76,15 @@ struct vfio_platform_reset_combo {
const char *module_name;
 };
 
+typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev);
+
+struct vfio_platform_reset_node {
+   struct list_head link;
+   char *compat;
+   struct module *owner;
+   vfio_platform_reset_fn_t reset;
+};
+
 extern int vfio_platform_probe_common(struct vfio_platform_device *vdev,
  struct device *dev);
 extern struct vfio_platform_device *vfio_platform_remove_common
@@ -89,4 +98,9 @@ extern int vfio_platform_set_irqs_ioctl(struct 
vfio_platform_device *vdev,
unsigned start, unsigned count,
void *data);
 
+extern int vfio_platform_register_reset(struct module *owner,
+   char *compat,
+   vfio_platform_reset_fn_t reset);
+extern int vfio_platform_unregister_reset(char *compat);
+
 #endif /* VFIO_PLATFORM_PRIVATE_H */
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/4] VFIO platform reset module rework

[Eric Auger]

This series fixes the current implementation by getting rid of the
usage of __symbol_get which caused a compilation issue with
CONFIG_MODULES disabled. On top of this, the usage of MODULE_ALIAS makes
possible to add a new reset module without being obliged to update the
framework. The new implementation relies on the reset module registering
its reset function to the vfio-platform driver.

The series is available at

https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.3-rc5-rework-xgbe-v2

Best Regards

Eric


Eric Auger (4):
  vfio: platform: add capability to register a reset function
  vfio: platform: reset: calxedaxgmac: add reset function registration
  vfio: platform: add compat in vfio_platform_device
  vfio: platform: use list of registered reset function

 .../platform/reset/vfio_platform_calxedaxgmac.c|  40 +++-
 drivers/vfio/platform/vfio_platform_common.c   | 112 -
 drivers/vfio/platform/vfio_platform_private.h  |  16 +++
 3 files changed, 140 insertions(+), 28 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] vfio: platform: use list of registered reset function

[Eric Auger]

Remove the static lookup table and use the dynamic list of registered
reset functions instead. Also load the reset module through its alias.
The reset struct module pointer is stored in vfio_platform_device.

This patch fixes the issue related to the usage of __symbol_get, which
besides from being moot, prevented compilation with CONFIG_MODULES
disabled.

Also usage of MODULE_ALIAS makes possible to add a new reset module
without needing to update the framework. This was suggested by Arnd.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
Reported-by: Arnd Bergmann <a...@arndb.de>
---
 drivers/vfio/platform/vfio_platform_common.c  | 46 +++
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 26 insertions(+), 21 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index 31a6a8c..f3b6299 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -27,37 +27,41 @@ struct list_head reset_list;
 LIST_HEAD(reset_list);
 static DEFINE_MUTEX(driver_lock);
 
-static const struct vfio_platform_reset_combo reset_lookup_table[] = {
-   {
-   .compat = "calxeda,hb-xgmac",
-   .reset_function_name = "vfio_platform_calxedaxgmac_reset",
-   .module_name = "vfio-platform-calxedaxgmac",
-   },
-};
+static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat,
+   struct module **module)
+{
+   struct vfio_platform_reset_node *iter;
+
+   list_for_each_entry(iter, _list, link) {
+   if (!strcmp(iter->compat, compat) &&
+   try_module_get(iter->owner)) {
+   *module = iter->owner;
+   return iter->reset;
+   }
+   }
+
+   return NULL;
+}
 
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   int (*reset)(struct vfio_platform_device *);
-   int i;
-
-   for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
-   request_module(reset_lookup_table[i].module_name);
-   reset = __symbol_get(
-   reset_lookup_table[i].reset_function_name);
-   if (reset) {
-   vdev->reset = reset;
-   return;
-   }
-   }
+   char modname[256];
+
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+   >reset_module);
+   if (!vdev->reset) {
+   snprintf(modname, 256, "vfio-reset:%s", vdev->compat);
+   request_module(modname);
+   vdev->reset = vfio_platform_lookup_reset(vdev->compat,
+>reset_module);
}
 }
 
 static void vfio_platform_put_reset(struct vfio_platform_device *vdev)
 {
if (vdev->reset)
-   symbol_put_addr(vdev->reset);
+   module_put(vdev->reset_module);
 }
 
 static int vfio_platform_regions_init(struct vfio_platform_device *vdev)
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index b274646..2070dcc 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -57,6 +57,7 @@ struct vfio_platform_device {
int refcnt;
struct mutexigate;
const char  *compat;
+   struct module   *reset_module;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/4] vfio: platform: reset: calxedaxgmac: add reset function registration

[Eric Auger]

This patch adds the reset function registration/unregistration.
Also a MODULE_ALIAS is added.

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 .../platform/reset/vfio_platform_calxedaxgmac.c| 40 --
 1 file changed, 38 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c 
b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
index 619dc7d..4f76b17 100644
--- a/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
+++ b/drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c
@@ -29,8 +29,7 @@
 #define DRIVER_VERSION  "0.1"
 #define DRIVER_AUTHOR   "Eric Auger <eric.au...@linaro.org>"
 #define DRIVER_DESC "Reset support for Calxeda xgmac vfio platform device"
-
-#define CALXEDAXGMAC_COMPAT "calxeda,hb-xgmac"
+#define COMPAT "calxeda,hb-xgmac"
 
 /* XGMAC Register definitions */
 #define XGMAC_CONTROL   0x  /* MAC Configuration */
@@ -80,6 +79,43 @@ int vfio_platform_calxedaxgmac_reset(struct 
vfio_platform_device *vdev)
 }
 EXPORT_SYMBOL_GPL(vfio_platform_calxedaxgmac_reset);
 
+static int __init vfio_platform_calxedaxgmac_init(void)
+{
+   int (*register_reset)(struct module *, char*,
+   vfio_platform_reset_fn_t);
+   int ret;
+
+   register_reset = symbol_get(vfio_platform_register_reset);
+   if (!register_reset)
+   return -EINVAL;
+
+   ret = register_reset(THIS_MODULE, COMPAT,
+   vfio_platform_calxedaxgmac_reset);
+
+   symbol_put(vfio_platform_register_reset);
+
+   return ret;
+}
+
+static void __exit vfio_platform_calxedaxgmac_exit(void)
+{
+   int (*unregister_reset)(char *);
+   int ret;
+
+   unregister_reset = symbol_get(vfio_platform_unregister_reset);
+   if (!unregister_reset)
+   return;
+
+   ret = unregister_reset(COMPAT);
+
+   symbol_put(vfio_platform_unregister_reset);
+}
+
+module_init(vfio_platform_calxedaxgmac_init);
+module_exit(vfio_platform_calxedaxgmac_exit);
+
+MODULE_ALIAS("vfio-reset:"  COMPAT);
+
 MODULE_VERSION(DRIVER_VERSION);
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR(DRIVER_AUTHOR);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/4] vfio: platform: add compat in vfio_platform_device

[Eric Auger]

Let's retrieve the compatibility string on probe and store it
in the vfio_platform_device struct

Signed-off-by: Eric Auger <eric.au...@linaro.org>
---
 drivers/vfio/platform/vfio_platform_common.c  | 15 ---
 drivers/vfio/platform/vfio_platform_private.h |  1 +
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_common.c 
b/drivers/vfio/platform/vfio_platform_common.c
index d36afc9..31a6a8c 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -38,16 +38,11 @@ static const struct vfio_platform_reset_combo 
reset_lookup_table[] = {
 static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
struct device *dev)
 {
-   const char *compat;
int (*reset)(struct vfio_platform_device *);
-   int ret, i;
-
-   ret = device_property_read_string(dev, "compatible", );
-   if (ret)
-   return;
+   int i;
 
for (i = 0 ; i < ARRAY_SIZE(reset_lookup_table); i++) {
-   if (!strcmp(reset_lookup_table[i].compat, compat)) {
+   if (!strcmp(reset_lookup_table[i].compat, vdev->compat)) {
request_module(reset_lookup_table[i].module_name);
reset = __symbol_get(
reset_lookup_table[i].reset_function_name);
@@ -538,6 +533,12 @@ int vfio_platform_probe_common(struct vfio_platform_device 
*vdev,
struct iommu_group *group;
int ret;
 
+   ret = device_property_read_string(dev, "compatible", >compat);
+   if (ret) {
+   pr_err("VFIO: cannot retrieve compat for %s\n", vdev->name);
+   return -EINVAL;
+   }
+
if (!vdev)
return -EINVAL;
 
diff --git a/drivers/vfio/platform/vfio_platform_private.h 
b/drivers/vfio/platform/vfio_platform_private.h
index 17323f0..b274646 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -56,6 +56,7 @@ struct vfio_platform_device {
u32 num_irqs;
int refcnt;
struct mutexigate;
+   const char  *compat;
 
/*
 * These fields should be filled by the bus specific binder
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] vfio/type1: Do not support IOMMUs that allow bypass

[Eric Auger]

Hi Alex,
On 10/15/2015 10:52 PM, Alex Williamson wrote:
> We can only provide isolation if DMA is forced through the IOMMU
> aperture.  Don't allow type1 to be used if this is not the case.
> 
> Signed-off-by: Alex Williamson 
> ---
> 
> Eric, I see a number of IOMMU drivers enable this, do the ones you
> care about for ARM set geometry.force_aperture?  Thanks,
I am currently using arm-smmu.c. I don't see force_aperture being set.

Best Regards

Eric
> 
> Alex
> 
>  drivers/vfio/vfio_iommu_type1.c |   12 
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 57d8c37..6afa9d4 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -728,6 +728,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
>   struct vfio_group *group, *g;
>   struct vfio_domain *domain, *d;
>   struct bus_type *bus = NULL;
> + struct iommu_domain_geometry geometry;
>   int ret;
>  
>   mutex_lock(>lock);
> @@ -762,6 +763,17 @@ static int vfio_iommu_type1_attach_group(void 
> *iommu_data,
>   goto out_free;
>   }
>  
> + /*
> +  * If a domain does not force DMA within the aperture, devices are not
> +  * isolated and type1 is not an appropriate IOMMU model.
> +  */
> + ret = iommu_domain_get_attr(domain->domain,
> + DOMAIN_ATTR_GEOMETRY, );
> + if (ret || !geometry.force_aperture) {
> + ret = -EPERM;
> + goto out_domain;
> + }
> +
>   if (iommu->nesting) {
>   int attr = 1;
>  
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] VFIO: platform: AMD xgbe reset module

[Eric Auger]

Hi Arnd,
On 10/16/2015 03:26 PM, Arnd Bergmann wrote:
> On Friday 16 October 2015 15:06:45 Eric Auger wrote:
> 
>>  I've since forgotten his answer, but the fact that
>>> __symbol_get() is only defined for modules makes it moot, we either need
>>> to make symbol_get() work or define __symbol_get() for non-module
>>> builds.
>> I currently don't see any solution for any of those. The only solution I
>> can see is someone registers the reset function pointer to vfio.
>>
>> I think we could keep the existing reset modules, do the request_module
>> from VFIO, using their module name registered in the lookup table. But
>> instead of having the reset function in the look-up table we would have
>> the reset modules register their reset function pointer to VFIO. I think
>> this could work around the symbol_get issue.
>>
>> This still leaves the layer violation argument though.
>>
>> Assuming this works, would that be an acceptable solution, although I
>> acknowledge this does not perfectly fit into the driver model?
> 
> I think it's possible to avoid the layering violation that way too,
> by loading the module based on the compatible string, with a module_alias.
> 
> static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
> struct device *dev)
> {
> const char *compat;
> int (*reset)(struct vfio_platform_device *);
> int ret, i;
>   char modname[256];
> 
> ret = device_property_read_string(dev, "compatible", );
> if (ret)
> return;
> 
>   reset = vfio_platform_lookup_reset(compat);
>   if (!reset) {
>   snprintf(modname, "vfio-reset:%s", compat);
>   request_module(modname);
>   reset = vfio_platform_lookup_reset(compat);
>   }
> 
>   vdev->reset = reset;
> }
> 
> ---
> 
> #define module_vfio_reset_handler(compat, reset)  \
> MODULE_ALIAS("vfio_reset" compat);\
> static int reset ## _module_init(void)
> \
> { \
>   return vfio_reset_register(THIS_MODULE, compat, );\
> }
> 
> I think that solution is good enough, as it avoids most of the
> problems with the current implementation but is a simple enough change.

That looks a good perspective. Thanks for the hint! Let's code now ;-)

Best Regards

Eric


> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] VFIO: platform: AMD xgbe reset module

[Eric Auger]

Dear all,
On 10/15/2015 06:53 PM, Alex Williamson wrote:
> On Thu, 2015-10-15 at 16:46 +0200, Eric Auger wrote:
>> Hi Arnd,
>> On 10/15/2015 03:59 PM, Arnd Bergmann wrote:
>>> On Thursday 15 October 2015 14:12:28 Christoffer Dall wrote:
>>>>>
>>>>> enum vfio_platform_op {
>>>>>   VFIO_PLATFORM_BIND,
>>>>>   VFIO_PLATFORM_UNBIND,
>>>>>   VFIO_PLATFORM_RESET,
>>>>> };
>>>>>
>>>>> struct platform_driver {
>>>>> int (*probe)(struct platform_device *);
>>>>> int (*remove)(struct platform_device *);
>>>>>   ...
>>>>>   int (*vfio_manage)(struct platform_device *, enum vfio_platform_op);
>>>>> struct device_driver driver;
>>>>> };
>>>>>
>>>>> This would integrate much more closely into the platform driver framework,
>>>>> just like the regular vfio driver integrates into the PCI framework.
>>>>> Unlike PCI however, you can't just use the generic driver framework to
>>>>> unbind the driver, because you still need device specific code.
>>>>>
>>>> Thanks for these suggestions, really helpful.
>>>>
>>>> What I don't understand in the latter example is how VFIO knows which
>>>> struct platform_driver to interact with?
>>>
>>> This would assume that the driver remains bound to the device, so VFIO
>>> gets a pointer to the device from somewhere (as it does today) and then
>>> follows the dev->driver pointer to get to the platform_driver.
> 
> The complexity of managing a bi-modal driver seems like far more than a
> little bit of code duplication in a device specific reset module and
> extends into how userspace makes devices available through vfio, so I
> think it's too late for that discussion.
>   
>>>> Also, just so I'm sure I understand correctly, VFIO_PLATFORM_UNBIND is
>>>> then called by VFIO before the VFIO driver unbinds from the device
>>>> (unbinding the platform driver from the device being a completely
>>>> separate thing)?
>>>
>>> This is where we'd need a little more changes for this approach. Instead
>>> of unbinding the device from its driver, the idea would be that the
>>> driver remains bound as far as the driver model is concerned, but
>>> it would be in a quiescent state where no other subsystem interacts with
>>> it (i.e. it gets unregistered from networking core or whichever it uses).
>>
>> Currently we use the same mechanism as for PCI, ie. unbind the native
>> driver and then bind VFIO platform driver in its place. Don't you think
>> changing this may be a pain for user-space tools that are designed to
>> work that way for PCI?
>>
>> My personal preference would be to start with your first proposal since
>> it looks (to me) less complex and "unknown" that the 2d approach.
>>
>> Let's wait for Alex opinion too...
> 
> I thought the reason we took the approach we have now is so that we
> don't have reset code loaded into the kernel unless we have a device
> that needs it.  Therefore we don't really want to preemptively load all
> the reset drivers and have them do a registration.  The unfortunate
> side-effect of that is the platform code needs to go looking for the
> driver.  We do that via the __symbol_get() trick, which only fails
> without modules because the underscore variant isn't defined in that
> case.  I remember asking Eric previously why we're using that rather
> than symbol_get(),

the rationale behind using __get_symbol is that the function takes a
char * as argument while the get_symbol macro takes the very symbol. I
needed to provide a string since this is what is stored in the lookup
table. Unfortunately I did not see the #if CONFIG_MODULES. I Should have
been warned about "__" and Alex doubts, sin of naïvety.

 I've since forgotten his answer, but the fact that
> __symbol_get() is only defined for modules makes it moot, we either need
> to make symbol_get() work or define __symbol_get() for non-module
> builds.
I currently don't see any solution for any of those. The only solution I
can see is someone registers the reset function pointer to vfio.

I think we could keep the existing reset modules, do the request_module
from VFIO, using their module name registered in the lookup table. But
instead of having the reset function in the look-up table we would have
the reset modules register their reset function pointer to VFIO. I think
this could work around the symbol_get issue.

This still leaves the layer violation argument though.

Assuming this works, would that be an acceptable solution, although I
acknowledge this does not perfectly fit into the driver model?

Best Regards

Eric
> 
> Otherwise, we should probably abandon the idea of these reset functions
> being modules and build them into the vfio platform driver (which would
> still be less loaded, dead code than a bi-modal host driver).  Thanks,

> 
> Alex
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] VFIO: platform: AMD xgbe reset module

[Eric Auger]

Hi Arnd,
On 10/14/2015 05:38 PM, Arnd Bergmann wrote:
> On Wednesday 14 October 2015 15:33:12 Eric Auger wrote:
>> --- a/drivers/vfio/platform/vfio_platform_common.c
>> +++ b/drivers/vfio/platform/vfio_platform_common.c
>> @@ -31,6 +31,11 @@ static const struct vfio_platform_reset_combo 
>> reset_lookup_table[] = {
>> .reset_function_name = "vfio_platform_calxedaxgmac_reset",
>> .module_name = "vfio-platform-calxedaxgmac",
>> },
>> +   {
>> +   .compat = "amd,xgbe-seattle-v1a",
>> +   .reset_function_name = "vfio_platform_amdxgbe_reset",
>> +   .module_name = "vfio-platform-amdxgbe",
>> +   },
>>  };
>>  
>>  static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
>>
> 
> This is causing build errors for me when CONFIG_MODULES is disabled.
Sorry about that and thanks for reporting the issue
> 
> Could this please be restructured so vfio_platform_get_reset does
> not attempt to call __symbol_get() but instead has the drivers
> register themselves properly to a subsystem?
OK

Could you elaborate about "has the drivers register themselves properly
to a subsystem".

My first proposal when coping with this problematic of being able to add
reset plugins to the vfio-platform driver was to create new drivers per
device requiring reset. but this was considered painful for end-users,
who needed to be aware of the right driver to bind - and I think that
makes sense - (https://lkml.org/lkml/2015/4/17/568) .

A naive question I dare to ask, wouldn't it be acceptable to make
vfio_platform depend on CONFIG_MODULES? Don't we disable modules for
security purpose? In that context would we use VFIO?

Best Regards

Eric
> 
> I don't see any way this could be fixed otherwise. The problem
> of course showed up with calxedaxgmac already, but I'd prefer not
> to see anything added there until the common code has been improved.
> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] VFIO: platform: AMD xgbe reset module

[Eric Auger]

Hi Arnd,
On 10/15/2015 03:59 PM, Arnd Bergmann wrote:
> On Thursday 15 October 2015 14:12:28 Christoffer Dall wrote:
>>>
>>> enum vfio_platform_op {
>>>   VFIO_PLATFORM_BIND,
>>>   VFIO_PLATFORM_UNBIND,
>>>   VFIO_PLATFORM_RESET,
>>> };
>>>
>>> struct platform_driver {
>>> int (*probe)(struct platform_device *);
>>> int (*remove)(struct platform_device *);
>>>   ...
>>>   int (*vfio_manage)(struct platform_device *, enum vfio_platform_op);
>>> struct device_driver driver;
>>> };
>>>
>>> This would integrate much more closely into the platform driver framework,
>>> just like the regular vfio driver integrates into the PCI framework.
>>> Unlike PCI however, you can't just use the generic driver framework to
>>> unbind the driver, because you still need device specific code.
>>>
>> Thanks for these suggestions, really helpful.
>>
>> What I don't understand in the latter example is how VFIO knows which
>> struct platform_driver to interact with?
> 
> This would assume that the driver remains bound to the device, so VFIO
> gets a pointer to the device from somewhere (as it does today) and then
> follows the dev->driver pointer to get to the platform_driver.

>  
>> Also, just so I'm sure I understand correctly, VFIO_PLATFORM_UNBIND is
>> then called by VFIO before the VFIO driver unbinds from the device
>> (unbinding the platform driver from the device being a completely
>> separate thing)?
> 
> This is where we'd need a little more changes for this approach. Instead
> of unbinding the device from its driver, the idea would be that the
> driver remains bound as far as the driver model is concerned, but
> it would be in a quiescent state where no other subsystem interacts with
> it (i.e. it gets unregistered from networking core or whichever it uses).

Currently we use the same mechanism as for PCI, ie. unbind the native
driver and then bind VFIO platform driver in its place. Don't you think
changing this may be a pain for user-space tools that are designed to
work that way for PCI?

My personal preference would be to start with your first proposal since
it looks (to me) less complex and "unknown" that the 2d approach.

Let's wait for Alex opinion too...

Thanks again

Eric




> 
>   Arnd
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] VFIO: platform: AMD xgbe reset module

[Eric Auger]

Hi Arnd,
On 10/15/2015 02:12 PM, Christoffer Dall wrote:
> On Thu, Oct 15, 2015 at 01:21:55PM +0200, Arnd Bergmann wrote:
>> On Thursday 15 October 2015 10:08:02 Eric Auger wrote:
>>> Hi Arnd,
>>> On 10/14/2015 05:38 PM, Arnd Bergmann wrote:
>>>> On Wednesday 14 October 2015 15:33:12 Eric Auger wrote:
>>>>> --- a/drivers/vfio/platform/vfio_platform_common.c
>>>>> +++ b/drivers/vfio/platform/vfio_platform_common.c
>>>>> @@ -31,6 +31,11 @@ static const struct vfio_platform_reset_combo 
>>>>> reset_lookup_table[] = {
>>>>> .reset_function_name = "vfio_platform_calxedaxgmac_reset",
>>>>> .module_name = "vfio-platform-calxedaxgmac",
>>>>> },
>>>>> +   {
>>>>> +   .compat = "amd,xgbe-seattle-v1a",
>>>>> +   .reset_function_name = "vfio_platform_amdxgbe_reset",
>>>>> +   .module_name = "vfio-platform-amdxgbe",
>>>>> +   },
>>>>>  };
>>>>>  
>>>>>  static void vfio_platform_get_reset(struct vfio_platform_device *vdev,
>>>>>
>>>>
>>>> This is causing build errors for me when CONFIG_MODULES is disabled.
>>> Sorry about that and thanks for reporting the issue
>>>>
>>>> Could this please be restructured so vfio_platform_get_reset does
>>>> not attempt to call __symbol_get() but instead has the drivers
>>>> register themselves properly to a subsystem?
>>> OK
>>>
>>> Could you elaborate about "has the drivers register themselves properly
>>> to a subsystem".
>>>
>>> My first proposal when coping with this problematic of being able to add
>>> reset plugins to the vfio-platform driver was to create new drivers per
>>> device requiring reset. but this was considered painful for end-users,
>>> who needed to be aware of the right driver to bind - and I think that
>>> makes sense - (https://lkml.org/lkml/2015/4/17/568) .
>>
>> Having multiple drivers indeed sucks, but your current approach isn't
>> that much better, as you still have two modules that are used to driver
>> the same hardware.
>>
>> I would expect that the same driver that is used for the normal
>> operation and that it calls a function to register itself to vfio
>> by passing a structure with the device and reset function pointer.
>>
>>> A naive question I dare to ask, wouldn't it be acceptable to make
>>> vfio_platform depend on CONFIG_MODULES? Don't we disable modules for
>>> security purpose? In that context would we use VFIO?
>>
>> I think a lot of embedded systems turn off modules to save a little
>> memory, speed up boot time and simplify their user space.
>>
>> Aside from that, the current method is highly unusual and looks a bit
>> fragile to me, as you are relying on internals of the module loader
>> code. It's also a layering violation as the generic code needs to be
>> patched for each device specific module that is added, and we try
>> to avoid that.
Many thanks for taking the time to write this down
>>
>> A possible solution could be something inside the xgbe driver like
>>
>>
>> static void xgbe_init_module(void)
>> {
>>  int ret = 0;
>>
>>  if (IS_ENABLED(CONFIG_AMD_XGBE_ETHERNET)
>>  ret = platform_driver_register(_driver);
>>  if (ret)
>>  return ret;
>>
>>  if (IS_ENABLED(CONFIG_VFIO_PLATFORM))
>>  ret = vfio_platform_register_reset(_of_match, 
>> xgbe_platform_reset);
>>
>>  return ret; 
>> }
>>
>> This way you have exactly one driver module that gets loaded for the
>> device and you can use it either with the platform_driver or through
>> vfio.
If I understand it correctly you still need 2 loaded modules (VFIO
driver & XGBE driver which implements the reset function) or am I
missing something?

I had a similar mechanism of registration in my PATCH v1 but I did the
registration from the reset module itself instead of in the native
driver, as you suggest here.

Best Regards

Eric

>>
>> A nicer way that would be a little more work would be to integrate
>> the reset infrastructure into 'struct platform_driver' framework,
>> by adding another callback to the it for doing the interaction with
>> vfio, something like
>>
>> enum vfio_platform_op {
>>  VFIO_PLATFORM_BIND,
>>