Re: [Xen-devel] [RFC PATCH 09/49] ARM: VGIC: change to level-IRQ compatible IRQ injection interface

[Julien Grall]

Hi,

On 12/02/18 14:24, Andre Przywara wrote:

What would you expect the caller to do on error? Except printing an
error message?


I don't know either. Comparing this to hardware, an IRQ is usually
fire-and-forget (separating the interrupt line from the interrupt state
here), so a device doesn't really handle the case when an IRQ does not
make it through (it can't know easily anyway). However the whole state
machine might get busted in the process (if no one lowers the line, for
instance).
So looking at this printing a message looks like the best choice.

I checked all users of vgic_inject_irq(), at the moment all IRQ numbers
passed in look safe: they are either hardcoded (timer, evtchn) or
validated before (hardware IRQs, when they are tied to a virtual IRQ).
So indeed we *should* never see an invalid IRQ number, at the moment.
I need to check how this changes with the ITS, though.

So we could change the prototype (back) to void, but print some error
message if the vgic_get_irq() call fails within vgic_inject_irq().


Sounds good to me. Make sure to have those message using the log level 
guest debug message. I might even be tempt to use dgprintk(...) here so 
they get dropped in non-debug build.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH 09/49] ARM: VGIC: change to level-IRQ compatible IRQ injection interface

[Julien Grall]

On 12/02/18 11:59, Andre Przywara wrote:

Hi,


Hi Andre,


On 12/02/18 11:15, Julien Grall wrote:

Hi Andre,

On 09/02/18 14:38, Andre Przywara wrote:

diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 5f47aa84a9..2fc6e19625 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -285,7 +285,7 @@ bool vgic_migrate_irq(struct vcpu *old, struct
vcpu *new, unsigned int irq)
   vgic_remove_irq_from_queues(old, p);
   irq_set_affinity(p->desc, cpumask_of(new->processor));
   spin_unlock_irqrestore(>arch.vgic.lock, flags);
-    vgic_vcpu_inject_irq(new, irq);
+    vgic_inject_irq(new->domain, new, irq, true);
   return true;
   }
   /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING
@@ -444,7 +444,7 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir,
enum gic_sgi_mode irqmode,
   sgir, target->list);
   continue;
   }
-    vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
+    vgic_inject_irq(d, d->vcpu[vcpuid], virq, true);
   }
   break;
   case SGI_TARGET_OTHERS:
@@ -453,12 +453,12 @@ bool vgic_to_sgi(struct vcpu *v, register_t
sgir, enum gic_sgi_mode irqmode,
   {
   if ( i != current->vcpu_id && d->vcpu[i] != NULL &&
    is_vcpu_online(d->vcpu[i]) )
-    vgic_vcpu_inject_irq(d->vcpu[i], virq);
+    vgic_inject_irq(d, d->vcpu[i], virq, true);
   }
   break;
   case SGI_TARGET_SELF:
   perfc_incr(vgic_sgi_self);
-    vgic_vcpu_inject_irq(d->vcpu[current->vcpu_id], virq);
+    vgic_inject_irq(d, current, virq, true);
   break;
   default:
   gprintk(XENLOG_WARNING,
@@ -518,13 +518,29 @@ void vgic_remove_irq_from_queues(struct vcpu *v,
struct pending_irq *p)
   gic_remove_from_lr_pending(v, p);
   }
   -void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
+int vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+    bool level)


Looking at the code after the series has been applied, no one is caring
about the return value of vgic_inject_irq. So what is the rationale
behind changing the return type from void to int?


The KVM version returns an error value, in particular when:
- the VGIC has not been initialized yet
- we can't determine the VCPU for a private interrupt
- the interrupt ID is invalid (SPI beyond limit, not mapped LPI)
In the moment it's not very useful for Xen: the first two conditions
don't really happen, consequently I removed those checks. But the third
check may become interesting once we get LPIs. Also since Xen currently
uses a void prototype for injection, *this* patch *now* doesn't exploit
the newly gained possibility of properly handling errors. From briefly
checking all the users, all of them seem to be in void functions, so
indeed an error return is not very useful.
The reasons I kept it in was to allow introduction of checks later. I
think having a function returning an error where some users ignore this
is better than the other way round.


I don't think it is much better. This is a way to expose yet another 
security issue because the return is not correctly checked (see XSA-246 
for instance). Any return value should be checked or have a comment 
explaining why it is fine.




So of course I can easily make this void, but I wonder what we do in
those cases where the SPI is not valid, for instance? Shall we print
some (rate-limited) warning?


I can understand why KVM needs such interface as the interrupt 
controller may be emulated QEMU. But I can't see why a SPI would not be 
valid in Xen context (except programming error). So could give an example?


What would you expect the caller to do on error? Except printing an 
error message?


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH 09/49] ARM: VGIC: change to level-IRQ compatible IRQ injection interface

[Andre Przywara]

Hi,

On 12/02/18 11:15, Julien Grall wrote:
> Hi Andre,
> 
> On 09/02/18 14:38, Andre Przywara wrote:
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index 5f47aa84a9..2fc6e19625 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -285,7 +285,7 @@ bool vgic_migrate_irq(struct vcpu *old, struct
>> vcpu *new, unsigned int irq)
>>   vgic_remove_irq_from_queues(old, p);
>>   irq_set_affinity(p->desc, cpumask_of(new->processor));
>>   spin_unlock_irqrestore(>arch.vgic.lock, flags);
>> -    vgic_vcpu_inject_irq(new, irq);
>> +    vgic_inject_irq(new->domain, new, irq, true);
>>   return true;
>>   }
>>   /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING
>> @@ -444,7 +444,7 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir,
>> enum gic_sgi_mode irqmode,
>>   sgir, target->list);
>>   continue;
>>   }
>> -    vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
>> +    vgic_inject_irq(d, d->vcpu[vcpuid], virq, true);
>>   }
>>   break;
>>   case SGI_TARGET_OTHERS:
>> @@ -453,12 +453,12 @@ bool vgic_to_sgi(struct vcpu *v, register_t
>> sgir, enum gic_sgi_mode irqmode,
>>   {
>>   if ( i != current->vcpu_id && d->vcpu[i] != NULL &&
>>    is_vcpu_online(d->vcpu[i]) )
>> -    vgic_vcpu_inject_irq(d->vcpu[i], virq);
>> +    vgic_inject_irq(d, d->vcpu[i], virq, true);
>>   }
>>   break;
>>   case SGI_TARGET_SELF:
>>   perfc_incr(vgic_sgi_self);
>> -    vgic_vcpu_inject_irq(d->vcpu[current->vcpu_id], virq);
>> +    vgic_inject_irq(d, current, virq, true);
>>   break;
>>   default:
>>   gprintk(XENLOG_WARNING,
>> @@ -518,13 +518,29 @@ void vgic_remove_irq_from_queues(struct vcpu *v,
>> struct pending_irq *p)
>>   gic_remove_from_lr_pending(v, p);
>>   }
>>   -void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
>> +int vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>> +    bool level)
> 
> Looking at the code after the series has been applied, no one is caring
> about the return value of vgic_inject_irq. So what is the rationale
> behind changing the return type from void to int?

The KVM version returns an error value, in particular when:
- the VGIC has not been initialized yet
- we can't determine the VCPU for a private interrupt
- the interrupt ID is invalid (SPI beyond limit, not mapped LPI)
In the moment it's not very useful for Xen: the first two conditions
don't really happen, consequently I removed those checks. But the third
check may become interesting once we get LPIs. Also since Xen currently
uses a void prototype for injection, *this* patch *now* doesn't exploit
the newly gained possibility of properly handling errors. From briefly
checking all the users, all of them seem to be in void functions, so
indeed an error return is not very useful.
The reasons I kept it in was to allow introduction of checks later. I
think having a function returning an error where some users ignore this
is better than the other way round.

So of course I can easily make this void, but I wonder what we do in
those cases where the SPI is not valid, for instance? Shall we print
some (rate-limited) warning?

Cheers,
Andre.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH 09/49] ARM: VGIC: change to level-IRQ compatible IRQ injection interface

[Julien Grall]

Hi Andre,

On 09/02/18 14:38, Andre Przywara wrote:

diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 5f47aa84a9..2fc6e19625 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -285,7 +285,7 @@ bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, 
unsigned int irq)
  vgic_remove_irq_from_queues(old, p);
  irq_set_affinity(p->desc, cpumask_of(new->processor));
  spin_unlock_irqrestore(>arch.vgic.lock, flags);
-vgic_vcpu_inject_irq(new, irq);
+vgic_inject_irq(new->domain, new, irq, true);
  return true;
  }
  /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING
@@ -444,7 +444,7 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum 
gic_sgi_mode irqmode,
  sgir, target->list);
  continue;
  }
-vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
+vgic_inject_irq(d, d->vcpu[vcpuid], virq, true);
  }
  break;
  case SGI_TARGET_OTHERS:
@@ -453,12 +453,12 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum 
gic_sgi_mode irqmode,
  {
  if ( i != current->vcpu_id && d->vcpu[i] != NULL &&
   is_vcpu_online(d->vcpu[i]) )
-vgic_vcpu_inject_irq(d->vcpu[i], virq);
+vgic_inject_irq(d, d->vcpu[i], virq, true);
  }
  break;
  case SGI_TARGET_SELF:
  perfc_incr(vgic_sgi_self);
-vgic_vcpu_inject_irq(d->vcpu[current->vcpu_id], virq);
+vgic_inject_irq(d, current, virq, true);
  break;
  default:
  gprintk(XENLOG_WARNING,
@@ -518,13 +518,29 @@ void vgic_remove_irq_from_queues(struct vcpu *v, struct 
pending_irq *p)
  gic_remove_from_lr_pending(v, p);
  }
  
-void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)

+int vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
+bool level)


Looking at the code after the series has been applied, no one is caring 
about the return value of vgic_inject_irq. So what is the rationale 
behind changing the return type from void to int?


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH 09/49] ARM: VGIC: change to level-IRQ compatible IRQ injection interface

[Andre Przywara]

At the moment vgic_vcpu_inject_irq() is the interface for Xen internal
code and virtual devices to inject IRQs into a guest. This interface has
two shortcomings:
1) It requires a VCPU pointer, which we may not know (and don't need!)
for shared interrupts. A second function (vgic_vcpu_inject_spi()), was
there to work around this issue.
2) This interface only really supports edge triggered IRQs, which is
what the Xen VGIC emulates only anyway. However this needs to and will
change, so we need to add the desired level (high or low) to the
interface.
This replaces the existing injection call (taking a VCPU and and IRQ
parameter) with a new one, taking domain, VCPU, IRQ and level parameters.
The VCPU can be NULL in case we don't know and don't care.
We change all call sites to use this new interface. This still doesn't
give us the missing level IRQ handling, but at least prepares the callers
to do the right thing later automatically.

Signed-off-by: Andre Przywara 
---
 xen/arch/arm/domain.c  |  4 ++--
 xen/arch/arm/gic-v3-lpi.c  |  2 +-
 xen/arch/arm/irq.c |  2 +-
 xen/arch/arm/time.c|  2 +-
 xen/arch/arm/vgic.c| 43 +--
 xen/arch/arm/vpl011.c  |  2 +-
 xen/arch/arm/vtimer.c  |  4 ++--
 xen/include/asm-arm/vgic.h |  4 ++--
 8 files changed, 35 insertions(+), 28 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 9ad4cd0a6e..e76cfdfe83 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -952,14 +952,14 @@ void vcpu_mark_events_pending(struct vcpu *v)
 if ( already_pending )
 return;
 
-vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
+vgic_inject_irq(v->domain, v, v->domain->arch.evtchn_irq, true);
 }
 
 /* The ARM spec declares that even if local irqs are masked in
  * the CPSR register, an irq should wake up a cpu from WFI anyway.
  * For this reason we need to check for irqs that need delivery,
  * ignoring the CPSR register, *after* calling SCHEDOP_block to
- * avoid races with vgic_vcpu_inject_irq.
+ * avoid races with vgic_inject_irq.
  */
 void vcpu_block_unless_event_pending(struct vcpu *v)
 {
diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c
index 84582157b8..efd5cd62fb 100644
--- a/xen/arch/arm/gic-v3-lpi.c
+++ b/xen/arch/arm/gic-v3-lpi.c
@@ -153,7 +153,7 @@ void vgic_vcpu_inject_lpi(struct domain *d, unsigned int 
virq)
 if ( vcpu_id >= d->max_vcpus )
   return;
 
-vgic_vcpu_inject_irq(d->vcpu[vcpu_id], virq);
+vgic_inject_irq(d, d->vcpu[vcpu_id], virq, true);
 }
 
 /*
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 29af10e82c..aa4e832cae 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -225,7 +225,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, 
int is_fiq)
  * The irq cannot be a PPI, we only support delivery of SPIs to
  * guests.
 */
-vgic_vcpu_inject_spi(info->d, info->virq);
+vgic_inject_irq(info->d, NULL, info->virq, true);
 goto out_no_end;
 }
 
diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
index 36f640f0c1..c11fcfeadd 100644
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -260,7 +260,7 @@ static void vtimer_interrupt(int irq, void *dev_id, struct 
cpu_user_regs *regs)
 
 current->arch.virt_timer.ctl = READ_SYSREG32(CNTV_CTL_EL0);
 WRITE_SYSREG32(current->arch.virt_timer.ctl | CNTx_CTL_MASK, CNTV_CTL_EL0);
-vgic_vcpu_inject_irq(current, current->arch.virt_timer.irq);
+vgic_inject_irq(current->domain, current, current->arch.virt_timer.irq, 
true);
 }
 
 /*
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 5f47aa84a9..2fc6e19625 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -285,7 +285,7 @@ bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, 
unsigned int irq)
 vgic_remove_irq_from_queues(old, p);
 irq_set_affinity(p->desc, cpumask_of(new->processor));
 spin_unlock_irqrestore(>arch.vgic.lock, flags);
-vgic_vcpu_inject_irq(new, irq);
+vgic_inject_irq(new->domain, new, irq, true);
 return true;
 }
 /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING
@@ -444,7 +444,7 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum 
gic_sgi_mode irqmode,
 sgir, target->list);
 continue;
 }
-vgic_vcpu_inject_irq(d->vcpu[vcpuid], virq);
+vgic_inject_irq(d, d->vcpu[vcpuid], virq, true);
 }
 break;
 case SGI_TARGET_OTHERS:
@@ -453,12 +453,12 @@ bool vgic_to_sgi(struct vcpu *v, register_t sgir, enum 
gic_sgi_mode irqmode,
 {
 if ( i != current->vcpu_id && d->vcpu[i] != NULL &&
  is_vcpu_online(d->vcpu[i]) )
-vgic_vcpu_inject_irq(d->vcpu[i], virq);
+vgic_inject_irq(d, d->vcpu[i], virq, true);
 }
 break;