Hi,
On 04/05/2017 03:46 PM, Mark Rutland wrote:
> On Wed, Apr 05, 2017 at 12:38:05PM +0200, Ralf Ramsauer wrote:
>> Hi,
>
> Hi,
>
>> I need spinlocks in inmates on ARM, so I simply included asm/spinlock.h.
>> Taking locks in inmates on ARM somehow doesn't work at all, but that's
>> not the main issue. As soon as I try to take locks, cells can not be
>> destroyed any longer, the whole system freezes on cell destroy.
>>
>> I was able to trace this down to the 'ldrex' instruction. This minimal
>> example triggers this behavior (tested on a TK1, can not test it on
>> other boards):
>>
>> #include <inmate.h>
>>
>> void inmate_main(void)
>> {
>> unsigned int foo = 0, bar = 0;
>>
>> printk("Foo!\n");
>> asm volatile("ldrex %0, [%1]\n\t"
>> : "=&r" (foo) : "r" (bar));
>
> That looks busted. The second parameter is the value of bar, not the
> address of bar, so we'll dereference NULL. Either that should be &bar,
> or a "Q" constraint should be used ("+Q" if you had a strex).
>
> What's the memory map of the inmate? Could that result in a
> (destructive) read from device memory? Or some asynchronous abort?
Inmate memory starts at guest physical zero, so that should be no
problem. This assembly snippet is not meant to do anything useful, I
just tried to trace down why spinlocks aren't working and this is the
instruction that seems to (reproducibly) cause the misbehavior.
I tried the same code on a Orange Pi zero together with Jailhouse and
experienced different behavior: It only prints "Foo!" and stops then.
And I'm able to destroy the cell as expected.
>
> Is the MMU on? Architecturally, ldrex/strex are only guaranteed to
>From the guest's point of view, the MMU is turned off.
> function on write-back cacheable memory, and I can't recall the
> permitted UNPREDICTABLE behaviours otherwise.
>
>>
>> printk("Bar!\n");
>> while(1);
>> asm volatile("wfi");
>> }
>>
>>
>> The example code successfully finishes, I can read Foo and Bar on my
>> serial console. But I am not able to destroy the cell afterwards. My
>> system completely hangs on "jailhouse cell destroy".
>
> What happens if you place a full-system DSB between the printks and hte
> ldrex, i.e.
>
> asm volatible ("dsb sy\n" ::: "memory");
>
> ... do you still see the "Bar!" print?
Yep, I do, on the TK1.
On the Orange Pi 0, behavior does not change: I only see "Foo!" in
either case.
Ralf
>
>>
>> Curiously the cell can be re-loaded and started over again. Leaving out
>> the ldrex instruction does not trigger the bug.
>>
>> - How can ldrex prevent my cell from being destroyed?
>>
>> Does it change some 'internal state' where jailhouse is not aware
>> about?
>>
>> - Why do spinlocks not work at all in inmates?
>>
>> spin_lock(&lock) never returns in inmates on a fresh spin lock.
>> Does the ARM implementation of spinlocks require some special
>> global initialisation that I don't know of?
>
> The only thing I can think of is that the location being operated upon
> should be in write-back cacheable memory (with shareability common to
> all agents that may access the region).
>
> Thanks,
> Mark.
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
