Re: Doorbell interrupt from bare-metal inmate to Linux via uoi_ivshmem (ARM64 / ZCU102)

2018-03-08 Thread Jan Kiszka 
On 2018-03-08 16:07, jasperkuijs...@gmail.com wrote:
> On Wednesday, March 7, 2018 at 6:22:31 PM UTC+1, J. Kiszka wrote:
>> On 2018-03-07 15:52, jasperkuijs...@gmail.com wrote:
>>> On Wednesday, March 7, 2018 at 2:41:29 PM UTC+1, jasperk...@gmail.com wrote:
 Hi,

 I'm trying to set up a system with a bare-metal guest/inmate, with shared 
 memory between Linux root cell and inmate, and an (IPI) doorbell interrupt 
 from inmate to Linux.

 My configs are based on zynqmp-zcu102 (root), gic-demo (inmate). Modified 
 to only have only one IVSHMEM device instead of two. 

 Many things are working; gic-demo demo application runs correctly and from 
 Linux I can access the shared memory via an uio device. However I'm stuck 
 at generating the doorbell interrupt from inmate to Linux. 

 # lsuio
 uio0: name=uio_ivshmem, version=0.0.1, events=0
 map[0]: addr=0xFC10, size=4096
 map[1]: addr=0x00080040, size=1048576

 #cat /proc/interrupts
 ...
 229:  0  0  0 GICv2 136 Edge  uio_ivshmem
 ...

 I modified the gic-demo application slightly to generate a doorbell in the 
 timer interrupt handler method. I attempt the generate the doorbell by 
 writing the doorbell register in the ivshmem register region (map0). 
 (https://github.com/henning-schild-work/ivshmem-guest-code/blob/master/device_spec.txt)

 unsigned long* reg = (unsigned long*)0xFC10;
 *(reg + 3) = 1; // doorbell at offset 12byte

 On Linux side the doorbell is seen (events=1) only as soon as the inmate 
 is destroyed, see log below. For the inmate, the gic-demo timer interrupt 
 is only handled once, likely because there is another interrupt pending.

 # ./jailhouse cell start inmate0
 Started cell "inmate0"
 # lsuio
 uio0: name=uio_ivshmem, version=0.0.1, events=0
 map[0]: addr=0xFC10, size=4096
 map[1]: addr=0x00080040, size=1048576
 # ./jailhouse cell shutdown inmate0
 Cell "inmate0" can be loaded
 # lsuio
 uio0: name=uio_ivshmem, version=0.0.1, events=0
 map[0]: addr=0xFC10, size=4096
 map[1]: addr=0x00080040, size=1048576
 # ./jailhouse cell destroy inmate0
 Closing cell "inmate0"
 Page pool usage after cell destruction: mem 43/996, remap 69/131072
 [  222.899818] ivshmem_handler irq: 229
 [  222.920548] Detected VIPT I-cache on CPU3
 [  222.920599] CPU3: Booted secondary processor [410fd034]
 [  222.934074] Destroyed Jailhouse cell "inmate0"
 # lsuio
 uio0: name=uio_ivshmem, version=0.0.1, events=1
 map[0]: addr=0xFC10, size=4096
 map[1]: addr=0x00080040, size=1048576

 Message "[  222.899818] ivshmem_handler irq: 229" originates from a printk 
 statement I added to "irqreturn_t ivshmem_handler(int irq, struct uio_info 
 *dev_info)", which seems to be called only when the cpu is returned to 
 Linux.

 It seems the doorbell is generated for the core I perform the register 
 write on. How do I generate doorbells for other cores? Am I using a 
 correct method to generate doorbells in the first place? Are there other 
 options to do so?

 Kind regards, Jasper
>>>
>>> I want to add that since my original post I have changed flags for the 
>>> IVSHMEM device register region to include JAILHOUSE_MEM_COMM_REGION.
>>
>> Thanks wrong.
>>
>>>
>>> /* IVSHMEM register region for 00:00.0 */ {
>>> .phys_start = 0xfc10,
>>> .virt_start = 0xfc10,
>>> .size = 4096,
>>> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
>>> JAILHOUSE_MEM_COMM_REGION,
>>> },
>>>
>>> The result is that the inmate no longer hangs when attempting to access the 
>>> register space (I discovered that even reading IVPosition resulted in hang).
>>>
>>> However, Linux still does not see the doorbell. I have tried i=0,1,2,3 in 
>>> line below: 
>>>
>>> mmio_write32(reg + (i*8) + 3, 1);
>>>
>>
>> Configuration of these setups is unfortunately a bit tricky.
>>
>> Please check first if you get ivshmem-net communication between root and
>> non-root cell using the original setup. That is known to work. Then
>> stepwise modify settings and test setups to your needs.
>>
>> The (a bit unfortunate) alternative is instrumenting the hypervisor /wrt
>> ivshmem interactions to find out what happens from its POV. The code
>> isn't very complex.
>>
>> Jan
>>
>> -- 
>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
>> Corporate Competence Center Embedded Linux
> 
> Hi Jan,
> 
> I managed to generate doorbells succesfully and they are picked up by the 
> root cell.
> 
> I still use:
> 
> unsigned int* reg = (unsigned int*)0xfc10; //address of /dev/uio0 map[0] 
> in root cell
>

Re: Doorbell interrupt from bare-metal inmate to Linux via uoi_ivshmem (ARM64 / ZCU102)

2018-03-08 Thread jasperkuijsten 
On Wednesday, March 7, 2018 at 6:22:31 PM UTC+1, J. Kiszka wrote:
> On 2018-03-07 15:52, jasperkuijs...@gmail.com wrote:
> > On Wednesday, March 7, 2018 at 2:41:29 PM UTC+1, jasperk...@gmail.com wrote:
> >> Hi,
> >>
> >> I'm trying to set up a system with a bare-metal guest/inmate, with shared 
> >> memory between Linux root cell and inmate, and an (IPI) doorbell interrupt 
> >> from inmate to Linux.
> >>
> >> My configs are based on zynqmp-zcu102 (root), gic-demo (inmate). Modified 
> >> to only have only one IVSHMEM device instead of two. 
> >>
> >> Many things are working; gic-demo demo application runs correctly and from 
> >> Linux I can access the shared memory via an uio device. However I'm stuck 
> >> at generating the doorbell interrupt from inmate to Linux. 
> >>
> >> # lsuio
> >> uio0: name=uio_ivshmem, version=0.0.1, events=0
> >> map[0]: addr=0xFC10, size=4096
> >> map[1]: addr=0x00080040, size=1048576
> >>
> >> #cat /proc/interrupts
> >> ...
> >> 229:  0  0  0 GICv2 136 Edge  uio_ivshmem
> >> ...
> >>
> >> I modified the gic-demo application slightly to generate a doorbell in the 
> >> timer interrupt handler method. I attempt the generate the doorbell by 
> >> writing the doorbell register in the ivshmem register region (map0). 
> >> (https://github.com/henning-schild-work/ivshmem-guest-code/blob/master/device_spec.txt)
> >>
> >> unsigned long* reg = (unsigned long*)0xFC10;
> >> *(reg + 3) = 1; // doorbell at offset 12byte
> >>
> >> On Linux side the doorbell is seen (events=1) only as soon as the inmate 
> >> is destroyed, see log below. For the inmate, the gic-demo timer interrupt 
> >> is only handled once, likely because there is another interrupt pending.
> >>
> >> # ./jailhouse cell start inmate0
> >> Started cell "inmate0"
> >> # lsuio
> >> uio0: name=uio_ivshmem, version=0.0.1, events=0
> >> map[0]: addr=0xFC10, size=4096
> >> map[1]: addr=0x00080040, size=1048576
> >> # ./jailhouse cell shutdown inmate0
> >> Cell "inmate0" can be loaded
> >> # lsuio
> >> uio0: name=uio_ivshmem, version=0.0.1, events=0
> >> map[0]: addr=0xFC10, size=4096
> >> map[1]: addr=0x00080040, size=1048576
> >> # ./jailhouse cell destroy inmate0
> >> Closing cell "inmate0"
> >> Page pool usage after cell destruction: mem 43/996, remap 69/131072
> >> [  222.899818] ivshmem_handler irq: 229
> >> [  222.920548] Detected VIPT I-cache on CPU3
> >> [  222.920599] CPU3: Booted secondary processor [410fd034]
> >> [  222.934074] Destroyed Jailhouse cell "inmate0"
> >> # lsuio
> >> uio0: name=uio_ivshmem, version=0.0.1, events=1
> >> map[0]: addr=0xFC10, size=4096
> >> map[1]: addr=0x00080040, size=1048576
> >>
> >> Message "[  222.899818] ivshmem_handler irq: 229" originates from a printk 
> >> statement I added to "irqreturn_t ivshmem_handler(int irq, struct uio_info 
> >> *dev_info)", which seems to be called only when the cpu is returned to 
> >> Linux.
> >>
> >> It seems the doorbell is generated for the core I perform the register 
> >> write on. How do I generate doorbells for other cores? Am I using a 
> >> correct method to generate doorbells in the first place? Are there other 
> >> options to do so?
> >>
> >> Kind regards, Jasper
> > 
> > I want to add that since my original post I have changed flags for the 
> > IVSHMEM device register region to include JAILHOUSE_MEM_COMM_REGION.
> 
> Thanks wrong.
> 
> > 
> > /* IVSHMEM register region for 00:00.0 */ {
> > .phys_start = 0xfc10,
> > .virt_start = 0xfc10,
> > .size = 4096,
> > .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
> > JAILHOUSE_MEM_COMM_REGION,
> > },
> > 
> > The result is that the inmate no longer hangs when attempting to access the 
> > register space (I discovered that even reading IVPosition resulted in hang).
> > 
> > However, Linux still does not see the doorbell. I have tried i=0,1,2,3 in 
> > line below: 
> > 
> > mmio_write32(reg + (i*8) + 3, 1);
> > 
> 
> Configuration of these setups is unfortunately a bit tricky.
> 
> Please check first if you get ivshmem-net communication between root and
> non-root cell using the original setup. That is known to work. Then
> stepwise modify settings and test setups to your needs.
> 
> The (a bit unfortunate) alternative is instrumenting the hypervisor /wrt
> ivshmem interactions to find out what happens from its POV. The code
> isn't very complex.
> 
> Jan
> 
> -- 
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux

Hi Jan,

I managed to generate doorbells succesfully and they are picked up by the root 
cell.

I still use:

unsigned int* reg = (unsigned int*)0xfc10; //address of /dev/uio0 map[0] in 
root cell
mmio_write32(reg + 3, 1);

However, I completely removed the memory region from the

Re: Doorbell interrupt from bare-metal inmate to Linux via uoi_ivshmem (ARM64 / ZCU102)

2018-03-07 Thread Jan Kiszka 
On 2018-03-07 15:52, jasperkuijs...@gmail.com wrote:
> On Wednesday, March 7, 2018 at 2:41:29 PM UTC+1, jasperk...@gmail.com wrote:
>> Hi,
>>
>> I'm trying to set up a system with a bare-metal guest/inmate, with shared 
>> memory between Linux root cell and inmate, and an (IPI) doorbell interrupt 
>> from inmate to Linux.
>>
>> My configs are based on zynqmp-zcu102 (root), gic-demo (inmate). Modified to 
>> only have only one IVSHMEM device instead of two. 
>>
>> Many things are working; gic-demo demo application runs correctly and from 
>> Linux I can access the shared memory via an uio device. However I'm stuck at 
>> generating the doorbell interrupt from inmate to Linux. 
>>
>> # lsuio
>> uio0: name=uio_ivshmem, version=0.0.1, events=0
>> map[0]: addr=0xFC10, size=4096
>> map[1]: addr=0x00080040, size=1048576
>>
>> #cat /proc/interrupts
>> ...
>> 229:  0  0  0 GICv2 136 Edge  uio_ivshmem
>> ...
>>
>> I modified the gic-demo application slightly to generate a doorbell in the 
>> timer interrupt handler method. I attempt the generate the doorbell by 
>> writing the doorbell register in the ivshmem register region (map0). 
>> (https://github.com/henning-schild-work/ivshmem-guest-code/blob/master/device_spec.txt)
>>
>> unsigned long* reg = (unsigned long*)0xFC10;
>> *(reg + 3) = 1; // doorbell at offset 12byte
>>
>> On Linux side the doorbell is seen (events=1) only as soon as the inmate is 
>> destroyed, see log below. For the inmate, the gic-demo timer interrupt is 
>> only handled once, likely because there is another interrupt pending.
>>
>> # ./jailhouse cell start inmate0
>> Started cell "inmate0"
>> # lsuio
>> uio0: name=uio_ivshmem, version=0.0.1, events=0
>> map[0]: addr=0xFC10, size=4096
>> map[1]: addr=0x00080040, size=1048576
>> # ./jailhouse cell shutdown inmate0
>> Cell "inmate0" can be loaded
>> # lsuio
>> uio0: name=uio_ivshmem, version=0.0.1, events=0
>> map[0]: addr=0xFC10, size=4096
>> map[1]: addr=0x00080040, size=1048576
>> # ./jailhouse cell destroy inmate0
>> Closing cell "inmate0"
>> Page pool usage after cell destruction: mem 43/996, remap 69/131072
>> [  222.899818] ivshmem_handler irq: 229
>> [  222.920548] Detected VIPT I-cache on CPU3
>> [  222.920599] CPU3: Booted secondary processor [410fd034]
>> [  222.934074] Destroyed Jailhouse cell "inmate0"
>> # lsuio
>> uio0: name=uio_ivshmem, version=0.0.1, events=1
>> map[0]: addr=0xFC10, size=4096
>> map[1]: addr=0x00080040, size=1048576
>>
>> Message "[  222.899818] ivshmem_handler irq: 229" originates from a printk 
>> statement I added to "irqreturn_t ivshmem_handler(int irq, struct uio_info 
>> *dev_info)", which seems to be called only when the cpu is returned to Linux.
>>
>> It seems the doorbell is generated for the core I perform the register write 
>> on. How do I generate doorbells for other cores? Am I using a correct method 
>> to generate doorbells in the first place? Are there other options to do so?
>>
>> Kind regards, Jasper
> 
> I want to add that since my original post I have changed flags for the 
> IVSHMEM device register region to include JAILHOUSE_MEM_COMM_REGION.

Thanks wrong.

> 
> /* IVSHMEM register region for 00:00.0 */ {
> .phys_start = 0xfc10,
> .virt_start = 0xfc10,
> .size = 4096,
> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
> JAILHOUSE_MEM_COMM_REGION,
> },
> 
> The result is that the inmate no longer hangs when attempting to access the 
> register space (I discovered that even reading IVPosition resulted in hang).
> 
> However, Linux still does not see the doorbell. I have tried i=0,1,2,3 in 
> line below: 
> 
> mmio_write32(reg + (i*8) + 3, 1);
> 

Configuration of these setups is unfortunately a bit tricky.

Please check first if you get ivshmem-net communication between root and
non-root cell using the original setup. That is known to work. Then
stepwise modify settings and test setups to your needs.

The (a bit unfortunate) alternative is instrumenting the hypervisor /wrt
ivshmem interactions to find out what happens from its POV. The code
isn't very complex.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Doorbell interrupt from bare-metal inmate to Linux via uoi_ivshmem (ARM64 / ZCU102)

2018-03-07 Thread jasperkuijsten 
On Wednesday, March 7, 2018 at 3:52:07 PM UTC+1, jasperk...@gmail.com wrote:
> On Wednesday, March 7, 2018 at 2:41:29 PM UTC+1, jasperk...@gmail.com wrote:
> > Hi,
> > 
> > I'm trying to set up a system with a bare-metal guest/inmate, with shared 
> > memory between Linux root cell and inmate, and an (IPI) doorbell interrupt 
> > from inmate to Linux.
> > 
> > My configs are based on zynqmp-zcu102 (root), gic-demo (inmate). Modified 
> > to only have only one IVSHMEM device instead of two. 
> > 
> > Many things are working; gic-demo demo application runs correctly and from 
> > Linux I can access the shared memory via an uio device. However I'm stuck 
> > at generating the doorbell interrupt from inmate to Linux. 
> > 
> > # lsuio
> > uio0: name=uio_ivshmem, version=0.0.1, events=0
> > map[0]: addr=0xFC10, size=4096
> > map[1]: addr=0x00080040, size=1048576
> > 
> > #cat /proc/interrupts
> > ...
> > 229:  0  0  0 GICv2 136 Edge  uio_ivshmem
> > ...
> > 
> > I modified the gic-demo application slightly to generate a doorbell in the 
> > timer interrupt handler method. I attempt the generate the doorbell by 
> > writing the doorbell register in the ivshmem register region (map0). 
> > (https://github.com/henning-schild-work/ivshmem-guest-code/blob/master/device_spec.txt)
> > 
> > unsigned long* reg = (unsigned long*)0xFC10;
> > *(reg + 3) = 1; // doorbell at offset 12byte
> > 
> > On Linux side the doorbell is seen (events=1) only as soon as the inmate is 
> > destroyed, see log below. For the inmate, the gic-demo timer interrupt is 
> > only handled once, likely because there is another interrupt pending.
> > 
> > # ./jailhouse cell start inmate0
> > Started cell "inmate0"
> > # lsuio
> > uio0: name=uio_ivshmem, version=0.0.1, events=0
> > map[0]: addr=0xFC10, size=4096
> > map[1]: addr=0x00080040, size=1048576
> > # ./jailhouse cell shutdown inmate0
> > Cell "inmate0" can be loaded
> > # lsuio
> > uio0: name=uio_ivshmem, version=0.0.1, events=0
> > map[0]: addr=0xFC10, size=4096
> > map[1]: addr=0x00080040, size=1048576
> > # ./jailhouse cell destroy inmate0
> > Closing cell "inmate0"
> > Page pool usage after cell destruction: mem 43/996, remap 69/131072
> > [  222.899818] ivshmem_handler irq: 229
> > [  222.920548] Detected VIPT I-cache on CPU3
> > [  222.920599] CPU3: Booted secondary processor [410fd034]
> > [  222.934074] Destroyed Jailhouse cell "inmate0"
> > # lsuio
> > uio0: name=uio_ivshmem, version=0.0.1, events=1
> > map[0]: addr=0xFC10, size=4096
> > map[1]: addr=0x00080040, size=1048576
> > 
> > Message "[  222.899818] ivshmem_handler irq: 229" originates from a printk 
> > statement I added to "irqreturn_t ivshmem_handler(int irq, struct uio_info 
> > *dev_info)", which seems to be called only when the cpu is returned to 
> > Linux.
> > 
> > It seems the doorbell is generated for the core I perform the register 
> > write on. How do I generate doorbells for other cores? Am I using a correct 
> > method to generate doorbells in the first place? Are there other options to 
> > do so?
> > 
> > Kind regards, Jasper
> 
> I want to add that since my original post I have changed flags for the 
> IVSHMEM device register region to include JAILHOUSE_MEM_COMM_REGION.
> 
> /* IVSHMEM register region for 00:00.0 */ {
> .phys_start = 0xfc10,
> .virt_start = 0xfc10,
> .size = 4096,
> .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
> JAILHOUSE_MEM_COMM_REGION,
> },
> 
> The result is that the inmate no longer hangs when attempting to access the 
> register space (I discovered that even reading IVPosition resulted in hang).
> 
> However, Linux still does not see the doorbell. I have tried i=0,1,2,3 in 
> line below: 
> 
> mmio_write32(reg + (i*8) + 3, 1);

Some additional comments..

Fixed my mistake in inmate0.c: "unsigned int" instead of "unsigned long" to 
access correct offsets via pointer aritmetics.. sorry.

But that still does not give working doorbells.

Following: 
https://github.com/siemens/jailhouse/blob/master/Documentation/inter-cell-communication.txt
 I also attempt to mmio_write the LSTATE register but no success.

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Doorbell interrupt from bare-metal inmate to Linux via uoi_ivshmem (ARM64 / ZCU102)

2018-03-07 Thread jasperkuijsten 
On Wednesday, March 7, 2018 at 2:41:29 PM UTC+1, jasperk...@gmail.com wrote:
> Hi,
> 
> I'm trying to set up a system with a bare-metal guest/inmate, with shared 
> memory between Linux root cell and inmate, and an (IPI) doorbell interrupt 
> from inmate to Linux.
> 
> My configs are based on zynqmp-zcu102 (root), gic-demo (inmate). Modified to 
> only have only one IVSHMEM device instead of two. 
> 
> Many things are working; gic-demo demo application runs correctly and from 
> Linux I can access the shared memory via an uio device. However I'm stuck at 
> generating the doorbell interrupt from inmate to Linux. 
> 
> # lsuio
> uio0: name=uio_ivshmem, version=0.0.1, events=0
> map[0]: addr=0xFC10, size=4096
> map[1]: addr=0x00080040, size=1048576
> 
> #cat /proc/interrupts
> ...
> 229:  0  0  0 GICv2 136 Edge  uio_ivshmem
> ...
> 
> I modified the gic-demo application slightly to generate a doorbell in the 
> timer interrupt handler method. I attempt the generate the doorbell by 
> writing the doorbell register in the ivshmem register region (map0). 
> (https://github.com/henning-schild-work/ivshmem-guest-code/blob/master/device_spec.txt)
> 
> unsigned long* reg = (unsigned long*)0xFC10;
> *(reg + 3) = 1; // doorbell at offset 12byte
> 
> On Linux side the doorbell is seen (events=1) only as soon as the inmate is 
> destroyed, see log below. For the inmate, the gic-demo timer interrupt is 
> only handled once, likely because there is another interrupt pending.
> 
> # ./jailhouse cell start inmate0
> Started cell "inmate0"
> # lsuio
> uio0: name=uio_ivshmem, version=0.0.1, events=0
> map[0]: addr=0xFC10, size=4096
> map[1]: addr=0x00080040, size=1048576
> # ./jailhouse cell shutdown inmate0
> Cell "inmate0" can be loaded
> # lsuio
> uio0: name=uio_ivshmem, version=0.0.1, events=0
> map[0]: addr=0xFC10, size=4096
> map[1]: addr=0x00080040, size=1048576
> # ./jailhouse cell destroy inmate0
> Closing cell "inmate0"
> Page pool usage after cell destruction: mem 43/996, remap 69/131072
> [  222.899818] ivshmem_handler irq: 229
> [  222.920548] Detected VIPT I-cache on CPU3
> [  222.920599] CPU3: Booted secondary processor [410fd034]
> [  222.934074] Destroyed Jailhouse cell "inmate0"
> # lsuio
> uio0: name=uio_ivshmem, version=0.0.1, events=1
> map[0]: addr=0xFC10, size=4096
> map[1]: addr=0x00080040, size=1048576
> 
> Message "[  222.899818] ivshmem_handler irq: 229" originates from a printk 
> statement I added to "irqreturn_t ivshmem_handler(int irq, struct uio_info 
> *dev_info)", which seems to be called only when the cpu is returned to Linux.
> 
> It seems the doorbell is generated for the core I perform the register write 
> on. How do I generate doorbells for other cores? Am I using a correct method 
> to generate doorbells in the first place? Are there other options to do so?
> 
> Kind regards, Jasper

I want to add that since my original post I have changed flags for the IVSHMEM 
device register region to include JAILHOUSE_MEM_COMM_REGION.

/* IVSHMEM register region for 00:00.0 */ {
.phys_start = 0xfc10,
.virt_start = 0xfc10,
.size = 4096,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_COMM_REGION,
},

The result is that the inmate no longer hangs when attempting to access the 
register space (I discovered that even reading IVPosition resulted in hang).

However, Linux still does not see the doorbell. I have tried i=0,1,2,3 in line 
below: 

mmio_write32(reg + (i*8) + 3, 1);

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Doorbell interrupt from bare-metal inmate to Linux via uoi_ivshmem (ARM64 / ZCU102)

2018-03-07 Thread jasperkuijsten 
Hi,

I'm trying to set up a system with a bare-metal guest/inmate, with shared 
memory between Linux root cell and inmate, and an (IPI) doorbell interrupt from 
inmate to Linux.

My configs are based on zynqmp-zcu102 (root), gic-demo (inmate). Modified to 
only have only one IVSHMEM device instead of two. 

Many things are working; gic-demo demo application runs correctly and from 
Linux I can access the shared memory via an uio device. However I'm stuck at 
generating the doorbell interrupt from inmate to Linux. 

# lsuio
uio0: name=uio_ivshmem, version=0.0.1, events=0
map[0]: addr=0xFC10, size=4096
map[1]: addr=0x00080040, size=1048576

#cat /proc/interrupts
...
229:  0  0  0 GICv2 136 Edge  uio_ivshmem
...

I modified the gic-demo application slightly to generate a doorbell in the 
timer interrupt handler method. I attempt the generate the doorbell by writing 
the doorbell register in the ivshmem register region (map0). 
(https://github.com/henning-schild-work/ivshmem-guest-code/blob/master/device_spec.txt)

unsigned long* reg = (unsigned long*)0xFC10;
*(reg + 3) = 1; // doorbell at offset 12byte

On Linux side the doorbell is seen (events=1) only as soon as the inmate is 
destroyed, see log below. For the inmate, the gic-demo timer interrupt is only 
handled once, likely because there is another interrupt pending.

# ./jailhouse cell start inmate0
Started cell "inmate0"
# lsuio
uio0: name=uio_ivshmem, version=0.0.1, events=0
map[0]: addr=0xFC10, size=4096
map[1]: addr=0x00080040, size=1048576
# ./jailhouse cell shutdown inmate0
Cell "inmate0" can be loaded
# lsuio
uio0: name=uio_ivshmem, version=0.0.1, events=0
map[0]: addr=0xFC10, size=4096
map[1]: addr=0x00080040, size=1048576
# ./jailhouse cell destroy inmate0
Closing cell "inmate0"
Page pool usage after cell destruction: mem 43/996, remap 69/131072
[  222.899818] ivshmem_handler irq: 229
[  222.920548] Detected VIPT I-cache on CPU3
[  222.920599] CPU3: Booted secondary processor [410fd034]
[  222.934074] Destroyed Jailhouse cell "inmate0"
# lsuio
uio0: name=uio_ivshmem, version=0.0.1, events=1
map[0]: addr=0xFC10, size=4096
map[1]: addr=0x00080040, size=1048576

Message "[  222.899818] ivshmem_handler irq: 229" originates from a printk 
statement I added to "irqreturn_t ivshmem_handler(int irq, struct uio_info 
*dev_info)", which seems to be called only when the cpu is returned to Linux.

It seems the doorbell is generated for the core I perform the register write 
on. How do I generate doorbells for other cores? Am I using a correct method to 
generate doorbells in the first place? Are there other options to do so?

Kind regards, Jasper

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
/*
 * Jailhouse, a Linux-based partitioning hypervisor
 *
 * Configuration for gic-demo inmate on Xilinx ZynqMP ZCU102 eval board:
 * 1 CPU, 64K RAM, 1 serial port
 *
 * Copyright (c) Siemens AG, 2016
 *
 * Authors:
 *  Jan Kiszka 
 *
 * This work is licensed under the terms of the GNU GPL, version 2.  See
 * the COPYING file in the top-level directory.
 */

#include 
#include 

#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])

struct {
	struct jailhouse_cell_desc cell;
	__u64 cpus[1];
	struct jailhouse_memory mem_regions[4];
struct jailhouse_pci_device pci_devices[1];
} __attribute__((packed)) config = {
	.cell = {
		.signature = JAILHOUSE_CELL_DESC_SIGNATURE,
		.revision = JAILHOUSE_CONFIG_REVISION,
		.name = "inmate0",
		.flags = JAILHOUSE_CELL_PASSIVE_COMMREG,

		.cpu_set_size = sizeof(config.cpus),
		.num_memory_regions = ARRAY_SIZE(config.mem_regions),
		.num_irqchips = 0,
		.pio_bitmap_size = 0,
		.num_pci_devices = ARRAY_SIZE(config.pci_devices),
	},

	.cpus = {
		0x8,
	},

	.mem_regions = {
		/* UART */ {
			.phys_start = 0xff01,
			.virt_start = 0xff01,
			.size = 0x1000,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* RAM */ {
			.phys_start = 0x80060,
			.virt_start = 0,
			.size = 0x0001,
			.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_LOADABLE,
		},
		/* IVSHMEM shared memory region for 00:00.0 */ {
		.phys_start = 0x80040,
		.virt_start = 0x80040,
		.size = 0x10,
		.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED,
		},
		/* IVSHMEM register region for 00:00.0 */ {
		.phys_start = 0x0fc10,
		.virt_start = 0x0fc10,
		.size = 4096,
		.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED,