Re: [Xen-devel] [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

[Oleksandr Andrushchenko]

On 06/14/2018 01:19 AM, Boris Ostrovsky wrote:

On 06/13/2018 07:57 AM, Oleksandr Andrushchenko wrote:

On 06/13/2018 05:58 AM, Boris Ostrovsky wrote:


On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

+
+static struct gntdev_dmabuf *
+dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)


The name of this routine implies (to me) that we are getting a wait
object but IIUIC we are getting a gntdev_dmabuf that we are going to
later associate with a wait object.


How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_
just to show to which functionality a routine belongs.


That's too long IMO. If you really want to keep the prefix then let's
keep this the way it is. Maybe it's just me who read it that way.

I'll rename it to dmabuf_exp_wait_obj_get_dmabuf.
As it already wants fd it seems to be clear that
the lookup will be based on it.



   +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct
gntdev_grant_map *map)
+{
+    mutex_lock(>lock);
+    list_del(>next);
+    gntdev_put_map(NULL /* already removed */, map);


Why not pass call gntdev_put_map(priv, map) and then not have this
routine at all?


Well, I wish I could, but the main difference when calling
gntdev_put_map(priv, map)
with priv != NULL and my code is that:

void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map
*map)
{
     [...]
     if (populate_freeable_maps && priv) {
     ^
         mutex_lock(>lock);
         list_del(>next);
         mutex_unlock(>lock);
     }
     [...]
}

and

#define populate_freeable_maps use_ptemod
                            ^^
which means the map will never be removed from the list in my case
because use_ptemod == false for dma-buf.
This is why I do that by hand, e.g. remove the item from the list
and then pass NULL for priv.

Also, I will remove gntdev_remove_map as I can now access
priv->lock and gntdev_put_map directly form gntdev-dmabuf.c


Yes, that's a good idea.


I really dislike the fact that we are taking a lock here that
gntdev_put_map() takes as well, although not with NULL argument. (And
yes, I see that gntdev_release() does it too.)


This can be re-factored later I guess?

OK.

-boris




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.8-testing test] 124158: regressions - trouble: broken/fail/pass

[osstest service owner]

flight 124158 xen-4.8-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124158/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-credit2  broken
 build-armhf-xsm  broken  in 124070
 build-i386-pvops  6 kernel-build   fail in 123844 REGR. vs. 123091
 build-armhf   6 xen-build  fail in 123844 REGR. vs. 123091
 build-armhf-xsm  5 host-build-prep fail in 124070 REGR. vs. 123091

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit2   2 hosts-allocate   broken pass in 124100
 test-amd64-amd64-xl-credit2   7 xen-boot fail in 123701 pass in 124158
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 xen-boot fail in 123701 pass 
in 124158
 test-amd64-amd64-livepatch7 xen-boot fail in 123701 pass in 124158
 test-amd64-amd64-pair   10 xen-boot/src_host fail in 123701 pass in 124158
 test-amd64-amd64-pair   11 xen-boot/dst_host fail in 123701 pass in 124158
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-boot fail in 123701 pass in 124158
 test-amd64-i386-rumprun-i386  7 xen-boot fail in 123701 pass in 124158
 test-amd64-i386-xl-qemuu-debianhvm-amd64 7 xen-boot fail in 123701 pass in 
124158
 test-amd64-i386-libvirt-xsm   7 xen-boot fail in 123701 pass in 124158
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-boot fail in 123701 pass in 124158
 test-amd64-i386-migrupgrade 10 xen-boot/src_host fail in 123701 pass in 124158
 test-amd64-i386-migrupgrade 11 xen-boot/dst_host fail in 123701 pass in 124158
 test-amd64-amd64-xl-multivcpu  7 xen-bootfail in 123701 pass in 124158
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 xen-boot fail in 123701 pass in 124158
 test-xtf-amd64-amd64-37 xen-boot fail in 123844 pass in 124158
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 13 guest-saverestore fail in 
124100 pass in 124158
 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 
123701
 test-amd64-amd64-xl-rtds 10 debian-install fail pass in 123844
 test-amd64-i386-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 
124070
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-saverestore.2 
fail pass in 124100

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked in 123844 n/a
 build-armhf-libvirt   1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-livepatch 1 build-check(1)   blocked in 123844 n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)  blocked in 123844 n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-migrupgrade   1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1) blocked in 123844 n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1) blocked in 123844 n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 1 build-check(1) blocked in 
123844 n/a
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked in 123844 n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-xl1 build-check(1)   blocked in 123844 n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 
123844 n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 
123844 n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
in 123844 n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked in 123844 n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)blocked in 123844 n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1) 

[Xen-devel] Is there a faster way to restore Virtual machine status in Xen?

[Chenjia (C)]

Dear XEN expert：
We meet some problem in our project: In our previous project, we 
use KVM, and we do some job like this:

We create KVM snapshot by “virsh snapshot-create $DomainName $SnapshotXml”, 
then do following job:
While(1)
{
Run “virsh snapshot-revert $DomainName  $Snapshot --running --force”
Do some job in 30 secends
}


Now our project is move to Xen, so we need to do same thing like in 
KVM, but we found that there is no “Snapshot” in xen , so we change the job 
like this in xen:
While(1)
{
Run “xl destroy win7_checkpointFile”
Run “xl restore  win7_checkpointFile”
Do some job in 30 secends
}

We found that” xl destroy “and “xl restore” spend 10 times longer 
than “virsh snapshot-revert”,  it is unacceptable in our project

So our question is that: Is there a faster way to restore Virtual 
machine status in Xen?



Our environment:
CPU: intel xeon E52620 2.40GHz with 24 core,
memory : 256G,
SSD hard disk.
Host machine: suse12
xen_version: 4.8.2
The VM is Win7 OS with 2G memory

Best regards

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 124162: regressions - FAIL

[osstest service owner]

flight 124162 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124162/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 124058

version targeted for testing:
 ovmf c30084fbac289b731d0bd102b0e91072c63ea029
baseline version:
 ovmf a05a8a5aa17da4bc7144706a9931d68beec1a61f

Last test of basis   124058  2018-06-11 03:10:30 Z2 days
Failing since124074  2018-06-11 16:41:40 Z2 days4 attempts
Testing same since   124162  2018-06-13 07:23:33 Z0 days1 attempts


People who touched revisions under test:
  Ard Biesheuvel 
  Benjamin You 
  Dandan Bi 
  Derek Lin 
  Dongao Guo 
  Hao Wu 
  Jaben Carsey 
  Laszlo Ersek 
  Liming Gao 
  Michael D Kinney 
  Michael Zimmermann 
  Yonghong Zhu 
  Yunhua Feng 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 441 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 124182: tolerable all pass - PUSHED

[osstest service owner]

flight 124182 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124182/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  243435bf67e8159495194f623b9e4d8c90140384
baseline version:
 xen  41339ecb5f18ca7ec7b0c914c952a0e1715ae511

Last test of basis   124108  2018-06-12 12:01:06 Z1 days
Testing same since   124182  2018-06-13 22:00:46 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   41339ecb5f..243435bf67  243435bf67e8159495194f623b9e4d8c90140384 -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH 06/12] xen-blkfront: add callbacks for PM suspend and hibernation

[Anchal Agarwal]

Hi Roger,
To answer your question, due to the lack of mentioned commit
(commit 12ea729645ac ("xen/blkback: unmap all persistent grants when
frontend gets disconnected") in the older dom0 kernels(<3.2),resume from
hibernation can fail on guest side. In the absence of the commit,
Persistant Grants are not unmapped immediately when frontend is 
disconnected from backend and hence leave the block device in an 
inconsistent state. To avoid this unstability and work with larger set 
of kernel versions, this approach had been used. Once you don't have 
any pending req/resp it is safer for guest to resume from hibernation.

Thanks,
Anchal

On Wed, Jun 13, 2018 at 10:24:28AM +0200, Roger Pau Monn?? wrote:
> On Tue, Jun 12, 2018 at 08:56:13PM +, Anchal Agarwal wrote:
> > From: Munehisa Kamata 
> > 
> > Add freeze and restore callbacks for PM suspend and hibernation support.
> > The freeze handler stops a block-layer queue and disconnect the frontend
> > from the backend while freeing ring_info and associated resources. The
> > restore handler re-allocates ring_info and re-connect to the backedend,
> > so the rest of the kernel can continue to use the block device
> > transparently.Also, the handlers are used for both PM
> > suspend and hibernation so that we can keep the existing suspend/resume
> > callbacks for Xen suspend without modification.
> > If a backend doesn't have commit 12ea729645ac ("xen/blkback: unmap all
> > persistent grants when frontend gets disconnected"), the frontend may see
> > massive amount of grant table warning when freeing resources.
> > 
> >  [   36.852659] deferring g.e. 0xf9 (pfn 0x)
> >  [   36.855089] xen:grant_table: WARNING: g.e. 0x112 still in use!
> > 
> > In this case, persistent grants would need to be disabled.
> > 
> > Ensure no reqs/rsps in rings before disconnecting. When disconnecting
> > the frontend from the backend in blkfront_freeze(), there still may be
> > unconsumed requests or responses in the rings, especially when the
> > backend is backed by network-based device. If the frontend gets
> > disconnected with such reqs/rsps remaining there, it can cause
> > grant warnings and/or losing reqs/rsps by freeing pages afterward.
> 
> I'm not sure why having pending requests can cause grant warnings or
> lose of requests. If handled properly this shouldn't be an issue.
> Linux blkfront already does live migration (which also involves a
> reconnection of the frontend) with pending requests and that doesn't
> seem to be an issue.
> 
> > This can lead resumed kernel into unrecoverable state like unexpected
> > freeing of grant page and/or hung task due to the lost reqs or rsps.
> > Therefore we have to ensure that there is no unconsumed requests or
> > responses before disconnecting.
> 
> Given that we have multiqueue, plus multipage rings, I'm not sure
> waiting for the requests on the rings to complete is a good idea.
> 
> Why can't you just disconnect the frontend and requeue all the
> requests in flight? When the frontend connects on resume those
> requests will be queued again.
> 
> Thanks, Roger.
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

[Boris Ostrovsky]

On 06/13/2018 07:57 AM, Oleksandr Andrushchenko wrote:
> On 06/13/2018 05:58 AM, Boris Ostrovsky wrote:
>>
>>
>> On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:
>>>
>>> +
>>> +static struct gntdev_dmabuf *
>>> +dmabuf_exp_wait_obj_get_by_fd(struct gntdev_dmabuf_priv *priv, int fd)
>>
>>
>> The name of this routine implies (to me) that we are getting a wait
>> object but IIUIC we are getting a gntdev_dmabuf that we are going to
>> later associate with a wait object.
>>
> How about dmabuf_exp_wait_obj_get_dmabuf_by_fd?
> I would like to keep function prefixes, e.g. dmabuf_exp_wait_obj_
> just to show to which functionality a routine belongs.


That's too long IMO. If you really want to keep the prefix then let's
keep this the way it is. Maybe it's just me who read it that way.


>>
>>>
>>>   +#ifdef CONFIG_XEN_GNTDEV_DMABUF
>>> +void gntdev_remove_map(struct gntdev_priv *priv, struct
>>> gntdev_grant_map *map)
>>> +{
>>> +    mutex_lock(>lock);
>>> +    list_del(>next);
>>> +    gntdev_put_map(NULL /* already removed */, map);
>>
>>
>> Why not pass call gntdev_put_map(priv, map) and then not have this
>> routine at all?
>>
> Well, I wish I could, but the main difference when calling
> gntdev_put_map(priv, map)
> with priv != NULL and my code is that:
>
> void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map
> *map)
> {
>     [...]
>     if (populate_freeable_maps && priv) {
>     ^
>         mutex_lock(>lock);
>         list_del(>next);
>         mutex_unlock(>lock);
>     }
>     [...]
> }
>
> and
>
> #define populate_freeable_maps use_ptemod
>                            ^^
> which means the map will never be removed from the list in my case
> because use_ptemod == false for dma-buf.
> This is why I do that by hand, e.g. remove the item from the list
> and then pass NULL for priv.
>
> Also, I will remove gntdev_remove_map as I can now access
> priv->lock and gntdev_put_map directly form gntdev-dmabuf.c


Yes, that's a good idea.

>
>> I really dislike the fact that we are taking a lock here that
>> gntdev_put_map() takes as well, although not with NULL argument. (And
>> yes, I see that gntdev_release() does it too.)
>>
> This can be re-factored later I guess?

OK.

-boris


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 07/15] xen/arm: increase MAX_MODULES

[Stefano Stabellini]

Xen boot modules need to account not just for Dom0 but also for a few
potential DomUs, each of them coming with their own kernel and initrd.
Increase MAX_MODULES to 32 to allow for more DomUs.

Signed-off-by: Stefano Stabellini 
---
 xen/include/asm-arm/setup.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index e190451..86aac0e 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -8,7 +8,7 @@
 
 #define NR_MEM_BANKS 128
 
-#define MAX_MODULES 5 /* Current maximum useful modules */
+#define MAX_MODULES 32 /* Current maximum useful modules */
 
 typedef enum {
 BOOTMOD_XEN,
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 06/15] xen/arm: add BOOTMOD_DOMU_KERNEL/RAMDISK

[Stefano Stabellini]

Introduce bootmod types for domU kernels and initrds.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/bootfdt.c  | 4 
 xen/arch/arm/setup.c| 2 ++
 xen/include/asm-arm/setup.h | 2 ++
 3 files changed, 8 insertions(+)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 8eba42c..6c88ec4 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -194,6 +194,10 @@ static void __init process_multiboot_node(const void *fdt, 
int node,
 kind = BOOTMOD_RAMDISK;
 else if ( fdt_node_check_compatible(fdt, node, "xen,xsm-policy") == 0 )
 kind = BOOTMOD_XSM;
+else if ( fdt_node_check_compatible(fdt, node, "multiboot,domU-kernel") == 
0 )
+kind = BOOTMOD_DOMU_KERNEL;
+else if ( fdt_node_check_compatible(fdt, node, "multiboot,domU-ramdisk") 
== 0 )
+kind = BOOTMOD_DOMU_RAMDISK;
 else
 kind = BOOTMOD_UNKNOWN;
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 1d6f6bf..82593c8 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -249,6 +249,8 @@ const char * __init 
boot_module_kind_as_string(bootmodule_kind kind)
 case BOOTMOD_FDT: return "Device Tree";
 case BOOTMOD_KERNEL:  return "Kernel";
 case BOOTMOD_RAMDISK: return "Ramdisk";
+case BOOTMOD_DOMU_KERNEL:  return "DomU Kernel";
+case BOOTMOD_DOMU_RAMDISK: return "DomU Ramdisk";
 case BOOTMOD_XSM: return "XSM";
 case BOOTMOD_UNKNOWN: return "Unknown";
 default: BUG();
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 0cc3330..e190451 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -16,6 +16,8 @@ typedef enum {
 BOOTMOD_KERNEL,
 BOOTMOD_RAMDISK,
 BOOTMOD_XSM,
+BOOTMOD_DOMU_KERNEL,
+BOOTMOD_DOMU_RAMDISK,
 BOOTMOD_UNKNOWN
 }  bootmodule_kind;
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] 4.11.0 RC1 panic

[Manuel Bouyer]

On Wed, Jun 13, 2018 at 03:59:19AM -0600, Jan Beulich wrote:
> >>> On 13.06.18 at 10:57,  wrote:
> > On Wed, Jun 13, 2018 at 02:07:29AM -0600, Jan Beulich wrote:
> >> 
> >> (XEN) Assertion '!page->linear_pt_count' failed at mm.c:596
> >> 
> >> In fact, there's no assertion with that expression anywhere I could
> >> see. Do you have any local patches in place?
> > 
> > Yes, 2 of them from you (the first one is where the assert is). See 
> > attached.
> 
> Oh, I had long dropped that first one, after you had said that it didn't
> trigger in a long time. It triggering with the other debugging patch is
> not unexpected. So please drop that patch at least for the time being.

So far I've not been able to make Xen panic with the new xen kernel.
Attached is a log of the serial console, in case you notice something.

I'll keep anita tests running in a loop overnight, in case it ends up
hitting an assert.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


xen_console.txt.gz
Description: application/gunzip
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 15/15] xen: support console_switching between Dom0 and DomUs on ARM

[Stefano Stabellini]

Today Ctrl-AAA is used to switch between Xen and Dom0. Extend the
mechanism to allow for switching between Xen, Dom0, and any of the
initial DomU created from Xen alongside Dom0 out of information provided
via device tree.

Signed-off-by: Stefano Stabellini 
CC: andrew.coop...@citrix.com
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: konrad.w...@oracle.com
CC: t...@xen.org
CC: wei.l...@citrix.com
---
 xen/drivers/char/console.c | 49 --
 1 file changed, 34 insertions(+), 15 deletions(-)

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index dc9e0bb..07a89c6 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -31,10 +31,13 @@
 #include 
 #include 
 #include 
+#include 
 
 #ifdef CONFIG_X86
 #include 
 #include 
+#else
+#include 
 #endif
 
 /* console: comma-separated list of console outputs. */
@@ -389,29 +392,48 @@ static void dump_console_ring_key(unsigned char key)
 free_xenheap_pages(buf, order);
 }
 
-/* CTRL- switches input direction between Xen and DOM0. */
+/*
+ * CTRL- switches input direction between Xen, Dom0 and
+ * DomUs.
+ */
 #define switch_code (opt_conswitch[0]-'a'+1)
-static int __read_mostly xen_rx = 1; /* FALSE => input passed to domain 0. */
+static int __read_mostly xen_rx = 1; /* 1 => input passed to domain 0. */
 
 static void switch_serial_input(void)
 {
-static char *input_str[2] = { "DOM0", "Xen" };
-xen_rx = !xen_rx;
-printk("*** Serial input -> %s", input_str[xen_rx]);
+xen_rx++;
+if ( xen_rx == max_init_domid + 1 )
+xen_rx = 0;
+
+if ( !xen_rx )
+printk("*** Serial input xen_rx=%d -> %s", xen_rx, "Xen");
+else
+printk("*** Serial input xen_rx=%d -> DOM%d", xen_rx, xen_rx - 1);
+
 if ( switch_code )
-printk(" (type 'CTRL-%c' three times to switch input to %s)",
-   opt_conswitch[0], input_str[!xen_rx]);
+printk(" (type 'CTRL-%c' three times to switch input)",
+   opt_conswitch[0]);
 printk("\n");
 }
 
 static void __serial_rx(char c, struct cpu_user_regs *regs)
 {
-if ( xen_rx )
+if ( xen_rx == 0 )
 return handle_keypress(c, regs);
 
-/* Deliver input to guest buffer, unless it is already full. */
-if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
-serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
+if ( xen_rx == 1 )
+{
+/* Deliver input to guest buffer, unless it is already full. */
+if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
+serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
+}
+#ifdef CONFIG_ARM
+else
+{
+struct domain *d = get_domain_by_id(xen_rx - 1);
+vpl011_read_char(d, c);
+}
+#endif
 /* Always notify the guest: prevents receive path from getting stuck. */
 send_global_virq(VIRQ_CONSOLE);
 
@@ -925,7 +947,7 @@ void __init console_endboot(void)
  * a useful 'how to switch' message.
  */
 if ( opt_conswitch[1] == 'x' )
-xen_rx = !xen_rx;
+xen_rx = 0;
 
 register_keyhandler('w', dump_console_ring_key,
 "synchronously dump console ring buffer (dmesg)", 0);
@@ -935,9 +957,6 @@ void __init console_endboot(void)
 "decrease log level threshold", 0);
 register_irq_keyhandler('G', _toggle_guest,
 "toggle host/guest log level adjustment", 0);
-
-/* Serial input is directed to DOM0 by default. */
-switch_serial_input();
 }
 
 int __init console_has(const char *device)
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 12/15] xen/arm: generate vpl011 node on device tree for domU

[Stefano Stabellini]

Introduce vpl011 support to guests started from Xen: it provides a
simple way to print output from a guest, as most guests come with a
pl011 driver. It is also able to provide a working console with
interrupt support.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c | 70 +
 1 file changed, 70 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index b4f560f..ff65057 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1470,6 +1470,70 @@ static int make_timer_domU_node(const struct domain *d, 
void *fdt)
 return res;
 }
 
+static void set_interrupt(gic_interrupt_t *interrupt, unsigned int irq,
+  unsigned int cpumask, unsigned int level)
+{
+__be32 *cells = (__be32 *) interrupt;
+int is_ppi = (irq < 32);
+
+irq -= (is_ppi) ? 16: 32; /* PPIs start at 16, SPIs at 32 */
+
+/* See linux Documentation/devictree/bindings/arm/gic.txt */
+dt_set_cell(, 1, is_ppi); /* is a PPI? */
+dt_set_cell(, 1, irq);
+dt_set_cell(, 1, (cpumask << 8) | level);
+}
+
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+static int make_vpl011_uart_node(const struct domain *d, void *fdt,
+ int addrcells, int sizecells)
+{
+int res;
+gic_interrupt_t intr;
+int reg_size = addrcells + sizecells;
+int nr_cells = reg_size;
+__be32 reg[nr_cells];
+__be32 *cells;
+
+res = fdt_begin_node(fdt, "sbsa-pl011");
+if (res)
+return res;
+
+res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");
+if (res)
+return res;
+
+cells = [0];
+dt_child_set_range(, addrcells, sizecells, GUEST_PL011_BASE,
+GUEST_PL011_SIZE);
+if (res)
+return res;
+res = fdt_property(fdt, "reg", reg, sizeof(reg));
+if (res)
+return res;
+
+set_interrupt(, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
+if (res)
+return res;
+
+res = fdt_property_cell(fdt, "interrupt-parent",
+PHANDLE_GIC);
+if (res)
+return res;
+
+/* Use a default baud rate of 115200. */
+fdt_property_u32(fdt, "current-speed", 115200);
+
+res = fdt_end_node(fdt);
+if (res)
+return res;
+
+return 0;
+}
+#endif
+
 #define DOMU_DTB_SIZE 4096
 static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
 {
@@ -1531,6 +1595,12 @@ static int prepare_dtb_domU(struct domain *d, struct 
kernel_info *kinfo)
 if ( ret )
 goto err;
 
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+ret = make_vpl011_uart_node(d, kinfo->fdt, addrcells, sizecells);
+if ( ret )
+goto err;
+#endif
+
 ret = fdt_end_node(kinfo->fdt);
 if ( ret < 0 )
 goto err;
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 13/15] xen/arm: Allow vpl011 to be used by DomU

[Stefano Stabellini]

Make vpl011 being able to be used without a userspace component in Dom0.
In that case, output is printed to the Xen serial and input is received
from the Xen serial one character at a time.

Call domain_vpl011_init during construct_domU.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c  |  9 +++-
 xen/arch/arm/vpl011.c| 98 +---
 xen/include/asm-arm/vpl011.h |  2 +
 3 files changed, 84 insertions(+), 25 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index ff65057..97f14ca 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2482,7 +2482,14 @@ int __init construct_domU(struct domain *d, struct 
dt_device_node *node)
 if ( rc < 0 )
 return rc;
 
-return __construct_domain(d, );
+rc = __construct_domain(d, );
+if ( rc < 0 )
+return rc;
+
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+rc = domain_vpl011_init(d, NULL);
+#endif
+return rc;
 }
 
 int __init construct_dom0(struct domain *d)
diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index a281eab..5f1dc7a 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -34,6 +34,8 @@
 #include 
 #include 
 
+static void vpl011_data_avail(struct domain *d);
+
 /*
  * Since pl011 registers are 32-bit registers, all registers
  * are handled similarly allowing 8-bit, 16-bit and 32-bit
@@ -77,6 +79,29 @@ static void vpl011_update_interrupt_status(struct domain *d)
 #endif
 }
 
+void vpl011_read_char(struct domain *d, char c)
+{
+unsigned long flags;
+XENCONS_RING_IDX in_cons, in_prod;
+struct xencons_interface *intf = d->arch.vpl011.ring_buf;
+
+VPL011_LOCK(d, flags);
+
+in_cons = intf->in_cons;
+in_prod = intf->in_prod;
+if (xencons_queued(in_prod, in_cons, sizeof(intf->in)) == sizeof(intf->in))
+{
+VPL011_UNLOCK(d, flags);
+return;
+}
+
+intf->in[xencons_mask(in_prod, sizeof(intf->in))] = c;
+intf->in_prod = in_prod + 1;
+
+VPL011_UNLOCK(d, flags);
+vpl011_data_avail(d);
+}
+
 static uint8_t vpl011_read_data(struct domain *d)
 {
 unsigned long flags;
@@ -166,9 +191,18 @@ static void vpl011_write_data(struct domain *d, uint8_t 
data)
 struct vpl011 *vpl011 = >arch.vpl011;
 struct xencons_interface *intf = vpl011->ring_buf;
 XENCONS_RING_IDX out_cons, out_prod;
+unsigned int fifo_level = 0;
 
 VPL011_LOCK(d, flags);
 
+if ( vpl011->ring_page == NULL )
+{
+printk("%c", data);
+if (data == '\n')
+printk("DOM%u: ", d->domain_id);
+goto done;
+}
+
 out_cons = intf->out_cons;
 out_prod = intf->out_prod;
 
@@ -184,13 +218,10 @@ static void vpl011_write_data(struct domain *d, uint8_t 
data)
 if ( xencons_queued(out_prod, out_cons, sizeof(intf->out)) !=
  sizeof (intf->out) )
 {
-unsigned int fifo_level;
-
 intf->out[xencons_mask(out_prod, sizeof(intf->out))] = data;
 out_prod += 1;
 smp_wmb();
 intf->out_prod = out_prod;
-
 fifo_level = xencons_queued(out_prod, out_cons, sizeof(intf->out));
 
 if ( fifo_level == sizeof(intf->out) )
@@ -205,14 +236,15 @@ static void vpl011_write_data(struct domain *d, uint8_t 
data)
  */
 vpl011->uartfr |= BUSY;
 }
-
-vpl011_update_tx_fifo_status(vpl011, fifo_level);
-
-vpl011_update_interrupt_status(d);
 }
 else
 gprintk(XENLOG_ERR, "vpl011: Unexpected OUT ring buffer full\n");
 
+done:
+vpl011_update_tx_fifo_status(vpl011, fifo_level);
+
+vpl011_update_interrupt_status(d);
+
 vpl011->uartfr &= ~TXFE;
 
 VPL011_UNLOCK(d, flags);
@@ -462,13 +494,30 @@ int domain_vpl011_init(struct domain *d, struct 
vpl011_init_info *info)
 if ( vpl011->ring_buf )
 return -EINVAL;
 
-/* Map the guest PFN to Xen address space. */
-rc =  prepare_ring_for_helper(d,
-  gfn_x(info->gfn),
-  >ring_page,
-  >ring_buf);
-if ( rc < 0 )
-goto out;
+if ( info != NULL )
+{
+/* Map the guest PFN to Xen address space. */
+rc =  prepare_ring_for_helper(d,
+gfn_x(info->gfn),
+>ring_page,
+>ring_buf);
+if ( rc < 0 )
+goto out;
+
+rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
+vpl011_notification);
+if ( rc < 0 )
+goto out2;
+
+vpl011->evtchn = info->evtchn = rc;
+} else {
+vpl011->ring_buf = xzalloc(struct xencons_interface);
+if ( vpl011->ring_buf == NULL )
+{
+rc = -EINVAL;
+goto out;
+}
+}
 
 rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
 if ( !rc )
@@ -477,13 +526,6 @@ int domain_vpl011_init(struct domain *d, struct 
vpl011_init_info *info)
 

[Xen-devel] [PATCH RFC 11/15] xen/arm: generate a simple device tree for domUs

[Stefano Stabellini]

Introduce functions to generate a basic domU device tree, similar to the
existing functions in tools/libxl/libxl_arm.c.

Rename existing prepare_dtb to prepare_dtb_dom0 to avoid confusion.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c | 195 +++-
 1 file changed, 193 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 02a7f94..b4f560f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1360,7 +1360,194 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
 return res;
 }
 
-static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
+static int make_gic_domU_node(const struct domain *d, void *fdt, int 
addrcells, int sizecells)
+{
+int res = 0;
+int reg_size = addrcells + sizecells;
+int nr_cells = reg_size * 2;
+__be32 reg[nr_cells];
+__be32 *cells;
+
+res = fdt_begin_node(fdt, "interrupt-controller");
+if ( res )
+return res;
+
+res = fdt_property_cell(fdt, "#address-cells", 0);
+if ( res )
+return res;
+
+res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+if ( res )
+return res;
+
+res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+if ( res )
+return res;
+
+if (gic_hw_version() == GIC_V3)
+{
+const uint64_t gicd_base = GUEST_GICV3_GICD_BASE;
+const uint64_t gicd_size = GUEST_GICV3_GICD_SIZE;
+const uint64_t gicr0_base = GUEST_GICV3_GICR0_BASE;
+const uint64_t gicr0_size = GUEST_GICV3_GICR0_SIZE;
+
+res = fdt_property_string(fdt, "compatible", "arm,gic-v3");
+if ( res )
+return res;
+
+cells = [0];
+dt_child_set_range(, addrcells, sizecells, gicd_base, gicd_size);
+dt_child_set_range(, addrcells, sizecells, gicr0_base, 
gicr0_size);
+res = fdt_property(fdt, "reg", reg, sizeof(reg));
+}
+else if (gic_hw_version() == GIC_V3)
+{
+const uint64_t gicd_base = GUEST_GICD_BASE;
+const uint64_t gicd_size = GUEST_GICD_SIZE;
+const uint64_t gicc_base = GUEST_GICC_BASE;
+const uint64_t gicc_size = GUEST_GICC_SIZE;
+
+res = fdt_property_string(fdt, "compatible", "arm,cortex-a15-gic");
+if ( res )
+return res;
+
+cells = [0];
+dt_child_set_range(, addrcells, sizecells, gicd_base, gicd_size);
+dt_child_set_range(, addrcells, sizecells, gicc_base, gicc_size);
+}
+
+res = fdt_property(fdt, "reg", reg, sizeof(reg));
+if (res)
+return res;
+
+res = fdt_property_cell(fdt, "linux,phandle", PHANDLE_GIC);
+if (res)
+return res;
+
+res = fdt_property_cell(fdt, "phandle", PHANDLE_GIC);
+if (res)
+return res;
+
+res = fdt_end_node(fdt);
+
+return res;
+}
+
+static int make_timer_domU_node(const struct domain *d, void *fdt)
+{
+int res;
+gic_interrupt_t intrs[3];
+
+res = fdt_begin_node(fdt, "timer");
+if ( res )
+return res;
+
+if (!is_64bit_domain(d))
+{
+res = fdt_property_string(fdt, "compatible", "arm,armv7-timer");
+if ( res )
+return res;
+} else {
+res = fdt_property_string(fdt, "compatible", "arm,armv8-timer");
+if ( res )
+return res;
+}
+
+set_interrupt_ppi(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf, 
DT_IRQ_TYPE_LEVEL_LOW);
+set_interrupt_ppi(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, 
DT_IRQ_TYPE_LEVEL_LOW);
+set_interrupt_ppi(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf, 
DT_IRQ_TYPE_LEVEL_LOW);
+
+res = fdt_property(fdt, "interrupts", intrs, sizeof (intrs[0]) * 3);
+if ( res )
+return res;
+
+res = fdt_property_cell(fdt, "interrupt-parent",
+PHANDLE_GIC);
+if (res)
+return res;
+
+res = fdt_end_node(fdt);
+return res;
+}
+
+#define DOMU_DTB_SIZE 4096
+static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
+{
+int addrcells, sizecells;
+int ret;
+
+addrcells = dt_child_n_addr_cells(dt_host);
+sizecells = dt_child_n_size_cells(dt_host);
+
+kinfo->fdt = xmalloc_bytes(DOMU_DTB_SIZE);
+if ( kinfo->fdt == NULL )
+return -ENOMEM;
+
+ret = fdt_create(kinfo->fdt, DOMU_DTB_SIZE);
+if ( ret < 0 )
+goto err;
+
+ret = fdt_finish_reservemap(kinfo->fdt);
+if ( ret < 0 )
+goto err;
+
+ret = fdt_begin_node(kinfo->fdt, "/");
+if ( ret < 0 )
+goto err;
+
+ret = fdt_property_cell(kinfo->fdt, "#address-cells", addrcells);
+if ( ret )
+goto err;
+
+ret = fdt_property_cell(kinfo->fdt, "#size-cells", sizecells);
+if ( ret )
+goto err;
+
+ret = make_chosen_node(kinfo);
+if ( ret )
+goto err;
+
+ret = make_hypervisor_node(d, kinfo, addrcells, sizecells);
+if ( ret )
+goto err;
+
+ret = 

[Xen-devel] [PATCH RFC 01/15] xen: allow console_io hypercalls from DomUs on ARM

[Stefano Stabellini]

This is very useful when starting multiple domains from Xen without
xenstore access. It will allow them to print out to the Xen console.

Signed-off-by: Stefano Stabellini 
CC: andrew.coop...@citrix.com
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: konrad.w...@oracle.com
CC: t...@xen.org
CC: wei.l...@citrix.com
CC: dgde...@tycho.nsa.gov
---
If there is a better way to do this with XSM, please advise.

---
 xen/drivers/char/console.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 0f05369..dc9e0bb 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -555,9 +555,11 @@ long do_console_io(int cmd, int count, 
XEN_GUEST_HANDLE_PARAM(char) buffer)
 long rc;
 unsigned int idx, len;
 
+#ifndef CONFIG_ARM
 rc = xsm_console_io(XSM_OTHER, current->domain, cmd);
 if ( rc )
 return rc;
+#endif
 
 switch ( cmd )
 {
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Stefano Stabellini]

Move a few constants defined by libxl_arm.c to
xen/include/public/arch-arm.h, so that they are together with the other
guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
Also, this way they can be reused by hypervisor code.

Signed-off-by: Stefano Stabellini 
CC: wei.l...@citrix.com
CC: ian.jack...@eu.citrix.com
---
 tools/libxl/libxl_arm.c   | 26 --
 xen/include/public/arch-arm.h | 26 ++
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 8af9f6f..89a417f 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -8,23 +8,6 @@
 #include 
 #include 
 
-/**
- * IRQ line type.
- * DT_IRQ_TYPE_NONE- default, unspecified type
- * DT_IRQ_TYPE_EDGE_RISING - rising edge triggered
- * DT_IRQ_TYPE_EDGE_FALLING- falling edge triggered
- * DT_IRQ_TYPE_EDGE_BOTH   - rising and falling edge triggered
- * DT_IRQ_TYPE_LEVEL_HIGH  - high level triggered
- * DT_IRQ_TYPE_LEVEL_LOW   - low level triggered
- */
-#define DT_IRQ_TYPE_NONE   0x
-#define DT_IRQ_TYPE_EDGE_RISING0x0001
-#define DT_IRQ_TYPE_EDGE_FALLING   0x0002
-#define DT_IRQ_TYPE_EDGE_BOTH   \
-(DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
-#define DT_IRQ_TYPE_LEVEL_HIGH 0x0004
-#define DT_IRQ_TYPE_LEVEL_LOW  0x0008
-
 static const char *gicv_to_string(libxl_gic_version gic_version)
 {
 switch (gic_version) {
@@ -165,18 +148,9 @@ static struct arch_info {
 {"xen-3.0-aarch64", "arm,armv8-timer", "arm,armv8" },
 };
 
-/*
- * The device tree compiler (DTC) is allocating the phandle from 1 to
- * onwards. Reserve a high value for the GIC phandle.
- */
-#define PHANDLE_GIC (65000)
-
 typedef uint32_t be32;
 typedef be32 gic_interrupt[3];
 
-#define ROOT_ADDRESS_CELLS 2
-#define ROOT_SIZE_CELLS 2
-
 #define PROP_INITRD_START "linux,initrd-start"
 #define PROP_INITRD_END "linux,initrd-end"
 
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index eb424e8..cb88168 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -456,6 +456,32 @@ typedef uint64_t xen_callback_t;
 #define PSCI_cpu_on  2
 #define PSCI_migrate 3
 
+/*
+ * The device tree compiler (DTC) is allocating the phandle from 1 to
+ * onwards. Reserve a high value for the GIC phandle.
+ */
+#define PHANDLE_GIC (65000)
+
+#define ROOT_ADDRESS_CELLS 2
+#define ROOT_SIZE_CELLS 2
+
+/**
+ * IRQ line type.
+ * DT_IRQ_TYPE_NONE- default, unspecified type
+ * DT_IRQ_TYPE_EDGE_RISING - rising edge triggered
+ * DT_IRQ_TYPE_EDGE_FALLING- falling edge triggered
+ * DT_IRQ_TYPE_EDGE_BOTH   - rising and falling edge triggered
+ * DT_IRQ_TYPE_LEVEL_HIGH  - high level triggered
+ * DT_IRQ_TYPE_LEVEL_LOW   - low level triggered
+ */
+#define DT_IRQ_TYPE_NONE   0x
+#define DT_IRQ_TYPE_EDGE_RISING0x0001
+#define DT_IRQ_TYPE_EDGE_FALLING   0x0002
+#define DT_IRQ_TYPE_EDGE_BOTH   \
+(DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
+#define DT_IRQ_TYPE_LEVEL_HIGH 0x0004
+#define DT_IRQ_TYPE_LEVEL_LOW  0x0008
+
 #endif
 
 #ifndef __ASSEMBLY__
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 04/15] xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node

[Stefano Stabellini]

In order to make make_memory_node and make_hypervisor_node more
reusable, do not pass them dt_host. As they only use it to calculate
addrcells and sizecells, pass addrcells and sizecells directly.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c   | 20 ++--
 xen/common/device_tree.c  |  6 +++---
 xen/include/xen/device_tree.h |  2 +-
 3 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 11cdf05..bb88e09 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -533,11 +533,11 @@ static int fdt_property_interrupts(void *fdt, 
gic_interrupt_t *intr,
 
 static int make_memory_node(const struct domain *d,
 void *fdt,
-const struct dt_device_node *parent,
+int addrcells, int sizecells,
 const struct kernel_info *kinfo)
 {
 int res, i;
-int reg_size = dt_child_n_addr_cells(parent) + 
dt_child_n_size_cells(parent);
+int reg_size = addrcells + sizecells;
 int nr_cells = reg_size*kinfo->mem.nr_banks;
 __be32 reg[nr_cells];
 __be32 *cells;
@@ -563,7 +563,7 @@ static int make_memory_node(const struct domain *d,
 dt_dprintk("  Bank %d: %#"PRIx64"->%#"PRIx64"\n",
i, start, start + size);
 
-dt_child_set_range(, parent, start, size);
+dt_child_set_range(, addrcells, sizecells, start, size);
 }
 
 res = fdt_property(fdt, "reg", reg, sizeof(reg));
@@ -579,7 +579,7 @@ static void evtchn_allocate(struct domain *d);
 
 static int make_hypervisor_node(struct domain *d,
 const struct kernel_info *kinfo,
-const struct dt_device_node *parent)
+int addrcells, int sizecells)
 {
 const char compat[] =
 "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
@@ -588,9 +588,6 @@ static int make_hypervisor_node(struct domain *d,
 gic_interrupt_t intr;
 __be32 *cells;
 int res;
-/* Convenience alias */
-int addrcells = dt_child_n_addr_cells(parent);
-int sizecells = dt_child_n_size_cells(parent);
 void *fdt = kinfo->fdt;
 
 dt_dprintk("Create hypervisor node\n");
@@ -615,7 +612,8 @@ static int make_hypervisor_node(struct domain *d,
 
 /* reg 0 is grant table space */
 cells = [0];
-dt_child_set_range(, parent, kinfo->gnttab_start, 
kinfo->gnttab_size);
+dt_child_set_range(, addrcells, sizecells,
+   kinfo->gnttab_start, kinfo->gnttab_size);
 res = fdt_property(fdt, "reg", reg,
dt_cells_to_size(addrcells + sizecells));
 if ( res )
@@ -1292,11 +1290,13 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
 
 if ( node == dt_host )
 {
+int addrcells = dt_child_n_addr_cells(node);
+int sizecells = dt_child_n_size_cells(node);
 /*
  * The hypervisor node should always be created after all nodes
  * from the host DT have been parsed.
  */
-res = make_hypervisor_node(d, kinfo, node);
+res = make_hypervisor_node(d, kinfo, addrcells, sizecells);
 if ( res )
 return res;
 
@@ -1308,7 +1308,7 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
 if ( res )
 return res;
 
-res = make_memory_node(d, kinfo->fdt, node, kinfo);
+res = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
 if ( res )
 return res;
 
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 7b009ea..8fc401d 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -112,11 +112,11 @@ void dt_set_range(__be32 **cellp, const struct 
dt_device_node *np,
 dt_set_cell(cellp, dt_n_size_cells(np), size);
 }
 
-void dt_child_set_range(__be32 **cellp, const struct dt_device_node *parent,
+void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
 u64 address, u64 size)
 {
-dt_set_cell(cellp, dt_child_n_addr_cells(parent), address);
-dt_set_cell(cellp, dt_child_n_size_cells(parent), size);
+dt_set_cell(cellp, addrcells, address);
+dt_set_cell(cellp, sizecells, size);
 }
 
 static void __init *unflatten_dt_alloc(unsigned long *mem, unsigned long size,
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 0aecbe0..01040a6 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -699,7 +699,7 @@ void dt_set_range(__be32 **cellp, const struct 
dt_device_node *np,
  * Write a range into a series of cells and update cellp to point to the
  * cell just after.
  */
-void dt_child_set_range(__be32 **cellp, const struct dt_device_node *parent,
+void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
  

[Xen-devel] [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Stefano Stabellini]

Move generic initializations out of construct_dom0 so that they can be
reused.

No functional changes in this patch.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c | 124 
 1 file changed, 67 insertions(+), 57 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4e4cd19..b31c563 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2092,73 +2092,27 @@ static void __init find_gnttab_region(struct domain *d,
kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
 }
 
-int __init construct_dom0(struct domain *d)
+int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
 {
-struct kernel_info kinfo = {};
 struct vcpu *saved_current;
-int rc, i, cpu;
+int i, cpu;
 
 struct vcpu *v = d->vcpu[0];
 struct cpu_user_regs *regs = >arch.cpu_info->guest_cpu_user_regs;
 
-/* Sanity! */
-BUG_ON(d->domain_id != 0);
-BUG_ON(d->vcpu[0] == NULL);
-BUG_ON(v->is_initialised);
-
-printk("*** LOADING DOMAIN 0 ***\n");
-if ( dom0_mem <= 0 )
-{
-warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR 
NOW\n");
-dom0_mem = MB(512);
-}
-
-
-iommu_hwdom_init(d);
-
-d->max_pages = ~0U;
-
-kinfo.unassigned_mem = dom0_mem;
-kinfo.d = d;
-
-rc = kernel_probe();
-if ( rc < 0 )
-return rc;
-
 #ifdef CONFIG_ARM_64
 /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain */
-if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT )
+if ( !(cpu_has_el1_32) && kinfo->type == DOMAIN_32BIT )
 {
 printk("Platform does not support 32-bit domain\n");
 return -EINVAL;
 }
-d->arch.type = kinfo.type;
 
 if ( is_64bit_domain(d) )
 vcpu_switch_to_aarch64_mode(v);
 
 #endif
 
-allocate_memory(d, );
-find_gnttab_region(d, );
-
-/* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
-rc = gic_map_hwdom_extra_mappings(d);
-if ( rc < 0 )
-return rc;
-
-rc = platform_specific_mapping(d);
-if ( rc < 0 )
-return rc;
-
-if ( acpi_disabled )
-rc = prepare_dtb(d, );
-else
-rc = prepare_acpi(d, );
-
-if ( rc < 0 )
-return rc;
-
 /*
  * The following loads use the domain's p2m and require current to
  * be a vcpu of the domain, temporarily switch
@@ -2171,20 +2125,18 @@ int __init construct_dom0(struct domain *d)
  * kernel_load will determine the placement of the kernel as well
  * as the initrd & fdt in RAM, so call it first.
  */
-kernel_load();
+kernel_load(kinfo);
 /* initrd_load will fix up the fdt, so call it before dtb_load */
-initrd_load();
-dtb_load();
+initrd_load(kinfo);
+dtb_load(kinfo);
 
 /* Now that we are done restore the original p2m and current. */
 set_current(saved_current);
 p2m_restore_state(saved_current);
 
-discard_initial_modules();
-
 memset(regs, 0, sizeof(*regs));
 
-regs->pc = (register_t)kinfo.entry;
+regs->pc = (register_t)kinfo->entry;
 
 if ( is_32bit_domain(d) )
 {
@@ -2202,14 +2154,14 @@ int __init construct_dom0(struct domain *d)
  */
 regs->r0 = 0; /* SBZ */
 regs->r1 = 0x; /* We use DTB therefore no machine id */
-regs->r2 = kinfo.dtb_paddr;
+regs->r2 = kinfo->dtb_paddr;
 }
 #ifdef CONFIG_ARM_64
 else
 {
 regs->cpsr = PSR_GUEST64_INIT;
 /* From linux/Documentation/arm64/booting.txt */
-regs->x0 = kinfo.dtb_paddr;
+regs->x0 = kinfo->dtb_paddr;
 regs->x1 = 0; /* Reserved for future use */
 regs->x2 = 0; /* Reserved for future use */
 regs->x3 = 0; /* Reserved for future use */
@@ -2235,6 +2187,64 @@ int __init construct_dom0(struct domain *d)
 return 0;
 }
 
+int __init construct_dom0(struct domain *d)
+{
+struct kernel_info kinfo = {};
+int rc;
+
+struct vcpu *v = d->vcpu[0];
+
+/* Sanity! */
+BUG_ON(d->domain_id != 0);
+BUG_ON(d->vcpu[0] == NULL);
+BUG_ON(v->is_initialised);
+
+printk("*** LOADING DOMAIN 0 ***\n");
+if ( dom0_mem <= 0 )
+{
+warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR 
NOW\n");
+dom0_mem = MB(512);
+}
+
+
+iommu_hwdom_init(d);
+
+d->max_pages = ~0U;
+
+kinfo.unassigned_mem = dom0_mem;
+kinfo.d = d;
+
+rc = kernel_probe();
+if ( rc < 0 )
+return rc;
+
+allocate_memory(d, );
+find_gnttab_region(d, );
+
+/* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
+rc = gic_map_hwdom_extra_mappings(d);
+if ( rc < 0 )
+return rc;
+
+rc = platform_specific_mapping(d);
+if ( rc < 0 )
+return rc;
+
+d->arch.type = kinfo.type;
+
+if ( acpi_disabled )
+rc = prepare_dtb(d, );
+else
+rc = prepare_acpi(d, );
+
+if 

[Xen-devel] [PATCH RFC 05/15] xen/arm: rename acpi_make_chosen_node to make_chosen_node

[Stefano Stabellini]

acpi_make_chosen_node is actually generic and can be reused. Rename it
to make_chosen_node and make it available to non-ACPI builds.

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c | 84 ++---
 1 file changed, 42 insertions(+), 42 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index bb88e09..4e4cd19 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -935,6 +935,47 @@ static int make_timer_node(const struct domain *d, void 
*fdt,
 return res;
 }
 
+static int make_chosen_node(const struct kernel_info *kinfo)
+{
+int res;
+const char *bootargs = NULL;
+const struct bootmodule *mod = kinfo->kernel_bootmodule;
+void *fdt = kinfo->fdt;
+
+dt_dprintk("Create chosen node\n");
+res = fdt_begin_node(fdt, "chosen");
+if ( res )
+return res;
+
+if ( mod && mod->cmdline[0] )
+{
+bootargs = >cmdline[0];
+res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
+if ( res )
+   return res;
+}
+
+/*
+ * If the bootloader provides an initrd, we must create a placeholder
+ * for the initrd properties. The values will be replaced later.
+ */
+if ( mod && mod->size )
+{
+u64 a = 0;
+res = fdt_property(kinfo->fdt, "linux,initrd-start", , sizeof(a));
+if ( res )
+return res;
+
+res = fdt_property(kinfo->fdt, "linux,initrd-end", , sizeof(a));
+if ( res )
+return res;
+}
+
+res = fdt_end_node(fdt);
+
+return res;
+}
+
 static int map_irq_to_domain(struct domain *d, unsigned int irq,
  bool need_mapping, const char *devname)
 
@@ -1420,47 +1461,6 @@ static int acpi_route_spis(struct domain *d)
 return 0;
 }
 
-static int acpi_make_chosen_node(const struct kernel_info *kinfo)
-{
-int res;
-const char *bootargs = NULL;
-const struct bootmodule *mod = kinfo->kernel_bootmodule;
-void *fdt = kinfo->fdt;
-
-dt_dprintk("Create chosen node\n");
-res = fdt_begin_node(fdt, "chosen");
-if ( res )
-return res;
-
-if ( mod && mod->cmdline[0] )
-{
-bootargs = >cmdline[0];
-res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
-if ( res )
-   return res;
-}
-
-/*
- * If the bootloader provides an initrd, we must create a placeholder
- * for the initrd properties. The values will be replaced later.
- */
-if ( mod && mod->size )
-{
-u64 a = 0;
-res = fdt_property(kinfo->fdt, "linux,initrd-start", , sizeof(a));
-if ( res )
-return res;
-
-res = fdt_property(kinfo->fdt, "linux,initrd-end", , sizeof(a));
-if ( res )
-return res;
-}
-
-res = fdt_end_node(fdt);
-
-return res;
-}
-
 static int acpi_make_hypervisor_node(const struct kernel_info *kinfo,
  struct membank tbl_add[])
 {
@@ -1532,7 +1532,7 @@ static int create_acpi_dtb(struct kernel_info *kinfo, 
struct membank tbl_add[])
 return ret;
 
 /* Create a chosen node for DOM0 */
-ret = acpi_make_chosen_node(kinfo);
+ret = make_chosen_node(kinfo);
 if ( ret )
 goto err;
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 00/15] dom0less step1: boot multiple domains from device tree

[Stefano Stabellini]

Hi all,

This is first step toward "dom0less" as discussed in the various
certifications related threads and discussions.

The goal of this series is to enable Xen to boot multiple domains in
parallel, in addition to dom0, out of information found on device tree.

The device tree based boot protocol is extended to carry information
about DomUs. Based on that information, Xen creates and starts one or
more DomUs. DomUs created this way don't have access to xenstore, as
xenstore is not started yet. This is actually OK, because this is meant
for mission critical applications that typically only access directly
assigned devices. They cannot tolerate interference or increased IRQ
latency due to PV protocols. Device assignment is not actually covered
by this series, it will be added later.

DomUs can print to the Xen serial using the CONSOLEIO hypercalls. A
virtual PL011 is also emulated for them so that they can use their
regular PL011 driver. This allows unmodified guests to run as Xen on ARM
guests -- no Xen support needed at all. Console input also comes from
the Xen serial: the Ctrl-AAA switching mechanism is extended to switch
among domUs, dom0, and Xen.

Cheers,

Stefano


Stefano Stabellini (15):
  xen: allow console_io hypercalls from DomUs on ARM
  xen/arm: move a few guest related #defines to public/arch-arm.h
  xen/arm: extend device tree based multiboot protocol
  xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node
  xen/arm: rename acpi_make_chosen_node to make_chosen_node
  xen/arm: add BOOTMOD_DOMU_KERNEL/RAMDISK
  xen/arm: increase MAX_MODULES
  xen/arm: probe domU kernels and initrds
  xen/arm: refactor construct_dom0
  xen/arm: introduce construct_domU
  xen/arm: generate a simple device tree for domUs
  xen/arm: generate vpl011 node on device tree for domU
  xen/arm: Allow vpl011 to be used by DomU
  xen/arm: call construct_domU from start_xen and start DomU VMs
  xen: support console_switching between Dom0 and DomUs on ARM

 docs/misc/arm/device-tree/booting.txt | 102 +++
 tools/libxl/libxl_arm.c   |  26 --
 xen/arch/arm/bootfdt.c|   4 +
 xen/arch/arm/domain_build.c   | 533 +++---
 xen/arch/arm/kernel.c |  54 
 xen/arch/arm/kernel.h |   2 +
 xen/arch/arm/setup.c  |  52 +++-
 xen/arch/arm/vpl011.c |  98 +--
 xen/common/device_tree.c  |   6 +-
 xen/drivers/char/console.c|  51 +++-
 xen/include/asm-arm/setup.h   |  10 +-
 xen/include/asm-arm/vpl011.h  |   2 +
 xen/include/asm-x86/setup.h   |   2 +
 xen/include/public/arch-arm.h |  26 ++
 xen/include/xen/device_tree.h |   2 +-
 15 files changed, 789 insertions(+), 181 deletions(-)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Stefano Stabellini]

Introduce support for the "xen,domU" compatible node on device tree.
Create new DomU VMs based on the information found on device tree under
"xen,domU".

Introduce a simple global variable named max_init_domid to keep track of
the initial allocated domids.

Move the discard_initial_modules after DomUs have been built

Signed-off-by: Stefano Stabellini 
---
 xen/arch/arm/domain_build.c |  2 --
 xen/arch/arm/setup.c| 35 ++-
 xen/include/asm-arm/setup.h |  2 ++
 xen/include/asm-x86/setup.h |  2 ++
 4 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 97f14ca..e2d370f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2545,8 +2545,6 @@ int __init construct_dom0(struct domain *d)
 if ( rc < 0 )
 return rc;
 
-discard_initial_modules();
-
 return __construct_domain(d, );
 }
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 98bdb24..3723704 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -63,6 +63,8 @@ static unsigned long opt_xenheap_megabytes __initdata;
 integer_param("xenheap_megabytes", opt_xenheap_megabytes);
 #endif
 
+domid_t __read_mostly max_init_domid = 0;
+
 static __used void init_done(void)
 {
 free_init_memory();
@@ -711,6 +713,8 @@ void __init start_xen(unsigned long boot_phys_offset,
 struct bootmodule *xen_bootmodule;
 struct domain *dom0;
 struct xen_domctl_createdomain dom0_cfg = {};
+struct dt_device_node *chosen;
+struct dt_device_node *node;
 
 dcache_line_bytes = read_dcache_line_bytes();
 
@@ -860,7 +864,7 @@ void __init start_xen(unsigned long boot_phys_offset,
 dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
 dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
 
-dom0 = domain_create(0, _cfg);
+dom0 = domain_create(max_init_domid++, _cfg);
 if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
 panic("Error creating domain 0");
 
@@ -886,6 +890,35 @@ void __init start_xen(unsigned long boot_phys_offset,
 
 domain_unpause_by_systemcontroller(dom0);
 
+chosen = dt_find_node_by_name(dt_host, "chosen");
+if ( chosen != NULL )
+{
+dt_for_each_child_node(chosen, node)
+{
+struct domain *d;
+struct xen_domctl_createdomain d_cfg = {};
+
+if ( !dt_device_is_compatible(node, "xen,domU") )
+continue;
+
+d_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
+d_cfg.arch.nr_spis = gic_number_lines() - 32;
+
+d = domain_create(max_init_domid++, _cfg);
+if ( IS_ERR(d))
+panic("Error creating domU");
+
+d->is_privileged = 0;
+d->target = NULL;
+
+if ( construct_domU(d, node) != 0)
+printk("Could not set up DOMU guest OS");
+
+domain_unpause_by_systemcontroller(d);
+}
+}
+discard_initial_modules();
+
 /* Switch on to the dynamically allocated stack for the idle vcpu
  * since the static one we're running on is about to be freed. */
 memcpy(idle_vcpu[0]->arch.cpu_info, get_cpu_info(),
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index e9f9905..578f3b9 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -56,6 +56,8 @@ struct bootinfo {
 
 extern struct bootinfo bootinfo;
 
+extern domid_t max_init_domid;
+
 void arch_init_memory(void);
 
 void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
index 19232af..2fb9529 100644
--- a/xen/include/asm-x86/setup.h
+++ b/xen/include/asm-x86/setup.h
@@ -73,4 +73,6 @@ extern bool opt_dom0_shadow;
 #endif
 extern bool dom0_pvh;
 
+#define max_init_domid (1)
+
 #endif
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Stefano Stabellini]

Extend the existing device tree based multiboot protocol to include
information regarding other domUs to boot.

Signed-off-by: Stefano Stabellini 
---
 docs/misc/arm/device-tree/booting.txt | 102 ++
 1 file changed, 102 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt 
b/docs/misc/arm/device-tree/booting.txt
index ce2d0dc..95255e5 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -119,3 +119,105 @@ For those you would hardcode the Xen commandline in the 
DTB under
 line by writing bootargs (as for native Linux).
 A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
 for Dom0 and bootargs for native Linux.
+
+
+Creating DomUs directly from Xen
+
+
+It is possible to have Xen create other domains, in addition to dom0,
+out of the information provided via device tree. A Kernel and initrd
+(optional) need to be specified for each guest.
+
+For each DomU to be created there needs to be one node under /chosen
+with the following properties:
+
+- compatible
+
+"xen,domU"
+
+- mem (optional)
+
+A string specifying the amount of RAM to allocate to the guest. If
+not specified it defaults to "64M". The format of the string is the same
+as the one for the mem= parameter in xl config files.
+
+- cpus (optional)
+
+A string specifying the number of vcpus to allocate to the guest. If
+not specified it defaults to "1".
+
+- #address-cells and #size-cells
+
+Both #address-cells and #size-cells need to be specified because
+both sub-nodes (described shortly) have reg properties.
+
+Under the "xen,domU" compatible node, one or more sub-nodes are present
+for the DomU kernel and ramdisk.
+
+The kernel sub-node has the following properties:
+
+- compatible
+
+"multiboot,domU-kernel"
+
+- reg
+
+Specifies the physical address of the kernel in RAM and its
+length.
+
+- bootargs (optional)
+
+Command line parameters for the guest kernel.
+
+The ramdisk sub-node has the following properties:
+
+- compatible
+
+"multiboot,domU-ramdisk"
+
+- reg
+
+Specifies the physical address of the ramdisk in RAM and its
+length.
+
+
+Example
+===
+
+domU1 {
+compatible = "xen,domU";
+#address-cells = <0x2>;
+#size-cells = <0x1>;
+mem = "128M";
+cpus = "2";
+
+module@0x4a00 {
+compatible = "multiboot,domU-kernel";
+reg = <0x0 0x4a00 0xff>;
+bootargs = "console=ttyAMA0 init=/bin/sh";
+};
+
+module@0x4b00 {
+compatible = "multiboot,domU-ramdisk";
+reg = <0x0 0x4b00 0xff>;
+};
+};
+
+domU2 {
+compatible = "xen,domU";
+#address-cells = <0x2>;
+#size-cells = <0x1>;
+mem = "64M";
+cpus = "1";
+
+module@0x4c00 {
+compatible = "multiboot,domU-kernel";
+reg = <0x0 0x4c00 0xff>;
+bootargs = "console=ttyAMA0 init=/bin/sh";
+};
+
+module@0x4d00 {
+compatible = "multiboot,domU-ramdisk";
+reg = <0x0 0x4d00 0xff>;
+};
+};
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

[Boris Ostrovsky]

On 06/13/2018 05:04 AM, Oleksandr Andrushchenko wrote:
> On 06/13/2018 06:14 AM, Boris Ostrovsky wrote:
>>
>>
>> On 06/12/2018 09:42 AM, Oleksandr Andrushchenko wrote:
>>
>>>   int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32
>>> fd)
>>>   {
>>> -    return -EINVAL;
>>> +    struct gntdev_dmabuf *gntdev_dmabuf;
>>> +    struct dma_buf_attachment *attach;
>>> +    struct dma_buf *dma_buf;
>>> +
>>> +    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
>>> +    if (IS_ERR(gntdev_dmabuf))
>>> +    return PTR_ERR(gntdev_dmabuf);
>>> +
>>> +    pr_debug("Releasing DMA buffer with fd %d\n", fd);
>>> +
>>> +    attach = gntdev_dmabuf->u.imp.attach;
>>> +
>>> +    if (gntdev_dmabuf->u.imp.sgt)
>>> +    dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
>>> + DMA_BIDIRECTIONAL);
>>> +    dma_buf = attach->dmabuf;
>>> +    dma_buf_detach(attach->dmabuf, attach);
>>> +    dma_buf_put(dma_buf);
>>> +
>>> +    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs,
>>> +  gntdev_dmabuf->nr_pages);
>>
>>
>>
>> Should you first end foreign access, before doing anything?
>>
> I am rolling back in reverse order here, so I think we first need
> to finish local activities with the buffer and then end foreign
> access.

Looking at gntdev_dmabuf_imp_to_refs(), the order is
    dmabuf_imp_alloc_storage()
    dma_buf_attach()
    dma_buf_map_attachment()
    dmabuf_imp_grant_foreign_access()

Or was I looking at wrong place?

-boris

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 08/13] arm: add ALL, QEMU, Rcar3 and MPSoC configs

[Stefano Stabellini]

Add a "Platform Support" choice with four kconfig options: QEMU, RCAR3,
MPSOC and ALL. They enable the required options for their hardware
platform. ALL enables all available platforms and it's the default. It
doesn't automatically select any of the related drivers, otherwise they
cannot be disabled. ALL is implemented by selecting hidden options
corresponding to QEMU, MPSOC and RCAR3.

In the case of the MPSOC that has a platform file under
arch/arm/platforms/, build the file if MPSOC.

Signed-off-by: Stefano Stabellini 
CC: artem_myga...@epam.com
CC: volodymyr_babc...@epam.com

---
Changes in v5:
- turn platform support into a choice
- add ALL

Changes in v4:
- fix GICv3/GICV3
- default y to all options
- build xilinx-zynqmp if MPSOC
---
 xen/arch/arm/Kconfig|  2 ++
 xen/arch/arm/platforms/Kconfig  | 55 +
 xen/arch/arm/platforms/Makefile |  2 +-
 3 files changed, 58 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/arm/platforms/Kconfig

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 2b87111..75cacfb 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -213,6 +213,8 @@ config ARM64_HARDEN_BRANCH_PREDICTOR
 config ARM32_HARDEN_BRANCH_PREDICTOR
 def_bool y if ARM_32 && HARDEN_BRANCH_PREDICTOR
 
+source "arch/arm/platforms/Kconfig"
+
 source "common/Kconfig"
 
 source "drivers/Kconfig"
diff --git a/xen/arch/arm/platforms/Kconfig b/xen/arch/arm/platforms/Kconfig
new file mode 100644
index 000..07c5930
--- /dev/null
+++ b/xen/arch/arm/platforms/Kconfig
@@ -0,0 +1,55 @@
+choice
+   prompt "Platform Support"
+   default ALL
+   ---help---
+   Choose which hardware platform to enable in Xen.
+
+   If unsure, choose ALL.
+
+config ALL
+   bool "All Platforms"
+   select MPSOC_PLATFORM
+   select QEMU_PLATFORM
+   select RCAR3_PLATFORM
+   ---help---
+   Enable support for all available hardware platforms. It doesn't
+   automatically select any of the related drivers.
+
+config QEMU
+   bool "QEMU aarch virt machine support"
+   depends on ARM_64
+   select QEMU_PLATFORM
+   select GICV3
+   select HAS_PL011
+   ---help---
+   Enable all the required drivers for QEMU aarch64 virt emulated
+   machine.
+
+config RCAR3
+   bool "Renesas RCar3 support"
+   depends on ARM_64
+   select RCAR3_PLATFORM
+   select HAS_SCIF
+   ---help---
+   Enable all the required drivers for Renesas RCar3
+
+config MPSOC
+   bool "Xilinx Ultrascale+ MPSoC support"
+   depends on ARM_64
+   select MPSOC_PLATFORM
+   select HAS_CADENCE_UART
+   select ARM_SMMU
+   ---help---
+   Enable all the required drivers for Xilinx Ultrascale+ MPSoC
+
+endchoice
+
+config QEMU_PLATFORM
+   bool
+
+config RCAR3_PLATFORM
+   bool
+
+config MPSOC_PLATFORM
+   bool
+
diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
index 80e555c..a79bdb9 100644
--- a/xen/arch/arm/platforms/Makefile
+++ b/xen/arch/arm/platforms/Makefile
@@ -8,4 +8,4 @@ obj-$(CONFIG_ARM_64) += seattle.o
 obj-y += sunxi.o
 obj-$(CONFIG_ARM_64) += thunderx.o
 obj-$(CONFIG_ARM_64) += xgene-storm.o
-obj-$(CONFIG_ARM_64) += xilinx-zynqmp.o
+obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 05/13] make it possible to enable/disable UART drivers

[Stefano Stabellini]

All the UART drivers are silent options. Add one line descriptions so
that can be de/selected via menuconfig.

Add an x86 dependency to HAS_EHCI: EHCI PCI has not been used on ARM. In
fact, it depends on PCI, and moreover we have drivers for several
embedded UARTs for various ARM boards.

NS16550 remains not selectable on x86.

Signed-off-by: Stefano Stabellini 
Acked-by: Julien Grall 
---
Changes in v4:
- improve commit message
- remove prompt for HAS_EHCI

Changes in v3:
- NS16550 prompt if ARM

Changes in v2:
- make HAS_EHCI depend on x86
---
 xen/drivers/char/Kconfig | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index cc78ec3..b1f07f8 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -1,11 +1,11 @@
 config HAS_NS16550
-   bool
+   bool "NS16550 UART driver" if ARM
default y
help
  This selects the 16550-series UART support. For most systems, say Y.
 
 config HAS_CADENCE_UART
-   bool
+   bool "Xilinx Cadence UART driver"
default y
depends on ARM_64
help
@@ -13,7 +13,7 @@ config HAS_CADENCE_UART
  based board, say Y.
 
 config HAS_MVEBU
-   bool
+   bool "Marvell MVEBU UART driver"
default y
depends on ARM_64
help
@@ -21,7 +21,7 @@ config HAS_MVEBU
  based board, say Y.
 
 config HAS_PL011
-   bool
+   bool "ARM PL011 UART driver"
default y
depends on ARM
help
@@ -29,7 +29,7 @@ config HAS_PL011
  an Integrator/PP2, Integrator/CP or Versatile platform, say Y.
 
 config HAS_EXYNOS4210
-   bool
+   bool "Samsung Exynos 4210 UART driver"
default y
depends on ARM_32
help
@@ -37,7 +37,7 @@ config HAS_EXYNOS4210
  Exynos based board, say Y.
 
 config HAS_OMAP
-   bool
+   bool "Texas Instruments OMAP UART driver"
default y
depends on ARM_32
help
@@ -45,7 +45,7 @@ config HAS_OMAP
  Instruments based CPU, say Y.
 
 config HAS_SCIF
-   bool
+   bool "SuperH SCI(F) UART driver"
default y
depends on ARM
help
@@ -54,6 +54,7 @@ config HAS_SCIF
 
 config HAS_EHCI
bool
+   depends on X86
help
  This selects the USB based EHCI debug port to be used as a UART. If
  you have an x86 based system with USB, say Y.
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 07/13] arm: add a tiny kconfig configuration

[Stefano Stabellini]

Add a tiny kconfig configuration. Enabled NULL and Credit schedulers.
It only carries non-default options (use make menuconfig or make
olddefconfig to produce a complete .config file).

Signed-off-by: Stefano Stabellini 

---
---
 xen/arch/arm/configs/tiny64.conf | 43 
 1 file changed, 43 insertions(+)
 create mode 100644 xen/arch/arm/configs/tiny64.conf

diff --git a/xen/arch/arm/configs/tiny64.conf b/xen/arch/arm/configs/tiny64.conf
new file mode 100644
index 000..e9a5e65
--- /dev/null
+++ b/xen/arch/arm/configs/tiny64.conf
@@ -0,0 +1,43 @@
+CONFIG_ARM_64=y
+CONFIG_ARM=y
+
+#
+# Architecture Features
+#
+# CONFIG_GICV3 is not set
+# CONFIG_MEM_ACCESS is not set
+# CONFIG_SBSA_VUART_CONSOLE is not set
+
+#
+# Common Features
+#
+# CONFIG_TMEM is not set
+
+#
+# Schedulers
+#
+# CONFIG_SCHED_CREDIT2 is not set
+# CONFIG_SCHED_RTDS is not set
+# CONFIG_SCHED_ARINC653 is not set
+CONFIG_SCHED_NULL=y
+CONFIG_SCHED_NULL_DEFAULT=y
+CONFIG_SCHED_DEFAULT="null"
+# CONFIG_SUPPRESS_DUPLICATE_SYMBOL_WARNINGS is not set
+
+#
+# Device Drivers
+#
+# CONFIG_HAS_NS16550 is not set
+# CONFIG_HAS_CADENCE_UART is not set
+# CONFIG_HAS_MVEBU is not set
+# CONFIG_HAS_PL011 is not set
+# CONFIG_HAS_SCIF is not set
+# CONFIG_ARM_SMMU is not set
+
+#
+# Debugging Options
+#
+# CONFIG_DEBUG is not set
+# CONFIG_FRAME_POINTER is not set
+# CONFIG_VERBOSE_DEBUG is not set
+# CONFIG_SCRUB_DEBUG is not set
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 02/13] arm: make it possible to disable HAS_GICV3

[Stefano Stabellini]

Today it is a silent option. This patch adds a one line description and
makes it optional.

Signed-off-by: Stefano Stabellini 
Acked-by: Julien Grall 
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: andrew.coop...@citrix.com

---
Changes in v3:
- remove any changes to MEM_ACCESS
- update commit message

Changes in v2:
- make HAS_GICv3 depend on ARM_64
- remove modifications to ARM_HDLCD kconfig, it has been removed
---
 xen/arch/arm/Kconfig | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 4dc7ef5..fb69a66 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -12,7 +12,6 @@ config ARM_32
 config ARM_64
def_bool y
depends on 64BIT
-   select HAS_GICV3
 
 config ARM
def_bool y
@@ -42,6 +41,13 @@ config ACPI
 
 config HAS_GICV3
bool
+   prompt "GICv3 driver"
+   depends on ARM_64
+   default y
+   ---help---
+
+ Driver for the ARM Generic Interrupt Controller v3.
+ If unsure, say Y
 
 config HAS_ITS
 bool
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 03/13] arm: rename HAS_GICV3 to GICV3

[Stefano Stabellini]

HAS_GICV3 has become selectable by the user. To mark the change, rename
the option from HAS_GICV3 to GICV3.

Suggested-by: Julien Grall 
Signed-off-by: Stefano Stabellini 
Acked-by: Julien Grall 

---
Changes in v3:
- no changes

Changes in v2:
- patch added
---
 xen/arch/arm/Kconfig   | 4 ++--
 xen/arch/arm/Makefile  | 4 ++--
 xen/arch/arm/vgic.c| 2 +-
 xen/arch/arm/vgic/vgic.c   | 2 +-
 xen/include/asm-arm/gic.h  | 4 ++--
 xen/include/asm-arm/vgic.h | 4 ++--
 6 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index fb69a66..66adce4 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -39,7 +39,7 @@ config ACPI
  Advanced Configuration and Power Interface (ACPI) support for Xen is
  an alternative to device tree on ARM64.
 
-config HAS_GICV3
+config GICV3
bool
prompt "GICv3 driver"
depends on ARM_64
@@ -52,7 +52,7 @@ config HAS_GICV3
 config HAS_ITS
 bool
 prompt "GICv3 ITS MSI controller support" if EXPERT = "y"
-depends on HAS_GICV3 && !NEW_VGIC
+depends on GICV3 && !NEW_VGIC
 
 config NEW_VGIC
bool
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index a9533b1..b9c2fb7 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -17,7 +17,7 @@ obj-y += domctl.o
 obj-$(EARLY_PRINTK) += early_printk.o
 obj-y += gic.o
 obj-y += gic-v2.o
-obj-$(CONFIG_HAS_GICV3) += gic-v3.o
+obj-$(CONFIG_GICV3) += gic-v3.o
 obj-$(CONFIG_HAS_ITS) += gic-v3-its.o
 obj-$(CONFIG_HAS_ITS) += gic-v3-lpi.o
 obj-y += guestcopy.o
@@ -51,7 +51,7 @@ ifneq ($(CONFIG_NEW_VGIC),y)
 obj-y += gic-vgic.o
 obj-y += vgic.o
 obj-y += vgic-v2.o
-obj-$(CONFIG_HAS_GICV3) += vgic-v3.o
+obj-$(CONFIG_GICV3) += vgic-v3.o
 obj-$(CONFIG_HAS_ITS) += vgic-v3-its.o
 endif
 obj-y += vm_event.o
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 3fafdd0..7a2c455 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -98,7 +98,7 @@ int domain_vgic_register(struct domain *d, int *mmio_count)
 {
 switch ( d->arch.vgic.version )
 {
-#ifdef CONFIG_HAS_GICV3
+#ifdef CONFIG_GICV3
 case GIC_V3:
 if ( vgic_v3_init(d, mmio_count) )
return -ENODEV;
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index a35449b..832632a 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -974,7 +974,7 @@ unsigned int vgic_max_vcpus(const struct domain *d)
 return min_t(unsigned int, MAX_VIRT_CPUS, vgic_vcpu_limit);
 }
 
-#ifdef CONFIG_HAS_GICV3
+#ifdef CONFIG_GICV3
 /* Dummy implementation to allow building without actual vGICv3 support. */
 void vgic_v3_setup_hw(paddr_t dbase,
   unsigned int nr_rdist_regions,
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 58b910f..22fa122 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -166,7 +166,7 @@
 
 #define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
 
-#ifdef CONFIG_HAS_GICV3
+#ifdef CONFIG_GICV3
 /*
  * GICv3 registers that needs to be saved/restored
  */
@@ -194,7 +194,7 @@ struct gic_v2 {
  */
 union gic_state_data {
 struct gic_v2 v2;
-#ifdef CONFIG_HAS_GICV3
+#ifdef CONFIG_GICV3
 struct gic_v3 v3;
 #endif
 };
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 2a58ea3..374fdaa 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -156,7 +156,7 @@ struct vgic_dist {
 struct pending_irq *pending_irqs;
 /* Base address for guest GIC */
 paddr_t dbase; /* Distributor base address */
-#ifdef CONFIG_HAS_GICV3
+#ifdef CONFIG_GICV3
 /* GIC V3 addressing */
 /* List of contiguous occupied by the redistributors */
 struct vgic_rdist_region {
@@ -359,7 +359,7 @@ unsigned int vgic_max_vcpus(const struct domain *d);
 void vgic_v2_setup_hw(paddr_t dbase, paddr_t cbase, paddr_t csize,
   paddr_t vbase, uint32_t aliased_offset);
 
-#ifdef CONFIG_HAS_GICV3
+#ifdef CONFIG_GICV3
 struct rdist_region;
 void vgic_v3_setup_hw(paddr_t dbase,
   unsigned int nr_rdist_regions,
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 13/13] xen: clarify the security-support status of Kconfig options on ARM

[Stefano Stabellini]

Signed-off-by: Stefano Stabellini 
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: andrew.coop...@citrix.com
---
 SUPPORT.md | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/SUPPORT.md b/SUPPORT.md
index c5ec849..f37a3e6 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -23,6 +23,16 @@ security supported. Other Kconfig options that do not depend 
on
 CONFIG_EXPERT or CONFIG_DEBUG are supported, if the related features are
 marked as supported in this document.
 
+On ARM, a wider range of Kconfig configurations is available to enable
+very small lines of code counts in the hypervisor. Not all possible
+combinations of kconfig options are security supported. Instead, a few
+pre-canned configurations have been added to xen/arch/arm/configs: they
+are security suppored. Configurations derived from the pre-canned files
+by adding non-listed options with their default values, or by enabling
+any of the platform options under "Platform Support" (and their
+dependent options) are security supported, unless stated
+otherwise.
+
 ## Host Architecture
 
 ### x86-64
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 12/13] xen: specify support for EXPERT and DEBUG Kconfig options

[Stefano Stabellini]

Add a clear statement about them, reflecting the current security
support status of Kconfig options (no changes to current policies).

Signed-off-by: Stefano Stabellini 
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: andrew.coop...@citrix.com
---
 SUPPORT.md | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/SUPPORT.md b/SUPPORT.md
index f96c38c..c5ec849 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -16,6 +16,13 @@ for the definitions of the support status levels etc.
 
 # Feature Support
 
+## Kconfig
+
+Kconfig options that depend on CONFIG_EXPERT or CONFIG_DEBUG are not
+security supported. Other Kconfig options that do not depend on
+CONFIG_EXPERT or CONFIG_DEBUG are supported, if the related features are
+marked as supported in this document.
+
 ## Host Architecture
 
 ### x86-64
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 11/13] xen: support the Null scheduler

[Stefano Stabellini]

The Null scheduler has been in the tree long enough to be marked
supported.

Signed-off-by: Stefano Stabellini 
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: andrew.coop...@citrix.com
CC: dfaggi...@suse.com
---
 SUPPORT.md | 2 +-
 xen/common/Kconfig | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/SUPPORT.md b/SUPPORT.md
index 264b23f..f96c38c 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -284,7 +284,7 @@ Currently only single-vcpu domains are supported.
 
 ### Null Scheduler
 
-Status: Experimental
+Status: Supported
 
 A very simple, very static scheduling policy
 that always schedules the same vCPU(s) on the same pCPU(s).
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index db6bb2d..bb1e831 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -203,7 +203,7 @@ config SCHED_ARINC653
  cores, targeted for avionics, drones, and medical devices.
 
 config SCHED_NULL
-   bool "Null scheduler support (EXPERIMENTAL)"
+   bool "Null scheduler support"
default y
---help---
  The null scheduler is a static, zero overhead scheduler,
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 06/13] arm: make it possible to disable the SMMU driver

[Stefano Stabellini]

Introduce a Kconfig option for the ARM SMMUv1 and SMMUv2 driver.

Signed-off-by: Stefano Stabellini 
Acked-by: Julien Grall 
Acked-by: Jan Beulich 
CC: jbeul...@suse.com

---
Changes in v3:
- rename SMMUv2 to ARM_SMMU
- improve help message
- use if ARM

Changes in v2:
- rename HAS_SMMUv2 to SMMUv2
- move SMMUv2 to xen/drivers/passthrough/Kconfig
---
 xen/drivers/passthrough/Kconfig  | 12 
 xen/drivers/passthrough/arm/Makefile |  2 +-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
index 8d90b67..a3c0649 100644
--- a/xen/drivers/passthrough/Kconfig
+++ b/xen/drivers/passthrough/Kconfig
@@ -1,3 +1,15 @@
 
 config HAS_PASSTHROUGH
bool
+
+if ARM
+config ARM_SMMU
+   bool "ARM SMMUv1 and v2 driver"
+   default y
+   ---help---
+ Support for implementations of the ARM System MMU architecture
+ versions 1 and 2.
+
+ Say Y here if your SoC includes an IOMMU device implementing the
+ ARM SMMU architecture.
+endif
diff --git a/xen/drivers/passthrough/arm/Makefile 
b/xen/drivers/passthrough/arm/Makefile
index f4cd26e..0156431 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -1,2 +1,2 @@
 obj-y += iommu.o
-obj-y += smmu.o
+obj-$(ARM_SMMU) += smmu.o
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 04/13] Make MEM_ACCESS configurable

[Stefano Stabellini]

Select MEM_ACCESS_ALWAYS_ON on x86 to mark that MEM_ACCESS is not
configurable on x86. Avoid selecting it on ARM.
Rename HAS_MEM_ACCESS to MEM_ACCESS everywhere. Add a prompt and a
description to MEM_ACCESS in xen/common/Kconfig.

The result is that the user-visible option is MEM_ACCESS, and it is
configurable only on ARM (disabled by default). At the moment the
arch-specific mem_access code remains enabled on ARM, even with
MEM_ACCESS=y.

The purpose is to reduce code size. The option doesn't depend on EXPERT
because it would be nice to ecurity-support configurations without
MEM_ACCESS and a non-expert should be able to disable it.

Suggested-by: Julien Grall 
Signed-off-by: Stefano Stabellini 
Acked-by: Jan Beulich 

CC: dgde...@tycho.nsa.gov
CC: andrew.coop...@citrix.com
CC: george.dun...@eu.citrix.com
CC: ian.jack...@eu.citrix.com
CC: jbeul...@suse.com
CC: julien.gr...@arm.com
CC: konrad.w...@oracle.com
CC: sstabell...@kernel.org
CC: t...@xen.org
CC: wei.l...@citrix.com

---
Changes in v5:
- change MEM_ACCESS_ALWAYS_ON to bool
- change default for MEM_ACCESS, default y if MEM_ACCESS_ALWAYS_ON

Changes in v4:
- remove HAS_MEM_ACCESS
- move MEM_ACCESS_ALWAYS_ON to common
- combile default and bool to def_bool

Changes in v3:
- keep HAS_MEM_ACCESS to mark that an arch can do MEM_ACCESS
- introduce MEM_ACCESS_ALWAYS_ON
- the main MEM_ACCESS option is in xen/common/Kconfig

Changes in v2:
- patch added
---
 tools/firmware/xen-dir/shim.config |  2 +-
 xen/arch/arm/Kconfig   |  1 -
 xen/arch/x86/Kconfig   |  2 +-
 xen/common/Kconfig | 10 +-
 xen/common/Makefile|  2 +-
 xen/common/domctl.c|  2 +-
 xen/include/xen/mem_access.h   |  4 ++--
 xen/include/xsm/dummy.h|  2 +-
 xen/include/xsm/xsm.h  |  4 ++--
 xen/xsm/dummy.c|  2 +-
 xen/xsm/flask/hooks.c  |  4 ++--
 11 files changed, 21 insertions(+), 14 deletions(-)

diff --git a/tools/firmware/xen-dir/shim.config 
b/tools/firmware/xen-dir/shim.config
index 4d5630f..21d7075 100644
--- a/tools/firmware/xen-dir/shim.config
+++ b/tools/firmware/xen-dir/shim.config
@@ -29,7 +29,7 @@ CONFIG_COMPAT=y
 CONFIG_CORE_PARKING=y
 CONFIG_HAS_ALTERNATIVE=y
 CONFIG_HAS_EX_TABLE=y
-CONFIG_HAS_MEM_ACCESS=y
+CONFIG_MEM_ACCESS=y
 CONFIG_HAS_MEM_PAGING=y
 CONFIG_HAS_MEM_SHARING=y
 CONFIG_HAS_PDX=y
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 66adce4..2b87111 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -17,7 +17,6 @@ config ARM
def_bool y
select HAS_ALTERNATIVE
select HAS_DEVICE_TREE
-   select HAS_MEM_ACCESS
select HAS_PASSTHROUGH
select HAS_PDX
 
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index f64fc56..9a85fe9 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -15,7 +15,7 @@ config X86
select HAS_GDBSX
select HAS_IOPORTS
select HAS_KEXEC
-   select HAS_MEM_ACCESS
+   select MEM_ACCESS_ALWAYS_ON
select HAS_MEM_PAGING
select HAS_MEM_SHARING
select HAS_NS16550
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 9043dce..db6bb2d 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -20,9 +20,17 @@ config HAS_DEVICE_TREE
 config HAS_EX_TABLE
bool
 
-config HAS_MEM_ACCESS
+config MEM_ACCESS_ALWAYS_ON
bool
 
+config MEM_ACCESS
+   def_bool MEM_ACCESS_ALWAYS_ON
+   prompt "Memory Access and VM events" if !MEM_ACCESS_ALWAYS_ON
+   ---help---
+
+ Framework to configure memory access types for guests and receive
+ related events in userspace.
+
 config HAS_MEM_PAGING
bool
 
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 24d4752..6f2b3fc 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -22,7 +22,7 @@ obj-y += lib.o
 obj-$(CONFIG_NEEDS_LIST_SORT) += list_sort.o
 obj-$(CONFIG_LIVEPATCH) += livepatch.o livepatch_elf.o
 obj-y += lzo.o
-obj-$(CONFIG_HAS_MEM_ACCESS) += mem_access.o
+obj-$(CONFIG_MEM_ACCESS) += mem_access.o
 obj-y += memory.o
 obj-y += monitor.o
 obj-y += multicall.o
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 9b7bc08..891ad58 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -1085,7 +1085,7 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
u_domctl)
 copyback = 1;
 break;
 
-#ifdef CONFIG_HAS_MEM_ACCESS
+#ifdef CONFIG_MEM_ACCESS
 case XEN_DOMCTL_set_access_required:
 if ( unlikely(current->domain == d) ) /* no domain_pause() */
 ret = -EPERM;
diff --git a/xen/include/xen/mem_access.h b/xen/include/xen/mem_access.h
index 5ab34c1..7e95eab 100644
--- a/xen/include/xen/mem_access.h
+++ b/xen/include/xen/mem_access.h
@@ -78,7 +78,7 @@ long p2m_set_mem_access_multi(struct domain *d,
  */
 int p2m_get_mem_access(struct domain *d, gfn_t gfn, xenmem_access_t *access);
 
-#ifdef CONFIG_HAS_MEM_ACCESS
+#ifdef CONFIG_MEM_ACCESS
 

[Xen-devel] [PATCH v6 01/13] arm: remove the ARM HDLCD driver

[Stefano Stabellini]

The ARM HDLCD driver is unused. The device itself can only be found on
Virtual Express boards that are for early development only. Remove the
driver.

Also remove vexpress_syscfg, now unused, and "select VIDEO" that is not
useful anymore.

Suggested-by: Julien Grall 
Signed-off-by: Stefano Stabellini 
Reviewed-by: Julien Grall 
---
Changes in v3:
- remove "select VIDEO"
- remove vexpress_syscfg
Changes in v2:
- patch added
---
 xen/arch/arm/Kconfig |   2 -
 xen/arch/arm/platforms/vexpress.c|  35 
 xen/drivers/video/Kconfig|   3 -
 xen/drivers/video/Makefile   |   1 -
 xen/drivers/video/arm_hdlcd.c| 281 ---
 xen/include/asm-arm/platforms/vexpress.h |   6 -
 6 files changed, 328 deletions(-)
 delete mode 100644 xen/drivers/video/arm_hdlcd.c

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 8174c0c..4dc7ef5 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -17,12 +17,10 @@ config ARM_64
 config ARM
def_bool y
select HAS_ALTERNATIVE
-   select HAS_ARM_HDLCD
select HAS_DEVICE_TREE
select HAS_MEM_ACCESS
select HAS_PASSTHROUGH
select HAS_PDX
-   select VIDEO
 
 config ARCH_DEFCONFIG
string
diff --git a/xen/arch/arm/platforms/vexpress.c 
b/xen/arch/arm/platforms/vexpress.c
index 70839d6..b6193f7 100644
--- a/xen/arch/arm/platforms/vexpress.c
+++ b/xen/arch/arm/platforms/vexpress.c
@@ -59,41 +59,6 @@ static inline int vexpress_ctrl_start(uint32_t *syscfg, int 
write,
 return 0;
 }
 
-int vexpress_syscfg(int write, int function, int device, uint32_t *data)
-{
-uint32_t *syscfg = (uint32_t *) FIXMAP_ADDR(FIXMAP_MISC);
-int ret = -1;
-
-set_fixmap(FIXMAP_MISC, maddr_to_mfn(V2M_SYS_MMIO_BASE),
-   PAGE_HYPERVISOR_NOCACHE);
-
-if ( syscfg[V2M_SYS_CFGCTRL/4] & V2M_SYS_CFG_START )
-goto out;
-
-/* clear the complete bit in the V2M_SYS_CFGSTAT status register */
-syscfg[V2M_SYS_CFGSTAT/4] = 0;
-
-if ( write )
-{
-/* write data */
-syscfg[V2M_SYS_CFGDATA/4] = *data;
-
-if ( vexpress_ctrl_start(syscfg, write, function, device) < 0 )
-goto out;
-} else {
-if ( vexpress_ctrl_start(syscfg, write, function, device) < 0 )
-goto out;
-else
-/* read data */
-*data = syscfg[V2M_SYS_CFGDATA/4];
-}
-
-ret = 0;
-out:
-clear_fixmap(FIXMAP_MISC);
-return ret;
-}
-
 /*
  * TODO: Get base address from the device tree
  * See arm,vexpress-reset node
diff --git a/xen/drivers/video/Kconfig b/xen/drivers/video/Kconfig
index 52e8ce6..41ca503 100644
--- a/xen/drivers/video/Kconfig
+++ b/xen/drivers/video/Kconfig
@@ -11,6 +11,3 @@ config VGA
  Enable VGA output for the Xen hypervisor.
 
  If unsure, say Y.
-
-config HAS_ARM_HDLCD
-   bool
diff --git a/xen/drivers/video/Makefile b/xen/drivers/video/Makefile
index 2bb91d6..2b3fc76 100644
--- a/xen/drivers/video/Makefile
+++ b/xen/drivers/video/Makefile
@@ -4,4 +4,3 @@ obj-$(CONFIG_VIDEO) += font_8x16.o
 obj-$(CONFIG_VIDEO) += font_8x8.o
 obj-$(CONFIG_VIDEO) += lfb.o
 obj-$(CONFIG_VGA) += vesa.o
-obj-$(CONFIG_HAS_ARM_HDLCD) += arm_hdlcd.o
diff --git a/xen/drivers/video/arm_hdlcd.c b/xen/drivers/video/arm_hdlcd.c
deleted file mode 100644
index e1174b2..000
--- a/xen/drivers/video/arm_hdlcd.c
+++ /dev/null
@@ -1,281 +0,0 @@
-/*
- * xen/drivers/video/arm_hdlcd.c
- *
- * Driver for ARM HDLCD Controller
- *
- * Stefano Stabellini 
- * Copyright (c) 2013 Citrix Systems.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include "font.h"
-#include "lfb.h"
-#include "modelines.h"
-
-#define HDLCD ((volatile uint32_t *) FIXMAP_ADDR(FIXMAP_MISC))
-
-#define HDLCD_INTMASK   (0x18/4)
-#define HDLCD_FBBASE(0x100/4)
-#define HDLCD_LINELENGTH(0x104/4)
-#define HDLCD_LINECOUNT (0x108/4)
-#define HDLCD_LINEPITCH (0x10C/4)
-#define HDLCD_BUS   (0x110/4)
-#define HDLCD_VSYNC (0x200/4)
-#define HDLCD_VBACK (0x204/4)
-#define HDLCD_VDATA (0x208/4)
-#define HDLCD_VFRONT(0x20C/4)
-#define HDLCD_HSYNC (0x210/4)
-#define HDLCD_HBACK (0x214/4)
-#define HDLCD_HDATA (0x218/4)
-#define HDLCD_HFRONT(0x21C/4)
-#define HDLCD_POLARITIES(0x220/4)
-#define HDLCD_COMMAND   (0x230/4)
-#define HDLCD_PF(0x240/4)

[Xen-devel] [PATCH v6 0/13] arm: more kconfig configurability and small default configs

[Stefano Stabellini]

Hi all,

This patch series is the first step toward building a small certifiable
Xen hypervisor for ARM boards.

The series makes a few changes to allow disabling more kconfig options:
most of them already exist but cannot be disabled. It also introduces a
reference kconfig for Renesas RCar (due to popular demand, candidate for
certifications), Xilinx MPSoC, and for QEMU aarch64 (not for
certifications, but useful for debugging).

The last three patches clarify and make changes to the security support
status of kconfig options in Xen. We might want to merge them into
previous patches, or move them earlier in the series. I am happy to do
that once we settled on the required wording for SUPPORT.md.

Cheers,

Stefano


Stefano Stabellini (13):
  arm: remove the ARM HDLCD driver
  arm: make it possible to disable HAS_GICV3
  arm: rename HAS_GICV3 to GICV3
  Make MEM_ACCESS configurable
  make it possible to enable/disable UART drivers
  arm: make it possible to disable the SMMU driver
  arm: add a tiny kconfig configuration
  arm: add ALL, QEMU, Rcar3 and MPSoC configs
  xen: add per-platform defaults for NR_CPUS
  xen: add cloc target
  xen: support the Null scheduler
  xen: specify support for EXPERT and DEBUG Kconfig options
  xen: clarify the security-support status of Kconfig options on ARM

 SUPPORT.md   |  19 ++-
 tools/firmware/xen-dir/shim.config   |   2 +-
 xen/Makefile |  12 ++
 xen/arch/Kconfig |   3 +
 xen/arch/arm/Kconfig |  17 +-
 xen/arch/arm/Makefile|   4 +-
 xen/arch/arm/configs/tiny64.conf |  43 +
 xen/arch/arm/platforms/Kconfig   |  55 ++
 xen/arch/arm/platforms/Makefile  |   2 +-
 xen/arch/arm/platforms/vexpress.c|  35 
 xen/arch/arm/vgic.c  |   2 +-
 xen/arch/arm/vgic/vgic.c |   2 +-
 xen/arch/x86/Kconfig |   2 +-
 xen/common/Kconfig   |  12 +-
 xen/common/Makefile  |   2 +-
 xen/common/domctl.c  |   2 +-
 xen/drivers/char/Kconfig |  15 +-
 xen/drivers/passthrough/Kconfig  |  12 ++
 xen/drivers/passthrough/arm/Makefile |   2 +-
 xen/drivers/video/Kconfig|   3 -
 xen/drivers/video/Makefile   |   1 -
 xen/drivers/video/arm_hdlcd.c| 281 ---
 xen/include/asm-arm/gic.h|   4 +-
 xen/include/asm-arm/platforms/vexpress.h |   6 -
 xen/include/asm-arm/vgic.h   |   4 +-
 xen/include/xen/mem_access.h |   4 +-
 xen/include/xsm/dummy.h  |   2 +-
 xen/include/xsm/xsm.h|   4 +-
 xen/xsm/dummy.c  |   2 +-
 xen/xsm/flask/hooks.c|   4 +-
 30 files changed, 194 insertions(+), 364 deletions(-)
 create mode 100644 xen/arch/arm/configs/tiny64.conf
 create mode 100644 xen/arch/arm/platforms/Kconfig
 delete mode 100644 xen/drivers/video/arm_hdlcd.c

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v6 10/13] xen: add cloc target

[Stefano Stabellini]

Add a Xen build target to count the lines of code of the source files
built. Uses `cloc' to do the job.

With Xen on ARM taking off in embedded, IoT, and automotive, we are
seeing more and more uses of Xen in constrained environments. Users and
system integrators want the smallest Xen and Dom0 configurations. Some
of these deployments require certifications, where you definitely want
the smallest lines of code count. I provided this patch to give us the
lines of code count for that purpose.

Use the .o.d files to account for all the built source files. Generate a
list for the `cloc' utility and invoke `cloc'.

Signed-off-by: Stefano Stabellini 
Acked-by: Jan Beulich 
CC: jbeul...@suse.com
CC: andrew.coop...@citrix.com
---
Changes in v4:
- use grep regex to get multiple source files from .d files

Changes in v3:
- remove build as dependecy for the cloc target

Changes in v2:
- change implementation to use .o.d to find built source files
---
 xen/Makefile | 12 
 1 file changed, 12 insertions(+)

diff --git a/xen/Makefile b/xen/Makefile
index 62d479c..338d5a3 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -267,3 +267,15 @@ $(KCONFIG_CONFIG):
 include/config/auto.conf.cmd: ;
 
 -include $(BASEDIR)/include/config/auto.conf.cmd
+
+.PHONY: cloc
+cloc:
+   $(eval tmpfile := $(shell mktemp))
+   $(foreach f, $(shell find $(BASEDIR) -name *.o.d), \
+   $(eval path := $(dir $(f))) \
+   $(eval names := $(shell grep -o "[a-zA-Z0-9_/-]*\.[cS]" $(f))) \
+   $(foreach sf, $(names), \
+   $(shell if test -f $(path)/$(sf) ; then echo 
$(path)/$(sf) >> $(tmpfile); fi;)))
+   cloc --list-file=$(tmpfile)
+   rm $(tmpfile)
+
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 0/10] arm: more kconfig configurability and small default configs

[Stefano Stabellini]

On Tue, 12 Jun 2018, Julien Grall wrote:
> > diff --git a/SUPPORT.md b/SUPPORT.md
> > index 264b23f..e70f35c 100644
> > --- a/SUPPORT.md
> > +++ b/SUPPORT.md
> > @@ -16,6 +16,18 @@ for the definitions of the support status levels etc.
> > # Feature Support
> >   +## Kconfig
> > +
> > +On x86, Kconfig options that depend on CONFIG_EXPERT are not security
> > +supported. Other Kconfig options that do not depend on CONFIG_EXPERT are
> > +supported, if the related features marked as supported in this document.
> > +
> > +On ARM, a wider range of Kconfig configurations is available to enable
> > +very small lines of code counts in the hypervisor. Not all possible
> > +combinations of kconfig options are security supported. Instead, a small
> > +set of pre-canned configurations is supported, see xen/arch/arm/configs.
> 
> I think we need to be more specific about CONFIG_EXPERT=y. This is still
> something we don't want to security support on Arm.

Agreed, I'll clarify.


> Furthermore, tiny.config by default will select the platform "ALL" but most of
> the user will tailor to a specific platform. That platform will select
> drivers. By reading your statement, this new config will not be security
> supported. Not sure if it is wanted.

It was easier to explain when we actually had one config file per
platform under xen/arch/arm/configs. I have rewritten the statement to
make it clear that we support the platforms listed under
xen/arch/arm/platforms/Kconfig and the precanned configurations under
xen/arch/arm/configs. Let's see how it goes.


> This also made me realize that in your tiny config you select NULL scheduler
> which is still marked as experimental in the Kconfig. It feels strange that
> you security support it in the tiny.config but not by default.

Damn. The NULL scheduler is definitely required and it has been in the
tree long enough. I'll add a separate patch for that.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 00/11] Add support for Hygon's Dhyana Family 18h processor

[Borislav Petkov]

On Sat, Jun 09, 2018 at 09:20:10PM +0800, Pu Wen wrote:
> As a new x86 CPU Vendor, Chengdu Haiguang IC Design Co., Ltd (Hygon)
> is a Joint Venture between AMD and Haiguang Information Technology Co.,
> Ltd., and aims at providing high performance x86 processor for China
> server market.
> 
> The first generation Hygon's processor(Dhyana) originates from AMD
> technology and shares most of the architecture with AMD's family 17h,
> but with different CPU Vendor ID("HygonGenuine")/PCIE Device Vendor ID
> (0x1D94)/Family series number(Family 18h).
> 
> To enable the support of Linux kernel to Hygon's CPU, we added a new
> vendor type (X86_VENDOR_HYGON, with value of 9) in arch/x86/include/
> asm/processor.h, and shared most of kernel support codes with AMD
> family 17h.
> 
> These patches have been applied and tested successfully in Hygon's
> Dhyana SoC silicon. Also tested on AMD's EPYC (Family 17h) processor
> works fine and makes no harm to existing codes.

Well, I don't like this diffstat:

 37 files changed, 183 insertions(+), 56 deletions(-)

which adds a lot of code checking this new vendor. But then it adds in
the AMD paths and I don't see it being any different from an AMD CPU. So
it does the same a Zen does but then it is Hygon.

So I'd prefer to *not* sprinkle those X86_VENDOR_HYGON checks everywhere
but simply have the vendor be X86_VENDOR_AMD and only the user-visible
reporting to show that it is Hygon. Because to the kernel it is an AMD
CPU - only the superficial attributes are something else. Oh well, and
PCI device IDs but that's like another CPU revision.

Thx.

-- 
Regards/Gruss,
Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 0/10] arm: more kconfig configurability and small default configs

[Stefano Stabellini]

On Wed, 13 Jun 2018, Jan Beulich wrote:
> >>> On 12.06.18 at 21:53,  wrote:
> > On Tue, 12 Jun 2018, Jan Beulich wrote:
> >> >> >> As a consequence of these changes, some options will become 
> >> >> >> user-visible
> >> >> >> and not dependent on CONFIG_EXPERT. It does not mean that Xen Project
> >> >> >> will security support all possible combinations of kconfig options.
> >> >> >> Instead, there will be a small set of pre-canned configurations that
> >> >> >> will be supported.  See: 
> >> >> >> https://marc.info/?l=xen-devel=152424389512432 
> >> >> > 
> >> >> > George, Ian, Jan, shall SUPPORT.MD be updated to reflect the Kconfig 
> >> >> > changes?
> >> >> > 
> >> >> > I am mostly thinking about the board support and the fact that more 
> >> >> > options on Arm are selectable by the users.
> >> >> 
> >> >> I think that would be very desirable, yes.
> >> > 
> >> > Do you want me to add a patch for that to this series, or should I do it
> >> > separately?
> >> 
> >> I think such a doc change should be right in a particular patch making
> >> things user selectable.
> > 
> > I have added the following to patch #5, the one introducing all the UART
> > Kconfigs on ARM. I think it is the one introducing more new options. I
> > removed Julien's ACK because of this change. Let me know if you think we
> > need more details in SUPPORT.md.
> > 
> > diff --git a/SUPPORT.md b/SUPPORT.md
> > index 264b23f..e70f35c 100644
> > --- a/SUPPORT.md
> > +++ b/SUPPORT.md
> > @@ -16,6 +16,18 @@ for the definitions of the support status levels etc.
> >  
> >  # Feature Support
> >  
> > +## Kconfig
> > +
> > +On x86, Kconfig options that depend on CONFIG_EXPERT are not security
> > +supported. Other Kconfig options that do not depend on CONFIG_EXPERT are
> > +supported, if the related features marked as supported in this document.
> 
> ..., if the related features are marked ...
> 
> > +On ARM, a wider range of Kconfig configurations is available to enable
> > +very small lines of code counts in the hypervisor. Not all possible
> > +combinations of kconfig options are security supported. Instead, a small
> > +set of pre-canned configurations is supported, see xen/arch/arm/configs.
> 
> Patch 5 does not add any EXPERT dependencies afaics, so this is at least
> misleading. I think the EXPERT rule should apply generically, and perhaps be
> introduced by (and discussed in the context of) a separate patch. I also
> think DEBUG should be mentioned alongside EXPERT.
> 
> The patch relaxing things for ARM would then add a relaxation paragraph
> here.

I'll do.

Actually, for simplicity, I'll modify the SUPPORT statement for ARM in a
separate independent patch (so I'll add two patches) for our convenience
in reviewing and patch handling. We can easily merge patches at commit
time, or in a follow-up patch series.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 124152: regressions - FAIL

[osstest service owner]

flight 124152 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124152/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 124093
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 16 guest-localmigrate/x10 fail 
REGR. vs. 124093

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 124093
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 124093
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 124093
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 124093
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 124093
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 124093
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 qemuu2ab09bf2f9f55b9fb8d2de6eb2ba2a8570e268e2
baseline version:
 qemuu2afc4e3df80d947dd1bd42ce80278f591b35c74a

Last test of basis   124093  2018-06-12 03:58:14 Z1 days
Testing same since   124152  2018-06-12 22:37:40 Z0 days1 attempts


People who touched revisions under test:
  Babu Moger 
  BALATON Zoltan 
  Bandan Das 
  Cédric Le Goater 
  David Gibson 
  David Hildenbrand 
  Eduardo Habkost 
  Fam Zheng 
  Gerd Hoffmann 
  Greg Kurz 
  Igor Mammedov 
  Joel Stanley 
  John Snow 
  Leandro Lupori 
  luporl 
  Marc-André Lureau 
  Mark Cave-Ayland 
  Michael S. Tsirkin 
  Nicholas Piggin 
  Paolo Bonzini 
  Peter Maydell 
  Philippe Mathieu-Daudé 
  Ross Zwisler 
  Suraj Jitindar Singh 
  Thomas Huth 
  Vladimir Sementsov-Ogievskiy 

jobs:
 

Re: [Xen-devel] [Qemu-devel] [PATCH v4 11/40] hw/xen: Use the IEC binary prefix definitions

[Richard Henderson]

On 06/13/2018 02:13 AM, Eric Blake wrote:
> Or spell it UINT64_C(1) if you don't want a cast.

Not unsigned is what I want most.


r~

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [libvirt test] 124159: regressions - FAIL

[osstest service owner]

flight 124159 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124159/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 123814
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 123814
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 123814
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 123814

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  9f1b1194b031d528f4b376371282ed61279165b0
baseline version:
 libvirt  076a2b409667dd9f716a2a2085e1ffea9d58fe8b

Last test of basis   123814  2018-06-05 04:19:23 Z8 days
Failing since123840  2018-06-06 04:19:28 Z7 days8 attempts
Testing same since   124159  2018-06-13 04:19:09 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Anya Harter 
  Brijesh Singh 
  Chen Hanxiao 
  Christian Ehrhardt 
  Cole Robinson 
  Daniel Nicoletti 
  Daniel P. Berrangé 
  Fabiano Fidêncio 
  Filip Alac 
  intrigeri 
  intrigeri 
  Jamie Strandboge 
  Jiri Denemark 
  John Ferlan 
  Julio Faracco 
  Ján Tomko 
  Katerina Koukiou 
  Laszlo Ersek 
  Marc Hartmayer 
  Martin Kletzander 
  Michal Privoznik 
  Peter Krempa 
  Radostin Stoyanov 
  Ramy Elkest 
  ramyelkest 
  Roman Bogorodskiy 
  Stefan Bader 
  Stefan Berger 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  fail
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm blocked 
 test-arm64-arm64-libvirt-xsm blocked 
 test-armhf-armhf-libvirt-xsm blocked 
 test-amd64-i386-libvirt-xsm  blocked 
 test-amd64-amd64-libvirt blocked 
 test-arm64-arm64-libvirt blocked 
 test-armhf-armhf-libvirt blocked 
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-libvirt-pairblocked 
 test-amd64-i386-libvirt-pair blocked 
 test-arm64-arm64-libvirt-qcow2   blocked 
 test-armhf-armhf-libvirt-raw blocked 
 test-amd64-amd64-libvirt-vhd blocked 



sg-report-flight on 

Re: [Xen-devel] [RFC PATCH 01/12] xen/manage: keep track of the on-going suspend mode

[Balbir Singh]

On Wed, Jun 13, 2018 at 6:56 AM, Anchal Agarwal  wrote:
> From: Munehisa Kamata 
>
> To differentiate between Xen suspend, PM suspend and PM hibernation,
> keep track of the on-going suspend mode by mainly using a new PM
> notifier. Since Xen suspend doesn't have corresponding PM event, its
> main logic is modfied to acquire pm_mutex and set the current mode.
>

Why do we need to differentiate between them? The changelog does not
explain how Xen Suspend is different from PM suspend. The difference
could be what is injected into the guest vs what the guest decides to
do. How do we use the new suspend_mode?

> Note that we may see deadlock if PM suspend/hibernation is interrupted
> by Xen suspend. PM suspend/hibernation depends on xenwatch thread to
> process xenbus state transactions, but the thread will sleep to wait
> pm_mutex which is already held by PM suspend/hibernation context in the
> scenario. Though, acquirng pm_mutex is still right thing to do, and we
> would need to modify Xen shutdown code to avoid the issue. This will be
> fixed by a separate patch.
>
> Signed-off-by: Munehisa Kamata 
> Signed-off-by: Anchal Agarwal 
> Reviewed-by: Sebastian Biemueller 
> Reviewed-by: Munehisa Kamata 
> Reviewed-by: Eduardo Valentin 
> ---
>  drivers/xen/manage.c | 58 
> 
>  1 file changed, 58 insertions(+)
>
> diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c
> index 8835065..8f9ea87 100644
> --- a/drivers/xen/manage.c
> +++ b/drivers/xen/manage.c
> @@ -13,6 +13,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>
>  #include 
>  #include 
> @@ -39,6 +40,16 @@ enum shutdown_state {
>  /* Ignore multiple shutdown requests. */
>  static enum shutdown_state shutting_down = SHUTDOWN_INVALID;
>
> +enum suspend_modes {
> +   NO_SUSPEND = 0,
> +   XEN_SUSPEND,
> +   PM_SUSPEND,
> +   PM_HIBERNATION,
> +};


Why do the enums range across namespaces -- between NO, XEN and PM?
Can we please be consistent XEN_WATCH_SUSPEND, XEN_PM_SUSPEND. etc?

Balbir Singh.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH 02/12] xen/manage: introduce helper function to know the on-going suspend mode

[Balbir Singh]

On Wed, Jun 13, 2018 at 6:56 AM, Anchal Agarwal  wrote:
> From: Munehisa Kamata 
>
> Introduce simple functions which help to know the on-going suspend mode
> so that other Xen-related code can behave differently according to the
> current suspend mode.

I'd squash this patch with the previous, the previous one just left
too many open questions about how suspend mode is used. Looks like
suspend mode is a state, but I am not sure if valid transitions are
defined/checked?

Balbir Singh.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-linus test] 124151: regressions - FAIL

[osstest service owner]

flight 124151 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124151/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 123554
 test-armhf-armhf-xl   7 xen-boot fail REGR. vs. 123554
 test-armhf-armhf-libvirt-xsm  7 xen-boot fail REGR. vs. 123554
 test-armhf-armhf-xl-multivcpu  7 xen-bootfail REGR. vs. 123554
 test-armhf-armhf-libvirt-raw  7 xen-boot fail REGR. vs. 123554
 test-armhf-armhf-libvirt  7 xen-boot fail REGR. vs. 123554
 test-armhf-armhf-xl-vhd   7 xen-boot fail REGR. vs. 123554
 test-armhf-armhf-xl-credit2   7 xen-boot fail REGR. vs. 123554
 test-armhf-armhf-xl-cubietruck  7 xen-boot   fail REGR. vs. 123554
 test-armhf-armhf-xl-xsm   7 xen-boot fail REGR. vs. 123554

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  7 xen-boot fail REGR. vs. 123554

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 123554
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 123554
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 123554
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 123554
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 123554
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 123554
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 linux19785cf93b6c4252981894394f2dbd35c5e5d1ec
baseline version:
 linux0512e0134582ef85dee77d51aae77dcd1edec495

Last test of basis   123554  2018-06-01 13:09:41 Z   12 days
Failing since123655  2018-06-03 01:45:35 Z   10 days   10 attempts
Testing same since   124151  2018-06-12 22:09:10 Z0 days1 attempts


1942 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  pass
 build-armhf-libvirt  pass

Re: [Xen-devel] How to deal with hypercalls returning -EFAULT

[Andrew Cooper]

On 13/06/18 16:27, Juergen Gross wrote:
> Currently the release of Xen 4.11 is blocked due to a sporadic failure
> of the OSSTEST guest-saverestore[.2]. During that test a hypercall
> issued by libxc via the Linux privcmd driver returns -EFAULT in spite
> of all hypercall buffers locked in memory via mlock() (or similar flags
> specified in a mmap() call).
>
> My analysis has revealed that modern Linux kernels might make such
> locked user pages unaccessible for very short periods of time. This can
> happen e.g. when pages are subject to compaction or migration.
>
> There are multiple ways to mitigate this problem:
>
> 1. Trying to switch page migration or compaction off in dom0.
>Pros: - no change in Xen necessary

Pro: can likely retrofitted to existing environments without further
code changes.

(Not that I disagree with your Con's in this case)

>Cons: - new cases might come up in the future
>  - easy to miss, failures are really very sporadic and might
>happen only after updating the kernel
>
> 2. Add a bandaid to Xen tools by retrying hypercalls which have failed
>with -EFAULT (either for all or only for some hypercalls)
>Pros: - no interface change necessary
>Cons: - not all hypercalls might be just repeatable
>  - problem isn't solved but just worked around

We'd have to whitelist hypercalls which are safe to repeat like this. 
Most wont be.  Any mutable operation which -EFAULTs can't safely be
restarted, because we can't distinguish an early fault (Xen reading the
parameters) from a late fault (Xen trying to update a userspace pointer
with the result).

>
> 3. Modify the interface to the privcmd driver to pass information about
>used buffers to the kernel in order to lock them there. Either add a
>new interface for hypercall buffer management or add the list of
>buffers to the privcmd ioctl data structure.
>Pros: - problem is really solved
>Cons: - split solution between kernel and Xen, both must be changed

To be clear, you mean suggesting changing libxc here, rather than the
hypervisor?

Getting this problem fixed properly would be a distinct improvement over
the whack-a-mole which has been played in the past.

>
> 4. Modify the interface between hypervisor and kernel: instead of just
>returning -EFAULT let the hypervisor behave more like copy_to_user by
>raising a page fault which can then be fixed up in the kernel. This
>change must be activated by the kernel, of course.
>Pros: - rather simple change in the kernel "doing the right thing"
>  - hypercall bounce buffer handling in libxc/libxencall can be
>switched off for a kernel supporting this chnage
>Cons: - split solution between kernel and Xen, both must be changed
>  - not sure how complex the required hypervisor change will be

Sadly, as I've just realised...

Con: Cannot be used to replace all -EFAULTs.

Faults when copying data in can be resolved by passing #PF to the
kernel, but faults when trying to update guest state (continuation, or
completion information) cannot be safely resumed at a later point.

>
> It should be noted that we can either select only one of above solutions
> or one of 3/4 and additionally one of 1/2 as a fallback for old kernels.
>
> How to proceed?

Much as I hate to say it (as I do like this idea), I don't idea 4 is a
viable alternative to 3.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] How to deal with hypercalls returning -EFAULT

[Ian Jackson]

Juergen Gross writes ("How to deal with hypercalls returning -EFAULT"):
> 3. Modify the interface to the privcmd driver to pass information about
>used buffers to the kernel in order to lock them there. Either add a
>new interface for hypercall buffer management or add the list of
>buffers to the privcmd ioctl data structure.
>Pros: - problem is really solved
>Cons: - split solution between kernel and Xen, both must be changed

I think this is the best approach.  There are presumably already
internal kernel interfaces which could be used by privcmd to implement
this.  All that's needed is to decide what the kernel API should look
like and implement it.  libxc doesn't really care very much what that
interface looks like, so it can be whatever is convenient for the
kernel.

> 2. Add a bandaid to Xen tools by retrying hypercalls which have failed
>with -EFAULT (either for all or only for some hypercalls)
>Pros: - no interface change necessary
>Cons: - not all hypercalls might be just repeatable
>  - problem isn't solved but just worked around

This may allow us to make some kind of progress in systems which are
fundamentally broken, but I don't think it is a tolerable long term
approach.

If we do this, we should also do your option (3), and the workaround
should only be enabled if the proper interface is not available.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] uapi header dependencies?

[Jan Beulich]

>>> On 13.06.18 at 17:03,  wrote:
> On 13/06/18 15:31, Jan Beulich wrote:
>> Jürgen, Boris,
>> 
>> aren't headers in uapi/ directories supposed to be self contained?
> 
> Hmm, there are many uapi headers including stuff from outside uapi.
> I guess it is okay to include headers which are accessible in the
> kernel _and_ under /usr/include or to put such includes under
> #ifdef __KERNEL__
> 
>> The addition done by David in fbc872c38c clearly isn't, as there's
>> no domid_t anywhere in other uapi/ headers afaict. No idea how
>> to reasonably adjust this; I'd in particular dislike using __u16 there,
>> but perhaps that's the only option we have.
> 
> include/uapi/xen/privcmd.h is especially evil: it is including the
> kernel-only xen/interface/xen.h

Oh, indeed - apparently in order to obtain domid_t and xen_pfn_t
(which is precisely what is missing from evtchn.h). What a mess.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] uapi header dependencies?

[Juergen Gross]

On 13/06/18 15:31, Jan Beulich wrote:
> Jürgen, Boris,
> 
> aren't headers in uapi/ directories supposed to be self contained?

Hmm, there are many uapi headers including stuff from outside uapi.
I guess it is okay to include headers which are accessible in the
kernel _and_ under /usr/include or to put such includes under
#ifdef __KERNEL__

> The addition done by David in fbc872c38c clearly isn't, as there's
> no domid_t anywhere in other uapi/ headers afaict. No idea how
> to reasonably adjust this; I'd in particular dislike using __u16 there,
> but perhaps that's the only option we have.

include/uapi/xen/privcmd.h is especially evil: it is including the
kernel-only xen/interface/xen.h

Xen tools avoid the resulting problems by using a private copy of that
file without the offending #include. OMG!


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH] cr-daily-branch, cr-publish-flight-logs: Tolerate failure to push harness

[Roger Pau Monné]

On Wed, Jun 13, 2018 at 02:51:56PM +0100, Ian Jackson wrote:
> Provide cr-publish-flight-logs --push-harness-try, which attempts the
> push but doesn't mind if it fails.
> 
> If we are not --real, tolerate failure to publish the flight logs.
> 
> Also, honour OSSTEST_PUSH_HARNESS which might contain '' or
> --push-harness or --push-harness-try.
> 
> CC: Roger Pau Monné 
> Signed-off-by: Ian Jackson 

Just tested this by manually killing a cr-daily-branch and it worked
properly, see:

http://logs.test-lab.xenproject.org/osstest/logs/124171/

So:

Tested-by: Roger Pau Monné 

Thanks!

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [osstest] add FreeBSD flight

[Roger Pau Monné]

Hello,

I've run a test flight of my FreeBSD osstest series today, the flight
shows all green:

http://logs.test-lab.xenproject.org/osstest/logs/124163/

The series can be found at:

git://xenbits.xen.org/people/royger/osstest.git freebsd_v18

AFAICT it's fully Acked. I've rebased it on top of current osstest
master branch. I think it's ready to go in unless there are
objections.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH] cr-daily-branch, cr-publish-flight-logs: Tolerate failure to push harness

[Ian Jackson]

Provide cr-publish-flight-logs --push-harness-try, which attempts the
push but doesn't mind if it fails.

If we are not --real, tolerate failure to publish the flight logs.

Also, honour OSSTEST_PUSH_HARNESS which might contain '' or
--push-harness or --push-harness-try.

CC: Roger Pau Monné 
Signed-off-by: Ian Jackson 
---
 cr-publish-flight-logs | 10 --
 cri-args-hostlists |  5 -
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/cr-publish-flight-logs b/cr-publish-flight-logs
index 45545ce..1e5a49d 100755
--- a/cr-publish-flight-logs
+++ b/cr-publish-flight-logs
@@ -27,9 +27,9 @@ our %c;
 readglobalconfig();
 
 my $push_harness = 0;
-if (@ARGV && $ARGV[0] eq '--push-harness') {
+if (@ARGV && $ARGV[0] =~ m{^--push-harness(?:-try)?$}) {
+$push_harness = $&;
 shift @ARGV;
-$push_harness = 1;
 }
 
 my $flight= shift @ARGV // '';
@@ -46,8 +46,14 @@ if ($push_harness) {
 
 exit 0 if (!$githost || !$gitdir);
 
+eval {
 system_checked("git push $githost:$gitdir HEAD:refs/heads/flight-$flight");
 system_checked("ssh $githost 'cd $gitdir && git update-server-info'");
+};
+if ($@) {
+   $push_harness eq '--push-harness-try' or die;
+   warn;
+}
 }
 
 sub copydir ($$) {
diff --git a/cri-args-hostlists b/cri-args-hostlists
index a75ff7b..7d23087 100644
--- a/cri-args-hostlists
+++ b/cri-args-hostlists
@@ -51,6 +51,8 @@ elif [ "x$1" = "x--like-real" ]; then
: ${OSSTEST_CRON_SETTINGS:=$dcs-real}
: ${OSSTEST_HTML_SUFFIX:=-play}
shift
+else
+   : ${OSSTEST_PUSH_HARNESS=--push-harness-try}
 fi 
 : ${OSSTEST_BLESSING:=play}
 
@@ -127,7 +129,8 @@ start_email () {
 publish_send_email () {
local flight=$1
exec >&2
-   ./cr-publish-flight-logs --push-harness $flight
+   ./cr-publish-flight-logs ${OSSTEST_PUSH_HARNESS- --push-harness} \
+   $flight
send_email tmp/$flight.email
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] uapi header dependencies?

[Jan Beulich]

Jürgen, Boris,

aren't headers in uapi/ directories supposed to be self contained?
The addition done by David in fbc872c38c clearly isn't, as there's
no domid_t anywhere in other uapi/ headers afaict. No idea how
to reasonably adjust this; I'd in particular dislike using __u16 there,
but perhaps that's the only option we have.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 03/17] cs-bisection-step: Refactor $subjobs calculations a bit

[Ian Jackson]

Parse the runvar name earlier, adding job and orgflight members to the
row hashes we got from the db.  This slightly unifies the call to
preparejob, but more relevantly, makes the effective job and flight
information available earlier.  That will be useful in a moment.

No functional change.

Signed-off-by: Ian Jackson 
---
 cs-bisection-step | 18 --
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/cs-bisection-step b/cs-bisection-step
index 0d0e08f..01ee6e5 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1197,6 +1197,17 @@ END
 my $subjobs= $jobq->fetchall_arrayref( {} );
 $jobq->finish();
 
+foreach my $subjob (@$subjobs) {
+my $jobspec= $subjob->{val};
+if ($jobspec =~ m/^(\d+)\.(.+)$/) {
+$subjob->{job}   = $2;
+$subjob->{orgflight} = $1;
+} else {
+$subjob->{job}   = $jobspec;
+$subjob->{orgflight} = $copyflight;
+}
+}
+
 # See if there's a build we can reuse
 
 my ($recipe) = $dbh_tests->selectrow_array(<{val};
-if ($jobspec =~ m/^(\d+)\.(.+)$/) {
-$target= preparejob($2, $1, 1);
-} else {
-$target= preparejob($jobspec, $copyflight, 1);
-}
+$target= preparejob($subjob->{job}, $subjob->{orgflight}, 1);
 $jobsetq->execute($target, $popflight, $popjob, $subjob->{name});
 }
 $jobsetq->finish();
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 14/17] dm restrict audit: install newer chiark-scripts for fishdescriptor

[Ian Jackson]

fishdescriptor is not in stretch or earlier.  It will be in buster,
and we expect that a suitable version will be available in
stretch-backports soon.

For jessie, use DebianExtraPackages to install the .deb from buster
(which is directly installable on jessie).

Deployment note: I have already copied the .deb to the images
directories in Massachusetts and Cambridge.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm   | 5 +
 production-config   | 2 ++
 production-config-cambridge | 2 ++
 3 files changed, 9 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 9789ca6..2c1e6ed 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1023,6 +1023,11 @@ sub preseed_create_guest ($$$;@) {
 $extra_packages = "pv-grub-menu";
 }
 }
+if (grep { m/_dmrestrict$/ && $r{$_} } keys %r and
+   $suite =~ m/stretch/) {
+   preseed_backports_packages($ho, $sfx, \%xopts, $suite,
+  qw(chiark-utils-bin));
+}
 
 my $preseed_file= preseed_base($ho, $sfx, $extra_packages, %xopts);
 $preseed_file.= (

[Xen-devel] [OSSTEST PATCH 15/17] dm restrict audit: Provide auditing script

[Ian Jackson]

ts-depriv-audit-qemu knows how to create a domain paused, and audit
its fds.  It uses
  * osstest-depriv-fd-collector, an on-test-host helper script
  * fishdescriptor, a new utility program in chiark-scripts
  * depriv-fd-checker, a new test program in xen.git

Signed-off-by: Ian Jackson 
---
 overlay/usr/local/bin/osstest-depriv-fd-collector |  94 +
 ts-depriv-audit-qemu  | 154 ++
 2 files changed, 248 insertions(+)
 create mode 100755 overlay/usr/local/bin/osstest-depriv-fd-collector
 create mode 100755 ts-depriv-audit-qemu

diff --git a/overlay/usr/local/bin/osstest-depriv-fd-collector 
b/overlay/usr/local/bin/osstest-depriv-fd-collector
new file mode 100755
index 000..b7444f8
--- /dev/null
+++ b/overlay/usr/local/bin/osstest-depriv-fd-collector
@@ -0,0 +1,94 @@
+#!/usr/bin/perl -w
+#
+# usage: osstest-depriv-fd-collector 
+#
+# audits that  has only depriv fds, eg for a depriv qemu
+#
+# output is series of lines
+# pass|failrepeated for each fd
+#[]  sockinfo...   as from fishdescriptor
+#generic unknown  
+#
+# fishdescriptor on every fd in 
+#check what kind of fd it is
+#
+# classes are
+#   generic
+#  (handled here)
+#   appendonly
+#   readonly
+#   privcmd
+#   gntdev
+#   evtchn
+#  (passed to fishdescriptor and depriv-fd-checker)
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2018 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict;
+
+die unless @ARGV==2;
+our ($pid,$checker) = @ARGV;
+die unless $pid =~ m{^\d+$};
+
+our @fdchecks;
+our @fdsockinfos;
+
+open F, "find /proc/$pid/fd/* -printf '%f %l\\n' |" or die $!;
+while () {
+chomp;
+die "$_ ?" unless s/^(\d+) //;
+my $fd = $1;
+if (m{^/dev/null$} ||
+   m{^pipe:} ||
+m{^anon_inode:\[(?:signalfd|eventfd|eventpoll)\]$}) {
+   print "generic pass $fd $_\n" or die $!;
+} elsif (m{^/var/log/xen/qemu-dm-.*\.log$}) {
+   push @fdchecks, [ $fd, 'appendonly', $_ ];
+} elsif (m{^/dev/urandom$} ||
+m{^/root/.*.iso}) {
+   push @fdchecks, [ $fd, 'readonly', $_ ];
+} elsif (m{^/dev/xen/(privcmd|gntdev|evtchn)$}) {
+   push @fdchecks, [ $fd, $1, $_ ];
+} elsif (m{^/dev/net/tun$}) {
+   push @fdchecks, [ $fd, 'tun', $_ ];
+} elsif (m{^socket:}) {
+   push @fdsockinfos, $fd;
+} else {
+   print "generic unknown $fd $_\n" or die $!;
+}
+}
+close F or die "$? $!";
+
+my @fish = (qw(fishdescriptor), $pid);
+my @chk = $checker;
+
+foreach my $fd (@fdsockinfos) {
+push @fish, $fd, qw(sockinfo);
+}
+
+foreach my $fdi (@fdchecks) {
+my ($fd, $class, $rhs) = @$fdi;
+my $fish_fd = $fd + 100;
+push @fish, "$fish_fd=$fd";
+push @chk, $class, $fish_fd, "$fd=$rhs";
+}
+
+my @cmd = (@fish, qw(exec), @chk);
+
+print STDERR "$0 running @cmd\n";
+
+exec @cmd or die $!;
diff --git a/ts-depriv-audit-qemu b/ts-depriv-audit-qemu
new file mode 100755
index 000..2405b69
--- /dev/null
+++ b/ts-depriv-audit-qemu
@@ -0,0 +1,154 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2018 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use DBI;
+BEGIN { unshift @INC, qw(.); }
+use Osstest;
+use Osstest::TestSupport;
+
+use Data::Dumper;
+$Data::Dumper::Useqq = 1;
+
+tsreadconfig();
+
+our $mode = shift @ARGV;
+our $modesubproc = ${*::}{"mode_$mode"};
+die "unknown mode $mode ?" unless $modesubproc;
+
+our ($ho,$gho) = ts_get_host_guest(@ARGV);
+
+
+our $data_re;
+our $fish_output;
+
+sub compile_data_re () {
+$data_re = join '|', map { chomp; qr{$_}; } ;
+}  
+
+sub fish_guest () {

[Xen-devel] [OSSTEST PATCH 13/17] toolstack: Provide guest_unpause

[Ian Jackson]

Only for xl (and xm) for now.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm  | 7 +++
 Osstest/Toolstack/xl.pm | 8 
 2 files changed, 15 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 2e0e892..0c57dd7 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -115,6 +115,7 @@ BEGIN {
   guest_find_domid guest_check_up guest_check_up_quick
   guest_get_state guest_await_reboot
   guest_await_shutdown guest_await_destroy guest_destroy
+ guest_unpause
   guest_vncsnapshot_begin guest_vncsnapshot_stash
  guest_check_remus_ok guest_editconfig
   guest_prepare_disk guest_unprepare_disk
@@ -1689,6 +1690,12 @@ sub guest_destroy ($) {
 guest_unprepare_disk($gho);
 }
 
+sub guest_unpause ($) {
+my ($gho) = @_;
+my $ho = $gho->{Host};
+toolstack($ho)->unpause($gho);
+}
+
 sub guest_await_destroy ($$) {
 my ($gho, $timeout) = @_;
 my $ho = $gho->{Host};
diff --git a/Osstest/Toolstack/xl.pm b/Osstest/Toolstack/xl.pm
index d3e3b0c..d31af8c 100644
--- a/Osstest/Toolstack/xl.pm
+++ b/Osstest/Toolstack/xl.pm
@@ -58,6 +58,14 @@ sub create_paused ($$) {
 my $cfg = $gho->{CfgPath};
 }
 
+sub unpause ($$) {
+my ($self,$gho) = @_;
+my $gn = $gho->{Name};
+target_cmd_root($self->{Host},
+   $self->{_VerboseCommand}." unpause $gn", 100);
+
+}
+
 sub consolecmd ($$) {
 my ($self,$gho) = @_;
 my $gn = $gho->{Name};
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 10/17] make-flight: Add a single test with restricted dm

[Ian Jackson]

As of this commit, this test is only functional: we don't do any
auditing yet.

Right now save/restore migration is broken with restricted dm, so
disable that test with the recipe flag.  See
  sg-run-job: Allow flight to specify recipe flag to disable migr tests

Run this only on recent enough Xen: the dm_restrict option was added
as an experimental feature in Xen 4.10 (currently as yet unreleased).
(With older libxl/xl, the option would be ignored and the test would
be a waste of time.)

Signed-off-by: Ian Jackson 
---
 make-flight | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/make-flight b/make-flight
index c29a612..ebc9156 100755
--- a/make-flight
+++ b/make-flight
@@ -442,6 +442,20 @@ do_hvm_debian_tests() {
   # For libvirt we only generate -xsm test case.
   if [ "x$qemuu_suffix" == "x-qemuu" ]; then
 do_hvm_debian_test_one ovmf xl ovmf false
+
+case $xenbranch in
+  xen-3.*-testing) dmrestrict=false ;;
+  xen-4.[0-9]-testing) dmrestrict=false ;;
+  *)   dmrestrict=true ;;
+esac
+
+if $dmrestrict; then
+  do_hvm_debian_test_one dmrestrict xl seabios false '' -dmrestrict '
+   debianhvm_dmrestrict=true
+   recipe_nomigrate=true
+   '
+fi
+
 for xsm in $xsms ; do
   do_hvm_debian_test_one debianhvm xl seabios $xsm
   if [ x$xsm = xtrue ]; then
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 07/17] sg-run-job: Allow flight to specify recipe flag to disable migr tests

[Ian Jackson]

We want this because there is no sensible way to probe whether a
restricted qemu can cope with save/restore or migration.  So we will
want to disable it in flight construction (depending on Xen version,
eventually).

Signed-off-by: Ian Jackson 
---
 sg-run-job | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 1355876..965e55d 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -591,6 +591,9 @@ proc run-job/test-livepatch {} {
 }
 
 proc test-guest-migr {g} {
+# guest is expected to be running and remains so
+if {[recipe-flag nomigrate]} return
+
 set to_reap [spawn-ts . = ts-migrate-support-check + host $g 1]
 set can_migrate [reap-ts $to_reap]
 set to_reap [spawn-ts . = ts-saverestore-support-check + host $g]
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 02/17] cs-bisection-step: Improve a message

[Ian Jackson]

Signed-off-by: Ian Jackson 
---
 cs-bisection-step | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cs-bisection-step b/cs-bisection-step
index 0be8bd0..0d0e08f 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1208,7 +1208,7 @@ END
 if ($cache_option and $cacheok and $recipe =~ m/^build/ and !@$subjobs) {
my $reusejob= $builds_investigated{$popjob};
if (!defined $reusejob) {
-   print STDERR "Searching for $popflight.$popjob to reuse...\n";
+   print STDERR "Searching for $popjob (like $copyflight) to 
reuse...\n";
$reusejob =
$dbh_tests->selectrow_hashref(

[Xen-devel] [OSSTEST PATCH 17/17] dm restrict audit: Document future plans

[Ian Jackson]

Signed-off-by: Ian Jackson 
---
 ts-depriv-audit-qemu | 16 
 1 file changed, 16 insertions(+)

diff --git a/ts-depriv-audit-qemu b/ts-depriv-audit-qemu
index 2405b69..81bd5c0 100755
--- a/ts-depriv-audit-qemu
+++ b/ts-depriv-audit-qemu
@@ -56,6 +56,9 @@ END
 /usr/local/lib/xen/bin/depriv-fd-checker
 END
 stashfilecontents($fish_output,"fish-info-paused.txt");
+
+# Ideally we would check other process properties too:
+# eg, check that qemu has chrooted; check its uid; etc.
 }
 
 sub packages () {
@@ -139,6 +142,19 @@ sub mode_ispaused () {
 audit_fish();
 }
 
+# In the future when migration works, we would like to audit the qemu
+# receiving the migration stream.  This auditing should be done just
+# before the qemu starts reading the stream, as the stream might be
+# hostile and might be able to take over the receiving qemu.
+# I intend the following approach:
+#install wrapper script for qemu, which:
+#  looks for  -incoming fd:%d   (libxl_dm.c:1416)
+#   substitutes a pipe which will cause qemu to block
+#   waits a fixed time
+#  maybe checks that qemu is reading that fd somehow
+#  runs the audit and leaves the output somewhere we can find it
+#  arranges for the blocking pipe thing to use cat to unblock qemu
+
 compile_data_re();
 $modesubproc->();
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 04/17] cs-bisection-step: Handle build job chains

[Ian Jackson]

cs-bisection-step assumes that every job it will need to create has a
unique name.  Indeed, in general, it is probably not reasonable to
expect it to work if that is not the case.

build-amd64-freebsd needs a previous build-amd64-freebsd.

Currently cs-bisection-step confuses itself into writing a flight
where build-amd64-freebsd uses itself as its freebsdbuildjob.
This naturally does not work very well.

I think the right approach is for cs-bisection-step to spot when it
its recursion through the jobs, via *job runvars, would descend into a
job name which it was already encoutered earlier in the dependency
chain, and, when that occurs, to simply skip regenerating that deeper
copy of the job.

That is achieved here by filtering the subjob out of the list, before
we go looking for jobs to reuse.  As a result, not only will no new
job be created, but the original deeper job will be reused because the
runvar value will not be updated.

FTR, the circular dependency produces this error from sg-execute-flight:

  wait for process failed: no children
  while executing
  "wait -nohang"
  (procedure "main_iteration" line 14)
  invoked from within
  "main_iteration"
  (procedure "main" line 8)
  invoked from within
  "main"
  (file "./sg-execute-flight" line 238)

This is because sg-execute-flight's algorithm assumes that if there
are no jobs running there must be some job whose dependencies are done.
That is true if the job graph has no cycles.

Improving the error message from sg-execute-flight is left as an
exercise for the future.

Signed-off-by: Ian Jackson 
---
 cs-bisection-step | 21 +
 1 file changed, 21 insertions(+)

diff --git a/cs-bisection-step b/cs-bisection-step
index 01ee6e5..a7e0336 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1133,6 +1133,7 @@ END
 
 our %jobs_created;
 our %builds_investigated; # $builds_investigated{$popjob} = 0, or {..row..}
+our %recursion_track;
 
 sub preparejob ($$$);
 sub preparejob ($$$) {
@@ -1144,6 +1145,11 @@ sub preparejob ($$$) {
return $jobs_created{$popjob};
 }
 
+local $recursion_track{$popjob} = {
+Depth => (1 + keys %recursion_track),
+Spec => "$copyflight.$popjob"
+};
+
 print STDERR "Need $popflight.$popjob (like $copyflight)\n";
 
 # Create a temporary table containing the runvars we want
@@ -1208,6 +1214,21 @@ END
 }
 }
 
+@$subjobs = grep {
+if ($recursion_track{ $_->{job} }) {
+print STDERR "Not recursively creating another $_->{job} (".
+(
+ join " -> ",
+ map { $_->{Spec} }
+ sort { $a->{Depth} <=> $b->{Depth} }
+ values %recursion_track
+). " -> $_->{orgflight}.$_->{job}\n";
+0;
+} else {
+1;
+}
+} @$subjobs;
+
 # See if there's a build we can reuse
 
 my ($recipe) = $dbh_tests->selectrow_array(

[Xen-devel] [OSSTEST PATCH 06/17] sg-run-job: Provide new recipe-flag facility

[Ian Jackson]

Individual recipes can now honour modifications requested by setting
runvars like recipe_.

No callers yet so no functional change.

Signed-off-by: Ian Jackson 
---
 sg-run-job | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 0a3f688..1355876 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -160,6 +160,13 @@ proc testid_matches_globs {testid globs} {
 return 0
 }
 
+proc recipe-flag {flagname {def 0}} {
+global flight jobinfo
+set name "recipe_$flagname"
+set val [jobdb::read-runvar $flight $jobinfo(job) $name $def]
+return [regexp {true|y|1} $val]
+}
+
 #-- test script handling --
 
 #   spwan-tsIFFAIL TESTID SCRIPT-ARGS...
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 01/17] db_retry: Honour new OSSTEST_DB_ALWAYS_ABORT variable

[Ian Jackson]

This is useful for debugging.

Signed-off-by: Ian Jackson 
---
 Osstest.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Osstest.pm b/Osstest.pm
index 2263786..3377ea3 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -330,6 +330,8 @@ sub db_retry ($$$;$$) {
last if $db_retry_stop eq 'abort';
next;
}
+die 'OSSTEST_DB_ALWAYS_ABORT set'
+if $ENV{'OSSTEST_DB_ALWAYS_ABORT'};
$committing = 1;
$dbh->commit();
};
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 00/12] Run dm_restrict=1 tests, and audit the qemu

[Ian Jackson]

From: Ian Jackson 

This series adds tests with the xl/libxl dm_restrict=1 feature
enabled (to tests of Xen 4.10 and later).

Save/restore and migration is disabled in this tests for now, as that
is broken in xen.git.

This series also adds auditing of the deprivileged qemu's file
descriptors.  That auditing depends on a new test utility in xen.git.
That utility is not yet present in any xen.git upstream branch and is
currently targeted at Xen 4.12.  Without it, the audit test will bomb
out (ie, be a test failure).  This is fine; when the tool is
available, the test should start to pass.

Ian Jackson (12):
  sg-run-job: Provide new recipe-flag facility
  sg-run-job: Allow flight to specify recipe flag to disable migr tests
  ts-xen-install: Create xen-qemuuser-range-base
  more_prepareguest_hvm: Honour guest runvar dmrestrict
  make-flight: Add a single test with restricted dm
  ts-xen-install: Honour DebianExtraPackages_
  toolstack: Provide guest_create_paused
  toolstack: Provide guest_unpause
  dm restrict audit: install newer chiark-scripts for fishdescriptor
  dm restrict audit: Provide auditing script
  dm restrict audit: Run ts-depriv-audit-qemu
  dm restrict audit: Document future plans

 Osstest/Debian.pm |   5 +
 Osstest/TestSupport.pm|  20 ++-
 Osstest/Toolstack/xl.pm   |  22 ++-
 README|   1 +
 make-flight   |  15 ++
 overlay/usr/local/bin/osstest-depriv-fd-collector |  94 
 production-config |   2 +
 production-config-cambridge   |   2 +
 sg-run-job|  18 +++
 ts-depriv-audit-qemu  | 170 ++
 ts-xen-install|   6 +
 11 files changed, 353 insertions(+), 2 deletions(-)
 create mode 100755 overlay/usr/local/bin/osstest-depriv-fd-collector
 create mode 100755 ts-depriv-audit-qemu

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 08/17] ts-xen-install: Create xen-qemuuser-range-base

[Ian Jackson]

It is fine to do in all jobs.  libxl will only use this if we ask it
to.

We do need to check whether the user exists already, since adduser
does not seem to have a mode which exits zero in that case.

Signed-off-by: Ian Jackson 
---
 ts-xen-install | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-xen-install b/ts-xen-install
index 9913e85..1ec236f 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -129,6 +129,9 @@ END
 }
 
 sub adjustconfig () {
+my $quser = 'xen-qemuuser-range-base';
+target_cmd_root($ho, "id $quser || adduser --system --uid=20 $quser");
+
 target_editfile_root($ho, "/etc/xen/xend-config.sxp",
 "xend-config.sxp", sub {
my (@domains) = (qw(localhost localhost.localdomain),
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 12/17] toolstack: Provide guest_create_paused

[Ian Jackson]

Only for xl (and xm) for now.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm  |  9 -
 Osstest/Toolstack/xl.pm | 14 +-
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index c843902..2e0e892 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -119,7 +119,7 @@ BEGIN {
  guest_check_remus_ok guest_editconfig
   guest_prepare_disk guest_unprepare_disk
   host_involves_pcipassthrough host_get_pcipassthrough_devs
-  toolstack guest_create
+  toolstack guest_create guest_create_paused
 
   await_webspace_fetch_byleaf create_webfile
   file_link_contents get_timeout
@@ -1702,6 +1702,13 @@ sub guest_create ($) {
 toolstack($ho,$gho)->create($gho);
 }
 
+sub guest_create_paused ($) {
+my ($gho) = @_;
+my $ho = $gho->{Host};
+guest_prepare_disk($gho);
+toolstack($ho,$gho)->create_paused($gho);
+}
+
 sub guest_prepare_disk ($) {
 my ($gho) = @_;
 
diff --git a/Osstest/Toolstack/xl.pm b/Osstest/Toolstack/xl.pm
index e179217..d3e3b0c 100644
--- a/Osstest/Toolstack/xl.pm
+++ b/Osstest/Toolstack/xl.pm
@@ -40,10 +40,22 @@ sub destroy ($$) {
 target_cmd_root($self->{Host}, $self->{_Command}." destroy $gn", 40);
 }
 
+sub _create ($$$) {
+my ($self,$gho,$options) = @_;
+my $cfg = $gho->{CfgPath};
+target_cmd_root($self->{Host},
+   $self->{_VerboseCommand}." create $options $cfg", 100);
+}
+
 sub create ($$) {
 my ($self,$gho) = @_;
+return $self->_create($gho,'');
+}
+
+sub create_paused ($$) {
+my ($self,$gho) = @_;
+return $self->_create($gho,'-p');
 my $cfg = $gho->{CfgPath};
-target_cmd_root($self->{Host}, $self->{_VerboseCommand}." create $cfg", 
100);
 }
 
 sub consolecmd ($$) {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 05/17] cs-bisection-step: Do explicitly set runvar for suppressed recursion

[Ian Jackson]

When we detect that we are considering a job which is identically
named to one earlier in the dependency chain, it could happen that the
final referencing runvar in the job at which we break the cycle is
actually to an unqualified job name.

(This cannot happen unless the cycle has more than 2 jobs, and
therefore more than one job name, because otherwise the job we would
be copying would have a self-reference.  So it cannot occur right
now.)

So when breaking the cycle, we should update the job we are building
to refer to the exact flight and job we want it to reuse.

The most convenient way to do this is to reorganise the new recursion
suppression code: we retain the suppressed entries in $subjobs, and
filter them as appropriate.

Signed-off-by: Ian Jackson 
---
 cs-bisection-step | 36 
 1 file changed, 20 insertions(+), 16 deletions(-)

diff --git a/cs-bisection-step b/cs-bisection-step
index a7e0336..05bfaa0 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1214,20 +1214,17 @@ END
 }
 }
 
-@$subjobs = grep {
-if ($recursion_track{ $_->{job} }) {
-print STDERR "Not recursively creating another $_->{job} (".
-(
- join " -> ",
- map { $_->{Spec} }
- sort { $a->{Depth} <=> $b->{Depth} }
- values %recursion_track
-). " -> $_->{orgflight}.$_->{job}\n";
-0;
-} else {
-1;
-}
-} @$subjobs;
+foreach my $subjob (@$subjobs) {
+next unless $recursion_track{ $subjob->{job} };
+$subjob->{suppress} = 1;
+print STDERR "Not recursively demanding another $subjob->{job} (".
+(
+ join " -> ",
+ map { $_->{Spec} }
+ sort { $a->{Depth} <=> $b->{Depth} }
+ values %recursion_track
+). " -> $subjob->{orgflight}.$subjob->{job}\n";
+}
 
 # See if there's a build we can reuse
 
@@ -1237,7 +1234,8 @@ END
 
 my $usejob;
 
-if ($cache_option and $cacheok and $recipe =~ m/^build/ and !@$subjobs) {
+if ($cache_option and $cacheok and $recipe =~ m/^build/
+and !grep { !$_->{suppress} } @$subjobs) {
my $reusejob= $builds_investigated{$popjob};
if (!defined $reusejob) {
print STDERR "Searching for $popjob (like $copyflight) to 
reuse...\n";
@@ -1309,7 +1307,13 @@ END
 END
 foreach my $subjob (@$subjobs) {
 my $target;
-$target= preparejob($subjob->{job}, $subjob->{orgflight}, 1);
+if ($subjob->{suppress}) {
+$target = "$subjob->{orgflight}.$subjob->{job}";
+print STDERR "Reusing $target for $subjob->{name}".
+" in $popflight.$popjob\n";
+} else {
+$target= preparejob($subjob->{job}, $subjob->{orgflight}, 1);
+}
 $jobsetq->execute($target, $popflight, $popjob, $subjob->{name});
 }
 $jobsetq->finish();
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 09/17] more_prepareguest_hvm: Honour guest runvar dmrestrict

[Ian Jackson]

Currently nothing sets this.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 4 
 1 file changed, 4 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index d8ed02d..c843902 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2066,6 +2066,10 @@ END
 $cfg .= "device_model_version='$devmodel'\n";
 }
 
+if (guest_var_boolean($gho, 'dmrestrict')) {
+$cfg .= "dm_restrict=1\n";
+}
+
 my $bios = $xopts{'Bios'};
 if (defined $bios) {
 $cfg .= "bios='$bios'\n";
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen: don't use privcmd_call() from xen_mc_flush()

[Jan Beulich]

>>> On 13.06.18 at 13:12,  wrote:
> On 13/06/18 12:20, Jan Beulich wrote:
> On 13.06.18 at 12:05,  wrote:
>> On 13.06.18 at 11:58,  wrote:
 Using privcmd_call() for a singleton multicall seems to be wrong, as
 privcmd_call() is using stac()/clac() to enable hypervisor access to
 Linux user space.

 Add a new xen_single_call() function to be use for that purpose.

 Reported-by: Jan Beulich 
 Signed-off-by: Juergen Gross 
>>>
>>> Reviewed-by: Jan Beulich 
>> 
>> Actually I've only now realized that this isn't a real problem right now:
>> PV can't use SMAP (we don't provide a virtualized version of it), and
>> HVM/PVH can't use multicalls (which may have to change for PVH Dom0,
>> so having the change in place is helpful anyway), so the whole
>> in-kernel logic to collect and issue batches should be unreachable there.
>> 
>> But perhaps the commit message would benefit from a little bit of
>> re-wording.
> 
> Hmm, right.
> 
> What about:
> 
> "Using privcmd_call() for a singleton multicall seems to be wrong, as
>  privcmd_call() is using stac()/clac() to enable hypervisor access to
>  Linux user space.
> 
>  Even if currently not a problem (pv domains can't use SMAP while HVM
>  and PVH domains can't use multicalls) things might change when
>  PVH dom0 support is added to the kernel."

SGTM

Jan




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [libvirt] Likely build race, "/usr/bin/ld: cannot find -lvirt"

[Ian Jackson]

Jiri Denemark writes ("Re: [libvirt] Likely build race, "/usr/bin/ld: cannot 
find -lvirt""):
> On Tue, Jun 12, 2018 at 07:57:40 -0500, Eric Blake wrote:
> > Can you add that line directly into Makefile.am, or does doing that 
> > cause automake to complain and/or omit its normal rules because it 
> > thinks you are overriding its defaults?
> 
> Yeah. It doesn't complain, but it omits its normal
> install-modLTLIBRARIES rule which mean nothing will be installed.
> However, the error is still reported so there are other libraries which
> are not in mod_LTLIBRARIES affected too.

I did a web search for "automake add dependency" and ended up at a
stackmumble page which referred to EXTRA_a_DEPENDENCIES.

See
  (automake-1) Program and Library Variables
which says


'maude_DEPENDENCIES'
'EXTRA_maude_DEPENDENCIES'
 It is also occasionally useful to have a target (program or
 library) depend on some other file that is not actually part of
 that target.  This can be done using the '_DEPENDENCIES' variable.
 Each target depends on the contents of such a variable, but no
 further interpretation is done.

 Since these dependencies are associated to the link rule used to
 create the programs they should normally list files used by the
 link command.  That is '*.$(OBJEXT)', '*.a', or '*.la' files for
 programs; '*.lo' and '*.la' files for Libtool libraries; and
 '*.$(OBJEXT)' files for static libraries.  In rare cases you may
 need to add other kinds of files such as linker scripts, but
 _listing a source file in '_DEPENDENCIES' is wrong_.  If some
 source file needs to be built before all the components of a
 program are built, consider using the 'BUILT_SOURCES' variable
 (*note Sources::).

 If '_DEPENDENCIES' is not supplied, it is computed by Automake.
 The automatically-assigned value is the contents of '_LDADD' or
 '_LIBADD', with most configure substitutions, '-l', '-L', '-dlopen'
 and '-dlpreopen' options removed.  The configure substitutions that
 are left in are only '$(LIBOBJS)' and '$(ALLOCA)'; these are left
 because it is known that they will not cause an invalid value for
 '_DEPENDENCIES' to be generated.

 '_DEPENDENCIES' is more likely used to perform conditional
 compilation using an 'AC_SUBST' variable that contains a list of
 objects.  *Note Conditional Sources::, and *note Conditional
 Libtool Sources::.

 The 'EXTRA_*_DEPENDENCIES' variable may be useful for cases where
 you merely want to augment the 'automake'-generated '_DEPENDENCIES'
 variable rather than replacing it.


Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [xen-4.8-testing test] 124100: regressions - FAIL

[Jan Beulich]

>>> On 13.06.18 at 13:18,  wrote:
> Jan asked me to investigate why we weren't getting a push on Xen 4.8.
> I investigated the failures in this osstest report.
> 
> I think we have a combination of:
> 
>  * Flaky armhf hardware
> 
>  * A real Xen-related heisenbug bug (but which is not a regression)
>(the EFAULT libxc/linux bug; CC to Juergen)
> 
>  * Mystery failures to make progress during local computation
>and I/O which look like Linux kernel bugs
> 
>  * Incompatibility between Xen 4.8 and osstest's approach to UEFI
>booting, now fixed.
> 
>  * A mystery libvirt heisenbug.  (Hence the CC to Jim.)

Thanks a lot for the analysis!

> Jan: I would be inclined to force push this.  OTOH, if we wait,
> eventually osstest will collect a set of flights which osstest's
> archeaologist can see justifies a push.

Considering

Last test of basis   123091  2018-05-23 07:11:28 Z   20 days
Failing since123345  2018-05-29 08:36:34 Z   14 days   13 attempts
Testing same since   123492  2018-05-31 20:14:51 Z   12 days   11 attempts

I'd favor a sufficiently justified (as it is now) force push, and then
an immediate release. As you say elsewhere, the problem with in
particular the albanas has been bad enough for it to be sufficiently
unpredictable when a normal push might happen (in fact I was
surprised by how quickly this happened on 4.7, as I had also
expressed in a reply to that flight's report).

For the release you'd need to tag qemu-trad and (as was asked by
Wei iirc on irc) mini-os, such that I could then push the version
update on the main tree.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable test] 124140: regressions - FAIL

[osstest service owner]

flight 124140 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124140/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 124090

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds16 guest-start/debian.repeat fail REGR. vs. 124090

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 124057
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 124057
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 124057
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 124090
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 124090
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 124090
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 124090
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 124090
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 124090
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 124090
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  41339ecb5f18ca7ec7b0c914c952a0e1715ae511
baseline version:
 xen  11535cdbc0ae5925a55b3e735447c30faaa6f63b

Last test of basis   124090  2018-06-12 01:51:41 Z1 days
Testing same since   124140  2018-06-12 17:06:49 Z0 days1 attempts

Re: [Xen-devel] [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

[Oleksandr Andrushchenko]

On 06/13/2018 03:03 PM, Oleksandr Andrushchenko wrote:

On 06/13/2018 03:02 PM, Boris Ostrovsky wrote:



On 06/13/2018 02:26 AM, Oleksandr Andrushchenko wrote:

On 06/13/2018 03:47 AM, Boris Ostrovsky wrote:



On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 


diff --git a/include/xen/mem-reservation.h 
b/include/xen/mem-reservation.h

new file mode 100644
index ..e0939387278d
--- /dev/null
+++ b/include/xen/mem-reservation.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include 
+#include 
+
+#include 
+#include 
+
+#include 
+#include 



I should have noticed this in the previous post but I suspect most 
of these includes belong in the C file. For example, there is no 
reason for hypercall.h here.



Yes, it seems that the header can only have
#include 
Will move the rest into the .c file



You may need something for clear_highpage() and maybe for Xen feature 
flags. But you'll find out for sure when you try to build. ;-)



#include 


Or even
#include 
according to [1]

;)

-boris




-boris



+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+    struct page **pages,
+    xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+   struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 
long count,

+    struct page **pages,
+    xen_pfn_t *frames)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+    __xenmem_reservation_va_mapping_update(count, pages, 
frames);

+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned 
long count,

+   struct page **pages)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+    __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif






[1] https://elixir.bootlin.com/linux/v4.17.1/ident/clear_highpage

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH v4 11/40] hw/xen: Use the IEC binary prefix definitions

[Eric Blake]

On 06/12/2018 04:10 PM, Richard Henderson wrote:

So there's tradeoffs either way, and you at least need to document in your
commit messages what auditing you have done that any type changes introduced by
your changes are safe.


I'm more concerned about unnecessary or unintended signed vs unsigned changes
than I am about width.  But if we're going to force a 64-bit type, use
(int64_t)1 not 1LL.  That way the type will match the existing PRId64 printf
markup.


Or spell it UINT64_C(1) if you don't want a cast.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

[Oleksandr Andrushchenko]

On 06/13/2018 03:02 PM, Boris Ostrovsky wrote:



On 06/13/2018 02:26 AM, Oleksandr Andrushchenko wrote:

On 06/13/2018 03:47 AM, Boris Ostrovsky wrote:



On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 


diff --git a/include/xen/mem-reservation.h 
b/include/xen/mem-reservation.h

new file mode 100644
index ..e0939387278d
--- /dev/null
+++ b/include/xen/mem-reservation.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include 
+#include 
+
+#include 
+#include 
+
+#include 
+#include 



I should have noticed this in the previous post but I suspect most 
of these includes belong in the C file. For example, there is no 
reason for hypercall.h here.



Yes, it seems that the header can only have
#include 
Will move the rest into the .c file



You may need something for clear_highpage() and maybe for Xen feature 
flags. But you'll find out for sure when you try to build. ;-)



#include 

;)

-boris




-boris



+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+    struct page **pages,
+    xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+   struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 
long count,

+    struct page **pages,
+    xen_pfn_t *frames)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+    __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned 
long count,

+   struct page **pages)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+    __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif






___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 3/9] xen/balloon: Share common memory reservation routines

[Boris Ostrovsky]

On 06/13/2018 02:26 AM, Oleksandr Andrushchenko wrote:

On 06/13/2018 03:47 AM, Boris Ostrovsky wrote:



On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 


diff --git a/include/xen/mem-reservation.h 
b/include/xen/mem-reservation.h

new file mode 100644
index ..e0939387278d
--- /dev/null
+++ b/include/xen/mem-reservation.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Xen memory reservation utilities.
+ *
+ * Copyright (c) 2003, B Dragovic
+ * Copyright (c) 2003-2004, M Williamson, K Fraser
+ * Copyright (c) 2005 Dan M. Smith, IBM Corporation
+ * Copyright (c) 2010 Daniel Kiper
+ * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
+ */
+
+#ifndef _XENMEM_RESERVATION_H
+#define _XENMEM_RESERVATION_H
+
+#include 
+#include 
+
+#include 
+#include 
+
+#include 
+#include 



I should have noticed this in the previous post but I suspect most of 
these includes belong in the C file. For example, there is no reason 
for hypercall.h here.



Yes, it seems that the header can only have
#include 
Will move the rest into the .c file



You may need something for clear_highpage() and maybe for Xen feature 
flags. But you'll find out for sure when you try to build. ;-)


-boris




-boris



+
+static inline void xenmem_reservation_scrub_page(struct page *page)
+{
+#ifdef CONFIG_XEN_SCRUB_PAGES
+    clear_highpage(page);
+#endif
+}
+
+#ifdef CONFIG_XEN_HAVE_PVMMU
+void __xenmem_reservation_va_mapping_update(unsigned long count,
+    struct page **pages,
+    xen_pfn_t *frames);
+
+void __xenmem_reservation_va_mapping_reset(unsigned long count,
+   struct page **pages);
+#endif
+
+static inline void xenmem_reservation_va_mapping_update(unsigned 
long count,

+    struct page **pages,
+    xen_pfn_t *frames)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+    __xenmem_reservation_va_mapping_update(count, pages, frames);
+#endif
+}
+
+static inline void xenmem_reservation_va_mapping_reset(unsigned long 
count,

+   struct page **pages)
+{
+#ifdef CONFIG_XEN_HAVE_PVMMU
+    if (!xen_feature(XENFEAT_auto_translated_physmap))
+    __xenmem_reservation_va_mapping_reset(count, pages);
+#endif
+}
+
+int xenmem_reservation_increase(int count, xen_pfn_t *frames);
+
+int xenmem_reservation_decrease(int count, xen_pfn_t *frames);
+
+#endif





___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 8/9] xen/gntdev: Implement dma-buf export functionality

[Oleksandr Andrushchenko]

On 06/13/2018 05:58 AM, Boris Ostrovsky wrote:



On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

1. Create a dma-buf from grant references provided by the foreign
    domain. By default dma-buf is backed by system memory pages, but
    by providing GNTDEV_DMA_FLAG_XXX flags it can also be created
    as a DMA write-combine/coherent buffer, e.g. allocated with
    corresponding dma_alloc_xxx API.
    Export the resulting buffer as a new dma-buf.

2. Implement waiting for the dma-buf to be released: block until the
    dma-buf with the file descriptor provided is released.
    If within the time-out provided the buffer is not released then
    -ETIMEDOUT error is returned. If the buffer with the file descriptor
    does not exist or has already been released, then -ENOENT is
    returned. For valid file descriptors this must not be treated as
    error.

3. Make gntdev's common code and structures available to dma-buf.

Signed-off-by: Oleksandr Andrushchenko 


---
  drivers/xen/gntdev-common.h |   4 +
  drivers/xen/gntdev-dmabuf.c | 470 +++-
  drivers/xen/gntdev.c    |  10 +
  3 files changed, 482 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/gntdev-common.h b/drivers/xen/gntdev-common.h
index a3408fd39b07..72f80dbce861 100644
--- a/drivers/xen/gntdev-common.h
+++ b/drivers/xen/gntdev-common.h
@@ -89,4 +89,8 @@ bool gntdev_account_mapped_pages(int count);
    int gntdev_map_grant_pages(struct gntdev_grant_map *map);
  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+void gntdev_remove_map(struct gntdev_priv *priv, struct 
gntdev_grant_map *map);

+#endif
+
  #endif
diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index dc57c6a25525..84cba67c6ad7 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -3,13 +3,53 @@
  /*
   * Xen dma-buf functionality for gntdev.
   *
+ * DMA buffer implementation is based on drivers/gpu/drm/drm_prime.c.
+ *
   * Copyright (c) 2018 Oleksandr Andrushchenko, EPAM Systems Inc.
   */
  +#include 
  #include 
  +#include 
+#include 
+
+#include "gntdev-common.h"
  #include "gntdev-dmabuf.h"
  +struct gntdev_dmabuf {
+    struct gntdev_dmabuf_priv *priv;
+    struct dma_buf *dmabuf;
+    struct list_head next;
+    int fd;
+
+    union {
+    struct {
+    /* Exported buffers are reference counted. */
+    struct kref refcount;
+
+    struct gntdev_priv *priv;
+    struct gntdev_grant_map *map;
+    } exp;
+    } u;
+
+    /* Number of pages this buffer has. */
+    int nr_pages;
+    /* Pages of this buffer. */
+    struct page **pages;
+};
+
+struct gntdev_dmabuf_wait_obj {
+    struct list_head next;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct completion completion;
+};
+
+struct gntdev_dmabuf_attachment {
+    struct sg_table *sgt;
+    enum dma_data_direction dir;
+};
+
  struct gntdev_dmabuf_priv {
  /* List of exported DMA buffers. */
  struct list_head exp_list;
@@ -23,17 +63,439 @@ struct gntdev_dmabuf_priv {
    /* Implementation of wait for exported DMA buffer to be released. */
  +static void dmabuf_exp_release(struct kref *kref);
+
+static struct gntdev_dmabuf_wait_obj *
+dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
+    struct gntdev_dmabuf *gntdev_dmabuf)
+{
+    struct gntdev_dmabuf_wait_obj *obj;
+
+    obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+    if (!obj)
+    return ERR_PTR(-ENOMEM);
+
+    init_completion(>completion);
+    obj->gntdev_dmabuf = gntdev_dmabuf;
+
+    mutex_lock(>lock);
+    list_add(>next, >exp_wait_list);
+    /* Put our reference and wait for gntdev_dmabuf's release to 
fire. */

+    kref_put(_dmabuf->u.exp.refcount, dmabuf_exp_release);
+    mutex_unlock(>lock);
+    return obj;
+}
+
+static void dmabuf_exp_wait_obj_free(struct gntdev_dmabuf_priv *priv,
+ struct gntdev_dmabuf_wait_obj *obj)
+{
+    struct gntdev_dmabuf_wait_obj *cur_obj, *q;
+
+    mutex_lock(>lock);
+    list_for_each_entry_safe(cur_obj, q, >exp_wait_list, next)
+    if (cur_obj == obj) {
+    list_del(>next);
+    kfree(obj);
+    break;
+    }
+    mutex_unlock(>lock);



Do we really need to walk the list?


It can be deleted without walking the list and no reason to do that walk.
Just an example of over-engineering here, thank you for spotting this.
And if we do, do we need the safe variant of the walk? We are holding 
the lock. Here and elsewhere.


You are perfectly right. I will not use safe variant of the walk, no 
need for that



+}
+
+static int dmabuf_exp_wait_obj_wait(struct gntdev_dmabuf_wait_obj *obj,
+    u32 wait_to_ms)
+{
+    if (wait_for_completion_timeout(>completion,
+    msecs_to_jiffies(wait_to_ms)) <= 0)
+    return -ETIMEDOUT;
+
+    return 0;
+}
+
+static void dmabuf_exp_wait_obj_signal(struct gntdev_dmabuf_priv *priv,
+   struct gntdev_dmabuf 

[Xen-devel] [linux-4.9 test] 124113: regressions - FAIL

[osstest service owner]

flight 124113 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124113/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  7 xen-bootfail REGR. vs. 122969
 test-amd64-amd64-xl-shadow7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-libvirt-xsm  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 7 xen-boot fail REGR. vs. 
122969
 test-amd64-amd64-xl-qemuu-win10-i386  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemut-ws16-amd64  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemut-win10-i386  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemut-debianhvm-amd64  7 xen-bootfail REGR. vs. 122969
 test-amd64-amd64-libvirt-vhd  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemut-win7-amd64  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-i386-pvgrub  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-libvirt  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-qemuu-nested-intel  7 xen-boot  fail REGR. vs. 122969
 test-amd64-amd64-xl-pvhv2-intel  7 xen-boot  fail REGR. vs. 122969
 test-amd64-amd64-xl-credit2   7 xen-boot fail REGR. vs. 122969
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 122969
 test-amd64-amd64-xl-qemuu-win7-amd64  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemuu-ws16-amd64  7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 122969
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. 
vs. 122969
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 xen-boot fail REGR. vs. 122969

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu  6 xen-install fail in 123970 pass in 124113
 test-armhf-armhf-xl-arndale   7 xen-boot fail in 124084 pass in 124113
 test-amd64-amd64-xl-xsm   7 xen-boot   fail pass in 123861
 test-amd64-amd64-xl-rtds  7 xen-boot   fail pass in 123861
 test-amd64-amd64-xl-qcow2 7 xen-boot   fail pass in 123861
 test-amd64-amd64-pygrub   7 xen-boot   fail pass in 123861
 test-amd64-amd64-rumprun-amd64  7 xen-boot fail pass in 123861
 test-amd64-amd64-xl-pvshim7 xen-boot   fail pass in 123861
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fail pass in 
123861
 test-amd64-amd64-xl   7 xen-boot   fail pass in 123861
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  7 xen-boot  fail pass in 123861
 test-amd64-amd64-xl-multivcpu  7 xen-boot  fail pass in 123861
 test-amd64-amd64-pair10 xen-boot/src_host  fail pass in 123861
 test-amd64-amd64-pair11 xen-boot/dst_host  fail pass in 123861
 test-amd64-amd64-examine  8 reboot fail pass in 123861
 test-amd64-amd64-amd64-pvgrub  7 xen-boot  fail pass in 123914
 test-amd64-amd64-libvirt-pair 10 xen-boot/src_host fail pass in 123970
 test-amd64-amd64-libvirt-pair 11 xen-boot/dst_host fail pass in 123970
 test-amd64-i386-rumprun-i386 17 rumprun-demo-xenstorels/xenstorels.repeat fail 
pass in 124084

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm 13 migrate-support-check fail in 123861 never pass
 test-amd64-i386-libvirt 13 migrate-support-check fail in 123861 never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail in 123861 never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail in 123861 never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 122969
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 122969
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   

[Xen-devel] [PATCH v3 9/9] x86/vmx: Don't leak EFER.NXE into guest context

[Andrew Cooper]

Intel hardware only uses 4 bits in MSR_EFER.  Changes to LME and LMA are
handled automatically via the VMENTRY_CTLS.IA32E_MODE bit.

SCE is handled by ad-hoc logic in context_switch(), vmx_restore_guest_msrs()
and vmx_update_guest_efer(), and works by altering the host SCE value to match
the setting the guest wants.  This works because, in HVM vcpu context, Xen
never needs to execute a SYSCALL or SYSRET instruction.

However, NXE has never been context switched.  Unlike SCE, NXE cannot be
context switched at vcpu boundaries because disabling NXE makes PTE.NX bits
reserved and cause a pagefault when encountered.  This means that the guest
always has Xen's setting in effect, irrespective of the bit it can see and
modify in its virtualised view of MSR_EFER.

This isn't a major problem for production operating systems because they, like
Xen, always turn the NXE on when it is available.  However, it does have an
observable effect on which guest PTE bits are valid, and whether
PFEC_insn_fetch is visible in a #PF error code.

Second generation VT-x hardware has host and guest EFER fields in the VMCS,
and support for loading and saving them automatically.  First generation VT-x
hardware needs to use MSR load/save lists to cause an atomic switch of
MSR_EFER on vmentry/exit.

Therefore we update vmx_init_vmcs_config() to find and use guest/host EFER
support when available (and MSR load/save lists on older hardware) and drop
all ad-hoc alteration of SCE.

There are two minor complications when selecting the EFER setting:
 * For shadow guests, NXE is a paging setting and must remain under host
   control, but this is fine as Xen also handles the pagefaults.
 * When the Unrestricted Guest control is clear, hardware doesn't tolerate LME
   and LMA being different.  This doesn't matter in practice as we intercept
   all writes to CR0 and reads from MSR_EFER, so can provide architecturally
   consistent behaviour from the guests point of view.

With changing how EFER is loaded, vmcs_dump_vcpu() needs adjusting.  Read EFER
from the appropriate information source, and identify when dumping the guest
EFER value which source was used.

As a result of fixing EFER context switching, we can remove the Intel-special
case from hvm_nx_enabled() and let guest_walk_tables() work with the real
guest paging settings.

Signed-off-by: Andrew Cooper 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Tim Deegan 
Reviewed-by: Kevin Tian 
---
CC: Jan Beulich 
CC: Wei Liu 

v2:
 * Fix a vmentry failure on Nehalem/Westmere hardware.  LME != LMA is actuall
   dependent on Unrestricted Guest (which is clear for Shadow guests as a side
   effect of clearing EPT).
 * Fix vmcs_dump_vcpu() to cope with the new EFER behaviour.
v3:
 * Don't opencode the existing vmx_unrestricted_guest() predicate.
 * Clear the MSR_EFER read intercept when possible.
---
 xen/arch/x86/domain.c  | 10 -
 xen/arch/x86/hvm/vmx/vmcs.c| 26 
 xen/arch/x86/hvm/vmx/vmx.c | 84 ++
 xen/include/asm-x86/hvm/hvm.h  |  4 +-
 xen/include/asm-x86/hvm/vmx/vmcs.h | 16 
 5 files changed, 102 insertions(+), 38 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 0ca820a..e817795 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1723,16 +1723,6 @@ void context_switch(struct vcpu *prev, struct vcpu *next)
 {
 __context_switch();
 
-if ( is_pv_domain(nextd) &&
- (is_idle_domain(prevd) ||
-  is_hvm_domain(prevd) ||
-  is_pv_32bit_domain(prevd) != is_pv_32bit_domain(nextd)) )
-{
-uint64_t efer = read_efer();
-if ( !(efer & EFER_SCE) )
-write_efer(efer | EFER_SCE);
-}
-
 /* Re-enable interrupts before restoring state which may fault. */
 local_irq_enable();
 
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index cb14b43..cdc1e77 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -342,8 +342,8 @@ static int vmx_init_vmcs_config(void)
 }
 
 min = VM_EXIT_ACK_INTR_ON_EXIT;
-opt = VM_EXIT_SAVE_GUEST_PAT | VM_EXIT_LOAD_HOST_PAT |
-  VM_EXIT_CLEAR_BNDCFGS;
+opt = (VM_EXIT_SAVE_GUEST_PAT | VM_EXIT_LOAD_HOST_PAT |
+   VM_EXIT_LOAD_HOST_EFER | VM_EXIT_CLEAR_BNDCFGS);
 min |= VM_EXIT_IA32E_MODE;
 _vmx_vmexit_control = adjust_vmx_controls(
 "VMExit Control", min, opt, MSR_IA32_VMX_EXIT_CTLS, );
@@ -383,7 +383,8 @@ static int vmx_init_vmcs_config(void)
 _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS;
 
 min = 0;
-opt = VM_ENTRY_LOAD_GUEST_PAT | VM_ENTRY_LOAD_BNDCFGS;
+opt = (VM_ENTRY_LOAD_GUEST_PAT | VM_ENTRY_LOAD_GUEST_EFER |
+   VM_ENTRY_LOAD_BNDCFGS);
 _vmx_vmentry_control = adjust_vmx_controls(
 "VMEntry Control", min, opt, MSR_IA32_VMX_ENTRY_CTLS, );
 
@@ -1148,6 +1149,8 @@ static int 

Re: [Xen-devel] [xen-4.8-testing test] 124100: regressions - FAIL

[Ian Jackson]

Jan asked me to investigate why we weren't getting a push on Xen 4.8.
I investigated the failures in this osstest report.

I think we have a combination of:

 * Flaky armhf hardware

 * A real Xen-related heisenbug bug (but which is not a regression)
   (the EFAULT libxc/linux bug; CC to Juergen)

 * Mystery failures to make progress during local computation
   and I/O which look like Linux kernel bugs

 * Incompatibility between Xen 4.8 and osstest's approach to UEFI
   booting, now fixed.

 * A mystery libvirt heisenbug.  (Hence the CC to Jim.)

Jan: I would be inclined to force push this.  OTOH, if we wait,
eventually osstest will collect a set of flights which osstest's
archeaologist can see justifies a push.

Jim: please read down to where I discuss
test-amd64-amd64-libvirt-pair.  If you have any insight I'd appreciate
it.  Let me know if you want me to preserve the logs, which will
otherwise expire in a few weeks.

Juergen: this is just FYI.

HTH.


osstest service owner writes ("[xen-4.8-testing test] 124100: regressions - 
FAIL"):
> flight 124100 xen-4.8-testing real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/124100/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  build-armhf-xsm  broken  in 
> 124070
>  build-armhf-xsm  5 host-build-prep fail in 124070 REGR. vs. 
> 123091
>  build-armhf   6 xen-build  fail in 123844 REGR. vs. 
> 123091

I haven't looked but I think these are the arndale bug.  It looked at
124070 and 123844 because it was trying to determine that other
failures were heisenbugs.


>  build-i386-pvops  6 kernel-build   fail in 123844 REGR. vs. 
> 123091

This is "git checkout 57a3ca7835962109d94533465a75e8c716b26845" taking
more than 4000 seconds (!) on albana1.

I have looked at the host logs for albana1 and there seem to be no
other build failures, except for libvirt ones (which is expected
because there is a race in the libvirt makefiles).

I looked at the logs for this particular failure.  osstest collected
ps output, which shows this:

14583 ?D00:00:03   0   3  0.6  1.7  1 balance_dirty_pages_ratel 
 \_ git checkout 57a3ca7835962109d94533465a75e8c716b26845

There is nothing unexpected or interesting in any of the logfiles.
Note that this host was not running Xen.  The kernel was the default
Debian jessie i386 kernel.

I have no real explanation.  This seems like it must be a bug in the
kernel.


> Tests which are failing intermittently (not blocking):
>  test-amd64-amd64-xl-credit2   7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 xen-boot fail in 123701 pass 
> in 124100
>  test-amd64-amd64-livepatch7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-amd64-pair   10 xen-boot/src_host fail in 123701 pass in 
> 124100
>  test-amd64-amd64-pair   11 xen-boot/dst_host fail in 123701 pass in 
> 124100
>  test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-i386-rumprun-i386  7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-i386-xl-qemuu-debianhvm-amd64 7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-i386-qemut-rhel6hvm-intel  7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-i386-libvirt-xsm   7 xen-boot fail in 123701 pass in 
> 124100
>  test-amd64-i386-migrupgrade 10 xen-boot/src_host fail in 123701 pass in 
> 124100
>  test-amd64-i386-migrupgrade 11 xen-boot/dst_host fail in 123701 pass in 
> 124100
>  test-amd64-amd64-xl-multivcpu  7 xen-bootfail in 123701 pass in 
> 124100
>  test-amd64-amd64-xl-qemuu-ovmf-amd64  7 xen-boot fail in 123701 pass in 
> 124100
>  test-xtf-amd64-amd64-37 xen-boot fail in 123844 pass in 
> 124100

I haven't looked at all of these, but I have looked at a few,
including the xtf test in 123844.  The jobs I looked at ran on one of
the albanas (the new uefi hosts).  These flights were after albana*
were put into service but before I taught osstest to avoid trying to
boot xen.gz from 4.9 and earlier on uefi hosts (by avoiding running
4.9 tests on those hosts at all).

So I think these failures are all understood and expected.  osstest is
fixed now, so they will not occur in new runs.  osstest is trying to
justify them as heisenbugs, by observing that they passed in 124100.

The wide range of affected tests means that osstest ends up looking
for a lot of other passes to justify these, and I think that is a big
part of the reason why the push is taking so long.


>  test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass 
> in 123701

From the log:

2018-06-12 20:59:40 Z executing ssh ... root@172.16.144.61 virsh migrate --live 
debian.guest.osstest xen+ssh://joubertin0
error: Timed out during operation: cannot acquire state change lock
2018-06-12 21:00:16 Z command nonzero 

Re: [Xen-devel] [PATCH] xen: don't use privcmd_call() from xen_mc_flush()

[Juergen Gross]

On 13/06/18 12:20, Jan Beulich wrote:
 On 13.06.18 at 12:05,  wrote:
> On 13.06.18 at 11:58,  wrote:
>>> Using privcmd_call() for a singleton multicall seems to be wrong, as
>>> privcmd_call() is using stac()/clac() to enable hypervisor access to
>>> Linux user space.
>>>
>>> Add a new xen_single_call() function to be use for that purpose.
>>>
>>> Reported-by: Jan Beulich 
>>> Signed-off-by: Juergen Gross 
>>
>> Reviewed-by: Jan Beulich 
> 
> Actually I've only now realized that this isn't a real problem right now:
> PV can't use SMAP (we don't provide a virtualized version of it), and
> HVM/PVH can't use multicalls (which may have to change for PVH Dom0,
> so having the change in place is helpful anyway), so the whole
> in-kernel logic to collect and issue batches should be unreachable there.
> 
> But perhaps the commit message would benefit from a little bit of
> re-wording.

Hmm, right.

What about:

"Using privcmd_call() for a singleton multicall seems to be wrong, as
 privcmd_call() is using stac()/clac() to enable hypervisor access to
 Linux user space.

 Even if currently not a problem (pv domains can't use SMAP while HVM
 and PVH domains can't use multicalls) things might change when
 PVH dom0 support is added to the kernel."


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 5/9] x86/vmx: Improvements to LBR MSR handling

[Andrew Cooper]

On 13/06/18 07:30, Jan Beulich wrote:
 On 12.06.18 at 18:33,  wrote:
>> On 12/06/18 10:00, Jan Beulich wrote:
>> On 12.06.18 at 10:51,  wrote:
 On 12/06/2018 09:15, Jan Beulich wrote:
 On 08.06.18 at 20:48,  wrote:
>> @@ -3106,14 +3104,13 @@ static int vmx_msr_write_intercept(unsigned int 
>> msr, 
 uint64_t msr_content)
>>  for ( ; (rc == 0) && lbr->count; lbr++ )
>>  for ( i = 0; (rc == 0) && (i < lbr->count); i++ )
>>  if ( (rc = vmx_add_guest_msr(v, lbr->base + i)) == 
>> 0 )
>> -{
>>  vmx_clear_msr_intercept(v, lbr->base + i, 
 VMX_MSR_RW);
>> -if ( lbr_tsx_fixup_needed )
>> -v->arch.hvm_vmx.lbr_fixup_enabled |= 
 FIXUP_LBR_TSX;
>> -if ( bdw_erratum_bdf14_fixup_needed )
>> -v->arch.hvm_vmx.lbr_fixup_enabled |=
>> -FIXUP_BDW_ERRATUM_BDF14;
>> -}
>> +
>> +v->arch.hvm_vmx.lbr_flags |= LBR_MSRS_INSERTED;
>> +if ( lbr_tsx_fixup_needed )
>> +v->arch.hvm_vmx.lbr_flags |= LBR_FIXUP_TSX;
>> +if ( bdw_erratum_bdf14_fixup_needed )
>> +v->arch.hvm_vmx.lbr_flags |= LBR_FIXUP_BDF14;
> Note how the setting of the flags previously depended on
> vmx_add_guest_msr() having returned success at least once.
 And?

 Unless this sequence returns fully successfully, we throw #MC into the
 guest without setting any kind of vMCE state.  It might be the least bad
 option we have available, but its also not reasonable to expect the
 guest to survive.

 The two ways to fail are ENOMEM which E2BIG.  The former is going to be
 causing other forms of chaos, and the latter isn't going to occur in
 practice because current codepaths in Xen use a maximum of ~40 or the
 256 available slots.  If in the unlikely case that we fail with ENOMEM
 on the first entry, all the fixup logic gets short circuited due to the
 missing memory allocation (so practically 0 extra overhead), and the
 guest will still malfunction.

 The error handling here is sufficiently poor that I'm not worried about
 changing one minor corner case.  I'm actually debating whether it would
 be better to make the allocation at vmcs construction time, to avoid
 runtime out-of-memory issues.
>>> With further improved MSR handling down the road, I assume we'll
>>> have some entries in the list in almost all cases, so yes, I think that
>>> would be desirable.
>> For performance reasons, we'll want to keep the size of the lists to an
>> absolute minimum.
>>
>> On a closer inspection, the only uses we currently have for the
>> load/save lists are this new EFER case (on Gen1 hardware), the Global
>> Perf Ctl (for vPMU, and we really should be using the load/save support
>> like EFER), and the LBR MSRs.
>>
>> Therefore, for on non-ancient hardware, a guest which doesn't touch
>> MSR_DEBUGCTL is not going to need the memory allocation, so perhaps an
>> up-front allocation isn't the wisest of options.  I'll keep this in mind
>> during the MSR work.
> Hmm, okay. In which case, if we anyway don't expect the guest to
> survive here in case of some failure, why don't we crash it right away
> instead of injecting #MC? Would make for a much easier to recognize
> cause of the guest crash.

Yeah - I was considering that as an option.  -ENOSPC is definitely a
hypervisor coding issue, and can't be remedied in place.  -ENOMEM is
unfortunate if a guest happens to hit it, but cleanly crashing is better
behaviour than the current #MC with no information.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen: don't use privcmd_call() from xen_mc_flush()

[Jan Beulich]

>>> On 13.06.18 at 12:05,  wrote:
 On 13.06.18 at 11:58,  wrote:
>> Using privcmd_call() for a singleton multicall seems to be wrong, as
>> privcmd_call() is using stac()/clac() to enable hypervisor access to
>> Linux user space.
>> 
>> Add a new xen_single_call() function to be use for that purpose.
>> 
>> Reported-by: Jan Beulich 
>> Signed-off-by: Juergen Gross 
> 
> Reviewed-by: Jan Beulich 

Actually I've only now realized that this isn't a real problem right now:
PV can't use SMAP (we don't provide a virtualized version of it), and
HVM/PVH can't use multicalls (which may have to change for PVH Dom0,
so having the change in place is helpful anyway), so the whole
in-kernel logic to collect and issue batches should be unreachable there.

But perhaps the commit message would benefit from a little bit of
re-wording.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 9/9] x86/vmx: Don't leak EFER.NXE into guest context

[Andrew Cooper]

On 12/06/18 09:54, Jan Beulich wrote:
 On 08.06.18 at 20:48,  wrote:
>> @@ -1646,22 +1637,71 @@ static void vmx_update_guest_cr(struct vcpu *v, 
>> unsigned int cr,
>>  
>>  static void vmx_update_guest_efer(struct vcpu *v)
>>  {
>> -unsigned long vm_entry_value;
>> +unsigned long entry_ctls, guest_efer = v->arch.hvm_vcpu.guest_efer,
>> +xen_efer = read_efer();
>> +
>> +if ( paging_mode_shadow(v->domain) )
>> +{
>> +/*
>> + * When using shadow pagetables, EFER.NX is a Xen-owned bit and is 
>> not
>> + * under guest control.
>> + */
>> +guest_efer &= ~EFER_NX;
>> +guest_efer |= xen_efer & EFER_NX;
>> +}
>> +
>> +if ( !(v->arch.hvm_vmx.secondary_exec_control &
>> +   SECONDARY_EXEC_UNRESTRICTED_GUEST) )
> !vmx_unrestricted_guest(v)
>
>> +{
>> +/*
>> + * When Unrestricted Guest is not enabled in the VMCS, hardware does
>> + * not tolerate the LME and LMA settings being different.  As writes
>> + * to CR0 are intercepted, it is safe to leave LME clear at this
>> + * point, and fix up both LME and LMA when CR0.PG is set.
>> + */
>> +if ( !(guest_efer & EFER_LMA) )
>> +guest_efer &= ~EFER_LME;
>> +}
>>  
>>  vmx_vmcs_enter(v);
>>  
>> -__vmread(VM_ENTRY_CONTROLS, _entry_value);
>> -if ( v->arch.hvm_vcpu.guest_efer & EFER_LMA )
>> -vm_entry_value |= VM_ENTRY_IA32E_MODE;
>> +/*
>> + * The intended guest running mode is derived from VM_ENTRY_IA32E_MODE,
>> + * which (architecturally) is the guest's LMA setting.
>> + */
>> +__vmread(VM_ENTRY_CONTROLS, _ctls);
>> +
>> +entry_ctls &= ~VM_ENTRY_IA32E_MODE;
>> +if ( guest_efer & EFER_LMA )
>> +entry_ctls |= VM_ENTRY_IA32E_MODE;
>> +
>> +__vmwrite(VM_ENTRY_CONTROLS, entry_ctls);
>> +
>> +/* We expect to use EFER loading in the common case, but... */
>> +if ( likely(cpu_has_vmx_efer) )
>> +__vmwrite(GUEST_EFER, guest_efer);
>> +
>> +/* ... on Gen1 VT-x hardware, we have to use MSR load/save lists 
>> instead. */
>>  else
>> -vm_entry_value &= ~VM_ENTRY_IA32E_MODE;
>> -__vmwrite(VM_ENTRY_CONTROLS, vm_entry_value);
>> +{
>> +/*
>> + * When the guests choice of EFER matches Xen's, remove the 
>> load/save
>> + * list entries.  It is unnecessary overhead, especially as this is
>> + * expected to be the common case for 64bit guests.
>> + */
>> +if ( guest_efer == xen_efer )
>> +{
>> +vmx_del_msr(v, MSR_EFER, VMX_MSR_HOST);
>> +vmx_del_msr(v, MSR_EFER, VMX_MSR_GUEST_LOADONLY);
>> +}
>> +else
>> +{
>> +vmx_add_msr(v, MSR_EFER, xen_efer, VMX_MSR_HOST);
>> +vmx_add_msr(v, MSR_EFER, guest_efer, VMX_MSR_GUEST_LOADONLY);
>> +}
>> +}
>>  
>>  vmx_vmcs_exit(v);
>> -
>> -if ( v == current )
>> -write_efer((read_efer() & ~EFER_SCE) |
>> -   (v->arch.hvm_vcpu.guest_efer & EFER_SCE));
>>  }
> As mentioned before, overall this would allow for disabling read intercepts in
> certain cases. If you don't want to do this right away that's certainly fine, 
> but
> could I talk you into at least adding a comment to this effect?

Apologies - that was a straight oversight.  Razvan thinks the monitor
side of things is actually fine, which was my concern with doing it
originally.

I've inserted the following fragment in the tail of this function, after
the vmx_vmcs_exit(v);

/*
 * If the guests virtualised view of MSR_EFER matches the value loaded
 * into hardware, clear the read intercept to avoid unnecessary VMExits.
 */
if ( guest_efer == v->arch.hvm_vcpu.guest_efer )
vmx_clear_msr_intercept(v, MSR_EFER, VMX_MSR_R);
else
vmx_set_msr_intercept(v, MSR_EFER, VMX_MSR_R);

and will quickly whip up an XTF test for some confirmation.

>
>> --- a/xen/include/asm-x86/hvm/vmx/vmcs.h
>> +++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
>> @@ -311,6 +311,8 @@ extern u64 vmx_ept_vpid_cap;
>>  (vmx_cpu_based_exec_control & CPU_BASED_MONITOR_TRAP_FLAG)
>>  #define cpu_has_vmx_pat \
>>  (vmx_vmentry_control & VM_ENTRY_LOAD_GUEST_PAT)
>> +#define cpu_has_vmx_efer \
>> +(vmx_vmentry_control & VM_ENTRY_LOAD_GUEST_EFER)
> I think this was asked before, but I'm concerned (of at least the 
> inconsistency)
> anyway: cpu_has_vmx_mpx, for example, checks both flags. Of course there's
> unlikely to be any hardware with just one of the two features, but what about
> buggy virtual environments we might run in?

I'm not worried about buggy virtual environments.  For one, its not
really our bug to care about, but irrespective, if an environment is
this buggy, it won't notice the setting we've made, and the vmentry will
be fine.

This, FYI, is exactly what happens with the Virtual NMI feature when
nested under Xen atm.  Some hypervisors fail to check 

[Xen-devel] [xen-unstable-coverity test] 124166: all pass - PUSHED

[osstest service owner]

flight 124166 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/124166/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen  41339ecb5f18ca7ec7b0c914c952a0e1715ae511
baseline version:
 xen  11535cdbc0ae5925a55b3e735447c30faaa6f63b

Last test of basis   124041  2018-06-10 09:20:02 Z3 days
Testing same since   124166  2018-06-13 09:18:59 Z0 days1 attempts


People who touched revisions under test:
  Ian Jackson 
  Julien Grall 

jobs:
 coverity-amd64   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   11535cdbc0..41339ecb5f  41339ecb5f18ca7ec7b0c914c952a0e1715ae511 -> 
coverity-tested/smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen: don't use privcmd_call() from xen_mc_flush()

[Jan Beulich]

>>> On 13.06.18 at 11:58,  wrote:
> Using privcmd_call() for a singleton multicall seems to be wrong, as
> privcmd_call() is using stac()/clac() to enable hypervisor access to
> Linux user space.
> 
> Add a new xen_single_call() function to be use for that purpose.
> 
> Reported-by: Jan Beulich 
> Signed-off-by: Juergen Gross 

Reviewed-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] 4.11.0 RC1 panic

[Jan Beulich]

>>> On 13.06.18 at 10:57,  wrote:
> On Wed, Jun 13, 2018 at 02:07:29AM -0600, Jan Beulich wrote:
>> 
>> (XEN) Assertion '!page->linear_pt_count' failed at mm.c:596
>> 
>> In fact, there's no assertion with that expression anywhere I could
>> see. Do you have any local patches in place?
> 
> Yes, 2 of them from you (the first one is where the assert is). See 
> attached.

Oh, I had long dropped that first one, after you had said that it didn't
trigger in a long time. It triggering with the other debugging patch is
not unexpected. So please drop that patch at least for the time being.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] xen: don't use privcmd_call() from xen_mc_flush()

[Juergen Gross]

Using privcmd_call() for a singleton multicall seems to be wrong, as
privcmd_call() is using stac()/clac() to enable hypervisor access to
Linux user space.

Add a new xen_single_call() function to be use for that purpose.

Reported-by: Jan Beulich 
Signed-off-by: Juergen Gross 
---
 arch/x86/include/asm/xen/hypercall.h | 25 +++--
 arch/x86/xen/multicalls.c|  6 +++---
 2 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/arch/x86/include/asm/xen/hypercall.h 
b/arch/x86/include/asm/xen/hypercall.h
index bfd882617613..6b2f90a0b149 100644
--- a/arch/x86/include/asm/xen/hypercall.h
+++ b/arch/x86/include/asm/xen/hypercall.h
@@ -209,24 +209,37 @@ extern struct { char _entry[32]; } hypercall_page[];
 })
 
 static inline long
-privcmd_call(unsigned call,
-unsigned long a1, unsigned long a2,
-unsigned long a3, unsigned long a4,
-unsigned long a5)
+xen_single_call(unsigned int call,
+   unsigned long a1, unsigned long a2,
+   unsigned long a3, unsigned long a4,
+   unsigned long a5)
 {
__HYPERCALL_DECLS;
__HYPERCALL_5ARG(a1, a2, a3, a4, a5);
 
-   stac();
asm volatile(CALL_NOSPEC
 : __HYPERCALL_5PARAM
 : [thunk_target] "a" (_page[call])
 : __HYPERCALL_CLOBBER5);
-   clac();
 
return (long)__res;
 }
 
+static inline long
+privcmd_call(unsigned int call,
+unsigned long a1, unsigned long a2,
+unsigned long a3, unsigned long a4,
+unsigned long a5)
+{
+   long res;
+
+   stac();
+   res = xen_single_call(call, a1, a2, a3, a4, a5);
+   clac();
+
+   return res;
+}
+
 static inline int
 HYPERVISOR_set_trap_table(struct trap_info *table)
 {
diff --git a/arch/x86/xen/multicalls.c b/arch/x86/xen/multicalls.c
index dc502ca8263e..2bce7958ce8b 100644
--- a/arch/x86/xen/multicalls.c
+++ b/arch/x86/xen/multicalls.c
@@ -80,9 +80,9 @@ void xen_mc_flush(void)
   and just do the call directly. */
mc = >entries[0];
 
-   mc->result = privcmd_call(mc->op,
- mc->args[0], mc->args[1], 
mc->args[2], 
- mc->args[3], mc->args[4]);
+   mc->result = xen_single_call(mc->op, mc->args[0], mc->args[1],
+mc->args[2], mc->args[3],
+mc->args[4]);
ret = mc->result < 0;
break;
 
-- 
2.13.7


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline baseline-only test] 74863: tolerable FAIL

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 74863 qemu-mainline real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/74863/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail like 74838
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail like 74838
 test-armhf-armhf-libvirt 12 guest-start  fail   like 74838
 test-armhf-armhf-libvirt-xsm 12 guest-start  fail   like 74838
 test-armhf-armhf-xl-multivcpu 12 guest-start  fail  like 74838
 test-armhf-armhf-xl-xsm  12 guest-start  fail   like 74838
 test-armhf-armhf-xl  12 guest-start  fail   like 74838
 test-armhf-armhf-xl-rtds 12 guest-start  fail   like 74838
 test-armhf-armhf-xl-midway   12 guest-start  fail   like 74838
 test-armhf-armhf-xl-credit2  12 guest-start  fail   like 74838
 test-amd64-amd64-xl-pvshim   12 guest-start  fail   like 74838
 test-amd64-amd64-qemuu-nested-intel 14 xen-boot/l1 fail like 74838
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 74838
 test-armhf-armhf-xl-vhd  10 debian-di-installfail   like 74838
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 74838
 test-armhf-armhf-libvirt-raw 10 debian-di-installfail   like 74838
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass

version targeted for testing:
 qemuu2afc4e3df80d947dd1bd42ce80278f591b35c74a
baseline version:
 qemuu0d2fa03dae4fbe185a082f361342b1e30aed4582

Last test of basis74838  2018-06-10 06:20:02 Z3 days
Testing same since74863  2018-06-12 22:49:27 Z0 days1 attempts


People who touched revisions under test:
  Alberto Garcia 
  Bruce Rogers 
  Jeff Cody 
  John Snow 
  Laurent Vivier 
  Mark Cave-Ayland 
  Max Reitz 
  Paolo Bonzini 
  Peter Maydell 
  Richard Henderson 
  Thomas Huth 
  Vladimir Sementsov-Ogievskiy 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  fail
 test-amd64-i386-xl   pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-xl-xsm  pass
 test-armhf-armhf-xl-xsm  fail
 test-amd64-i386-xl-xsm   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-amd64-xl-pvhv2-amdpass
 

[Xen-devel] [distros-debian-squeeze test] 74864: tolerable FAIL

[Platform Team regression test user]

flight 74864 distros-debian-squeeze real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/74864/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-i386-squeeze-netboot-pygrub 10 debian-di-install fail like 
74786
 test-amd64-i386-i386-squeeze-netboot-pygrub 10 debian-di-install fail like 
74786
 test-amd64-amd64-amd64-squeeze-netboot-pygrub 10 debian-di-install fail like 
74786
 test-amd64-i386-amd64-squeeze-netboot-pygrub 10 debian-di-install fail like 
74786

baseline version:
 flight   74786

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-squeeze-netboot-pygrubfail
 test-amd64-i386-amd64-squeeze-netboot-pygrub fail
 test-amd64-amd64-i386-squeeze-netboot-pygrub fail
 test-amd64-i386-i386-squeeze-netboot-pygrub  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 9/9] xen/gntdev: Implement dma-buf import functionality

[Oleksandr Andrushchenko]

On 06/13/2018 06:14 AM, Boris Ostrovsky wrote:



On 06/12/2018 09:42 AM, Oleksandr Andrushchenko wrote:


  int gntdev_dmabuf_imp_release(struct gntdev_dmabuf_priv *priv, u32 fd)
  {
-    return -EINVAL;
+    struct gntdev_dmabuf *gntdev_dmabuf;
+    struct dma_buf_attachment *attach;
+    struct dma_buf *dma_buf;
+
+    gntdev_dmabuf = dmabuf_imp_find_unlink(priv, fd);
+    if (IS_ERR(gntdev_dmabuf))
+    return PTR_ERR(gntdev_dmabuf);
+
+    pr_debug("Releasing DMA buffer with fd %d\n", fd);
+
+    attach = gntdev_dmabuf->u.imp.attach;
+
+    if (gntdev_dmabuf->u.imp.sgt)
+    dma_buf_unmap_attachment(attach, gntdev_dmabuf->u.imp.sgt,
+ DMA_BIDIRECTIONAL);
+    dma_buf = attach->dmabuf;
+    dma_buf_detach(attach->dmabuf, attach);
+    dma_buf_put(dma_buf);
+
+    dmabuf_imp_end_foreign_access(gntdev_dmabuf->u.imp.refs,
+  gntdev_dmabuf->nr_pages);




Should you first end foreign access, before doing anything?


I am rolling back in reverse order here, so I think we first need
to finish local activities with the buffer and then end foreign
access.

-boris



+ dmabuf_imp_free_storage(gntdev_dmabuf);
+    return 0;
  }



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [xen-unstable test] 123379: regressions - FAIL

[Juergen Gross]

On 13/06/18 10:58, Andrew Cooper wrote:
> On 13/06/18 09:52, Juergen Gross wrote:
>> On 12/06/18 17:58, Juergen Gross wrote:
>>> On 08/06/18 12:12, Juergen Gross wrote:
 On 07/06/18 13:30, Juergen Gross wrote:
> On 06/06/18 11:40, Juergen Gross wrote:
>> On 06/06/18 11:35, Jan Beulich wrote:
>> On 05.06.18 at 18:19,  wrote:
>>  test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 14 
>> guest-saverestore.2 
 I thought I would reply again with the key point from my earlier mail
 highlighted, and go a bit further.  The first thing to go wrong in
 this was:

 2018-05-30 22:12:49.320+: xc: Failed to get types for pfn batch 
 (14 = Bad address): Internal error
 2018-05-30 22:12:49.483+: xc: Save failed (14 = Bad address): 
 Internal error
 2018-05-30 22:12:49.648+: libxl-save-helper: complete r=-1: Bad 
 address

 You can see similar messages in the other logfile:

 2018-05-30 22:12:49.650+: libxl: 
 libxl_stream_write.c:350:libxl__xc_domain_save_done: Domain 3:saving 
 domain: domain responded to suspend request: Bad address

 All of these are reports of the same thing: xc_get_pfn_type_batch at
 xc_sr_save.c:133 failed with EFAULT.  I'm afraid I don't know why.

 There is no corresponding message in the host's serial log nor the
 dom0 kernel log.
>>> I vaguely recall from the time when I had looked at the similar Windows
>>> migration issues that the guest is already in the process of being 
>>> cleaned
>>> up when these occur. Commit 2dbe9c3cd2 ("x86/mm: silence a pointless
>>> warning") intentionally suppressed a log message here, and the
>>> immediately following debugging code (933f966bcd x86/mm: add
>>> temporary debugging code to get_page_from_gfn_p2m()) was reverted
>>> a little over a month later. This wasn't as a follow-up to another patch
>>> (fix), but following the discussion rooted at
>>> https://lists.xenproject.org/archives/html/xen-devel/2017-06/msg00324.html
>> That was -ESRCH, not -EFAULT.
> I've looked a little bit more into this.
>
> As we are seeing EFAULT being returned by the hypervisor this either
> means the tools are specifying an invalid address (quite unlikely)
> or the buffers are not as MAP_LOCKED as we wish them to be.
>
> Is there a way to see whether the host was experiencing some memory
> shortage, so the buffers might have been swapped out?
>
> man mmap tells me: "This implementation will try to populate (prefault)
> the whole range but the mmap call doesn't fail with ENOMEM if this
> fails. Therefore major faults might happen later on."
>
> And: "One should use mmap(2) plus mlock(2) when major faults are not
> acceptable after the initialization of the mapping."
>
> With osdep_alloc_pages() in tools/libs/call/linux.c touching all the
> hypercall buffer pages before doing the hypercall I'm not sure this
> could be an issue.
>
> Any thoughts on that?
 Ian, is there a chance to dedicate a machine to a specific test trying
 to reproduce the problem? In case we manage to get this failure in a
 reasonable time frame I guess the most promising approach would be to
 use a test hypervisor producing more debug data. If you think this is
 worth doing I can write a patch.
>>> Trying to reproduce the problem in a limited test environment finally
>>> worked: doing a loop of "xl save -c" produced the problem after 198
>>> iterations.
>>>
>>> I have asked a SUSE engineer doing kernel memory management if he
>>> could think of something. His idea is that maybe some kthread could be
>>> the reason for our problem, e.g. trying page migration or compaction
>>> (at least on the test machine I've looked at compaction of mlocked
>>> pages is allowed: /proc/sys/vm/compact_unevictable_allowed is 1).
>>>
>>> In order to be really sure nothing in the kernel can temporarily
>>> switch hypercall buffer pages read-only or invalid for the hypervisor
>>> we'll have to modify the privcmd driver interface: it will have to
>>> gain knowledge which pages are handed over to the hypervisor as buffers
>>> in order to be able to lock them accordingly via get_user_pages().
>>>
>>> While this is a possible explanation of the fault we are seeing it might
>>> be related to another reason. So I'm going to apply some modifications
>>> to the hypervisor to get some more diagnostics in order to verify the
>>> suspected kernel behavior is really the reason for the hypervisor to
>>> return EFAULT.
>> I was lucky. Took only 39 iterations this time.
>>
>> The debug data confirms the theory that the kernel is setting the PTE to
>> invalid or read only for a short amount of time:
>>
>> (XEN) fixup for address 7ffb9904fe44, error_code 0002:
>> (XEN) 

Re: [Xen-devel] [xen-unstable test] 123379: regressions - FAIL

[Andrew Cooper]

On 13/06/18 09:52, Juergen Gross wrote:
> On 12/06/18 17:58, Juergen Gross wrote:
>> On 08/06/18 12:12, Juergen Gross wrote:
>>> On 07/06/18 13:30, Juergen Gross wrote:
 On 06/06/18 11:40, Juergen Gross wrote:
> On 06/06/18 11:35, Jan Beulich wrote:
> On 05.06.18 at 18:19,  wrote:
>  test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 14 
> guest-saverestore.2 
>>> I thought I would reply again with the key point from my earlier mail
>>> highlighted, and go a bit further.  The first thing to go wrong in
>>> this was:
>>>
>>> 2018-05-30 22:12:49.320+: xc: Failed to get types for pfn batch (14 
>>> = Bad address): Internal error
>>> 2018-05-30 22:12:49.483+: xc: Save failed (14 = Bad address): 
>>> Internal error
>>> 2018-05-30 22:12:49.648+: libxl-save-helper: complete r=-1: Bad 
>>> address
>>>
>>> You can see similar messages in the other logfile:
>>>
>>> 2018-05-30 22:12:49.650+: libxl: 
>>> libxl_stream_write.c:350:libxl__xc_domain_save_done: Domain 3:saving 
>>> domain: domain responded to suspend request: Bad address
>>>
>>> All of these are reports of the same thing: xc_get_pfn_type_batch at
>>> xc_sr_save.c:133 failed with EFAULT.  I'm afraid I don't know why.
>>>
>>> There is no corresponding message in the host's serial log nor the
>>> dom0 kernel log.
>> I vaguely recall from the time when I had looked at the similar Windows
>> migration issues that the guest is already in the process of being 
>> cleaned
>> up when these occur. Commit 2dbe9c3cd2 ("x86/mm: silence a pointless
>> warning") intentionally suppressed a log message here, and the
>> immediately following debugging code (933f966bcd x86/mm: add
>> temporary debugging code to get_page_from_gfn_p2m()) was reverted
>> a little over a month later. This wasn't as a follow-up to another patch
>> (fix), but following the discussion rooted at
>> https://lists.xenproject.org/archives/html/xen-devel/2017-06/msg00324.html
> That was -ESRCH, not -EFAULT.
 I've looked a little bit more into this.

 As we are seeing EFAULT being returned by the hypervisor this either
 means the tools are specifying an invalid address (quite unlikely)
 or the buffers are not as MAP_LOCKED as we wish them to be.

 Is there a way to see whether the host was experiencing some memory
 shortage, so the buffers might have been swapped out?

 man mmap tells me: "This implementation will try to populate (prefault)
 the whole range but the mmap call doesn't fail with ENOMEM if this
 fails. Therefore major faults might happen later on."

 And: "One should use mmap(2) plus mlock(2) when major faults are not
 acceptable after the initialization of the mapping."

 With osdep_alloc_pages() in tools/libs/call/linux.c touching all the
 hypercall buffer pages before doing the hypercall I'm not sure this
 could be an issue.

 Any thoughts on that?
>>> Ian, is there a chance to dedicate a machine to a specific test trying
>>> to reproduce the problem? In case we manage to get this failure in a
>>> reasonable time frame I guess the most promising approach would be to
>>> use a test hypervisor producing more debug data. If you think this is
>>> worth doing I can write a patch.
>> Trying to reproduce the problem in a limited test environment finally
>> worked: doing a loop of "xl save -c" produced the problem after 198
>> iterations.
>>
>> I have asked a SUSE engineer doing kernel memory management if he
>> could think of something. His idea is that maybe some kthread could be
>> the reason for our problem, e.g. trying page migration or compaction
>> (at least on the test machine I've looked at compaction of mlocked
>> pages is allowed: /proc/sys/vm/compact_unevictable_allowed is 1).
>>
>> In order to be really sure nothing in the kernel can temporarily
>> switch hypercall buffer pages read-only or invalid for the hypervisor
>> we'll have to modify the privcmd driver interface: it will have to
>> gain knowledge which pages are handed over to the hypervisor as buffers
>> in order to be able to lock them accordingly via get_user_pages().
>>
>> While this is a possible explanation of the fault we are seeing it might
>> be related to another reason. So I'm going to apply some modifications
>> to the hypervisor to get some more diagnostics in order to verify the
>> suspected kernel behavior is really the reason for the hypervisor to
>> return EFAULT.
> I was lucky. Took only 39 iterations this time.
>
> The debug data confirms the theory that the kernel is setting the PTE to
> invalid or read only for a short amount of time:
>
> (XEN) fixup for address 7ffb9904fe44, error_code 0002:
> (XEN) Pagetable walk from 7ffb9904fe44:
> (XEN)  L4[0x0ff] = 000458da6067 00019190
> (XEN)  L3[0x1ee] = 000457d26067 

Re: [Xen-devel] 4.11.0 RC1 panic

[Manuel Bouyer]

On Wed, Jun 13, 2018 at 02:07:29AM -0600, Jan Beulich wrote:
> 
> (XEN) Assertion '!page->linear_pt_count' failed at mm.c:596
> 
> In fact, there's no assertion with that expression anywhere I could
> see. Do you have any local patches in place?

Yes, 2 of them from you (the first one is where the assert is). See attached.

> In any event, to take
> care of the other assertion you've hit below an updated debugging
> patch. I hope I didn't overlook any further (cascade) ones.

Will rebuild with it, I'll keep you informed.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--
(besides a more reliable confirmation - or otherwise - that this indeed
is an issue with 32-bit guests only).

While I think I have ruled out the TLB flush time stamp setting still
happening too early / wrongly in certain cases, there's a small
debugging patch that I would hope could help prove this one or the
other way (see below).

Btw: You've said earlier that there wouldn't be a domain number in
the panic message. However,

(XEN) RFLAGS: 00010246   CONTEXT: hypervisor (d14v3)

has it (at the end: domain 14, vCPU 3). Just in case this helps
identifying further useful pieces of information.

Jan

--- xen/arch/x86/mm.c.orig
+++ xen/arch/x86/mm.c
@@ -578,7 +578,11 @@ static inline void set_tlbflush_timestam
  */
 if ( !(page->count_info & PGC_page_table) ||
  !shadow_mode_enabled(page_get_owner(page)) )
+{
+/* NB: This depends on WRAP_MASK in flushtlb.c to be <= 0x. */
+ASSERT(!page->linear_pt_count);
 page_set_tlbflush_timestamp(page);
+}
 }
 
 const char __section(".bss.page_aligned.const") __aligned(PAGE_SIZE)


$NetBSD: $

Commit df8234fd2c ("replace vCPU's dirty CPU mask by numeric ID") was
too lax in two respects: First of all it didn't consider the case of a
vCPU not having a valid dirty CPU in the descriptor table TLB flush
case. This is the issue Manual has run into with NetBSD.

Additionally reads of ->dirty_cpu for other than the current vCPU are at
risk of racing with scheduler actions, i.e. single atomic reads need to
be used there. Obviously the non-init write sites then better also use
atomic writes.

Having to touch the descriptor table TLB flush code here anyway, take
the opportunity and switch it to be at most one flush_tlb_mask()
invocation.

Reported-by: Manuel Bouyer 
Signed-off-by: Jan Beulich 

--- xen/arch/x86/domain.c.orig
+++ xen/arch/x86/domain.c
@@ -1631,7 +1631,7 @@ static void __context_switch(void)
  */
 if ( pd != nd )
 cpumask_set_cpu(cpu, nd->dirty_cpumask);
-n->dirty_cpu = cpu;
+write_atomic(>dirty_cpu, cpu);
 
 if ( !is_idle_domain(nd) )
 {
@@ -1687,7 +1687,7 @@ static void __context_switch(void)
 
 if ( pd != nd )
 cpumask_clear_cpu(cpu, pd->dirty_cpumask);
-p->dirty_cpu = VCPU_CPU_CLEAN;
+write_atomic(>dirty_cpu, VCPU_CPU_CLEAN);
 
 per_cpu(curr_vcpu, cpu) = n;
 }
--- xen/arch/x86/mm.c.orig
+++ xen/arch/x86/mm.c
@@ -1202,11 +1202,23 @@ void put_page_from_l1e(l1_pgentry_t l1e,
  unlikely(((page->u.inuse.type_info & PGT_count_mask) != 0)) &&
  (l1e_owner == pg_owner) )
 {
+cpumask_t *mask = this_cpu(scratch_cpumask);
+
+cpumask_clear(mask);
+
 for_each_vcpu ( pg_owner, v )
 {
-if ( pv_destroy_ldt(v) )
-flush_tlb_mask(cpumask_of(v->dirty_cpu));
+unsigned int cpu;
+
+if ( !pv_destroy_ldt(v) )
+continue;
+cpu = read_atomic(>dirty_cpu);
+if ( is_vcpu_dirty_cpu(cpu) )
+__cpumask_set_cpu(cpu, mask);
 }
+
+if ( !cpumask_empty(mask) )
+flush_tlb_mask(mask);
 }
 put_page(page);
 }
@@ -2979,13 +2991,18 @@ static inline int vcpumask_to_pcpumask(
 
 while ( vmask )
 {
+unsigned int cpu;
+
 vcpu_id = find_first_set_bit(vmask);
 vmask &= ~(1UL << vcpu_id);
 vcpu_id += vcpu_bias;
 if ( (vcpu_id >= d->max_vcpus) )
 return 0;
-if ( ((v = d->vcpu[vcpu_id]) != NULL) && vcpu_cpu_dirty(v) )
-__cpumask_set_cpu(v->dirty_cpu, pmask);
+if ( (v = d->vcpu[vcpu_id]) == NULL )
+continue;
+cpu = read_atomic(>dirty_cpu);
+if ( is_vcpu_dirty_cpu(cpu) )
+__cpumask_set_cpu(cpu, pmask);
 }
 }
 }
--- xen/include/xen/sched.h.orig
+++ xen/include/xen/sched.h
@@ -795,10 +795,15 @@ static inline int vcpu_runnable(struct v
  atomic_read(>domain->pause_count));
 }
 
-static inline bool vcpu_cpu_dirty(const struct vcpu *v)
+static inline bool is_vcpu_dirty_cpu(unsigned int cpu)
 {
 BUILD_BUG_ON(NR_CPUS >= VCPU_CPU_CLEAN);
-return v->dirty_cpu != VCPU_CPU_CLEAN;
+return cpu != VCPU_CPU_CLEAN;
+}
+

[Xen-devel] [PATCH V2 2/2] x86/altp2m: Fixed domain crash with INVALID_ALTP2M EPTP index

[Razvan Cojocaru]

vcpu_altp2m(v).p2midx can become INVALID_ALTP2M with normal
usage (in altp2m_vcpu_reset()), which can then result in that
value being __vmwritten() in EPTP_INDEX by vmx_vcpu_update_eptp().
The value can then end up being __vmread() in vmx_vmexit_handler()
which then calls BUG_ON(idx >= MAX_ALTP2M). Since MAX_ALTP2M is
currently 10 and INVALID_ALTP2M is #defined as 0x, the
domain will always crash in this case.

Signed-off-by: Razvan Cojocaru 

---
Cc: Jun Nakajima 
Cc: Kevin Tian 
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Tamas K Lengyel 
---
 xen/arch/x86/hvm/vmx/vmx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 9707514..c7f3925 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -3592,7 +3592,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
 }
 }
 
-if ( idx != vcpu_altp2m(v).p2midx )
+if ( idx != INVALID_ALTP2M && idx != vcpu_altp2m(v).p2midx )
 {
 BUG_ON(idx >= MAX_ALTP2M);
 atomic_dec(_get_altp2m(v)->active_vcpus);
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2 1/2] xen/altp2m: set access_required properly for all altp2ms

[Razvan Cojocaru]

For the hostp2m, access_required starts off as 0, then it can be
set with xc_domain_set_access_required(). However, all the altp2ms
set it to 1 on init, and ignore both the hostp2m and the hypercall.
This patch sets access_required to the value from the hostp2m
on altp2m init, and propagates the values received via hypercall
to all the active altp2ms, when applicable.

Signed-off-by: Razvan Cojocaru 

---
Cc: Stefano Stabellini 
Cc: Julien Grall 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Tim Deegan 
Cc: Wei Liu 
Cc: George Dunlap 
Cc: Tamas K Lengyel 
---
Changes since V1:
 - Corrected typos in the commit message.
 - Moved arch_domain_set_access_required() to x86/mm/mem_access.c
   and arm/mem_access.c.
 - Fixed wrong tab type.
 - Removed the d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) check.
 - Moved the setting of p2m_get_hostp2m(d)->access_required to
   the arch_domain_set_access_required() function.
---
 xen/arch/arm/mem_access.c|  5 +
 xen/arch/x86/mm/mem_access.c | 18 ++
 xen/arch/x86/mm/p2m.c|  3 ++-
 xen/common/domctl.c  |  4 ++--
 xen/include/xen/domain.h |  2 ++
 5 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/mem_access.c b/xen/arch/arm/mem_access.c
index ae2686f..a59c6ef 100644
--- a/xen/arch/arm/mem_access.c
+++ b/xen/arch/arm/mem_access.c
@@ -453,6 +453,11 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn,
 return ret;
 }
 
+void arch_domain_set_access_required(struct domain *d, bool access_required)
+{
+p2m_get_hostp2m(d)->access_required = access_required;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index c0cd017..6811572 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -465,6 +465,24 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, 
xenmem_access_t *access)
 return _p2m_get_mem_access(p2m, gfn, access);
 }
 
+void arch_domain_set_access_required(struct domain *d, bool access_required)
+{
+unsigned int i;
+
+p2m_get_hostp2m(d)->access_required = access_required;
+
+if ( !altp2m_active(d) )
+return;
+
+for ( i = 0; i < MAX_ALTP2M; i++ )
+{
+struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
+
+if ( p2m )
+p2m->access_required = access_required;
+}
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index c53cab4..8e9fbb5 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -199,6 +199,7 @@ static int p2m_init_altp2m(struct domain *d)
 {
 unsigned int i;
 struct p2m_domain *p2m;
+struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
 
 mm_lock_init(>arch.altp2m_list_lock);
 for ( i = 0; i < MAX_ALTP2M; i++ )
@@ -210,7 +211,7 @@ static int p2m_init_altp2m(struct domain *d)
 return -ENOMEM;
 }
 p2m->p2m_class = p2m_alternate;
-p2m->access_required = 1;
+p2m->access_required = hostp2m->access_required;
 _atomic_set(>active_vcpus, 0);
 }
 
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 9b7bc08..37f174f 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -1092,8 +1092,8 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
u_domctl)
 else
 {
 domain_pause(d);
-p2m_get_hostp2m(d)->access_required =
-op->u.access_required.access_required;
+arch_domain_set_access_required(d,
+op->u.access_required.access_required);
 domain_unpause(d);
 }
 break;
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 177cb35..9df53a6 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -66,6 +66,8 @@ void arch_domain_unpause(struct domain *d);
 
 int arch_domain_soft_reset(struct domain *d);
 
+void arch_domain_set_access_required(struct domain *d, bool access_required);
+
 int arch_set_info_guest(struct vcpu *, vcpu_guest_context_u);
 void arch_get_info_guest(struct vcpu *, vcpu_guest_context_u);
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-linus bisection] complete test-armhf-armhf-xl

[osstest service owner]

branch xen-unstable
xenbranch xen-unstable
job test-armhf-armhf-xl
testid xen-boot

Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  linux 
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
  Bug introduced:  8efcf34a263965e471e304f94d1f6799d42a
  Bug not present: 786b71f5b754273ccef6d9462e52062b3e1f9877
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/124161/


  (Revision log too long, omitted.)


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/linux-linus/test-armhf-armhf-xl.xen-boot.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/linux-linus/test-armhf-armhf-xl.xen-boot 
--summary-out=tmp/124161.bisection-summary --basis-template=123554 
--blessings=real,real-bisect linux-linus test-armhf-armhf-xl xen-boot
Searching for failure / basis pass:
 124092 fail [host=cubietruck-metzinger] / 124013 [host=arndale-lakeside] 
123937 [host=arndale-lakeside] 123871 [host=arndale-metrocentre] 123792 
[host=cubietruck-gleizes] 123655 [host=cubietruck-braque] 123554 
[host=arndale-bluewater] 123438 [host=arndale-westfield] 123370 
[host=arndale-lakeside] 123310 ok.
Failure / basis pass flights: 124092 / 123310
Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 8efcf34a263965e471e304f94d1f6799d42a 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
11535cdbc0ae5925a55b3e735447c30faaa6f63b
Basis pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
fc5805daef091240cd5fc06634a8bcdb2f3bb843
Generating revisions with ./adhoc-revtuple-generator  
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git#786b71f5b754273ccef6d9462e52062b3e1f9877-8efcf34a263965e471e304f94d1f6799d42a
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen.git#43139135a8938de44f66333831d3a8655d07663a-43139135a8938de44f66333831d3a8655d07663a
 
git://xenbits.xen.org/xen.git#fc5805daef091240cd5fc06634a8bcdb2f3bb843-11535cdbc0ae5925a55b3e735447c30faaa6f63b
adhoc-revtuple-generator: tree discontiguous: linux-2.6
Loaded 1002 nodes in revision graph
Searching for test results:
 123188 [host=cubietruck-gleizes]
 123218 [host=arndale-bluewater]
 123271 [host=arndale-metrocentre]
 123310 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
fc5805daef091240cd5fc06634a8bcdb2f3bb843
 123370 [host=arndale-lakeside]
 123438 [host=arndale-westfield]
 123554 [host=arndale-bluewater]
 123655 [host=cubietruck-braque]
 123792 [host=cubietruck-gleizes]
 123937 [host=arndale-lakeside]
 123871 [host=arndale-metrocentre]
 124013 [host=arndale-lakeside]
 124047 fail irrelevant
 124066 fail irrelevant
 124106 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
35fcb982ea16c40619fee8bba4789a94d824521e
 124153 fail 8efcf34a263965e471e304f94d1f6799d42a 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
11535cdbc0ae5925a55b3e735447c30faaa6f63b
 124088 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
fc5805daef091240cd5fc06634a8bcdb2f3bb843
 124160 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
11535cdbc0ae5925a55b3e735447c30faaa6f63b
 124117 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
9fa730c3576e7f009417c3e25f50b362ec222725
 124096 fail irrelevant
 124105 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
06f542f8f2e446c01bd0edab51e9450af7f6e05b
 124129 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
12915da5732444c8c891d19773ea1df1858d00bd
 124092 fail 8efcf34a263965e471e304f94d1f6799d42a 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
43139135a8938de44f66333831d3a8655d07663a 
11535cdbc0ae5925a55b3e735447c30faaa6f63b
 124147 pass 786b71f5b754273ccef6d9462e52062b3e1f9877 

Re: [Xen-devel] [RFC PATCH 06/12] xen-blkfront: add callbacks for PM suspend and hibernation

[Roger Pau Monné]

On Tue, Jun 12, 2018 at 08:56:13PM +, Anchal Agarwal wrote:
> From: Munehisa Kamata 
> 
> Add freeze and restore callbacks for PM suspend and hibernation support.
> The freeze handler stops a block-layer queue and disconnect the frontend
> from the backend while freeing ring_info and associated resources. The
> restore handler re-allocates ring_info and re-connect to the backedend,
> so the rest of the kernel can continue to use the block device
> transparently.Also, the handlers are used for both PM
> suspend and hibernation so that we can keep the existing suspend/resume
> callbacks for Xen suspend without modification.
> If a backend doesn't have commit 12ea729645ac ("xen/blkback: unmap all
> persistent grants when frontend gets disconnected"), the frontend may see
> massive amount of grant table warning when freeing resources.
> 
>  [   36.852659] deferring g.e. 0xf9 (pfn 0x)
>  [   36.855089] xen:grant_table: WARNING: g.e. 0x112 still in use!
> 
> In this case, persistent grants would need to be disabled.
> 
> Ensure no reqs/rsps in rings before disconnecting. When disconnecting
> the frontend from the backend in blkfront_freeze(), there still may be
> unconsumed requests or responses in the rings, especially when the
> backend is backed by network-based device. If the frontend gets
> disconnected with such reqs/rsps remaining there, it can cause
> grant warnings and/or losing reqs/rsps by freeing pages afterward.

I'm not sure why having pending requests can cause grant warnings or
lose of requests. If handled properly this shouldn't be an issue.
Linux blkfront already does live migration (which also involves a
reconnection of the frontend) with pending requests and that doesn't
seem to be an issue.

> This can lead resumed kernel into unrecoverable state like unexpected
> freeing of grant page and/or hung task due to the lost reqs or rsps.
> Therefore we have to ensure that there is no unconsumed requests or
> responses before disconnecting.

Given that we have multiqueue, plus multipage rings, I'm not sure
waiting for the requests on the rings to complete is a good idea.

Why can't you just disconnect the frontend and requeue all the
requests in flight? When the frontend connects on resume those
requests will be queued again.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 7/9] xen/gntdev: Add initial support for dma-buf UAPI

[Oleksandr Andrushchenko]

On 06/13/2018 04:49 AM, Boris Ostrovsky wrote:



On 06/12/2018 09:41 AM, Oleksandr Andrushchenko wrote:


diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index a09db23e9663..e82660d81d7e 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -48,6 +48,9 @@
  #include 
    #include "gntdev-common.h"
+#ifdef CONFIG_XEN_GNTDEV_DMABUF
+#include "gntdev-dmabuf.h"
+#endif
    MODULE_LICENSE("GPL");
  MODULE_AUTHOR("Derek G. Murray , "
@@ -566,6 +569,15 @@ static int gntdev_open(struct inode *inode, 
struct file *flip)

  INIT_LIST_HEAD(>freeable_maps);
  mutex_init(>lock);
  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+    priv->dmabuf_priv = gntdev_dmabuf_init();
+    if (IS_ERR(priv->dmabuf_priv)) {
+    ret = PTR_ERR(priv->dmabuf_priv);
+    kfree(priv);
+    return ret;
+    }
+#endif
+
  if (use_ptemod) {
  priv->mm = get_task_mm(current);
  if (!priv->mm) {
@@ -616,8 +628,13 @@ static int gntdev_release(struct inode *inode, 
struct file *flip)

  WARN_ON(!list_empty(>freeable_maps));
  mutex_unlock(>lock);
  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+    gntdev_dmabuf_fini(priv->dmabuf_priv);
+#endif
+
  if (use_ptemod)
  mmu_notifier_unregister(>mn, priv->mm);
+
  kfree(priv);
  return 0;
  }
@@ -987,6 +1004,107 @@ static long gntdev_ioctl_grant_copy(struct 
gntdev_priv *priv, void __user *u)

  return ret;
  }
  +#ifdef CONFIG_XEN_GNTDEV_DMABUF
+static long
+gntdev_ioctl_dmabuf_exp_from_refs(struct gntdev_priv *priv,
+  struct ioctl_gntdev_dmabuf_exp_from_refs __user *u)



Didn't we agree that this code moves to gntdev-dmabuf.c ?


Sure, didn't think we want IOCTL's code to be moved as well,
but that does make sense - will move all

-boris


Thank you,
Oleksandr

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel