Re: [Xen-devel] [PATCH] xen-netfront: remove warning when unloading module

[Wei Liu]

CC netfront maintainers.

On Mon, Nov 20, 2017 at 11:41:09AM +0100, Eduardo Otubo wrote:
> When unloading module xen_netfront from guest, dmesg would output
> warning messages like below:
> 
>   [  105.236836] xen:grant_table: WARNING: g.e. 0x903 still in use!
>   [  105.236839] deferring g.e. 0x903 (pfn 0x35805)
> 
> This problem relies on netfront and netback being out of sync. By the time
> netfront revokes the g.e.'s netback didn't have enough time to free all of
> them, hence displaying the warnings on dmesg.
> 
> The trick here is to make netfront to wait until netback frees all the g.e.'s
> and only then continue to cleanup for the module removal, and this is done by
> manipulating both device states.
> 
> Signed-off-by: Eduardo Otubo 
> ---
>  drivers/net/xen-netfront.c | 11 +++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> index 8b8689c6d887..b948e2a1ce40 100644
> --- a/drivers/net/xen-netfront.c
> +++ b/drivers/net/xen-netfront.c
> @@ -2130,6 +2130,17 @@ static int xennet_remove(struct xenbus_device *dev)
>  
>   dev_dbg(>dev, "%s\n", dev->nodename);
>  
> + xenbus_switch_state(dev, XenbusStateClosing);
> + while (xenbus_read_driver_state(dev->otherend) != XenbusStateClosing){
> + cpu_relax();
> + schedule();
> + }
> + xenbus_switch_state(dev, XenbusStateClosed);
> + while (dev->xenbus_state != XenbusStateClosed){
> + cpu_relax();
> + schedule();
> + }
> +
>   xennet_disconnect_backend(info);
>  
>   unregister_netdev(info->netdev);
> -- 
> 2.13.6
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] tools/libxl: mark hvm mmio area as reserved in e820 map

[Wei Liu]

On Fri, Nov 17, 2017 at 12:47:33PM +0100, Juergen Gross wrote:
> Make sure the HVM mmio area (especially console and Xenstore pages) is
> marked as "reserved" in the guest's E820 map, as otherwise conflicts
> might arise later, e.g. when hotplugging memory into the guest.
> 
> Signed-off-by: Juergen Gross <jgr...@suse.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

> ---
> This is a bugfix for PVH and HVM guests. Please consider for 4.10.

I agree this is 4.10 material.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] x86/hvm: Don't corrupt the HVM context stream when writing the MSR record

[Wei Liu]

On Thu, Nov 16, 2017 at 10:45:16PM +, Andrew Cooper wrote:
> Ever since it was introduced in c/s bd1f0b45ff, hvm_save_cpu_msrs() has had a
> bug whereby it corrupts the HVM context stream if some, but fewer than the
> maximum number of MSRs are written.
> 
> _hvm_init_entry() creates an hvm_save_descriptor with length for
> msr_count_max, but in the case that we write fewer than max, h->cur only moves
> forward by the amount of space used, causing the subsequent
> hvm_save_descriptor to be written within the bounds of the previous one.
> 
> To resolve this, reduce the length reported by the descriptor to match the
> actual number of bytes used.
> 
> A typical failure on the destination side looks like:
> 
> (XEN) HVM4 restore: CPU_MSR 0
> (XEN) HVM4.0 restore: not enough data left to read 56 MSR bytes
> (XEN) HVM4 restore: failed to load entry 20/0
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] tools/libxc: Fix restoration of PV MSRs after migrate

[Wei Liu]

On Thu, Nov 16, 2017 at 09:13:22PM +, Andrew Cooper wrote:
> There are two bugs in process_vcpu_msrs() which clearly demonstrate that I
> didn't test this bit of Migration v2 very well when writing it...
> 
> vcpu->msrsz is always expected to be a multiple of xen_domctl_vcpu_msr_t
> records in a spec-compliant stream, so the modulo yields 0 for the msr_count,
> rather than the actual number sent in the stream.
> 
> Passing 0 for the msr_count causes the hypercall to exit early, and hides the
> fact that the guest handle is inserted into the wrong field in the domctl
> union.
> 
> The reason that these bugs have gone unnoticed for so long is that the only
> MSRs passed like this for PV guests are the AMD DBGEXT MSRs, which only exist
> in fairly modern hardware, and whose use doesn't appear to be implemented in
> any contemporary PV guests.
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] x86/hvm: Don't ignore unknown MSRs in the migration stream

[Wei Liu]

On Thu, Nov 16, 2017 at 07:15:32PM +, Andrew Cooper wrote:
> Doing so amounts to silent state corruption, and must be avoided.
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] tools: xentoolcore_restrict_all: Do deregistration before close

[Wei Liu]

On Tue, Nov 14, 2017 at 12:15:42PM +, Ian Jackson wrote:
> Closing the fd before unhooking it from the list runs the risk that a
> concurrent thread calls xentoolcore_restrict_all will operate on the
> old fd value, which might refer to a new fd by then.  So we need to do
> it in the other order.
> 
> Sadly this weakens the guarantee provided by xentoolcore_restrict_all
> slight, but not (I think) in a problematic way.  It would be possible

slightly

> to implement the previous guarantee, but it would involve replacing
> all of the close() calls in all of the individual osdep parts of all
> of the individual libraries with calls to a new function which does
>dup2("/dev/null", thing->fd);
>pthread_mutex_lock(_lock);
>thing->fd = -1;
>pthread_mutex_unlock(_lock);
>close(fd);
> which would be terribly tedious.
> 
> Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/4 v3 for-4.10] libxl: Fix the bug introduced in commit "libxl: use correct type modifier for vuart_gfn"

[Wei Liu]

On Tue, Nov 14, 2017 at 05:12:26PM +0530, Bhupinder Thakur wrote:
> Hi,
> 
> On 14 Nov 2017 3:35 pm, "Wei Liu" <wei.l...@citrix.com> wrote:
> 
> > On Mon, Nov 13, 2017 at 03:56:23PM +, Julien Grall wrote:
> > > Hi Wei,
> > >
> > > Sorry I missed that e-mail.
> > >
> > > On 10/31/2017 10:07 AM, Wei Liu wrote:
> > > > Change the tag to for-4.10.
> > > >
> > > > Julien, this is needed to fix vuart emulation.
> > >
> > > To confirm, only patch #1 is candidate for Xen 4.10, right? The rest
> > will be
> > > queued for Xen 4.11?
> > >
> >
> > I think so.
> >
> > Bhupinder, can you confirm that?
> >
> 
> Yes. Only first patch is required for fixing the compilation issue.
> 

Thanks. I will commit the first patch and put the rest in my for-next
queue.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] libs/evtchn: Remove active handler on clean-up or failure

[Wei Liu]

On Tue, Nov 14, 2017 at 12:14:14PM +, Julien Grall wrote:
> Hi,
> 
> On 14/11/17 11:51, Ian Jackson wrote:
> > Ross Lagerwall writes ("Re: [PATCH for-4.10] libs/evtchn: Remove active 
> > handler on clean-up or failure"):
> > > On 11/10/2017 05:10 PM, Julien Grall wrote:
> > > > Commit 89d55473ed16543044a31d1e0d4660cf5a3f49df 
> > > > "xentoolcore_restrict_all:
> > > > Implement for libxenevtchn" added a call to register allowing to
> > > > restrict the event channel.
> > > > 
> > > > However, the call to deregister the handler was not performed if open
> > > > failed or when closing the event channel. This will result to corrupt
> > > > the list of handlers and potentially crash the application later one.
> > 
> > Sorry for not spotting this during review.
> > The fix is correct as far as it goes, so:
> > 
> > Acked-by: Ian Jackson 
> > 
> > > > The call to xentoolcore_deregister_active_handle is done at the same
> > > > place as for the grants. But I am not convinced this is thread safe as
> > > > there are potential race between close the event channel and restict
> > > > handler. Do we care about that?
> > ...
> > > However, I think it should call xentoolcore__deregister_active_handle()
> > > _before_ calling osdep_evtchn_close() to avoid trying to restrict a
> > > closed fd or some other fd that happens to have the same number.
> > 
> > You are right.  But this slightly weakens the guarantee provided by
> > xentoolcore_restrict_all.
> > 
> > > I think all the other libs need to be fixed as well, unless there was a
> > > reason it was done this way.
> > 
> > I will send a further patch.  In the meantime I suggest we apply
> > Julien's fix.
> 
> I am going to leave the decision to you and Wei. It feels a bit odd to
> release-ack my patch :).

We can only commit patches that are both acked and release-acked. The
latter gives RM control over when the patch should be applied.
Sometimes it is better to wait until something else happens (like
getting the tree to a stable state).

That's how I used release-ack anyway.

For this particular patch, my interpretation of what you just said
is you've given us release-ack and we can apply this patch anytime. I
will commit it soon.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/4 v3 for-4.10] libxl: Fix the bug introduced in commit "libxl: use correct type modifier for vuart_gfn"

[Wei Liu]

On Mon, Nov 13, 2017 at 03:56:23PM +, Julien Grall wrote:
> Hi Wei,
> 
> Sorry I missed that e-mail.
> 
> On 10/31/2017 10:07 AM, Wei Liu wrote:
> > Change the tag to for-4.10.
> > 
> > Julien, this is needed to fix vuart emulation.
> 
> To confirm, only patch #1 is candidate for Xen 4.10, right? The rest will be
> queued for Xen 4.11?
> 

I think so. 

Bhupinder, can you confirm that?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] Config.mk: Update QEMU changeset

[Wei Liu]

On Mon, Nov 13, 2017 at 12:27:32PM +, Anthony PERARD wrote:
> New commits:
> - xen/pt: allow QEMU to request MSI unmasking at bind time
> To fix a passthrough bug.
> - ui/gtk: Fix deprecation of vte_terminal_copy_clipboard
> A build fix.
> 
> Signed-off-by: Anthony PERARD 
> ---
> Should already be released-acked.

Applied.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [qemu-upstream-unstable test] 116118: FAIL

[Wei Liu]

On Mon, Nov 13, 2017 at 11:52:12AM +, Julien Grall wrote:
> Hi,
> 
> On 11/13/2017 06:44 AM, osstest service owner wrote:
> > flight 116118 qemu-upstream-unstable real [real]
> > http://logs.test-lab.xenproject.org/osstest/logs/116118/
> > 
> > Failures and problems with tests :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >   build-armhf-pvopsbroken  in 
> > 115713
> >   build-armhf-pvops  4 host-install(4) broken in 115713 REGR. vs. 
> > 114457
> 
> Looking at the test result, build-armhf-pvops seems to have passed nicely
> the past few weeks but one time.
> 
> So I am not sure why we are blocking here. Mostly the  one.

host-install failure is probably not related to change in code. It is
trying to install a host to do test. In this case, to build kernel.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] docs: update hvmlite.markdown

[Wei Liu]

On Mon, Nov 13, 2017 at 03:26:13AM -0700, Jan Beulich wrote:
> >>> On 12.11.17 at 12:03,  wrote:
> > --- a/docs/misc/hvmlite.markdown
> > +++ b/docs/misc/hvmlite.markdown
> > @@ -1,6 +1,3 @@
> > -**NOTE**: this document will be merged into `pvh.markdown` once PVH is 
> > replaced
> > -with the HVMlite implementation.
> > -
> 
> This being stale, wouldn't it then be better to rename the doc to
> pvh.markdown at the same time? Either way

Will do.

> Acked-by: Jan Beulich 
> 

Thanks.

> Jan
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] 答复: Re: Help：Can xen restore several snapshots more faster at same time？

[Wei Liu]

Please avoid top-posting.

On Mon, Nov 13, 2017 at 08:25:16AM +, Chenjia (C) wrote:
> 1.   is there some way to improve the xenstored process performance?
> 

The latest version of Cxenstored and Oxenstored have improved
transaction handling. Not sure which version you're using.

> 2.   We also try the virsh tool to restore several xen at same
> time ,we fount the virsh can restore 40+ snapshot(1G per snapshot) at
> same time, the performance is good when snapshot is all in ramdisk.
> But we can’t let all the snapshot is always in ramdisk for it is too
> big. Is there some way to reduce these virsh snapshots space?(these
> snapshot is same: same OS, same config, but need with different IP
> address)

For libvirt question you need to ask on libvirt list.

> 
> Would you please offer us some feedback? Thanks.
> By the way, can we talk with Chinese if possible?

Sorry but communication on mailing list needs to be in English so other
people can join the discussion and / or provide suggestions.

> 
> 发件人: HUANG SHENGQIANG
> 发送时间: 2017年11月6日 18:32
> 收件人: Chenjia (C) <chenji...@huawei.com<mailto:chenji...@huawei.com>>
> 主题: FW: Re: [Xen-devel] Help：Can xen restore several snapshots more faster at 
> same time？
> 
> --
> HUANG SHENGQIANG HUANG SHENGQIANG
> M: 
> +86-18201587800<tel:+86-18201587800>(优先)/+1-6046180423<tel:+1-6046180423>(海外出差)
> E: huang.shengqi...@huawei.com<mailto:huang.shengqi...@huawei.com>
> 产品与解决方案-交换机与企业网关解决方案架构与设计部
> Products & Solutions-Switch & Enterprise Gateway Solution Architecture & 
> Design Dept
> From:Wei Liu
> To:HUANG SHENGQIANG,
> Cc:xen-de...@lists.xenproject.org,Wangjianbing,Wei Liu,
> Date:2017-11-06 18:28:58
> Subject:Re: [Xen-devel] Help：Can xen restore several snapshots more faster at 
> same time？
> 
> On Mon, Nov 06, 2017 at 04:38:51AM +, HUANG SHENGQIANG wrote:
> > Dear XEN expert,
> >
> > I find a blocker issue in my project. Would you please offer us some 
> > feedback?
> >
> > The description from my development team:
> > we need restore as much as xen snapshot at same times, but we found ‘xl 
> > restore’ command is work linearly,  if we want to restore a new xen 
> > snapshot, we need to wait for the previous snapshot finish it’s work. We 
> > try to debug the xl source ,we found the follow information:
> > [cid:image001.png@01D356F6.B8EE87E0]
> >
> 
> Please don't send pictures.
> 
> > When an snapshot is being restore, we can see another process is blocked.  
> > We trying to delete the acquire_lock from the source code , then we see all 
> > the snapshots are being restore at same time, but restore is still very 
> > slow, one snapshot needs about 25 seconds  to finish restore(our 
> > environment is cpu E52620,  256G memory, SSD hard disk. The snapshot is 
> > Win7 OS with 2G memory).
> >
> 
> There is a lock in xl as you can see in the stack trace.
> 
> > So , does xen have the way to restore more faster when several snapshot is 
> > restore at same time? Why KVM can restore several snapshot fast at same 
> > time? (We try the same experiment in KVM, we got we can restore about 50+ 
> > snapshot in 20S. )
> >
> 
> Part of the toolstack needs to be reworked. You can start by removing
> the lock in xl to see what breaks.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH for-4.10] docs: update hvmlite.markdown

[Wei Liu]

Remove stale paragraph and escape underscore.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
Cc: Jan Beulich <jbeul...@suse.com>
Cc: Andrew Cooper <andrew.coop...@citrix.com>
Cc: Roger Pau Monne <roger@citrix.com>
---
 docs/misc/hvmlite.markdown | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/docs/misc/hvmlite.markdown b/docs/misc/hvmlite.markdown
index b2557f7b09..e85fb15374 100644
--- a/docs/misc/hvmlite.markdown
+++ b/docs/misc/hvmlite.markdown
@@ -1,6 +1,3 @@
-**NOTE**: this document will be merged into `pvh.markdown` once PVH is replaced
-with the HVMlite implementation.
-
 # x86/HVM direct boot ABI #
 
 Since the Xen entry point into the kernel can be different from the
@@ -38,7 +35,7 @@ All other processor registers and flag bits are unspecified. 
The OS is in
 charge of setting up it's own stack, GDT and IDT.
 
 The format of the boot start info structure (pointed to by %ebx) can be found
-in xen/include/public/arch-x86/hvm/start_info.h
+in xen/include/public/arch-x86/hvm/start\_info.h
 
 Other relevant information needed in order to boot a guest kernel
 (console page address, xenstore event channel...) can be obtained
@@ -53,7 +50,7 @@ AP startup can be performed using hypercalls or the local 
APIC if present.
 The following VCPU hypercalls can be used in order to bring up secondary vCPUs:
 
  * `VCPUOP_initialise` is used to set the initial state of the vCPU. The
-   argument passed to the hypercall must be of the type vcpu_hvm_context.
+   argument passed to the hypercall must be of the type vcpu\_hvm\_context.
See `public/hvm/hvm_vcpu.h` for the layout of the structure. Note that
this hypercall allows starting the vCPU in several modes (16/32/64bits),
regardless of the mode the BSP is currently running on.
@@ -71,7 +68,7 @@ PVHv2 guests that have access to hardware (either emulated or 
real) will also
 have ACPI tables with the description of the hardware that's available to the
 guest. This applies to both privileged and unprivileged guests. A pointer to
 the position of the RSDP in memory (if present) can be fetched from the start
-info structure that's passed at boot time (field rsdp_paddr).
+info structure that's passed at boot time (field rsdp\_paddr).
 
 Description of paravirtualized devices will come from XenStore, just as it's
 done for HVM guests.
@@ -93,5 +90,5 @@ Interrupts from paravirtualized devices are delivered using 
event channels, see
 [Event Channel Internals][event_channels] for more detailed information about
 event channels. Delivery of those interrupts can be configured in the same way
 as HVM guests, check xen/include/public/hvm/params.h and
-xen/include/public/hvm/hvm_op.h for more information about available delivery
+xen/include/public/hvm/hvm\_op.h for more information about available delivery
 methods.
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 07/31] xenpm: Clarify xenpm usage

[Wei Liu]

On Thu, Nov 09, 2017 at 07:09:57PM +0200, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshche...@epam.com>
> 
> CPU frequencies are in kHz. So, correct displayed text.
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshche...@epam.com>
> CC: Ian Jackson <ian.jack...@eu.citrix.com>
> CC: Wei Liu <wei.l...@citrix.com>
> CC: Stefano Stabellini <sstabell...@kernel.org>
> CC: Julien Grall <julien.gr...@linaro.org>
> ---
>  tools/misc/xenpm.c | 6 +++---

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mini-os: add a coding style file

[Wei Liu]

On Thu, Nov 09, 2017 at 01:35:49PM +0100, Juergen Gross wrote:
> On 09/11/17 13:31, Wei Liu wrote:
> > On Thu, Nov 09, 2017 at 01:10:12PM +0100, Juergen Gross wrote:
> >> Since carving out Mini-OS from the Xen repository there hasn't been a
> >> description of the preferred coding style. Copy the Xen CODING_STYLE
> >> file.
> >>
> > 
> > I welcome such addition. I have no opinion in actual style used though.
> > I just want consistency. :-)
> 
> Is this an Ack?
> 

Yes.

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mini-os: add a coding style file

[Wei Liu]

On Thu, Nov 09, 2017 at 01:10:12PM +0100, Juergen Gross wrote:
> Since carving out Mini-OS from the Xen repository there hasn't been a
> description of the preferred coding style. Copy the Xen CODING_STYLE
> file.
> 

I welcome such addition. I have no opinion in actual style used though.
I just want consistency. :-)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

[Wei Liu]

On Wed, Nov 08, 2017 at 05:24:14PM +0100, Olaf Hering wrote:
> On Thu, Oct 26, Olaf Hering wrote:
> 
> > > If not, then out-of-tree packages are going to have compatibility
> > > problems with this change.
> > Only if they use Requires=proc-xen.mount.
> 
> Any other objections to this change?
> 
> How to proceed with this?

Regardless of the decision whether we should backport this to older
branch, I think we should accept this patch going forward to avoid
breakage.

But is there really no way to ask nicely to see if systemd would accept
a change in behaviour? That is, to make proc-xen.mount (or any attempt
to mount API fs) a nop when xenfs is added to API file system.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] libevtchn: fix build on non-Linux hosts

[Wei Liu]

On Wed, Nov 08, 2017 at 12:52:57PM +, Roger Pau Monne wrote:
> Non-Linux hosts (where osdep_evtchn_restrict is not yet supported)
> made use of errno without including errno.h, fix this by including the
> header.
> 
> Signed-off-by: Roger Pau Monné <roger@citrix.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable bisection] complete test-amd64-amd64-i386-pvgrub

[Wei Liu]

On Tue, Nov 07, 2017 at 05:24:32PM +, Julien Grall wrote:
> Hi Wei,
> 
> On 07/11/17 15:13, Wei Liu wrote:
> > On Tue, Nov 07, 2017 at 03:09:07PM +, Julien Grall wrote:
> >> Hi Wei,
> >>
> >> On 06/11/17 14:55, Wei Liu wrote:
> >>> On Mon, Nov 06, 2017 at 01:47:56PM +, osstest service owner wrote:
> >>>> branch xen-unstable
> >>>> xenbranch xen-unstable
> >>>> job test-amd64-amd64-i386-pvgrub
> >>>> testid guest-start
> >>>>
> >>>> Tree: linux git://xenbits.xen.org/linux-pvops.git
> >>>> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> >>>> Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> >>>> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> >>>> Tree: xen git://xenbits.xen.org/xen.git
> >>>>
> >>>> *** Found and reproduced problem changeset ***
> >>>>
> >>>> Bug is in tree:  xen git://xenbits.xen.org/xen.git
> >>>> Bug introduced:  f48b5449dabc770acdde6d25cfbd265cfb71034d
> >>>> Bug not present: 86cf189a957129ea1ad6468fe9a0887b9e2819f3
> >>>> Last fail repro: 
> >>>> http://logs.test-lab.xenproject.org/osstest/logs/115612/
> >>>>
> >>>>
> >>>> commit f48b5449dabc770acdde6d25cfbd265cfb71034d
> >>>> Author: Wei Liu <wei.l...@citrix.com>
> >>>> Date:   Thu Oct 12 20:19:07 2017 +0100
> >>>> tools/dombuilder: Switch to using gfn terminology for console 
> >>>> and xenstore rings
> >>>> The sole use of xc_dom_translated() and xc_dom_p2m() outside of 
> >>>> the domain
> >>>> builder is for libxl_dom() to translate the console and xenstore 
> >>>> pfns back
> >>>> into useful values.  PV guest pfns are only interesting to the 
> >>>> domain builder,
> >>>> and gfns are the address space used by all other hypercalls.
> >>>> Renaming the fields in xc_dom_image is deliberate, as it will 
> >>>> cause
> >>>> out-of-tree users of the dombuilder to notice the different 
> >>>> semantics.
> >>>> Correct the terminology throughout xc_dom_gnttab{_hvm,}_seed(), 
> >>>> which are all
> >>>> using gfns despite the existing variable names.
> >>>> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
> >>>> Reviewed-by: Roger Pau Monn?? <roger@citrix.com>
> >>>> Acked-by: Wei Liu <wei.l...@citrix.com>
> >>>> Tested-by: Julien Grall <julien.gr...@arm.com>
> >>>> Release-acked-by: Julien Grall <julien.gr...@linaro.org>
> >>>> [ wei: fix stubdom build ]
> >>>> Signed-off-by: Wei Liu <wei.l...@citrix.com>
> >>>
> >>> This has broken pvgrub. The problem is more than just the name of the
> >>> variables. I have reverted this and its successor patch.
> >>
> >> It looks like osstest is still broken after the patches you reverted (see
> >> [1] and [2]).
> >>
> >> AFAICT, the only series between the two flights is the dombuilder, there 
> >> are
> >> 2 patches not reverted.
> >>
> >> Do you have an idea of what's going on?
> >>
> >> Cheers,
> >>
> >> [1] http://logs.test-lab.xenproject.org/osstest/logs/115624/
> >> [2]
> >> https://lists.xenproject.org/archives/html/xen-devel/2017-11/msg00391.html
> >>
> > 
> > test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. 
> > vs.  115526
> > test-armhf-armhf-xl-vhd 15 guest-start/debian.repeat fail REGR. vs.  
> > 115526
> 
> The log for the xl-vhd contains ([1])
> 
> libxl: error: libxl_bootloader.c:283:bootloader_local_detached_cb: Domain 
> 11:unable to detach locally attached disk
> libxl: error: libxl_create.c:1246:domcreate_rebuild_done: Domain 11:cannot 
> (re-)build domain: -3
> libxl: debug: libxl_domain.c:1138:devices_destroy_cb: Domain 11:Forked pid 
> 5103 for destroy of domain
> libxl: debug: libxl_create.c:1683:do_domain_create: Domain 0:ao 0x5d6e8: 
> inprogress: poller=0x56ad8, flags=i
> libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5d6e8: complete, 
> rc=-3
> libxl: debug: libxl_event.c:1838:libxl__ao__dest

Re: [Xen-devel] [PATCH for-next 6/9] kconfig: add llvm coverage option

[Wei Liu]

On Wed, Nov 08, 2017 at 04:07:35AM -0700, Jan Beulich wrote:
> >>> On 08.11.17 at 09:49, <roger@citrix.com> wrote:
> > On Wed, Nov 08, 2017 at 01:13:29AM -0700, Jan Beulich wrote:
> >> >>> On 26.10.17 at 12:10, <wei.l...@citrix.com> wrote:
> >> > On Thu, Oct 26, 2017 at 11:08:21AM +0100, Roger Pau Monné wrote:
> >> >> On Thu, Oct 26, 2017 at 11:03:13AM +0100, Wei Liu wrote:
> >> >> > On Thu, Oct 26, 2017 at 10:19:35AM +0100, Roger Pau Monne wrote:
> >> >> > >  config GCOV
> >> >> > > bool "Gcov Support"
> >> >> > > depends on !LIVEPATCH
> >> >> > 
> >> >> > && !LLVM_COVERAGE
> >> >> 
> >> >> That was my idea, but sadly that's not possible because you generate a
> >> >> circular dependency. The best solution that I found is to only mark
> >> >> one as exclusive (ie: have depends !GCOV in the LLVM_COVERAGE option
> >> >> below).
> >> > 
> >> > In that case, why not just use "choice" to let user pick the
> >> > implementation?
> >> 
> >> Plus then this choice should probably have an auto-detect option
> >> just like gcov's gcc version one - at least I assume that it should
> >> be pretty straightforward to tell at build time which variant to use.
> >> In fact - what would happen if one enabled the wrong option for
> >> the compiler in use? IOW I wonder whether auto-detection (and
> >> then also for the gcc version) should be the only valid mode).
> > 
> > I don't plan to introduce llvm/clang version choices here, I think
> > autodetection should be fine.
> 
> And I didn't think about version choices for clang here anyway -
> the point really was just about gcc vs clang.
> 
> > I can remove the gcc ones also, leaving this as a boolean choice (ie:
> > enable code coverage support), but I would like to have some
> > confirmation from the gcc side.
> 
> So let's ask Wei: What was the reason for the Kconfig level
> version choice here in the first place? After all, if the wrong one
> is being selected, the build will fail afaict due to the #error
> directives in the version specific files.
> 

The #error on versions was added in later iterations IIRC. Looking back
I think it would make sense to just have autodetect.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable bisection] complete test-amd64-amd64-i386-pvgrub

[Wei Liu]

On Tue, Nov 07, 2017 at 03:09:07PM +, Julien Grall wrote:
> Hi Wei,
> 
> On 06/11/17 14:55, Wei Liu wrote:
> > On Mon, Nov 06, 2017 at 01:47:56PM +, osstest service owner wrote:
> > > branch xen-unstable
> > > xenbranch xen-unstable
> > > job test-amd64-amd64-i386-pvgrub
> > > testid guest-start
> > > 
> > > Tree: linux git://xenbits.xen.org/linux-pvops.git
> > > Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> > > Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> > > Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> > > Tree: xen git://xenbits.xen.org/xen.git
> > > 
> > > *** Found and reproduced problem changeset ***
> > > 
> > >Bug is in tree:  xen git://xenbits.xen.org/xen.git
> > >Bug introduced:  f48b5449dabc770acdde6d25cfbd265cfb71034d
> > >Bug not present: 86cf189a957129ea1ad6468fe9a0887b9e2819f3
> > >Last fail repro: 
> > > http://logs.test-lab.xenproject.org/osstest/logs/115612/
> > > 
> > > 
> > >commit f48b5449dabc770acdde6d25cfbd265cfb71034d
> > >Author: Wei Liu <wei.l...@citrix.com>
> > >Date:   Thu Oct 12 20:19:07 2017 +0100
> > >tools/dombuilder: Switch to using gfn terminology for console and 
> > > xenstore rings
> > >The sole use of xc_dom_translated() and xc_dom_p2m() outside of 
> > > the domain
> > >builder is for libxl_dom() to translate the console and xenstore 
> > > pfns back
> > >into useful values.  PV guest pfns are only interesting to the 
> > > domain builder,
> > >and gfns are the address space used by all other hypercalls.
> > >Renaming the fields in xc_dom_image is deliberate, as it will cause
> > >out-of-tree users of the dombuilder to notice the different 
> > > semantics.
> > >Correct the terminology throughout xc_dom_gnttab{_hvm,}_seed(), 
> > > which are all
> > >using gfns despite the existing variable names.
> > >Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
> > >Reviewed-by: Roger Pau Monn?? <roger@citrix.com>
> > >Acked-by: Wei Liu <wei.l...@citrix.com>
> > >Tested-by: Julien Grall <julien.gr...@arm.com>
> > >Release-acked-by: Julien Grall <julien.gr...@linaro.org>
> > >[ wei: fix stubdom build ]
> > >Signed-off-by: Wei Liu <wei.l...@citrix.com>
> > 
> > This has broken pvgrub. The problem is more than just the name of the
> > variables. I have reverted this and its successor patch.
> 
> It looks like osstest is still broken after the patches you reverted (see
> [1] and [2]).
> 
> AFAICT, the only series between the two flights is the dombuilder, there are
> 2 patches not reverted.
> 
> Do you have an idea of what's going on?
> 
> Cheers,
> 
> [1] http://logs.test-lab.xenproject.org/osstest/logs/115624/
> [2]
> https://lists.xenproject.org/archives/html/xen-devel/2017-11/msg00391.html
> 

test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs.  
115526
test-armhf-armhf-xl-vhd 15 guest-start/debian.repeat fail REGR. vs.  115526 

These aren't related to dombuilder at first glance.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Libvirt config converter can't handle file not ending with new line

[Wei Liu]

On Mon, Nov 06, 2017 at 09:41:01PM -0700, Jim Fehlig wrote:
> On 10/30/2017 06:17 AM, Wei Liu wrote:
> > Hi Jim
> > 
> > I discover a problem when using xen_xl converter. When the file in
> > question doesn't end with a new line, I get the following error:
> > 
> >error: configuration file syntax error: memory conf:53: expecting a value
> 
> I'm not able to reproduce this issue. The libvirt.git tree I tried was a bit
> dated, but even after updating to latest master I can't reproduce.
> 
> > After digging a bit (but haven't read libvirt code), it appears that the
> > file didn't end with a new line.
> 
> I tried several files without ending new lines, going both directions
> (domxml-to-native and domxml-from-native), but didn't see the mentioned
> error. Perhaps your config is revealing another bug which is being
> improperly reported. Can you provide an example of the problematic config?
> 

I tried to get the exact file that caused the problem but it is already
destroyed by osstest.

A similar file:

http://logs.test-lab.xenproject.org/osstest/logs/115436/test-amd64-amd64-libvirt-pair/debian.guest.osstest.cfg

If you hexdump -C it, you can see the last character is 0a. Remove it and
feed the file into the converter.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 1/9] gcov: return ENOSYS for unimplemented gcov domctl

[Wei Liu]

On Tue, Nov 07, 2017 at 09:41:58AM +, Roger Pau Monné wrote:
> > Okay, so EOPNOTSUPP is it then, which is also my preference
> > (due to there being so many uses of EINVAL elsewhere). I've
> > merely mentioned that EINVAL would be suitable since,
> > technically speaking, the value in a "sub-operation" field being
> > invalid is no different from this being the case for the value in
> > any other field.
> 
> If I don't get any more comments I will re-send this patch separately
> using EOPNOTSUPP instead of ENOSYS. I will also keep the Acks gathered
> so far unless anyone objects.
> 

Please send a new patch. I believe this one is already applied.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable bisection] complete test-amd64-amd64-i386-pvgrub

[Wei Liu]

On Mon, Nov 06, 2017 at 01:47:56PM +, osstest service owner wrote:
> branch xen-unstable
> xenbranch xen-unstable
> job test-amd64-amd64-i386-pvgrub
> testid guest-start
> 
> Tree: linux git://xenbits.xen.org/linux-pvops.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> Tree: xen git://xenbits.xen.org/xen.git
> 
> *** Found and reproduced problem changeset ***
> 
>   Bug is in tree:  xen git://xenbits.xen.org/xen.git
>   Bug introduced:  f48b5449dabc770acdde6d25cfbd265cfb71034d
>   Bug not present: 86cf189a957129ea1ad6468fe9a0887b9e2819f3
>   Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/115612/
> 
> 
>   commit f48b5449dabc770acdde6d25cfbd265cfb71034d
>   Author: Wei Liu <wei.l...@citrix.com>
>   Date:   Thu Oct 12 20:19:07 2017 +0100
>   
>   tools/dombuilder: Switch to using gfn terminology for console and 
> xenstore rings
>   
>   The sole use of xc_dom_translated() and xc_dom_p2m() outside of the 
> domain
>   builder is for libxl_dom() to translate the console and xenstore pfns 
> back
>   into useful values.  PV guest pfns are only interesting to the domain 
> builder,
>   and gfns are the address space used by all other hypercalls.
>   
>   Renaming the fields in xc_dom_image is deliberate, as it will cause
>   out-of-tree users of the dombuilder to notice the different semantics.
>   
>   Correct the terminology throughout xc_dom_gnttab{_hvm,}_seed(), which 
> are all
>   using gfns despite the existing variable names.
>   
>   Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
>   Reviewed-by: Roger Pau Monn?? <roger@citrix.com>
>   Acked-by: Wei Liu <wei.l...@citrix.com>
>   Tested-by: Julien Grall <julien.gr...@arm.com>
>   Release-acked-by: Julien Grall <julien.gr...@linaro.org>
>   [ wei: fix stubdom build ]
>   Signed-off-by: Wei Liu <wei.l...@citrix.com>

This has broken pvgrub. The problem is more than just the name of the
variables. I have reverted this and its successor patch.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] x86/cpuid: Minor fixups missed from previous work

[Wei Liu]

On Mon, Nov 06, 2017 at 05:35:10AM -0700, Jan Beulich wrote:
>  >>> On 03.11.17 at 18:35, <andrew.coop...@citrix.com> wrote:
> > * Add more feature names to ./xen-cpuid
> >  * Vertically align the magic comments in cpufeatureset.h
> > 
> > Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
> 
> Acked-by: Jan Beulich <jbeul...@suse.com>
> 
> 

Acked-by: Wei Liu <wei.l...@citrix.com>


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Help：Can xen restore several snapshots more faster at same time？

[Wei Liu]

On Mon, Nov 06, 2017 at 04:38:51AM +, HUANG SHENGQIANG wrote:
> Dear XEN expert,
> 
> I find a blocker issue in my project. Would you please offer us some feedback?
> 
> The description from my development team:
> we need restore as much as xen snapshot at same times, but we found ‘xl 
> restore’ command is work linearly,  if we want to restore a new xen snapshot, 
> we need to wait for the previous snapshot finish it’s work. We try to debug 
> the xl source ,we found the follow information:
> [cid:image001.png@01D356F6.B8EE87E0]
> 

Please don't send pictures.

> When an snapshot is being restore, we can see another process is blocked.  We 
> trying to delete the acquire_lock from the source code , then we see all the 
> snapshots are being restore at same time, but restore is still very slow, one 
> snapshot needs about 25 seconds  to finish restore(our environment is cpu 
> E52620,  256G memory, SSD hard disk. The snapshot is Win7 OS with 2G memory).
> 

There is a lock in xl as you can see in the stack trace.

> So , does xen have the way to restore more faster when several snapshot is 
> restore at same time? Why KVM can restore several snapshot fast at same time? 
> (We try the same experiment in KVM, we got we can restore about 50+ snapshot 
> in 20S. )
> 

Part of the toolstack needs to be reworked. You can start by removing
the lock in xl to see what breaks.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable-smoke test] 115515: regressions - FAIL

[Wei Liu]

On Fri, Nov 03, 2017 at 12:46:03PM +, Wei Liu wrote:
> On Fri, Nov 03, 2017 at 10:54:58AM +0000, Wei Liu wrote:
> > On Fri, Nov 03, 2017 at 10:34:44AM +, Julien Grall wrote:
> > > Hi,
> > > 
> > > On 03/11/17 10:29, osstest service owner wrote:
> > > > flight 115515 xen-unstable-smoke real [real]
> > > > http://logs.test-lab.xenproject.org/osstest/logs/115515/
> > > > 
> > > > Regressions :-(
> > > > 
> > > > Tests which did not succeed and are blocking,
> > > > including tests which could not be run:
> > > >   test-amd64-amd64-libvirt 15 guest-saverestorefail REGR. 
> > > > vs. 115490
> > > >   test-amd64-amd64-xl-qemuu-debianhvm-i386 13 guest-saverestore fail 
> > > > REGR. vs. 115490
> > > > 
> > > > Tests which did not succeed, but are not blocking:
> > > >   test-amd64-amd64-libvirt 13 migrate-support-checkfail   
> > > > never pass
> > > >   test-armhf-armhf-xl  13 migrate-support-checkfail   
> > > > never pass
> > > >   test-armhf-armhf-xl  14 saverestore-support-checkfail   
> > > > never pass
> > > > 
> > > > version targeted for testing:
> > > >   xen  29028ed8db226ad3b7bc576c1e8891983acaa3ff
> > > > baseline version:
> > > >   xen  ff93dc55431517ed29c70dbff6721c6b0803acf9
> > > 
> > > The only difference between the two is the series from Andrew about the
> > > tools. Any idea, why it would break?
> > > 
> > 
> > This
> > 
> > libxl-save-helper: debug: starting restore: Success
> > xc: detail: fd 9, dom 3, hvm 0, pae 0, stream_type 0
> > xc: info: Found x86 HVM domain from Xen 4.10
> > xc: info: Restoring domain
> > domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, scratch gfn=0x148850
> > xc: error: panic: xc_dom_boot.c:392: xc_dom_gnttab_hvm_seed: failed to add 
> > gnttab to physmap [errno=22]
> > : Internal error
> 
> My bisection shows the following commit is the culprit.
> 
> 
> 9ff6dbfa7576cc1c5d6f9a3c59c69a8503e36f11 is the first bad commit
> commit 9ff6dbfa7576cc1c5d6f9a3c59c69a8503e36f11
> Author: Andrew Cooper <andrew.coop...@citrix.com>
> Date:   Thu Oct 12 20:19:09 2017 +0100
> 
> tools/dombuilder: Prevent failures of xc_dom_gnttab_init()
> 
> Recent changes in grant table configuration have caused calls to
> xc_dom_gnttab_init() to fail if not proceeded with a call to
> xc_domain_set_gnttab_limits().  This is backwards from the point of view 
> of
> 3rd party dombuilder users.
> 
> Add max_{grant,maptrack}_frames parameters to struct xc_dom_image, and 
> require
> them to be set by callers using xc_dom_gnttab_init().  Libxl, which uses
> xc_dom_gnttab_init() itself is updated appropriately.
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
> Acked-by: Wei Liu <wei.l...@citrix.com>
> Tested-by: Julien Grall <julien.gr...@arm.com>
> Reviewed-by: Juergen Gross <jgr...@suse.com>
> Release-acked-by: Julien Grall <julien.gr...@linaro.org>
> 

I have reverted this since I won't have time to fix it this afternoon
and I don't want to block the pushgate during w/e.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 1/2] ts-debian-di-install: use gho to pick d-i

[Wei Liu]

The original code used ho which gave us the host suite, but we wanted
the guest suite.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-debian-di-install | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index 6007971..361a171 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -152,8 +152,8 @@ sub setup_netboot($$$)
die if $r{ "$gho->{Guest}_netboot_kernel" }
|| $r{ "$gho->{Guest}_netboot_ramdisk" };
 
-   my $di_path = $c{TftpPath}.'/'.$ho->{Tftp}{DiBase}.'/'.${arch}.'/'.
-   debian_guest_di_version($ho).'-'.$ho->{Suite};
+   my $di_path = $c{TftpPath}.'/'.$gho->{Tftp}{DiBase}.'/'.${arch}.'/'.
+   debian_guest_di_version($gho).'-'.$gho->{Suite};
 
 if (${arch} =~ m/amd64|i386/) {
$kernel = "$di_path/vmlinuz-xen";
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 2/2] make-flight: guest should use jessie to test pvgrub

[Wei Liu]

Stretch has 64bit feature enabled for ext4, which pvgrub can't cope.
We want to continue to test pvgrub, so specify jessie in the guest
suite field.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 make-flight | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/make-flight b/make-flight
index 76620c1..9c5399b 100755
--- a/make-flight
+++ b/make-flight
@@ -578,8 +578,8 @@ do_pvgrub_tests () {
   job_create_test test-$xenarch$kern-$dom0arch-amd64-pvgrub \
 test-debian-di xl $xenarch $dom0arch\
   debian_arch=amd64 \
-  debian_suite=$guestsuite  \
-  debian_di_version=$guest_di_version   \
+  debian_suite=jessie   \
+  debian_di_version=`getconfig_TftpDiVersion_suite jessie`  \
   debian_method=netboot \
   debian_bootloader=pvgrub  \
   all_hostflags=$most_hostflags \
@@ -587,8 +587,8 @@ do_pvgrub_tests () {
   job_create_test test-$xenarch$kern-$dom0arch-i386-pvgrub  \
 test-debian-di xl $xenarch $dom0arch\
   debian_arch=i386  \
-  debian_suite=$guestsuite  \
-  debian_di_version=$guest_di_version   \
+  debian_suite=jessie   \
+  debian_di_version=`getconfig_TftpDiVersion_suite jessie`  \
   debian_method=netboot \
   debian_bootloader=pvgrub  \
   all_hostflags=$most_hostflags
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 0/2] Keep pvgrub tests

[Wei Liu]

These are extra patches to replace

make-flight: don't test pvgrub for Xen XXX

so that we can keep pvgrub tests.

See
http://logs.test-lab.xenproject.org/osstest/logs/115518/


Wei Liu (2):
  ts-debian-di-install: use gho to pick d-i
  make-flight: guest should use jessie to test pvgrub

 make-flight  | 8 
 ts-debian-di-install | 4 ++--
 2 files changed, 6 insertions(+), 6 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable-smoke test] 115515: regressions - FAIL

[Wei Liu]

On Fri, Nov 03, 2017 at 10:54:58AM +, Wei Liu wrote:
> On Fri, Nov 03, 2017 at 10:34:44AM +, Julien Grall wrote:
> > Hi,
> > 
> > On 03/11/17 10:29, osstest service owner wrote:
> > > flight 115515 xen-unstable-smoke real [real]
> > > http://logs.test-lab.xenproject.org/osstest/logs/115515/
> > > 
> > > Regressions :-(
> > > 
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > >   test-amd64-amd64-libvirt 15 guest-saverestorefail REGR. vs. 
> > > 115490
> > >   test-amd64-amd64-xl-qemuu-debianhvm-i386 13 guest-saverestore fail 
> > > REGR. vs. 115490
> > > 
> > > Tests which did not succeed, but are not blocking:
> > >   test-amd64-amd64-libvirt 13 migrate-support-checkfail   
> > > never pass
> > >   test-armhf-armhf-xl  13 migrate-support-checkfail   
> > > never pass
> > >   test-armhf-armhf-xl  14 saverestore-support-checkfail   
> > > never pass
> > > 
> > > version targeted for testing:
> > >   xen  29028ed8db226ad3b7bc576c1e8891983acaa3ff
> > > baseline version:
> > >   xen  ff93dc55431517ed29c70dbff6721c6b0803acf9
> > 
> > The only difference between the two is the series from Andrew about the
> > tools. Any idea, why it would break?
> > 
> 
> This
> 
> libxl-save-helper: debug: starting restore: Success
> xc: detail: fd 9, dom 3, hvm 0, pae 0, stream_type 0
> xc: info: Found x86 HVM domain from Xen 4.10
> xc: info: Restoring domain
> domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, scratch gfn=0x148850
> xc: error: panic: xc_dom_boot.c:392: xc_dom_gnttab_hvm_seed: failed to add 
> gnttab to physmap [errno=22]
> : Internal error

My bisection shows the following commit is the culprit.


9ff6dbfa7576cc1c5d6f9a3c59c69a8503e36f11 is the first bad commit
commit 9ff6dbfa7576cc1c5d6f9a3c59c69a8503e36f11
Author: Andrew Cooper <andrew.coop...@citrix.com>
Date:   Thu Oct 12 20:19:09 2017 +0100

tools/dombuilder: Prevent failures of xc_dom_gnttab_init()

Recent changes in grant table configuration have caused calls to
xc_dom_gnttab_init() to fail if not proceeded with a call to
xc_domain_set_gnttab_limits().  This is backwards from the point of view of
3rd party dombuilder users.

    Add max_{grant,maptrack}_frames parameters to struct xc_dom_image, and 
require
them to be set by callers using xc_dom_gnttab_init().  Libxl, which uses
xc_dom_gnttab_init() itself is updated appropriately.

Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
Acked-by: Wei Liu <wei.l...@citrix.com>
Tested-by: Julien Grall <julien.gr...@arm.com>
Reviewed-by: Juergen Gross <jgr...@suse.com>
Release-acked-by: Julien Grall <julien.gr...@linaro.org>


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable-smoke test] 115515: regressions - FAIL

[Wei Liu]

On Fri, Nov 03, 2017 at 10:34:44AM +, Julien Grall wrote:
> Hi,
> 
> On 03/11/17 10:29, osstest service owner wrote:
> > flight 115515 xen-unstable-smoke real [real]
> > http://logs.test-lab.xenproject.org/osstest/logs/115515/
> > 
> > Regressions :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >   test-amd64-amd64-libvirt 15 guest-saverestorefail REGR. vs. 
> > 115490
> >   test-amd64-amd64-xl-qemuu-debianhvm-i386 13 guest-saverestore fail REGR. 
> > vs. 115490
> > 
> > Tests which did not succeed, but are not blocking:
> >   test-amd64-amd64-libvirt 13 migrate-support-checkfail   never 
> > pass
> >   test-armhf-armhf-xl  13 migrate-support-checkfail   never 
> > pass
> >   test-armhf-armhf-xl  14 saverestore-support-checkfail   never 
> > pass
> > 
> > version targeted for testing:
> >   xen  29028ed8db226ad3b7bc576c1e8891983acaa3ff
> > baseline version:
> >   xen  ff93dc55431517ed29c70dbff6721c6b0803acf9
> 
> The only difference between the two is the series from Andrew about the
> tools. Any idea, why it would break?
> 

This

libxl-save-helper: debug: starting restore: Success
xc: detail: fd 9, dom 3, hvm 0, pae 0, stream_type 0
xc: info: Found x86 HVM domain from Xen 4.10
xc: info: Restoring domain
domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, scratch gfn=0x148850
xc: error: panic: xc_dom_boot.c:392: xc_dom_gnttab_hvm_seed: failed to add 
gnttab to physmap [errno=22]
: Internal error

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10 v2 3/5] tools/dombuilder: Switch to using gfn terminology for console and xenstore rings

[Wei Liu]

On Thu, Oct 12, 2017 at 08:19:07PM +0100, Andrew Cooper wrote:
> The sole use of xc_dom_translated() and xc_dom_p2m() outside of the domain
> builder is for libxl_dom() to translate the console and xenstore pfns back
> into useful values.  PV guest pfns are only interesting to the domain builder,
> and gfns are the address space used by all other hypercalls.
> 
> Renaming the fields in xc_dom_image is deliberate, as it will cause
> out-of-tree users of the dombuilder to notice the different semantics.
> 
> Correct the terminology throughout xc_dom_gnttab{_hvm,}_seed(), which are all
> using gfns despite the existing variable names.
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>
> Reviewed-by: Roger Pau Monné <roger@citrix.com>
> Acked-by: Wei Liu <wei.l...@citrix.com>
> Tested-by: Julien Grall <julien.gr...@arm.com>
> ---
> CC: Ian Jackson <ian.jack...@eu.citrix.com>
> CC: Julien Grall <julien.gr...@arm.com>
> 
> v2:
>  * More style fixes
> ---
>  tools/libxc/include/xc_dom.h  | 10 +++--
>  tools/libxc/xc_dom_arm.c  | 12 +--
>  tools/libxc/xc_dom_boot.c | 45 
> ++-
>  tools/libxc/xc_dom_compat_linux.c |  4 ++--
>  tools/libxc/xc_dom_x86.c  | 45 
> ---
>  tools/libxl/libxl_dom.c   | 11 +++---
>  6 files changed, 63 insertions(+), 64 deletions(-)
> 
> diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
> index cdcdd07..5907559 100644
> --- a/tools/libxc/include/xc_dom.h
> +++ b/tools/libxc/include/xc_dom.h
> @@ -94,8 +94,6 @@ struct xc_dom_image {
>  struct xc_dom_seg devicetree_seg;
>  struct xc_dom_seg start_info_seg; /* HVMlite only */
>  xen_pfn_t start_info_pfn;
> -xen_pfn_t console_pfn;
> -xen_pfn_t xenstore_pfn;
>  xen_pfn_t shared_info_pfn;
>  xen_pfn_t bootstack_pfn;
>  xen_pfn_t pfn_alloc_end;
> @@ -103,6 +101,14 @@ struct xc_dom_image {
>  xen_vaddr_t bsd_symtab_start;
>  
>  /*
> + * Details for the toolstack-prepared rings.
> + *
> + * *_gfn fields are allocated by the domain builder.
> + */
> +xen_pfn_t console_gfn;
> +xen_pfn_t xenstore_gfn;
> +
> +/*

Stubdom build is broken by this change:

mini-os.c:756:23: note: in expansion of macro ‘BOOTSEC_LOCATION’
 mbi.drives_addr = BOOTSEC_LOCATION + (60 * 1024);
   ^~~~
gcc -mno-red-zone -O1 -fno-omit-frame-pointer -O1 -fno-omit-frame-pointer  -m64 
-mno-red-zone -fno-reorder-blocks -fno-asynchronous-unwind-tables -m64 
-DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
 -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable 
-Wno-unused-local-typedefs   -fno-stack-protector -fno-exceptions -fno-builtin 
-Wall -Werror -Wredundant-decls -Wno-for
mat -Wno-redundant-decls -Wformat -fno-stack-protector -fgnu89-inline 
-Wstrict-prototypes -Wnested-externs -Wpointer-arith -Winline -g 
-D__INSIDE_MINIOS__ -m64 -mno-red-zone -fno-reorder-blocks -fno-
asynchronous-unwind-tables -DCONFIG_PARAVIRT -DCONFIG_SPARSE_BSS 
-DCONFIG_BLKFRONT -DCONFIG_TPMFRONT -DCONFIG_TPM_TIS -DCONFIG_TPMBACK 
-DCONFIG_XENBUS -D__XEN_INTERFACE_VERSION__=0x00030205 -isystem
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include -D__MINIOS__ 
-DHAVE_LIBC -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include/posix -isystem 
/local/work/COMMITTER/
xen.git/stubdom/../tools/xenstore/include  -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include/x86 -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include/x86/x8
6_64 -U __linux__ -U __FreeBSD__ -U __sun__ -nostdinc -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include/posix -isystem 
/local/work/COMMITTER/xen.git/stubdom/cross-root-x86_64/x
86_64-xen-elf/include -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include -isystem 
/local/work/COMMITTER/xen.git/stubdom/lwip-x86_64/src/include -isystem 
/local/work/COMMITTER/xen.git/stubdom/lwip-x86_6
4/src/include/ipv4 -I/local/work/COMMITTER/xen.git/stubdom/include 
-I/local/work/COMMITTER/xen.git/stubdom/../xen/include -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include -D__
MINIOS__ -DHAVE_LIBC -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include/posix -isystem 
/local/work/COMMITTER/xen.git/stubdom/../tools/xenstore/include  -isystem 
/local/work/COMM
ITTER/xen.git/stubdom/../extras/mini-os/include/x86 -isystem 
/local/work/COMMITTER/xen.git/stubdom/../extras/mini-os/include/x86/x86_64 -c 
gntmap.c -o /local/work/COMMITTER/xen.git/stubdom/mini-os-x8
6_64-vtpmmgr/gntmap.o
In file included from 
/local/work/COMMITTER/xen.git/stubdom/include/xen/xen.h:30:0,
 

Re: [Xen-devel] [PATCH v3 4/7] libxl: support mapping static shared memory areas during domain creation

[Wei Liu]

On Thu, Oct 19, 2017 at 10:36:32AM +0800, Zhongze Liu wrote:
> Add libxl__sshm_add to map shared pages from one DomU to another, The mapping
> process involves the follwing steps:
> 
>   * Set defaults and check for further errors in the static_shm configs:
> overlapping areas, invalid ranges, duplicated master domain,
> no master domain etc.
>   * Write infomation of static shared memory areas into the appropriate
> xenstore paths.
>   * Use xc_domain_add_to_physmap_batch to do the page sharing.
>   * Set the refcount of the shared region accordingly
> 
> Temporarily mark this as unsupported on x86 because calling p2m_add_foregin on
> two domU's is currently not allowd on x86 (see the comments in
> x86/mm/p2m.c:p2m_add_foregin for more details).
> 
> This is for the proposal "Allow setting up shared memory areas between VMs
> from xl config file" (see [1]).
> 
> [1] https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html
> 
> Signed-off-by: Zhongze Liu <blacksk...@gmail.com>
> 
> Cc: Wei Liu <wei.l...@citrix.com>
> Cc: Ian Jackson <ian.jack...@eu.citrix.com>
> Cc: Stefano Stabellini <sstabell...@kernel.org>
> Cc: Julien Grall <julien.gr...@arm.com>
> Cc: xen-devel@lists.xen.org
> ---
>   V3:
>   * unmap the successfully mapped pages whenever rc != 0


A general note: please properly capitalise the comments.

> ---
>  tools/libxl/Makefile |   2 +-
>  tools/libxl/libxl_arch.h |   6 +
>  tools/libxl/libxl_arm.c  |  15 ++
>  tools/libxl/libxl_create.c   |  27 +++
>  tools/libxl/libxl_internal.h |  13 ++
>  tools/libxl/libxl_sshm.c | 395 
> +++
>  tools/libxl/libxl_x86.c  |  18 ++
>  7 files changed, 475 insertions(+), 1 deletion(-)
>  create mode 100644 tools/libxl/libxl_sshm.c
> 
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> index 5a861f72cb..91bc70cda2 100644
> --- a/tools/libxl/Makefile
> +++ b/tools/libxl/Makefile
[...]
> +
> +/* check if the sshm slave configs in @sshm overlap */
> +int libxl__sshm_check_overlap(libxl__gc *gc, uint32_t domid,
> + libxl_static_shm *sshms, int len)
> +{
> +
> +const libxl_static_shm **slave_sshms = NULL;
> +int num_slaves;
> +int i;
> +
> +if (!len) return 0;
> +
> +slave_sshms = libxl__calloc(gc, len, sizeof(slave_sshms[0]));
> +num_slaves = 0;
> +for (i = 0; i < len; ++i) {
> +if (sshms[i].role == LIBXL_SSHM_ROLE_SLAVE)
> +slave_sshms[num_slaves++] = sshms + i;
> +}
> +qsort(slave_sshms, num_slaves, sizeof(slave_sshms[0]), sshm_range_cmp);
> +
> +for (i = 0; i < num_slaves - 1; ++i) {
> +if (slave_sshms[i+1]->begin < slave_sshms[i]->end) {
> +SSHM_ERROR(domid, slave_sshms[i+1]->id, "slave ranges overlap.");
> +return ERROR_INVAL;
> +}
> +}
> +
> +return 0;
> +}
> +
> +/*   libxl__sshm_do_map -- map pages into slave's physmap
> + *
> + *   This functions maps
> + * mater gfn: [@msshm->begin + @sshm->offset, @msshm->end + 
> @sshm->offset)

"master"

This is confusing. What if offset is not page aligned?

> + *   into
> + * slave gfn: [@sshm->begin, @sshm->end)
> + *
> + *   The gfns of the pages that are successfully mapped will be stored
> + *   in @mapped, and the number of the gfns will be stored in @nmapped.
> + *
> + *   The caller have to guarentee that sshm->begin < sshm->end */
> +static int libxl__sshm_do_map(libxl__gc *gc, uint32_t mid, uint32_t sid,
> +  libxl_static_shm *sshm, libxl_static_shm 
> *msshm,
> +  xen_pfn_t *mapped, unsigned int *nmapped)
> +{
> +int rc;
> +int i;
> +unsigned int num_mpages, num_spages, num_success, offset;
> +int *errs;
> +xen_ulong_t *idxs;
> +xen_pfn_t *gpfns;
> +
> +num_mpages = (msshm->end - msshm->begin) >> XC_PAGE_SHIFT;
> +num_spages = (sshm->end - sshm->begin) >> XC_PAGE_SHIFT;
> +offset = sshm->offset >> XC_PAGE_SHIFT;
> +
> +/* Check range. Test offset < mpages first to avoid overflow */
> +if ((offset >= num_mpages) || (num_mpages - offset < num_spages)) {
> +SSHM_ERROR(sid, sshm->id, "exceeds master's address space.");
> +rc = ERROR_INVAL;
> +goto out;
> +}
> +
> +/* fill out the pfn's and do the mapping */
> +errs = libxl__calloc(gc, num_spages, sizeof(int));
> +idxs = libxl__calloc(gc, num_spages, sizeof(xen_ulong_t));
> +gpfns

Re: [Xen-devel] [PATCH v3 5/7] libxl: support unmapping static shared memory areas during domain destruction

[Wei Liu]

On Thu, Oct 19, 2017 at 10:36:33AM +0800, Zhongze Liu wrote:
> Add libxl__sshm_del to unmap static shared memory areas mapped by
> libxl__sshm_add during domain creation. The unmapping process is:
> 
> * For a master: decrease the refcount of the sshm region, if the refcount
>   reaches 0, cleanup the whole sshm path.
> * For a slave: unmap the shared pages, and cleanup related xs entries.
>   decrease the refcount of the sshm region, if the refcount reaches 0,
>   cleanup the whole sshm path.
> 

This appears to be in line with what we discussed.

I would like to see some explanation for: if one or more of the things
the code does fail half way, the system is still going to be in a
consistent state. Most notably, there isn't going to be any page leaked
with uncleaned refs.

> +
> +rc = libxl__xs_transaction_commit(gc, );
> +if (!rc) break;
> +if (rc < 0) goto out;
> + isretry = true;

Indentation.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Commit moratorium to staging

[Wei Liu]

On Tue, Oct 31, 2017 at 04:52:37PM +, Roger Pau Monné wrote:
> 
> I have to admit I have no idea why Windows clears the STS power bit
> and then completely ignores it on certain occasions.
> 
> I'm also afraid I have no idea how to debug Windows in order to know
> why this event is acknowledged but ignored.
> 
> I've also tried to reproduce the same with a Debian guest, by doing
> the same amount of save/restores and migrations, and finally issuing a
> xl trigger  power, but Debian has always worked fine and
> shut down.
> 
> Any comments are welcome.

After googling around, some articles suggest Windows can ignore ACPI
events under certain circumstances. Is it worth checking in the Windows
event log to see if an event is received but ignored for reason X?

For Windows Server 2012:
https://serverfault.com/questions/534042/windows-2012-how-to-make-power-button-work-in-every-cases

Can't find anything for Windows Server 2016.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] gdbsx: prefer privcmd character device

[Wei Liu]

Cc Julien and change tag. I think this is safe to be applied to 4.10.

On Tue, Oct 31, 2017 at 01:58:07PM -0700, Elena Ufimtseva wrote:
> On Tue, Oct 31, 2017 at 03:25:39PM +0000, Wei Liu wrote:
> > On Tue, Oct 31, 2017 at 10:20:11AM -0500, Doug Goldstein wrote:
> > > Prefer using the character device over the proc file if the character
> > > device exists.
> > > 
> > > CC: Elena Ufimtseva <elena.ufimts...@oracle.com>
> > > CC: Ian Jackson <ian.jack...@eu.citrix.com>
> > > CC: Stefano Stabellini <stefano.stabell...@eu.citrix.com>
> > > CC: Wei Liu <wei.l...@citrix.com>
> > > Signed-off-by: Doug Goldstein <car...@cardoe.com>
> > > ---
> > > So this was originally submitted with 9c89dc95201 and 7d418eab3b6 and
> > > was rejected since the goal was to convert gdbsx to use libxc but that
> > > hasn't happened. /dev/xen/privcmd should be preferred and this change
> > > makes that happen. It would be nice if we landed this with the plan
> > > to convert gdbsx happening when it happens.
> > 
> > Oh well... I think this is fine.
> > 
> > Elena has the final verdict.
> 
> I think this is fine.
> I will look into the conversion and relevant discussions if I find them and
> see what can be done.
> 
> Thanks!
> 
> Meanwhile,
> Reviewed-by: Elena Ufimtseva <elena.ufimts...@oracle.com>
> 
> Elena

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [OSSTEST PATCH v2 00/19] Upgrade to Stretch

[Wei Liu]

On Tue, Oct 31, 2017 at 06:42:28PM +, Wei Liu wrote:
> On Tue, Oct 31, 2017 at 01:51:44PM +0000, Wei Liu wrote:
> > First version of this series can be found at [0].
> > 
> > This version contains workaround for Arndale boards. They are now 
> > functional.
> > 
> > A bunch of test cases failed:
> > 
> > 1. Rumpkernel tests -- I've sent an email to Antti for advice.
> > 2. Windows tests -- They don't look different from normal flights. 
> > 3. memdisk-try-append -- Osstest couldn't find some file. I don't think it 
> > is
> >related to the code I modified.
> > 4. guest-localmigrate/x10 for xl-qcow2 test -- Guest kernel bug.
> > 5. nested hvm amd, pvhv2 -- Expected failure.
> > 
> > Example flight:
> > http://logs.test-lab.xenproject.org/osstest/logs/115404/
> > 
> > The armhf d-i failure is fixed with an additional patch ("Skip bootloader
> > installaion for arm32 on Stretch) on top of the code for 15404, in:
> > 
> > http://logs.test-lab.xenproject.org/osstest/logs/115404/
> 
> This should be 115433.

And a complete run of the series:

http://logs.test-lab.xenproject.org/osstest/logs/115436/

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC for-next] x86/mm: introduce and use virt_to_xen_l4e

[Wei Liu]

Avoid open-coding in a lot of places.

No functional change.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
Is this patch useful or is open-coding preferred?
---
 xen/arch/x86/mm.c  | 14 +++---
 xen/arch/x86/x86_64/mm.c   | 36 +---
 xen/include/asm-x86/page.h |  5 +
 3 files changed, 29 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index a20fdcaea4..cd70aba60c 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -363,7 +363,7 @@ void __init arch_init_memory(void)
 if ( l3tab )
 {
 const l3_pgentry_t *l3idle =
-l4e_to_l3e(idle_pg_table[l4_table_offset(split_va)]);
+l4e_to_l3e(*virt_to_xen_l4e(split_va));
 
 for ( i = 0; i < l3_table_offset(split_va); ++i )
 l3tab[i] = l3idle[i];
@@ -1575,12 +1575,12 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn,
 
 /* Slot 256: RO M2P (if applicable). */
 l4t[l4_table_offset(RO_MPT_VIRT_START)] =
-ro_mpt ? idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)]
+ro_mpt ? *virt_to_xen_l4e(RO_MPT_VIRT_START)
: l4e_empty();
 
 /* Slot 257: PCI MMCFG. */
 l4t[l4_table_offset(PCI_MCFG_VIRT_START)] =
-idle_pg_table[l4_table_offset(PCI_MCFG_VIRT_START)];
+*virt_to_xen_l4e(PCI_MCFG_VIRT_START);
 
 /* Slot 258: Self linear mappings. */
 ASSERT(!mfn_eq(l4mfn, INVALID_MFN));
@@ -1609,7 +1609,7 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn,
 l4_pgentry_t *next;
 
 memcpy([l4_table_offset(XEN_VIRT_START)],
-   _pg_table[l4_table_offset(XEN_VIRT_START)],
+   virt_to_xen_l4e(XEN_VIRT_START),
(ROOT_PAGETABLE_FIRST_XEN_SLOT + root_pgt_pv_xen_slots -
 l4_table_offset(XEN_VIRT_START)) * sizeof(*l4t));
 
@@ -1629,7 +1629,7 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn,
   : ROOT_PAGETABLE_XEN_SLOTS);
 
 memcpy([l4_table_offset(XEN_VIRT_START)],
-   _pg_table[l4_table_offset(XEN_VIRT_START)],
+   virt_to_xen_l4e(XEN_VIRT_START),
(ROOT_PAGETABLE_FIRST_XEN_SLOT + slots -
 l4_table_offset(XEN_VIRT_START)) * sizeof(*l4t));
 }
@@ -1643,7 +1643,7 @@ bool fill_ro_mpt(mfn_t mfn)
 if ( !l4e_get_intpte(l4tab[l4_table_offset(RO_MPT_VIRT_START)]) )
 {
 l4tab[l4_table_offset(RO_MPT_VIRT_START)] =
-idle_pg_table[l4_table_offset(RO_MPT_VIRT_START)];
+*virt_to_xen_l4e(RO_MPT_VIRT_START);
 ret = true;
 }
 unmap_domain_page(l4tab);
@@ -4476,7 +4476,7 @@ static l3_pgentry_t *virt_to_xen_l3e(unsigned long v)
 {
 l4_pgentry_t *pl4e;
 
-pl4e = _pg_table[l4_table_offset(v)];
+pl4e = virt_to_xen_l4e(v);
 if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
 {
 bool locking = system_state > SYS_STATE_boot;
diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index 34cd8457cf..87edfe0a7e 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -133,7 +133,7 @@ static int m2p_mapped(unsigned long spfn)
 l2_pgentry_t *l2_ro_mpt;
 
 va = RO_MPT_VIRT_START + spfn * sizeof(*machine_to_phys_mapping);
-l3_ro_mpt = l4e_to_l3e(idle_pg_table[l4_table_offset(va)]);
+l3_ro_mpt = l4e_to_l3e(*virt_to_xen_l4e(va));
 
 switch ( l3e_get_flags(l3_ro_mpt[l3_table_offset(va)]) &
  (_PAGE_PRESENT |_PAGE_PSE))
@@ -166,7 +166,7 @@ static int share_hotadd_m2p_table(struct mem_hotadd_info 
*info)
   v += n << PAGE_SHIFT )
 {
 n = L2_PAGETABLE_ENTRIES * L1_PAGETABLE_ENTRIES;
-l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(v)])[
+l3e = l4e_to_l3e(*virt_to_xen_l4e(v))[
 l3_table_offset(v)];
 if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) )
 continue;
@@ -193,7 +193,7 @@ static int share_hotadd_m2p_table(struct mem_hotadd_info 
*info)
   v != RDWR_COMPAT_MPT_VIRT_END;
   v += 1 << L2_PAGETABLE_SHIFT )
 {
-l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(v)])[
+l3e = l4e_to_l3e(*virt_to_xen_l4e(v))[
 l3_table_offset(v)];
 if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) )
 continue;
@@ -226,7 +226,7 @@ static void destroy_compat_m2p_mapping(struct 
mem_hotadd_info *info)
 if ( emap > ((RDWR_COMPAT_MPT_VIRT_END - RDWR_COMPAT_MPT_VIRT_START) >> 2) 
)
 emap = (RDWR_COMPAT_MPT_VIRT_END - RDWR_COMPAT_MPT_VIRT_START) >> 2;
 
-l3_ro_mpt = 
l4e_to_l3e(idle_pg_table[l4_table_offset(HIRO_COMPAT_MPT_VIRT_START)]);
+l3_ro_mpt = l4e_to_l3e(*virt_to_xen_l4e(HIRO_COMPAT_MPT_VIRT_START));
 
 
ASSERT(l3e_get_flags(l3_ro_mpt[l3_table_offset(HIRO_COMPAT_MPT_VIRT_START)]) & 
_PAGE_PRESENT);
 
@@ -261,7 +261,7 @@ static void destr

Re: [Xen-devel] [OSSTEST PATCH v2 00/19] Upgrade to Stretch

[Wei Liu]

On Tue, Oct 31, 2017 at 01:51:44PM +, Wei Liu wrote:
> First version of this series can be found at [0].
> 
> This version contains workaround for Arndale boards. They are now functional.
> 
> A bunch of test cases failed:
> 
> 1. Rumpkernel tests -- I've sent an email to Antti for advice.
> 2. Windows tests -- They don't look different from normal flights. 
> 3. memdisk-try-append -- Osstest couldn't find some file. I don't think it is
>related to the code I modified.
> 4. guest-localmigrate/x10 for xl-qcow2 test -- Guest kernel bug.
> 5. nested hvm amd, pvhv2 -- Expected failure.
> 
> Example flight:
> http://logs.test-lab.xenproject.org/osstest/logs/115404/
> 
> The armhf d-i failure is fixed with an additional patch ("Skip bootloader
> installaion for arm32 on Stretch) on top of the code for 15404, in:
> 
> http://logs.test-lab.xenproject.org/osstest/logs/115404/

This should be 115433.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] common/multicall: Increase debugability for bad hypercalls

[Wei Liu]

On Tue, Oct 31, 2017 at 05:18:52PM +, Andrew Cooper wrote:
> While investigating an issue (in a new codepath I'd introduced, as it turns
> out), leaving interrupts disabled manifested as a subsequent op in the
> multicall failing a check_lock() test.
> 
> The codepath would have hit the ASSERT_NOT_IN_ATOMIC on the return-to-guest
> path, had it not hit the check_lock() first.
> 
> Call ASSERT_NOT_IN_ATOMIC() after each operation in the multicall, to make
> failures more obvious.
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] gdbsx: prefer privcmd character device

[Wei Liu]

On Tue, Oct 31, 2017 at 10:20:11AM -0500, Doug Goldstein wrote:
> Prefer using the character device over the proc file if the character
> device exists.
> 
> CC: Elena Ufimtseva <elena.ufimts...@oracle.com>
> CC: Ian Jackson <ian.jack...@eu.citrix.com>
> CC: Stefano Stabellini <stefano.stabell...@eu.citrix.com>
> CC: Wei Liu <wei.l...@citrix.com>
> Signed-off-by: Doug Goldstein <car...@cardoe.com>
> ---
> So this was originally submitted with 9c89dc95201 and 7d418eab3b6 and
> was rejected since the goal was to convert gdbsx to use libxc but that
> hasn't happened. /dev/xen/privcmd should be preferred and this change
> makes that happen. It would be nice if we landed this with the plan
> to convert gdbsx happening when it happens.

Oh well... I think this is fine.

Elena has the final verdict.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/5] libxl: add PV sound device

[Wei Liu]

On Tue, Oct 31, 2017 at 04:51:48PM +0200, Oleksandr Grytsov wrote:
> > > +
> > > +if (params->sample_rates) {
> > > +// calculate required string size;
> >
> > Coding style.
> 
> 
> Sorry, could you specify more precisely what has to be changed in this
> place?
> 

We use /* ... */ for comments.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 10/19] ts-debian-fixup: use correct resume device

[Wei Liu]

See code comment for explanation.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-debian-fixup | 12 
 1 file changed, 12 insertions(+)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index 288766d..9a63c61 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -175,6 +175,18 @@ sub otherfixupcfg () {
 $extra .= " iommu=soft";
 }
 
+# There might be stale entries in /etc/initramfs-tools/conf.d/resume which
+# get stored in the initramfs. That introduces delay in guest booting which
+# might cause tests to fail.
+#
+# This is particularly prominent in stretch when it tries to scan for the
+# inexistent device(s) for a long time. See also Debian bug #784810.
+#
+# Override that in kernel command line with the correct swap partition.
+$cfg =~ m/'phy:.+-swap,(xvda\d+),.*'/;
+$extra .= " resume=/dev/$1";
+logm("change resume device to $1");
+
 $cfg =~ s/^extra\s*=\s*['"](.*)['"]/extra = '$1 $extra'/mg;
 };
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 12/19] ts-xen-build-prep: install e2fslibs-dev

[Wei Liu]

The in-tree libfsimage ext2fs implementation can't handle 64bit
enabled ext4, which is the default in stretch.

Installing e2fslibs-dev causes libfsimage to pick up the packaged
ext2fs implementation.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-xen-build-prep | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 80bfb30..8b4b08a 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -225,6 +225,12 @@ sub prep () {
 push(@packages, qw(texinfo autopoint libpciaccess-dev));
 }
 
+# The in-tree ext4 support in libfsimage can't cope with 64bit ext4 on
+# 32bit build. Use the packaged library.
+if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
+push(@packages, qw(e2fslibs-dev));
+}
+
 target_install_packages($ho, @packages);
 target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates");
 # workaround for Debian #595728
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 15/19] Add clk_ignore_unused for stretch for arm hosts

[Wei Liu]

Without that parameter we lose uart output.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index e7fb020..2bfd5ae 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -240,7 +240,7 @@ END
push @xenkopt, $xenkopt;
# http://bugs.xenproject.org/xen/bug/45
push @xenkopt, "clk_ignore_unused"
-   if $ho->{Suite} =~ m/wheezy|jessie/;
+   if $ho->{Suite} =~ m/wheezy|jessie|stretch/;
 
$xenkopt = join ' ', @xenkopt;
logm("Dom0 Linux options: $xenkopt");
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 19/19] Switch to Debian Stretch

[Wei Liu]

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 Osstest.pm| 2 +-
 production-config | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest.pm b/Osstest.pm
index ceb62ca..f2e4dfa 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -86,7 +86,7 @@ our %c = qw(
 
 Images images
 
-DebianSuite jessie
+DebianSuite stretch
 DebianMirrorSubpath debian
 
 TestHostKeypairPath id_rsa_osstest
diff --git a/production-config b/production-config
index 0e6a51e..251d82f 100644
--- a/production-config
+++ b/production-config
@@ -93,10 +93,12 @@ TftpNetbootGroup osstest
 # Update with ./mg-debian-installer-update(-all)
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2017-04-06
+TftpDiVersion_stretch 2017-10-11
 
 # For ISO installs
 DebianImageVersion_wheezy 7.2.0
 DebianImageVersion_jessie 8.2.0
+DebianImageVersion_stretch 9.2.1
 
 # These should normally be the same.
 # Update with ./mg-cpu-microcode-update
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 13/19] TestSupport: add dpkg option when installing packages

[Wei Liu]

Upgrading configuration file of nbd-client is controlled by dpkg in
stretch. Add dpkg option to keep old configuration file(s).

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 Osstest/TestSupport.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 6d5e667..238be9e 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -527,7 +527,8 @@ sub target_run_apt {
 my ($ho, @aptopts) = @_;
 target_cmd_root($ho,
 "DEBIAN_PRIORITY=critical UCF_FORCE_CONFFOLD=y \\
-with-lock-ex -w /var/lock/osstest-apt apt-get @aptopts", 3000);
+with-lock-ex -w /var/lock/osstest-apt \\
+apt-get -o Dpkg::Options::=\"--force-confold\" @aptopts", 3000);
 }
 sub target_install_packages {
 my ($ho, @packages) = @_;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 17/19] Skip bootloader installation for arm32 in Stretch

[Wei Liu]

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 Osstest/Debian.pm | 5 +
 1 file changed, 5 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 2c3bcf4..b2d5007 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1027,6 +1027,11 @@ END
$preseed_file.= (<<END);
 d-i nobootloader/confirmation_common boolean true
 END
+# Stretch has a different method to skip bootloader installation
+$preseed_file.= (<<END);
+d-i grub-installer/skip boolean true
+d-i lilo-installer/skip boolean true
+END
 
 # Debian Bug #771949 means that update-menu-list always
 # generates a full absolute path to the kernel + initrd, while
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 16/19] Set mac address in interfaces(5) if force-mac-address is set

[Wei Liu]

ff9e0d8cbd generated a udev rule for setting the mac address. But that
udev rule is not copied into the target so reboot after installation
will fail.

We can copy the udev rule to target system so the reboot after
installation works, but then the generated udev rules will end up in
initramfs, which means the guest (which uses host's initrd)  will use
the same rune to set conflicting mac address.

Put the mac address in interfaces(5). We still need to keep the udev
rule for the initrd overlay otherwise host installation will fail.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 Osstest/Debian.pm | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 2bfd5ae..2c3bcf4 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1207,6 +1207,16 @@ END
preseed_hook_command($ho, 'late_command', $sfx, $cmds);
 }
 
+my $wantphysif= get_host_property($ho,'interface force','auto');
+if ($wantphysif ne 'auto' && $ho->{Flags}{'force-mac-address'}) {
+   preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+sed -i 's/\\(iface\\s*$wantphysif\\s*inet.*\\)/\\1\\nhwaddress 
$ho->{Ether}/' /target/etc/network/interfaces
+END
+}
+
 if ( $ho->{Flags}{'need-uboot-bootscr'} ) {
my @bootargs = uboot_common_kernel_bootargs($ho);
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 18/19] make-flight: don't test pvgrub for Xen XXX

[Wei Liu]

XXX Need to pin down the version of Xen when the upgrade to stretch is
complete because osstest configuration is branched for each version.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 make-flight | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index 76620c1..b0f1e69 100755
--- a/make-flight
+++ b/make-flight
@@ -827,7 +827,22 @@ test_matrix_do_one () {
   #do_passthrough_tests
 
   do_pygrub_tests
-  do_pvgrub_tests
+
+  # pvgrub1 tests for version < XXX only
+  case "$xenbranch" in
+  xen-4.3-testing) test_pvgrub=y ;;
+  xen-4.4-testing) test_pvgrub=y ;;
+  xen-4.5-testing) test_pvgrub=y ;;
+  xen-4.6-testing) test_pvgrub=y ;;
+  xen-4.7-testing) test_pvgrub=y ;;
+  xen-4.8-testing) test_pvgrub=y ;;
+  xen-4.9-testing) test_pvgrub=y ;;
+  *)   test_pvgrub=n ;;
+  esac
+
+  if [ x$test_pvgrub = xy ]; then
+  do_pvgrub_tests
+  fi
 
   do_xtf_tests
   do_livepatch_tests
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 11/19] ts-debian-hvm-install: disable new nic naming scheme

[Wei Liu]

This is required to fix nested hvm test. The L1 host is installed by
this script. We want the L1 host to not use the new nic naming scheme.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-debian-hvm-install | 13 +
 1 file changed, 13 insertions(+)

diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 54d5d1c..afe8dc9 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -114,6 +114,19 @@ set -ex
 in-target sed -i 's/^deb *cdrom/#&/g' /etc/apt/sources.list
 END
 
+# Do not use "Predictable Network Interface Names" -- this can break
+# nested HVM tests.
+# 
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
+#
+# See also
+# https://www.debian.org/releases/stretch/example-preseed.txt
+my $netifnames = "";
+$netifnames = <{Suite} =~ m/stretch/;
+d-i debian-installer/add-kernel-opts string net.ifnames=0
+END
+
+$preseed_file .= "$netifnames";
+
 $preseed_file .= preseed_hook_cmds();
 
 return $preseed_file;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 14/19] ts-guests-nbd-mirror: make it work with stretch

[Wei Liu]

On the server side, only add oldstyle= and port= on Wheezy and Jessie.
Stretch doesn't support or need those anymore.

On the client side, generate new style configuration file.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-guests-nbd-mirror | 47 +++
 1 file changed, 43 insertions(+), 4 deletions(-)

diff --git a/ts-guests-nbd-mirror b/ts-guests-nbd-mirror
index 3032204..7bcc02c 100755
--- a/ts-guests-nbd-mirror
+++ b/ts-guests-nbd-mirror
@@ -60,15 +60,19 @@ sub configserver () {
 [generic]
 user = root
 END
-$scfg .= <{Suite} =~ m/sarge|lenny|squeeze/;
+
+$scfg .= <{Suite} =~ m/wheezy|jessie/;
 oldstyle = true
 END
+
 foreach my $v (@vols) {
$v->{Port}= unique_incrementing_runvar("${srvhost}_nextport",4000);
$v->{Path}= "/dev/$v->{Gho}{Vg}/$v->{Lv}";
$scfg.=<<END;
 [export$v->{Ix}]
 exportname = $v->{Path}
+END
+   $scfg.=<{Suite} =~ m/wheezy|jessie/;
 port = $v->{Port}
 END
 }
@@ -79,9 +83,7 @@ END
 target_install_packages($sho, qw(nbd-server));
 }
 
-sub configclient () {
-target_cmd_root($cho, "dpkg --purge nbd-client ||:");
-
+sub configclient_pre_stretch () {
 my $mydaemon= '/root/nbd-client-async';
 target_putfilecontents_root_stash($cho,10,<<'END',$mydaemon);
 #!/bin/sh
@@ -107,7 +109,44 @@ NBD_PORT[$v->{Ix}]=$v->{Port}
 END
 }
 target_putfilecontents_root_stash($cho,10,$ccfg,"/etc/nbd-client");
+}
+
+sub configclient_stretch () {
+my $ccfg = <<END;
+# generated by $0
+END
+
+foreach my $v (@vols) {
+   my $nbddev = "nbd$v->{Ix}";
+   $ccfg .= <<END;
+$nbddev $sho->{Name} export$v->{Ix}
+END
+}
+
+target_putfilecontents_root_stash($cho,10,$ccfg,"/etc/nbdtab");
+}
+
+sub configclient () {
+target_cmd_root($cho, "dpkg --purge nbd-client ||:");
+
+if ($cho->{Suite} !~ m/stretch/) {
+configclient_pre_stretch();
+} else {
+configclient_stretch();
+}
+
 target_install_packages($cho, qw(nbd-client));
+
+if ($cho->{Suite} !~ m/squeeze|wheezy|jessie/) {
+   foreach my $v (@vols) {
+   my $nbddev = "nbd$v->{Ix}";
+   target_cmd_root($cho, <<END);
+mkdir -p /dev/$v->{Gho}{Vg}
+if ! test -L $v->{Path}; then ln -s /dev/$nbddev $v->{Path}; fi
+END
+   }
+   target_cmd_root($cho, "/etc/init.d/nbd-client start");
+}
 }
 
 sub shuffleconfigs () {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 03/19] ts-xen-build-prep: install packages for suites >jessie

[Wei Liu]

Stubdom build needs texinfo.

Libvirt build needs autopoint.

QEMU build needs libpciaccess-dev.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-xen-build-prep | 4 
 1 file changed, 4 insertions(+)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 7718cff..80bfb30 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -221,6 +221,10 @@ sub prep () {
# jessie (>jessie?)
push(@packages, "libnl-route-3-dev");
 }
+if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
+push(@packages, qw(texinfo autopoint libpciaccess-dev));
+}
+
 target_install_packages($ho, @packages);
 target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates");
 # workaround for Debian #595728
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 08/19] ts-guests-nbd-mirror: use target_{get, put}file_root to transfter cfg

[Wei Liu]

The original code used target_cmd_output_root which caused a trailing
new line to be deleted, which caused libvirt converter to fail.

It wasn't discovered until now because we appended too many "\n".

Use target_{get,put}file_root to do the job.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-guests-nbd-mirror | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/ts-guests-nbd-mirror b/ts-guests-nbd-mirror
index ca8300d..3032204 100755
--- a/ts-guests-nbd-mirror
+++ b/ts-guests-nbd-mirror
@@ -115,8 +115,11 @@ sub shuffleconfigs () {
my $gn= $gns[$i];
my $gho= $ghos[$i];
my $cfgpath= $r{ "$gho->{Guest}_cfgpath" };
-   my $cfgdata= target_cmd_output_root($sho,"cat $cfgpath");
-   target_putfilecontents_root_stash($cho,10,$cfgdata,$cfgpath);
+   my $file= $cfgpath;
+   $file=~ s,/,-,g;
+   $file= "$stash/".hostnamepath($cho)."--$file";
+   target_getfile_root($sho, 60, $cfgpath, $file);
+   target_putfile_root($cho, 60, $file, $cfgpath);
 }
 }
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 00/19] Upgrade to Stretch

[Wei Liu]

First version of this series can be found at [0].

This version contains workaround for Arndale boards. They are now functional.

A bunch of test cases failed:

1. Rumpkernel tests -- I've sent an email to Antti for advice.
2. Windows tests -- They don't look different from normal flights. 
3. memdisk-try-append -- Osstest couldn't find some file. I don't think it is
   related to the code I modified.
4. guest-localmigrate/x10 for xl-qcow2 test -- Guest kernel bug.
5. nested hvm amd, pvhv2 -- Expected failure.

Example flight:
http://logs.test-lab.xenproject.org/osstest/logs/115404/

The armhf d-i failure is fixed with an additional patch ("Skip bootloader
installaion for arm32 on Stretch) on top of the code for 15404, in:

http://logs.test-lab.xenproject.org/osstest/logs/115404/

Cc: Julien Grall <julien.gr...@arm.com>

[0] <20171020103840.32762-1-wei.l...@citrix.com>

Wei Liu (19):
  gitignore: ignore vim swap file
  ts-xen-build-prep: only install w3c-dtd-xhtml for suites jessie
  ts-xen-install: install some packages on stretch
  Debian.pm: use sysvinit-core instead of systemd
  ts-leak-check: suppress systemd-shim, which leaks in stretch
  ts-host-install: don't use the new nic naming scheme
  ts-guests-nbd-mirror: use target_{get,put}file_root to transfter cfg
  ts-debian-fixup: merge origin extra= to our own
  ts-debian-fixup: use correct resume device
  ts-debian-hvm-install: disable new nic naming scheme
  ts-xen-build-prep: install e2fslibs-dev
  TestSupport: add dpkg option when installing packages
  ts-guests-nbd-mirror: make it work with stretch
  Add clk_ignore_unused for stretch for arm hosts
  Set mac address in interfaces(5) if force-mac-address is set
  Skip bootloader installation for arm32 in Stretch
  make-flight: don't test pvgrub for Xen XXX
  Switch to Debian Stretch

 .gitignore |  1 +
 Osstest.pm |  2 +-
 Osstest/Debian.pm  | 19 --
 Osstest/TestSupport.pm |  3 ++-
 make-flight| 17 +++-
 production-config  |  2 ++
 ts-debian-fixup| 14 -
 ts-debian-hvm-install  | 13 
 ts-guests-nbd-mirror   | 54 --
 ts-host-install|  4 
 ts-leak-check  |  1 +
 ts-xen-build-prep  | 15 +-
 ts-xen-install |  3 +++
 13 files changed, 135 insertions(+), 13 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 01/19] gitignore: ignore vim swap file

[Wei Liu]

Signed-off-by: Wei Liu <wei.l...@citrix.com>
Acked-by: Ian Jackson <ian.jack...@eu.citrix.com>
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 425506b..f7e5b77 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,6 @@
 *~
 *.bak
+*.swp
 tmp
 *.tmp
 bisection.ps
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 02/19] ts-xen-build-prep: only install w3c-dtd-xhtml for suites

[Wei Liu]

That package is not included in Stretch.

That package was installed because libvirt build needed it. However
libvirt builds fine without it in Stretch.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-xen-build-prep | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 3e98364..7718cff 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -206,9 +206,12 @@ sub prep () {
   libglib2.0-dev liblzma-dev pkg-config
   autoconf automake libtool xsltproc
   libxml2-utils libxml2-dev
-  libdevmapper-dev w3c-dtd-xhtml libxml-xpath-perl
+  libdevmapper-dev libxml-xpath-perl
   ccache nasm checkpolicy ebtables);
 
+if ($ho->{Suite} =~ m/squeeze|wheezy|jessie/) {
+   push(@packages, "w3c-dtd-xhtml");
+}
 if ($ho->{Suite} !~ m/squeeze|wheezy/) {
push(@packages, qw(ocaml-nox ocaml-findlib));
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 04/19] ts-xen-install: install some packages on stretch

[Wei Liu]

The "route" command is now in that package.

libnl is needed when running xl.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-xen-install | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-xen-install b/ts-xen-install
index ec907c5..d4c25c7 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -57,6 +57,9 @@ sub packages () {
libsdl1.2debian libglib2.0-0 liblzma5
   qemu-utils
netcat-openbsd));
+if ($ho->{Suite} =~ m/stretch/) {
+target_install_packages($ho, 'net-tools libnl-route-3-200');
+}
 if ($ho->{Suite} =~ m/jessie/) {
 target_install_packages($ho, 'libnl-route-3-200');
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 09/19] ts-debian-fixup: merge origin extra= to our own

[Wei Liu]

The original extra= was not removed, so there were two extra= in the
resulting config file.

It wasn't a problem for xl because the second extra= took precedence.
However libvirt tests would only pick up the first extra= --  they
worked by chance.

Fix this issue by merging the original.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 ts-debian-fixup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index f29971d..288766d 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -175,7 +175,7 @@ sub otherfixupcfg () {
 $extra .= " iommu=soft";
 }
 
-$cfg .= "\nextra='$extra'\n";
+$cfg =~ s/^extra\s*=\s*['"](.*)['"]/extra = '$1 $extra'/mg;
 };
 
 sub writecfg () {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 05/19] Debian.pm: use sysvinit-core instead of systemd

[Wei Liu]

Install that packages for suites >wheezy, because they use systemd as
the default init.

Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 845027a..e7fb020 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -827,7 +827,7 @@ sub preseed_base ($$$;@) {
 
 # Systemd doesn't honor osstest-confirm-booted service, which
 # breaks ts-leak-check.  Fall back to SysV init for now.
-if ( $suite =~ /jessie/ ) {
+if ( $suite !~ /squeeze|wheezy/ ) {
preseed_hook_command($ho, 'late_command', $sfx, <<END)
 in-target apt-get install -y sysvinit-core
 END
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 07/19] ts-host-install: don't use the new nic naming scheme

[Wei Liu]

Signed-off-by: Wei Liu <wei.l...@citrix.com>
Acked-by: Ian Jackson <ian.jack...@eu.citrix.com>
---
 ts-host-install | 4 
 1 file changed, 4 insertions(+)

diff --git a/ts-host-install b/ts-host-install
index 11c14a7..7339858 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -271,6 +271,10 @@ END
 # why this is repeated.
 push @hocmdline, "console=$console" unless $console eq "NONE";
 
+# Don't use "Predictable Network Interface Names"
+# 
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
+push @hocmdline, "net.ifnames=0" if $ho->{Suite} =~ m/stretch/;
+
 push @hocmdline,
 get_host_property($ho, "linux-boot-append $ho->{Suite}", ''),
 get_host_property($ho, "linux-boot-append $ho->{Suite} $r{arch}", '');
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v2 06/19] ts-leak-check: suppress systemd-shim, which leaks in stretch

[Wei Liu]

Signed-off-by: Wei Liu <wei.l...@citrix.com>
Acked-by: Ian Jackson <ian.jack...@eu.citrix.com>
---
 ts-leak-check | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-leak-check b/ts-leak-check
index 678d069..41e6245 100755
--- a/ts-leak-check
+++ b/ts-leak-check
@@ -202,6 +202,7 @@ xenstore /vm
 xenstore /libxl
 
 process .* udevd
+process .* /.+/systemd-shim
 
 file /var/run/xenstored/db
 file /var/run/xenstored/db.debug
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 3/7] libxl: introduce a new structure to represent static shared memory regions

[Wei Liu]

On Thu, Oct 19, 2017 at 10:36:31AM +0800, Zhongze Liu wrote:
> Add a new structure to the IDL familiy to represent static shared memory 
> regions
> as proposed in the proposal "Allow setting up shared memory areas between VMs
> from xl config file" (see [1]).
> 
> [1] https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html
> 
> Signed-off-by: Zhongze Liu <blacksk...@gmail.com>
> Reviewed-by: Stefano Stabellini <sstabell...@kernel.org>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 1/7] libxc: add xc_domain_remove_from_physmap to wrap XENMEM_remove_from_physmap

[Wei Liu]

On Thu, Oct 19, 2017 at 10:36:29AM +0800, Zhongze Liu wrote:
> This is for the proposal "Allow setting up shared memory areas between VMs
> from xl config file". See:
> 
>   https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html
> 
> Then plan is to use XENMEM_add_to_physmap_batch to map the shared pages from
> one domU to another and use XENMEM_remove_from_physmap to cancel the sharing.
> A wrapper to XENMEM_add_to_physmap_batch was added in the following commit:
> 
>   commit 20e725e9364cff4a29945f66986ecd88cca8743d
> 
> Now add the wrapper to XENMEM_remove_from_physmap.
> 
> Signed-off-by: Zhongze Liu <blacksk...@gmail.com>
> Reviewed-by: Stefano Stabellini <sstabell...@kernel.org>
> Acked-by: Wei Liu <wei.l...@citrix.com>
> 
> Cc: Ian Jackson <ian.jack...@eu.citrix.com>
> Cc: Wei Liu <wei.l...@citrix.com>
> Cc: Stefano Stabellini <sstabell...@kernel.org>
> Cc: Julien Grall <julien.gr...@arm.com>
> Cc: xen-devel@lists.xen.org
> ---
>  tools/libxc/include/xenctrl.h |  4 
>  tools/libxc/xc_domain.c   | 11 +++
>  2 files changed, 15 insertions(+)
> 
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index 666db0b919..0d8364ea4b 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -1415,6 +1415,10 @@ int xc_domain_add_to_physmap_batch(xc_interface *xch,
> xen_pfn_t *gfpns,
> int *errs);
>  
> +int xc_domain_remove_from_physmap(xc_interface *xch,
> +  domid_t domid,

We recently made changes to use uint32_t for domid.

You can keep my ack after the change.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] MAINTAINERS: Make Christian Lindig maintainer for ocaml tools

[Wei Liu]

On Tue, Oct 17, 2017 at 05:44:11PM +0100, Ian Jackson wrote:
> oxenstored is our default implementation of xenstore, for platforms
> that have ocaml support.  We need it to be maintained.  Dave Scott,
> the only existing maintainer, has had limited availability.
> 
> Christian has been reveiwing patches and offering opinions where
> necessary, although activity in this area has been quiet and there has
> not been a great deal of new development.
> 
> Christian's contributions have been sensible and I think it would be a
> good idea now to formally make him a maintainer.
> 
> CC: Christian Lindig <christian.lin...@citrix.com>
> CC: David Scott <d...@recoil.org>
> Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] common/spinlock: Improve the output from check_lock() if it trips

[Wei Liu]

On Tue, Oct 31, 2017 at 10:49:10AM +, Andrew Cooper wrote:
> If check_lock() triggers, a crash will occur.  Instead of simply identifying
> "the irq context was different", indicate the expected and current irq
> context.
> 
> Signed-off-by: Andrew Cooper <andrew.coop...@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 for-next 02/12] pci: introduce a type to store a SBDF

[Wei Liu]

On Wed, Oct 18, 2017 at 12:40:24PM +0100, Roger Pau Monne wrote:
> That provides direct access to all the members that constitute a SBDF.
> The only function switched to use it is hvm_pci_decode_addr, because
> it makes following patches simpler.
> 
> Suggested-by: Andrew Cooper <andrew.coop...@citrix.com>
> Signed-off-by: Roger Pau Monné <roger@citrix.com>
> Reviewed-by: Paul Durrant <paul.durr...@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 for-next 08/12] xen: introduce rangeset_consume_ranges

[Wei Liu]

On Wed, Oct 18, 2017 at 12:40:30PM +0100, Roger Pau Monne wrote:
> This function allows to iterate over a rangeset while removing the
> processed regions.
> 
> This will be used in order to split processing of large memory areas
> when mapping them into the guest p2m.
> 
> Signed-off-by: Roger Pau Monné <roger@citrix.com>

Reviewed-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 4/4] libxl: move ibxl_devid_to_device_... to LIBXL_DEFINE_DEVID_TO_DEVICE

[Wei Liu]

On Thu, Oct 05, 2017 at 12:30:48PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/4] libxl: move libxl__device_from_ to LIBXL_DEFINE_DEVICE_FROM_TYPE

[Wei Liu]

On Thu, Oct 05, 2017 at 12:30:47PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> LIBXL_DEFINE_DEVICE_FROM_TYPE uses libxl__..._devtype.type to
> be assigned as device and backend type.
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/4] libxl: use libxl__device_kind in LIBXL_DEFINE_UPDATE_DEVID

[Wei Liu]

On Thu, Oct 05, 2017 at 12:30:46PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> Use libxl__..._devtype.type to update device id.
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/4] libxl: use libxl__device_kind to get device XS entry

[Wei Liu]

On Thu, Oct 05, 2017 at 12:30:45PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> On adding to XS name of device is taken from
> libxl__device_kind enum. On getting device from XS
> the name is hardcoded. It leads to potential
> mistmatch errors. The patch is using libxl__device_kind
> everywere to have one source of device name.
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/4 v3] libxl: Change the type of console_mfn to xen_pfn_t

[Wei Liu]

On Tue, Oct 31, 2017 at 12:25:06PM +0530, Bhupinder Thakur wrote:
> Currently the type of console mfn is unsigned long in libxl. This may be
> an issue for 32-bit toolstack running on 64-bit Xen, where the pfn are
> 64 bit. To ensure that console_mfn can hold any valid 64-bit pfn, the
> type of console_mfn is changed to xen_pfn_t.
> 
> Also the name console_mfn is misleading as it is actually a gfn. This
> patch also modifies the name to console_gfn.
> 
> Signed-off-by: Bhupinder Thakur <bhupinder.tha...@linaro.org>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/4 v3 for-4.10] libxl: Fix the bug introduced in commit "libxl: use correct type modifier for vuart_gfn"

[Wei Liu]

Change the tag to for-4.10.

Julien, this is needed to fix vuart emulation.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 4/4 v3] xenconsole: Define and use a macro XEN_INVALID_PFN instead of -1

[Wei Liu]

On Tue, Oct 31, 2017 at 12:25:08PM +0530, Bhupinder Thakur wrote:
> xenconsole will use a new macro XEN_INVALID_PFN instead of -1 for 
> initializing ring-ref.
> Since the type of ring_ref is changed to xen_pfn_t (which is an unsigned 
> value) assigning -1
> appeared to be confusing. For clarity, XEN_INVALID_PFN is introduced.
> 
> Signed-off-by: Bhupinder Thakur <bhupinder.tha...@linaro.org>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/6] libxl: add backend type to vkb

[Wei Liu]

On Thu, Oct 05, 2017 at 12:07:08PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> New field backend_type is added to vkb device
> in order to have QEMU and user space backend
> simultaneously. Each vkb backend shall read
> appropriate XS entry and service only own
> frontends.
> 
> Signed-off-by: Oleksandr Grytsov 
> ---
>  tools/libxl/libxl_create.c  |  4 
>  tools/libxl/libxl_dm.c  |  2 ++
>  tools/libxl/libxl_types.idl |  7 +++
>  tools/libxl/libxl_vkb.c | 10 +-
>  tools/xl/xl_parse.c |  4 
>  5 files changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index f813114..7268f7f 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -1349,6 +1349,7 @@ static void domcreate_launch_dm(libxl__egc *egc, 
> libxl__multidev *multidev,
>  }
>  
>  libxl_device_vkb_init();
> +vkb.backend_type = LIBXL_VKB_BACKEND_QEMU;

Hmm... See below.

>  libxl__device_add(gc, domid, __vkb_devtype, );
>  libxl_device_vkb_dispose();
>  
> @@ -1376,6 +1377,9 @@ static void domcreate_launch_dm(libxl__egc *egc, 
> libxl__multidev *multidev,
>  for (i = 0; i < d_config->num_vfbs; i++) {
>  libxl__device_add(gc, domid, __vfb_devtype,
>_config->vfbs[i]);
> +}
> +
> +for (i = 0; i < d_config->num_vkbs; i++) {
>  libxl__device_add(gc, domid, __vkb_devtype,
>_config->vkbs[i]);
>  }
> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 98f89a9..d8b0ee7 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -1728,6 +1728,8 @@ static int 
> libxl__vfb_and_vkb_from_hvm_guest_config(libxl__gc *gc,
>  
>  vkb->backend_domid = 0;
>  vkb->devid = 0;
> +vkb->backend_type = LIBXL_VKB_BACKEND_QEMU;
> +
>  return 0;
>  }
>  
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index cd0c06f..65cd81a 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -240,6 +240,12 @@ libxl_checkpointed_stream = 
> Enumeration("checkpointed_stream", [
>  (2, "COLO"),
>  ])
>  
> +libxl_vkb_backend = Enumeration("vkb_backend", [
> +(0, "UNKNOWN"),
> +(1, "QEMU"),
> +(2, "LINUX")
> +])

Originally this is only internal detail, but now you want to expose
this.  You need to set the default value for this; otherwise you could
break migration.

And then you also need to provide a setdefault function for
libxl_device_vkb.

Also I'm a bit confused because the LINUX type is not used in this
series.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/6] libxl: move vkb device to libxl_vkb.c

[Wei Liu]

On Thu, Oct 05, 2017 at 12:07:06PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> Logically it is better to move vkb to
> separate file as vkb device used not only by vfb
> and console.
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/6] libxl: fix vkb XS entry and type

[Wei Liu]

On Thu, Oct 05, 2017 at 12:07:07PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> vkb has vkbd name in XS.
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 4/5] xl: add vsnd CLI commands

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:23PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov <oleksandr_gryt...@epam.com>
> 
> Add CLI commands to attach, detach and list virtual sound devices
> 
> Signed-off-by: Oleksandr Grytsov <oleksandr_gryt...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/5] xl: add PV sound condif parser

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:22PM +0300, Oleksandr Grytsov wrote:
> +static void parse_vsnd_card_config(const XLU_Config *config,
> +   XLU_ConfigValue *card_value,
> +   libxl_domain_config *d_config)
> +{
> +int ret;
> +
> +XLU_ConfigList *card_list;
> +
> +// get card

Please delete this.

> +ret = xlu_cfg_value_get_list(config, card_value,  _list, 0);
> +
> +if (ret) {
> +fprintf(stderr, "Failed to get vsnd card list: %s\n", strerror(ret));
> +goto out;
> +}
> +
> +libxl_device_vsnd *vsnd;
> +
> +vsnd = ARRAY_EXTEND_INIT(d_config->vsnds,
> + d_config->num_vsnds,
> + libxl_device_vsnd_init);
> +
> +const char *card_item;
> +int item = 0;
> +

Mixing code and declarations will break.

> +while ((card_item = xlu_cfg_get_listitem(card_list, item++)) != NULL) {
> +ret = parse_vsnd_item(vsnd, card_item);
> +if (ret) goto out;
> +}
> +
> +ret = 0;
> +
> +out:
> +if (ret) exit(EXIT_FAILURE);
> +}

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/5] libxl: add vsnd list and info

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:21PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> Add getting vsnd list amd info API
> 
> Signed-off-by: Oleksandr Grytsov 

Same comments for previous patch apply here, too.

> ---
>  tools/libxl/libxl.h |  10 ++
>  tools/libxl/libxl_types.idl |  19 +++
>  tools/libxl/libxl_utils.h   |   3 +
>  tools/libxl/libxl_vsnd.c| 359 
> +++-
>  4 files changed, 388 insertions(+), 3 deletions(-)
> 
>  
[...]
> +static int libxl__sample_rates_from_string(libxl__gc *gc, const char *str,
> +   libxl_vsnd_params *params)
> +{
> +char *tmp = libxl__strdup(gc, str);
> +
> +params->num_sample_rates = 0;
> +params->sample_rates = NULL;
> +
> +char *p = strtok(tmp, " ,");
> +
> +while (p != NULL) {
> +params->sample_rates = realloc(params->sample_rates,

libxl__realloc(NOGC, ...)

[...]
> +
> +static int libxl__pcm_from_xenstore(libxl__gc *gc, const char *path,
> +libxl_vsnd_pcm *pcm)
> +{
> +libxl_ctx *ctx = libxl__gc_owner(gc);

You can use CTX throughout.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/5] libxl: add PV sound device

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:20PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> Add PV sound device described in sndif.h
> 
> Signed-off-by: Oleksandr Grytsov 

[...]
>  
>  libxl__console_backend = Enumeration("console_backend", [
> diff --git a/tools/libxl/libxl_vsnd.c b/tools/libxl/libxl_vsnd.c
> new file mode 100644
> index 000..26885f9
> --- /dev/null
> +++ b/tools/libxl/libxl_vsnd.c
> @@ -0,0 +1,307 @@
> +/*
> + * Copyright (C) 2016 EPAM Systems Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "libxl_internal.h"
> +#include "xen/io/sndif.h"
> +

Use  -- this is not a local header.

> +
> +static unsigned int libxl__rates_to_str_vsnd(char *str, uint32_t 
> *sample_rates,
> + int num_sample_rates)
> +{
> +unsigned int len;
> +int i;
> +
> +len = 0;
> +
> +if (num_sample_rates == 0) {
> +return len;
> +}

Coding style.

> +
> +for (i = 0; i < num_sample_rates - 1; i++) {
> +if (str) {
> +len += sprintf([len], "%u,", sample_rates[i]);

libxl__sprintf(NOGC, ...)

> +} else {
> +len += snprintf(NULL, 0, "%u,", sample_rates[i]);
> +}
> +}
> +
> +if (str) {
> +len += sprintf([len], "%u", sample_rates[i]);
> +} else {
> +len += snprintf(NULL, 0, "%u", sample_rates[i]);
> +}
> +
> +return len;
> +}
> +
[...]
> +
> +static int libxl__set_params_vsnd(libxl__gc *gc, char *path,
> +  libxl_vsnd_params *params, flexarray_t 
> *front)
> +{
> +char *buffer;
> +int len;
> +int rc;
> +
> +if (params->sample_rates) {
> +// calculate required string size;

Coding style.

> +len = libxl__rates_to_str_vsnd(NULL, params->sample_rates,
> +   params->num_sample_rates);
> +
> +if (len) {
> +buffer = libxl__malloc(gc, len + 1);
> +
> +libxl__rates_to_str_vsnd(buffer, params->sample_rates,
> + params->num_sample_rates);
> +rc = flexarray_append_pair(front,
> +   
> GCSPRINTF("%s"XENSND_FIELD_SAMPLE_RATES,
> + path), buffer);
> +if (rc) return rc;

goto out please.

Please fix these coding style issues throughout this series.j

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC v2 0/8] Live migration for VMs with QEMU backed local storage

[Wei Liu]

Also CC QEMU maintainers.

Is there any new QEMU patch required?

On Wed, Oct 18, 2017 at 10:26:27AM -0700, Bruno Alvisio wrote:
> I am reviving this thread about the migration of VMs with local storage. I 
> have worked on a solution to be able to migrate VMs that use QEMU as the 
> backend disk driver. I have adapted the migration flow and piggybacked on the 
> “drive-mirroring” capability already provided by QEMU.
> 
> Overview
> 1. The “xl migrate” command has an additional “-q” flag. When provided the 
> local storage of the VM is mirrored to the destination during the migration 
> process.
> 2. Internally, the modification consists on adding a new 
> libxl__stream_read_state struct to the libxl__domain_create_state structure 
> and libxl__stream_read_state structure to the libxl__domain_save_state struct.
> 3. Migration flow can now be divided into three phases:
>a. Phase One: Copies the necessary state to start a QEMU process on the 
> destination. It is started with the “-incoming defer” option.
>b. Phase Two: Disk is mirrored using the QEMU embedded NBD server.
>c. Phase Three: Once the disk is completely mirrored, virtual RAM of the 
> domain is live migrated to the destination. This phase most closely resembles 
> to the current migration flow.
> 4. If the “-q” option is not provided the migration is equivalent to the 
> current migration flow.
> 
> The new migration flow has follows the following major sequence of steps:
> 1. 1st stream copies the QEMU devices RAM from source to destination.
> 2. QEMU process is started on the destination with the option “-incoming 
> defer”. (This creates the QEMU process but it doesn’t start running the main 
> loop until “migrate incoming” command is executed)
> 3. “drive mirror” QMP command is executed so that the disk is mirrored to the 
> destination node.
> 4. An event listener waits for the QMP BLOCK_JOB_READY event sent by QEMU 
> which signals that the "disk mirror job" is complete.
> 5. 2nd Stream copies the virtual RAM from source to destination including 
> QEMU state. At this point, the VM is suspended on source.
> 6. “migrate incoming” QMP command is executed on destination.
> 7. VM is restored in destination.
> 
> This is sample configuration file that I have used to test my branch:
> 
> name="tinycore"
> disk=['/home/balvisio/tinycore.img,raw,xvda,w']
> memory=128
> builder='hvm'
> vcpus=1
> vfb = ['type=vnc']
> vif= ['bridge=xenbr0']
> boot='b'
> acpi=1
> device_model_version='qemu-xen'
> serial='pty'
> vnc=1
> xen_platform_pci=0
> 
> Notes
> 
> 1. Note that the configuration file uses "xen_platform_pci=0”. This is 
> necessary so that the block device is seen by QEMU. Further modification 
> should be made for the case "xen_platform_pci=1” if we still want to use NBD 
> mirroring capability provided by QEMU.
> 2. The current branch has still many hardcoded values. Many of the can be 
> easily removed:
>   a. Port used for disk mirroring (11000)
> b. Name of the block devices. (ide0-hd0) Currently the branch only 
> supports VM with on disk drive.
> c. Live migration memory transfer: The pages transferred by libxc is 
> hardcoded. Only a VM with 128 MB of memory is supported.
> 
> Here is a link to the branch in Github: 
> https://github.com/balvisio/xen/tree/feature/local_storage_migration
> 
> Any feedback/suggestion is appreciated.
> 

I haven't had time to review this series in detail, but I think the
series could use some clean-up and improvement to the commit message.
Please refer to

https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches

Currently all patches are missing commit message explaining why the
change is needed as well as your SoB. In case you're not aware, we also
have the requirement each individual patch should build.

There are coding style issues in your patches, please refer to
libxl/CODING_STYLE. The same style applies to xl, too.

Furthermore, patches should be cleaned up before posting -- I see a few
debug printf's got added and removed, as well as a few TODOs with open
questions. Preferably the open questions should be mentioned in the
cover letter to catch attentions.

How this helps. Any more questions, feel free to ask.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 7/9] coverage: introduce support for llvm profiling

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:36AM +0100, Roger Pau Monne wrote:
> Introduce the functionality in order to fill the hooks of the
> cov_sysctl_ops struct.
> 
> Signed-off-by: Roger Pau Monné <roger@citrix.com>
> ---
> Cc: Andrew Cooper <andrew.coop...@citrix.com>
> Cc: George Dunlap <george.dun...@eu.citrix.com>
> Cc: Ian Jackson <ian.jack...@eu.citrix.com>
> Cc: Jan Beulich <jbeul...@suse.com>
> Cc: Konrad Rzeszutek Wilk <konrad.w...@oracle.com>
> Cc: Stefano Stabellini <sstabell...@kernel.org>
> Cc: Tim Deegan <t...@xen.org>
> Cc: Wei Liu <wei.l...@citrix.com>
> Cc: llvm-...@lists.llvm.org
> ---
> Note that the file that contains the helpers is under a BSD 2-clause
> license. This is done so it can be shared with other OSes that use the
> llvm/clang compiler.

It would be helpful if you can put a reference to the original code
somewhere, either in commit message or in the comment section of the
file.

> ---
>  xen/common/coverage/Makefile |   2 +-
>  xen/common/coverage/llvm.c   | 148 
> +++
>  xen/include/public/sysctl.h  |   6 ++
>  3 files changed, 155 insertions(+), 1 deletion(-)
>  create mode 100644 xen/common/coverage/llvm.c
> 
> diff --git a/xen/common/coverage/Makefile b/xen/common/coverage/Makefile
> index e4541a1233..f2ffb2b8de 100644
> --- a/xen/common/coverage/Makefile
> +++ b/xen/common/coverage/Makefile
> @@ -11,5 +11,5 @@ obj-$(CONFIG_GCOV_FORMAT_AUTODETECT) += $(call 
> cc-ifversion,lt,0x040700, \
>   gcc_4_9.o, $(call 
> cc-ifversion,lt,0x07, \
>   gcc_5.o, gcc_7.o
>  else
> -obj-y += coverage.o
> +obj-y += llvm.o coverage.o
>  endif
> diff --git a/xen/common/coverage/llvm.c b/xen/common/coverage/llvm.c
> new file mode 100644
> index 00..c8905070a3
> --- /dev/null
> +++ b/xen/common/coverage/llvm.c
> +#include 
> +#include 
> +#include 
> +#include 
> +

Order.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 5/9] coverage: introduce generic file

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:34AM +0100, Roger Pau Monne wrote:
> 
> diff --git a/xen/common/coverage/Makefile b/xen/common/coverage/Makefile
> index f68d050eca..0e0510679e 100644
> --- a/xen/common/coverage/Makefile
> +++ b/xen/common/coverage/Makefile
> @@ -1,4 +1,4 @@
> -obj-y += gcov_base.o gcov.o
> +obj-y += gcov_base.o gcov.o coverage.o
>  obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_3_4.o
>  obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_4_7.o
>  obj-$(CONFIG_GCOV_FORMAT_4_9) += gcc_4_9.o
> diff --git a/xen/common/coverage/coverage.c b/xen/common/coverage/coverage.c
> new file mode 100644
> index 00..1dec6944be
> --- /dev/null
> +++ b/xen/common/coverage/coverage.c
> @@ -0,0 +1,71 @@
> +/*
> + * Generic functionality for coverage analysis.
> + *
> + * Copyright (C) 2017 Citrix Systems R
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms and conditions of the GNU General Public
> + * License, version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public
> + * License along with this program; If not, see 
> .
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 

Please sort this.

The rest appears to be pure code motion to me. It looks fine.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 3/9] gcov: rename sysctl and functions

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:32AM +0100, Roger Pau Monne wrote:
> Change gcov to cov.
> 
> Signed-off-by: Roger Pau Monné <roger@citrix.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 2/9] gcov: rename folder and header to coverage

[Wei Liu]

> diff --git a/xen/include/xen/gcov.h b/xen/include/xen/coverage.h
> similarity index 100%
> rename from xen/include/xen/gcov.h
> rename to xen/include/xen/coverage.h

Please also rename the define guard in this file.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 9/9] coverage: add documentation for LLVM coverage

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:38AM +0100, Roger Pau Monne wrote:
> Signed-off-by: Roger Pau Monné <roger@citrix.com>
> ---
> Cc: Andrew Cooper <andrew.coop...@citrix.com>
> Cc: George Dunlap <george.dun...@eu.citrix.com>
> Cc: Ian Jackson <ian.jack...@eu.citrix.com>
> Cc: Jan Beulich <jbeul...@suse.com>
> Cc: Konrad Rzeszutek Wilk <konrad.w...@oracle.com>
> Cc: Stefano Stabellini <sstabell...@kernel.org>
> Cc: Tim Deegan <t...@xen.org>
> Cc: Wei Liu <wei.l...@citrix.com>
> ---
>  docs/misc/coverage.markdown | 47 
> +
>  1 file changed, 47 insertions(+)
> 
> diff --git a/docs/misc/coverage.markdown b/docs/misc/coverage.markdown
> index 0a32c48f4b..565644631a 100644
> --- a/docs/misc/coverage.markdown
> +++ b/docs/misc/coverage.markdown
> @@ -8,6 +8,8 @@ information. Every basic block in the code will be 
> instrumented by the compiler
>  to compute these statistics. It should not be used in production as it slows
>  down your hypervisor.
>  
> +# GCOV (GCC coverage)
> +
>  ## Enable coverage
>  
>  Test coverage support can be turned on compiling Xen with the `coverage` 
> option set
> @@ -87,3 +89,48 @@ blob extracted from xencov!**
>  * See output in a browser
>  
>  firefox cov/index.html
> +
> +# LLVM coverage
> +
> +## Enable coverage
> +
> +Coverage can be enabled using a Kconfig option, from the top-level directory
> +use the following command to display the Kconfig menu:
> +
> +gmake -C xen menuconfig clang=y
> +
> +The LLVM coverage option can be found inside of the "Debugging Options"
> +section. After enabling it just compile Xen as you would normally do:
> +
> +   gmake xen clang=y
> +

It can be a bit confusing when make and gmake appear in the same
document. I suggest you use make all over and add a footnote saying it
should be gmake on FreeBSD.

This footnote should apply to both gcov and llvm-cov -- I believe you
should be able to build gcov support on FreeBSD, too.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] tools/xenstored: Check number of strings passed to do_control()

[Wei Liu]

On Fri, Oct 27, 2017 at 04:32:15PM +, Pawel Wieczorkiewicz wrote:
> It is possible to send a zero-string message body to xenstore's
> XS_CONTROL handling function. Then the number of strings is used
> for an array allocation. This leads to a crash in strcmp() in a
> CONTROL sub-command invocation loop.
> The output of xs_count_string() should be verified and all 0 or
> negative values should be rejected with an EINVAL. At least the
> sub-command name must be specified.
> 
> The xenstore crash can only be triggered from within dom0 (there
> is a check in do_control() rejecting all non-dom0 requests with
> an EACCES).
> 
> Testing: reproduced with the following command:
> python -c 'print 16*"\x00"' | nc -U $XENSTORED_RUNDIR/socket
> 
> Signed-off-by: Pawel Wieczorkiewicz <wipa...@amazon.de>
> Reviewed-by: Martin Pohlack <mpohl...@amazon.de>

Acked-by: Wei Liu <wei.l...@citrix.com>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 for-4.10] scripts: introduce a script for build test

[Wei Liu]

Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>
Signed-off-by: Wei Liu <wei.l...@citrix.com>
---
Cc: Andrew Cooper <andrew.coop...@citrix.com>
Cc: George Dunlap <george.dun...@eu.citrix.com>
Cc: Ian Jackson <ian.jack...@eu.citrix.com>
Cc: Jan Beulich <jbeul...@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.w...@oracle.com>
Cc: Stefano Stabellini <sstabell...@kernel.org>
Cc: Tim Deegan <t...@xen.org>
Cc: Wei Liu <wei.l...@citrix.com>
Cc: Julien Grall <julien.gr...@arm.com>
Cc: Anthony PERARD <anthony.per...@citrix.com>

v4:
1. Check, save/restore $?.
2. Don't use trap, check exit code directly.
3. More error messages.

v3:
1. Use git-clean in default rune.
2. Print more friendly message.
3. Restore HEAD automatically.
---
 scripts/build-test.sh | 71 +++
 1 file changed, 71 insertions(+)
 create mode 100755 scripts/build-test.sh

diff --git a/scripts/build-test.sh b/scripts/build-test.sh
new file mode 100755
index 00..413540c13b
--- /dev/null
+++ b/scripts/build-test.sh
@@ -0,0 +1,71 @@
+#!/bin/sh
+
+# Run command on every commit within the range specified. If no command is
+# provided, use the default one to clean and build the whole tree.
+#
+# The default rune is rather simple. To do a cross-build, please put your usual
+# build rune in a shell script and invoke it with this script.
+
+if ! test -f xen/common/kernel.c; then
+echo "Please run this script from top-level directory"
+exit 1
+fi
+
+if test $# -lt 2 ; then
+echo "Usage: $0   [CMD]"
+exit 1
+fi
+
+status=`git status -s`
+if test -n "$status"; then
+echo "Tree is dirty, aborted"
+exit 1
+fi
+
+BASE=$1; shift
+TIP=$1; shift
+
+ORIG_BRANCH=`git symbolic-ref -q --short HEAD`
+if test $? -ne 0; then
+echo "Detached HEAD, aborted"
+exit 1
+fi
+
+git rev-list $BASE..$TIP | nl -ba | tac | \
+while read num rev; do
+echo "Testing $num $rev"
+
+git checkout $rev
+ret=$?
+if test $ret -ne 0; then
+echo "Failed to checkout $num $rev with $ret"
+exit $ret
+fi
+
+if test $# -eq 0 ; then
+git clean -fdx && ./configure && make -j4
+else
+"$@"
+fi
+ret=$?
+if test $ret -ne 0; then
+echo "Failed at $num $rev with $ret"
+exit $ret
+fi
+echo
+done
+
+ret=$?
+
+echo "Restoring original HEAD"
+git checkout $ORIG_BRANCH
+gco_ret=$?
+if test $gco_ret -ne 0; then
+echo "Failed to restore orignal HEAD. Check tree status before doing 
anything else!"
+exit $gco_ret
+fi
+
+if test $ret -eq 0; then
+echo "ok."
+fi
+exit $ret
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 for-4.10] scripts: introduce a script for build test

[Wei Liu]

On Mon, Oct 30, 2017 at 03:14:04PM +, Ian Jackson wrote:
> Wei Liu writes ("[PATCH v3 for-4.10] scripts: introduce a script for build 
> test"):
> > Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>
> > Signed-off-by: Wei Liu <wei.l...@citrix.com>
> ...
> ...
> > +trap "echo Restoring original HEAD ; git checkout $ORIG_BRANCH" EXIT
> 
> This will smash the whole script's exit status.  I think you need to
> save/restore $?.  Be careful with your quoting.  Normally it is better
> for the argument to trap to be ''-quoted rather than "", to avoid it
> being expanded twice (and, the first time, too soon).
> 
> Also, if this fails, it leaves the failure message buried in a scrool
> of make -j4 output, where the user probably won't see it.  And it
> prints exactly the same message on success and failure.  On failure
> you should print the failing commitid, and exit nonzero.
> 
> On success you should print some reassuring `ok' message.
> 

Right. I've addressed your comments and will send out a new version
soon.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 for-4.10] scripts: introduce a script for build test

[Wei Liu]

On Wed, Oct 25, 2017 at 05:00:21PM +0100, Wei Liu wrote:
> Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>
> Signed-off-by: Wei Liu <wei.l...@citrix.com>
> ---
> Cc: Andrew Cooper <andrew.coop...@citrix.com>
> Cc: George Dunlap <george.dun...@eu.citrix.com>
> Cc: Ian Jackson <ian.jack...@eu.citrix.com>
> Cc: Jan Beulich <jbeul...@suse.com>
> Cc: Konrad Rzeszutek Wilk <konrad.w...@oracle.com>
> Cc: Stefano Stabellini <sstabell...@kernel.org>
> Cc: Tim Deegan <t...@xen.org>
> Cc: Wei Liu <wei.l...@citrix.com>
> Cc: Julien Grall <julien.gr...@arm.com>
> Cc: Anthony PERARD <anthony.per...@citrix.com>
> 
> v3:
> 1. Use git-clean in default rune.
> 2. Print more friendly message.
> 3. Restore HEAD automatically.

Ping?

> ---
>  scripts/build-test.sh | 50 ++
>  1 file changed, 50 insertions(+)
>  create mode 100755 scripts/build-test.sh
> 
> diff --git a/scripts/build-test.sh b/scripts/build-test.sh
> new file mode 100755
> index 00..f55ec5d4fa
> --- /dev/null
> +++ b/scripts/build-test.sh
> @@ -0,0 +1,50 @@
> +#!/bin/sh
> +
> +# Run command on every commit within the range specified. If no command is
> +# provided, use the default one to clean and build the whole tree.
> +#
> +# The default rune is rather simple. To do a cross-build, please put your 
> usual
> +# build rune in a shell script and invoke it with this script.
> +
> +if ! test -f xen/common/kernel.c; then
> +echo "Please run this script from top-level directory"
> +exit 1
> +fi
> +
> +if test $# -lt 2 ; then
> +echo "Usage: $0   [CMD]"
> +exit 1
> +fi
> +
> +status=`git status -s`
> +if test -n "$status"; then
> +echo "Tree is dirty, aborted"
> +exit 1
> +fi
> +
> +BASE=$1; shift
> +TIP=$1; shift
> +
> +ORIG_BRANCH=`git symbolic-ref -q --short HEAD`
> +if test $? -ne 0; then
> +echo "Detached HEAD, aborted"
> +exit 1
> +fi
> +
> +trap "echo Restoring original HEAD ; git checkout $ORIG_BRANCH" EXIT
> +
> +git rev-list $BASE..$TIP | nl -ba | tac | \
> +while read num rev; do
> +echo "Testing $num $rev"
> +git checkout $rev
> +if test $# -eq 0 ; then
> +git clean -fdx && ./configure && make -j4
> +else
> +"$@"
> +fi
> +if test $? -ne 0; then
> +echo "Failed at $num $rev"
> +exit 1
> +fi
> +echo
> +done
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Libvirt config converter can't handle file not ending with new line

[Wei Liu]

Hi Jim

I discover a problem when using xen_xl converter. When the file in
question doesn't end with a new line, I get the following error:

  error: configuration file syntax error: memory conf:53: expecting a value


After digging a bit (but haven't read libvirt code), it appears that the
file didn't end with a new line.

I have worked around this, but I think this is an issue that should be
fixed.

Thanks,
Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/5 v2] libxl: Change the type of console_mfn to xen_pfn_t

[Wei Liu]

On Mon, Oct 30, 2017 at 02:42:57PM +0530, Bhupinder Thakur wrote:
> Hi,
> 
> On 26 October 2017 at 16:47, Andrew Cooper <andrew.coop...@citrix.com> wrote:
> > On 26/10/17 12:13, Wei Liu wrote:
> >> On Wed, Oct 25, 2017 at 02:57:05PM +0530, Bhupinder Thakur wrote:
> >>> Currently the type of console mfn is unsigned long in libxl. This may be
> >>> an issue for 32-bit toolstack running on 64-bit Xen, where the pfn are
> >>> 64 bit. To ensure that console_mfn can hold any valid 64-bit pfn, the
> >>> type of console_mfn is changed to xen_pfn_t.
> >>>
> >>> Signed-off-by: Bhupinder Thakur <bhupinder.tha...@linaro.org>
> >>> ---
> >>> CC: Ian Jackson <ian.jack...@eu.citrix.com>
> >>> CC: Wei Liu <wei.l...@citrix.com>
> >>> CC: Stefano Stabellini <sstabell...@kernel.org>
> >>> CC: Julien Grall <julien.gr...@arm.com>
> >>>
> >>> This patch is as per the review of commit fa1f157
> >>> libxl: Fix the bug introduced in commit "libxl: use correct type
> >>>
> >>>  tools/libxl/libxl_console.c  | 2 +-
> >>>  tools/libxl/libxl_dom.c  | 2 +-
> >>>  tools/libxl/libxl_internal.h | 2 +-
> >>>  3 files changed, 3 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/tools/libxl/libxl_console.c b/tools/libxl/libxl_console.c
> >>> index 6bfc0e5..f2ca689 100644
> >>> --- a/tools/libxl/libxl_console.c
> >>> +++ b/tools/libxl/libxl_console.c
> >>> @@ -329,7 +329,7 @@ int libxl__device_console_add(libxl__gc *gc, uint32_t 
> >>> domid,
> >>>  flexarray_append(ro_front, "port");
> >>>  flexarray_append(ro_front, GCSPRINTF("%"PRIu32, 
> >>> state->console_port));
> >>>  flexarray_append(ro_front, "ring-ref");
> >>> -flexarray_append(ro_front, GCSPRINTF("%lu", state->console_mfn));
> >>> +flexarray_append(ro_front, GCSPRINTF("%"PRIu_xen_pfn, 
> >>> state->console_mfn));
> >> Actually, please consider changing console_mfn to console_pfn.
> >
> > If you are going to make this change, then it is a gfn, not a pfn.
> > (console_pfn would be as equally wrong for PV guests as console_mfn is
> > currently wrong for HVM guest.)
> 
> Changing console_mfn to console_gfn will require changes in many
> files. Should I go ahead and change all the files?
> 

$ cd libxl && git grep \\bconsole_mfn | wc -l
14

Not too bad, so please go ahead.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

[Wei Liu]

On Thu, Oct 26, 2017 at 04:45:38PM +0100, Andrew Cooper wrote:
> On 26/10/17 16:25, Olaf Hering wrote:
> > An upcoming change in systemd will mount xenfs right away, along with
> > all other system mounts. This improves the detection of the
> > virtualization environment, which is currently racy. Some parts of
> > systemd rely on the presence of /proc/xen/capabilities, which will only
> > exist if xenfs is mounted. Since xenfs is mounted by the proc-xen.mount
> > unit, it will be processed very late. Other units may be processed
> > earlier, and if they make use of ConditionVirtualization*= failures may
> > occour.
> >
> > Unfortunately mounting xenfs by systemd as an API filesystem will lead
> > to errors when proc-xen.mount is processed. Since that mount point
> > already exists the unit is considered as failed, and other units that
> > depend on proc-xen.mount will not start. To avoid this the existing
> > proc-xen.mount will be converted into proc-xen.service, which just
> > mounts xenfs manually. All dependencies are updated by this change.
> >
> > The existing conditionals in proc-xen.mount will prevent failures with
> > existing systemd based installations:
> > ConditionPathExists=!/proc/xen/capabilities will prevent execution with
> > a new systemd that mounts xenfs. And this conditional, in combination
> > with ConditionPathExists=/proc/xen, will trigger execution with an old
> > systemd.
> >
> > An absolute path to the mount binary has to be used. /bin/mount is
> > expected to be universally available, nowaways it is a symlink to
> > /usr/bin/mount.
> >
> > Signed-off-by: Olaf Hering 
> 
> Can't all information be obtained from /sys/hypervisor?  If not, how
> hard would it be to make happen?

It is not possible to tell from /sys/hypervisor at the moment.

There is /sys/hypervisor/properties/features but that only contains
feature bits from hypervisor. And XENFEAT_dom0 is not a reliable
indicator for Dom0.

I suppose it requires a small mount of work to make /sys/hypervisor
contain such information, but that won't get propagated to older kernel
so we need to deal with the problem at hand nonetheless.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] tools/hotplug: create XEN_LOG_DIR at runtime

[Wei Liu]

On Fri, Oct 27, 2017 at 07:52:37PM +0300, Andrii Anisov wrote:
> From: Andrii Anisov <andrii_ani...@epam.com>
> 
> /var/log could be a tmpfs mount point, so create xen subfolder at runtime.
> 
> Signed-off-by: Andrii Anisov <andrii_ani...@epam.com>
> Reviewed-by: Volodymyr Babchuk <volodymyr_babc...@epam.com>
> Reviewed-by: Oleksandr Andrushchenko <oleksandr_andrushche...@epam.com>

Acked-by: Wei Liu <wei.l...@citrix.com>

Julien I think we should apply this for 4.10.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

[Wei Liu]

On Thu, Oct 26, 2017 at 05:25:36PM +0200, Olaf Hering wrote:
> An upcoming change in systemd will mount xenfs right away, along with
> all other system mounts. This improves the detection of the
> virtualization environment, which is currently racy. Some parts of
> systemd rely on the presence of /proc/xen/capabilities, which will only
> exist if xenfs is mounted. Since xenfs is mounted by the proc-xen.mount
> unit, it will be processed very late. Other units may be processed
> earlier, and if they make use of ConditionVirtualization*= failures may
> occour.
> 
> Unfortunately mounting xenfs by systemd as an API filesystem will lead
> to errors when proc-xen.mount is processed. Since that mount point
> already exists the unit is considered as failed, and other units that
> depend on proc-xen.mount will not start. To avoid this the existing
> proc-xen.mount will be converted into proc-xen.service, which just
> mounts xenfs manually. All dependencies are updated by this change.
> 
> The existing conditionals in proc-xen.mount will prevent failures with
> existing systemd based installations:
> ConditionPathExists=!/proc/xen/capabilities will prevent execution with
> a new systemd that mounts xenfs. And this conditional, in combination
> with ConditionPathExists=/proc/xen, will trigger execution with an old
> systemd.
> 
> An absolute path to the mount binary has to be used. /bin/mount is
> expected to be universally available, nowaways it is a symlink to
> /usr/bin/mount.
> 
> Signed-off-by: Olaf Hering 

The code itself looks OK at a glance.

What is unsaid is that we need to backport this to older Xen releases so
that old Xen can work with new systemd.

Ian and Jan?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 4/5] tools: libxendevicemodel: Provide xendevicemodel_add_to_physmap

[Wei Liu]

On Fri, Oct 20, 2017 at 10:22:55AM +0100, Paul Durrant wrote:
> > -Original Message-
> > From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> > Ross Lagerwall
> > Sent: 18 October 2017 15:04
> > To: Xen-devel <xen-devel@lists.xen.org>
> > Cc: Ross Lagerwall <ross.lagerw...@citrix.com>; Ian Jackson
> > <ian.jack...@citrix.com>; Wei Liu <wei.l...@citrix.com>
> > Subject: [Xen-devel] [PATCH v1 4/5] tools: libxendevicemodel: Provide
> > xendevicemodel_add_to_physmap
> > 
> > Signed-off-by: Ross Lagerwall <ross.lagerw...@citrix.com>
> > ---
> >  tools/libs/devicemodel/Makefile |  2 +-
> >  tools/libs/devicemodel/core.c   | 17 +
> >  tools/libs/devicemodel/include/xendevicemodel.h | 13 +
> >  tools/libs/devicemodel/libxendevicemodel.map|  5 +
> >  4 files changed, 36 insertions(+), 1 deletion(-)
> > 
> > diff --git a/tools/libs/devicemodel/Makefile
> > b/tools/libs/devicemodel/Makefile
> > index 342371a..5b2df7a 100644
> > --- a/tools/libs/devicemodel/Makefile
> > +++ b/tools/libs/devicemodel/Makefile
> > @@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
> >  include $(XEN_ROOT)/tools/Rules.mk
> > 
> >  MAJOR= 1
> > -MINOR= 1
> > +MINOR= 2
> >  SHLIB_LDFLAGS += -Wl,--version-script=libxendevicemodel.map
> > 
> >  CFLAGS   += -Werror -Wmissing-prototypes
> > diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
> > index b66d4f9..2a23077 100644
> > --- a/tools/libs/devicemodel/core.c
> > +++ b/tools/libs/devicemodel/core.c
> > @@ -564,6 +564,23 @@ int xendevicemodel_shutdown(
> >  return xendevicemodel_op(dmod, domid, 1, , sizeof(op));
> >  }
> > 
> > +int xendevicemodel_add_to_physmap(
> > +xendevicemodel_handle *dmod, domid_t domid, uint64_t idx, uint64_t
> > gpfn)
> 
> Do you really want this to be single page? Populating VRAM is not going to be 
> fast if you need to make 16MB / 4K hypercalls to do it.

This. Since we're introducing a new interface please consider giving it
the ability to batch.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel