Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

2017-11-08 Thread Olaf Hering
On Wed, Nov 08, Wei Liu wrote: > But is there really no way to ask nicely to see if systemd would accept > a change in behaviour? That is, to make proc-xen.mount (or any attempt > to mount API fs) a nop when xenfs is added to API file system. I have considered that as well. If the failing unit

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

2017-11-08 Thread Olaf Hering
On Thu, Oct 26, Olaf Hering wrote: > > If not, then out-of-tree packages are going to have compatibility > > problems with this change. > Only if they use Requires=proc-xen.mount. Any other objections to this change? How to proceed with this? Olaf signature.asc Description

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

2017-10-26 Thread Olaf Hering
On Thu, Oct 26, Andrew Cooper wrote: > I've never really understood why xenfs exists in the first place > (although I expect the answer is "Because that is how someone did it in > the past"), and I'm not aware of any other project which needs its own > custom filesystem driver for device nodes.

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

2017-10-26 Thread Olaf Hering
On Thu, Oct 26, Andrew Cooper wrote: > Can't all information be obtained from /sys/hypervisor?  If not, how > hard would it be to make happen? Likely not that hard. Not sure why that was not added in the first place. > What happens to all the software which currently has a dependency on >

[Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

2017-10-26 Thread Olaf Hering
, will trigger execution with an old systemd. An absolute path to the mount binary has to be used. /bin/mount is expected to be universally available, nowaways it is a symlink to /usr/bin/mount. Signed-off-by: Olaf Hering <o...@aepfle.de> --- based on 4.10.0-rc2 Please run autogen.sh: tools/config

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-10-11 Thread Olaf Hering
On Wed, Oct 11, Olaf Hering wrote: > -#define MAX_BATCH_SIZE 1024 /* up to 1024 pages (4MB) at a time */ > +#define MAX_BATCH_SIZE SUPERPAGE_1GB_NR_PFNS /* up to 1GB at a time */ Actually the error is something else, I missed this in the debug output: xc: error: Failed to get types f

[Xen-devel] [PATCH v10 0/3] tools/libxc: use superpages

2017-10-11 Thread Olaf Hering
comments v2: split into individual commits based on staging c39cf093fc ("x86/asm: add .file directives") Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/libxc: add API for bitmap access for restore tools/libxc: use superpages during restore of HVM gu

[Xen-devel] [PATCH v10 3/3] tools/libxc: use superpages during restore of HVM guest

2017-10-11 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 30 +-

[Xen-devel] [PATCH v10 2/3] tools/libxc: add API for bitmap access for restore

2017-10-11 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v10 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-10-11 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-10-11 Thread Olaf Hering
On Fri, Sep 08, Olaf Hering wrote: > A related question: is it save to increase MAX_BATCH_SIZE from 1024 to > (256*1024) to transfer a whole gigabyte at a time? That way it will be > easier to handle holes within a 1GB superpage. To answer my own question: This change leads to t

Re: [Xen-devel] [PULL 1/2] xen-disk: use g_new0 to fix build

2017-09-25 Thread Olaf Hering
On Wed, Sep 20, Stefano Stabellini wrote: > From: Olaf Hering <o...@aepfle.de> > g_malloc0_n is available since glib-2.24. To allow build with older glib > versions use the generic g_new0, which is already used in many other > places in the code. > Fixes commit 3284fad728 (&

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-08 Thread Olaf Hering
On Wed, Sep 06, Andrew Cooper wrote: > The stream has always been in-order for the first pass (even in the > legacy days), and I don't forsee that changing. Reliance on the order > was suggested by both myself and Jan during the early design. A related question: is it save to increase

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-06 Thread Olaf Hering
On Wed, Sep 06, Andrew Cooper wrote: > If a PVH guest has got MTRRs disabled, then it genuinely can run on an > unshattered 1G superpage at 0. Ok, the code will detect the holes and will release memory as needed. I will drop these two lines. Olaf signature.asc Description: PGP signature

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-06 Thread Olaf Hering
On Wed, Sep 06, Andrew Cooper wrote: > I still fail to understand why you need the bitmaps at all? You can > calculate everything you need from the pfn list alone, which will also > let you spot the presence or absence of the VGA hole. These bitmaps track if a range has been allocated as

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-06 Thread Olaf Hering
On Wed, Sep 06, Andrew Cooper wrote: > On 01/09/17 17:08, Olaf Hering wrote: > > +/* No superpage in 1st 2MB due to VGA hole */ > > +xc_sr_set_bit(0, >x86_hvm.restore.attempted_1g); > > +xc_sr_set_bit(0, >x86_hvm.restore.attempted_2m); > This is false for

Re: [Xen-devel] [PATCH v9 2/3] tools/libxc: add API for bitmap access for restore

2017-09-06 Thread Olaf Hering
On Wed, Sep 06, Andrew Cooper wrote: > On 01/09/17 17:08, Olaf Hering wrote: > > +static inline bool pfn_is_populated(struct xc_sr_context *ctx, xen_pfn_t > > pfn) > > +static inline int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t > > pfn) > Why ar

Re: [Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-06 Thread Olaf Hering
Am Wed, 6 Sep 2017 12:34:10 +0100 schrieb Wei Liu : > > +struct x86_hvm_sp { > Forgot to ask: what does sp stand for? superpage. I will check if there is room to expand this string. > > + * Try to allocate superpages. > > + * This works without memory map only if the pfns

[Xen-devel] [PATCH] libxc/bitops: correct comment for bitmap_size

2017-09-05 Thread Olaf Hering
The returned value represents now units of bytes instead of longs. Fixes commit 11d0044a16 ("tools/libxc: Modify bitmap operations to take void pointers") Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_bitops.h | 2 +- 1 file changed, 1 insertion(+), 1 deletio

[Xen-devel] [PATCH v9 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-01 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 26 ++-

[Xen-devel] [PATCH v9 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-09-01 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

[Xen-devel] [PATCH v9 0/3] tools/libxc: use superpages

2017-09-01 Thread Olaf Hering
ate_pfn v3: clear pointer in xc_sr_bitmap_free some coding style changes use getdomaininfo.max_pages to avoid Over-allocation check trim bitmap function names, drop trailing "_bit" add some comments v2: split into individual commits based on staging c39cf093fc ("x86/asm: add

[Xen-devel] [PATCH v9 2/3] tools/libxc: add API for bitmap access for restore

2017-09-01 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

Re: [Xen-devel] [PATCH v8 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-01 Thread Olaf Hering
On Fri, Sep 01, Olaf Hering wrote: > +static int x86_hvm_populate_pfns(struct xc_sr_context *ctx, unsigned count, > +/* > + * If this next pfn is within another 1GB superpage it is > required > + * to scan the entire previous superpage beca

[Xen-devel] [PATCH v8 3/3] tools/libxc: use superpages during restore of HVM guest

2017-09-01 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 25 ++-

[Xen-devel] [PATCH v8 2/3] tools/libxc: add API for bitmap access for restore

2017-09-01 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v8 0/3] tools/libxc: use superpages

2017-09-01 Thread Olaf Hering
avoid Over-allocation check trim bitmap function names, drop trailing "_bit" add some comments v2: split into individual commits based on staging c39cf093fc ("x86/asm: add .file directives") Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/li

[Xen-devel] [PATCH v8 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-09-01 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

[Xen-devel] [PATCH v7 2/3] tools/libxc: add API for bitmap access for restore

2017-08-31 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v7 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-08-31 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

[Xen-devel] [PATCH v7 0/3] tools/libxc: use superpages

2017-08-31 Thread Olaf Hering
ling "_bit" add some comments v2: split into individual commits Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/libxc: add API for bitmap access for restore tools/libxc: use superpages during restore of HVM guest tools/libxc/xc_dom_x86.c

[Xen-devel] [PATCH v7 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-31 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 25 ++-

[Xen-devel] ballooning specific PFNs in a HVM domU

2017-08-31 Thread Olaf Hering
Does the Linux kernel provide an API to claim specific pages? Right now it just does alloc_page(), which I think returns any random page that happens to be unused. I want to create a specific memory layout with holes to verify my migration patches. Olaf signature.asc Description: PGP signature

Re: [Xen-devel] [PATCH v6 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-30 Thread Olaf Hering
On Wed, Aug 30, Wei Liu wrote: > > Can this actually happen with the available senders? If not, this is > > again the missing memory map. > Probably not now, but as said, you shouldn't rely on the structure of > the stream unless it is stated in the spec. Well, what can happen with todays

Re: [Xen-devel] [PATCH v6 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-30 Thread Olaf Hering
On Wed, Aug 30, Wei Liu wrote: > As far as I can tell the algorithm in the patch can't handle: > > 1. First pfn in a batch points to start of second 1G address space > 2. Second pfn in a batch points to a page in the middle of first 1G > 3. Guest can only use 1G ram In which way does it not

Re: [Xen-devel] [PATCH v6 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-29 Thread Olaf Hering
On Sat, Aug 26, Olaf Hering wrote: > +static int x86_hvm_populate_pfns(struct xc_sr_context *ctx, unsigned count, > +/* > + * Scan the entire superpage because several batches will fit into > + * a superpage, and it is unknown which pfn triggered th

[Xen-devel] [PATCH v6 0/3] tools/libxc: use superpages

2017-08-26 Thread Olaf Hering
plit into individual commits Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/libxc: add API for bitmap access for restore tools/libxc: use superpages during restore of HVM guest tools/libxc/xc_dom_x86.c| 5 - tools/libxc/xc_private.h

[Xen-devel] [PATCH v6 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-08-26 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

[Xen-devel] [PATCH v6 2/3] tools/libxc: add API for bitmap access for restore

2017-08-26 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v6 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-26 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 25 +++-

Re: [Xen-devel] [PATCH v5 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-25 Thread Olaf Hering
On Fri, Aug 25, Olaf Hering wrote: > +static int x86_hvm_populate_pfns(struct xc_sr_context *ctx, unsigned count, > + const xen_pfn_t *original_pfns, > + const uint32_t *types) > +{ > +while ( min_pfn <

[Xen-devel] [PATCH v5 0/3] tools/libxc: use superpages

2017-08-25 Thread Olaf Hering
locate_pfn v3: clear pointer in xc_sr_bitmap_free some coding style changes use getdomaininfo.max_pages to avoid Over-allocation check trim bitmap function names, drop trailing "_bit" add some comments v2: split into individual commits Olaf Hering (3): tools/libxc: move SUPERP

[Xen-devel] [PATCH v5 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-25 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 26 ++--

[Xen-devel] [PATCH v5 2/3] tools/libxc: add API for bitmap access for restore

2017-08-25 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v5 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-08-25 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

[Xen-devel] [PATCH v4 2/3] tools/libxc: add API for bitmap access for restore

2017-08-25 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_sr_co

[Xen-devel] [PATCH v4 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-25 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 26 ++--

[Xen-devel] [PATCH v4 0/3] tools/libxc: use superpages

2017-08-25 Thread Olaf Hering
ome coding style changes use getdomaininfo.max_pages to avoid Over-allocation check trim bitmap function names, drop trailing "_bit" add some comments v2: split into individual commits Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/libxc: add A

[Xen-devel] [PATCH v4 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-08-25 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> Acked-by: Wei Liu <wei.l...@citrix.com> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 ++

Re: [Xen-devel] [PATCH v3 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-25 Thread Olaf Hering
On Fri, Aug 25, Olaf Hering wrote: > I think with the new check of max_pages an overallocation can not happen > anymore. If at some point the domU still has room for a superpage, it > will be allocated. In case the batch does not fully fill the superpage, > the holes will be freed.

Re: [Xen-devel] [PATCH v3 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-25 Thread Olaf Hering
On Fri, Aug 25, Wei Liu wrote: > Maybe a middle ground is to scan the batch to see if pfns can be fit > into a whole super page? I don't think you can get a batch as big as 1G > but there should be a lot of 2M batches? I think with the new check of max_pages an overallocation can not happen

Re: [Xen-devel] [PATCH v3 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-25 Thread Olaf Hering
On Fri, Aug 25, Wei Liu wrote: > I'm still unconvinced this works all the time because it still needs the > assumption that the stream contains contiguous pfns. This is how it is done today. If the pfns start to arrive in another order the format has to be changed to send a memory layout in

[Xen-devel] [PATCH v3 2/3] tools/libxc: add API for bitmap access for restore

2017-08-24 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.c | 41 +++ tools/libxc/xc_sr_common.h

[Xen-devel] [PATCH v3 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-24 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 25 +++--

[Xen-devel] [PATCH v3 0/3] tools/libxc: use superpages

2017-08-24 Thread Olaf Hering
v2: split into individual commits Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/libxc: add API for bitmap access for restore tools/libxc: use superpages during restore of HVM guest tools/libxc/xc_dom_x86.c| 5 - tools/libxc/xc_private.h|

[Xen-devel] [PATCH v3 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-08-24 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 + 2 files changed, 5 insertions(+), 5 del

Re: [Xen-devel] [PATCH v2 2/3] tools/libxc: add API for bitmap access for restore

2017-08-24 Thread Olaf Hering
On Thu, Aug 17, Olaf Hering wrote: > Extend API for managing bitmaps. Each bitmap is now represented by a > generic struct xc_sr_bitmap. > +static inline bool xc_sr_bitmap_resize(struct xc_sr_bitmap *bm, unsigned > long bits) > +static inline void xc_sr_bitmap_free(struct x

Re: [Xen-devel] [PATCH v2 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-23 Thread Olaf Hering
On Wed, Aug 23, Olaf Hering wrote: > The value of p2m_size does not represent the actual number of pages > assigned to a domU. This info is stored in getdomaininfo.max_pages, > which is currently not used by restore. I will see if using this value > will avoid triggering the Over-allo

Re: [Xen-devel] [PATCH v2 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-23 Thread Olaf Hering
On Wed, Aug 23, Wei Liu wrote: > On Tue, Aug 22, 2017 at 05:53:25PM +0200, Olaf Hering wrote: > > In my testing I have seen the case of over-allocation. Thats why I > > implemented the freeing of unpopulated parts. It would be nice to know > > how many pages are actually comi

Re: [Xen-devel] [PATCH v2 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-23 Thread Olaf Hering
On Tue, Aug 22, Olaf Hering wrote: > In my testing I have seen the case of over-allocation. Thats why I > implemented the freeing of unpopulated parts. It would be nice to know > how many pages are actually coming. I think this info is not available. If the receiving dom0 recognize

Re: [Xen-devel] [PATCH v2 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-22 Thread Olaf Hering
On Tue, Aug 22, Wei Liu wrote: > On Thu, Aug 17, 2017 at 07:01:33PM +0200, Olaf Hering wrote: > > +/* No superpage in 1st 2MB due to VGA hole */ > > +xc_sr_set_bit(0, >x86_hvm.restore.attempted_1g); > > +xc_sr_set_bit(0, >x86_hvm.restore.attempted_2m); >

[Xen-devel] [PATCH v2 0/3] tools/libxc: use superpages

2017-08-17 Thread Olaf Hering
Using superpages on the receiving dom0 will avoid performance regressions. Olaf v2: split into individual commits Olaf Hering (3): tools/libxc: move SUPERPAGE macros to common header tools/libxc: add API for bitmap access for restore tools/libxc: use superpages during restore of HVM

[Xen-devel] [PATCH v2 3/3] tools/libxc: use superpages during restore of HVM guest

2017-08-17 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.h | 15 +++

[Xen-devel] [PATCH v2 1/3] tools/libxc: move SUPERPAGE macros to common header

2017-08-17 Thread Olaf Hering
The macros SUPERPAGE_2MB_SHIFT and SUPERPAGE_1GB_SHIFT will be used by other code in libxc. Move the macros to a header file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_dom_x86.c | 5 - tools/libxc/xc_private.h | 5 + 2 files changed, 5 insertions(+), 5 del

[Xen-devel] [PATCH v2 2/3] tools/libxc: add API for bitmap access for restore

2017-08-17 Thread Olaf Hering
Extend API for managing bitmaps. Each bitmap is now represented by a generic struct xc_sr_bitmap. Switch the existing populated_pfns to this API. Signed-off-by: Olaf Hering <o...@aepfle.de> --- tools/libxc/xc_sr_common.c | 41 +++ tools/libxc/xc_sr_common.h

Re: [Xen-devel] [PATCH v1] tools/libxc: use superpages during restore of HVM guest

2017-08-04 Thread Olaf Hering
On Fri, Aug 04, Wei Liu wrote: > Can you split this patch into several: > 1. code movement > 2. refactoring / introduction of new hooks > 3. implementing the new algorithm I tried that, it did not work well. But, I can try again if required. Olaf signature.asc Description: PGP signature

Re: [Xen-devel] [PATCH v1] tools/libxc: use superpages during restore of HVM guest

2017-08-03 Thread Olaf Hering
On Wed, Aug 02, Olaf Hering wrote: > +++ b/tools/libxc/xc_sr_restore_x86_hvm.c > +#define SUPERPAGE_2MB_SHIFT 9 > +#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT) > +#define SUPERPAGE_1GB_SHIFT 18 > +#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1G

Re: [Xen-devel] backport docs changes for Xen 4.9.1

2017-08-03 Thread Olaf Hering
On Thu, Aug 03, Jan Beulich wrote: > >>> On 01.08.17 at 11:43, wrote: > > Please backport the following changes for docs/ for the Xen 4.9.1 > > release: > > > > aa4eb460bc docs: add pod variant of xl-numa-placement > > 458df9f374 docs: add pod variant of

[Xen-devel] [PATCH v1] tools/libxc: use superpages during restore of HVM guest

2017-08-02 Thread Olaf Hering
that is received. In case a pfn was not populated on the sending side it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering <o...@aepfle.de> --- based on RELEASE-4.9.0 tools/libxc/xc_sr_co

Re: [Xen-devel] [PATCH] vtpmmgr: make inline functions static

2017-08-01 Thread Olaf Hering
Ping On Fri, Jun 23, Olaf Hering wrote: > gcc7 is more strict with functions marked as inline. They are not > automatically inlined. Instead a function call is generated, but the > actual code is not visible by the linker. > > Do a mechanical change and mark every 'inline' as

[Xen-devel] Stable Branch Maintainer in Wiki and MAINTAINERS

2017-08-01 Thread Olaf Hering
According to https://wiki.xenproject.org/wiki/Xen_Project_Maintenance_Releases in the "Stable Branch Maintainer" section someone is supposed to be added to the MAINTAINERS file. Where in the staging-4.9 branch was this change done? I guess an equivalent of 1f4ea16035 ("update Xen version to

[Xen-devel] backport docs changes for Xen 4.9.1

2017-08-01 Thread Olaf Hering
Please backport the following changes for docs/ for the Xen 4.9.1 release: aa4eb460bc docs: add pod variant of xl-numa-placement 458df9f374 docs: add pod variant of xl-network-configuration.5 4359b86f31 docs: add pod variant of xen-pv-channel.7 55924baf22 docs: correct paragraph indention in

[Xen-devel] [PATCH v2] xen-disk: use g_new0 to fix build

2017-07-28 Thread Olaf Hering
g_malloc0_n is available since glib-2.24. To allow build with older glib versions use the generic g_new0, which is already used in many other places in the code. Fixes commit 3284fad728 ("xen-disk: add support for multi-page shared rings") Signed-off-by: Olaf Hering <o...@aepfl

Re: [Xen-devel] [Qemu-devel] [PATCH] xen-disk: use g_malloc0 to fix build

2017-07-28 Thread Olaf Hering
On Fri, Jul 28, Eric Blake wrote: > This version is prone to multiplication overflow (well, maybe not, but > you have to audit for that). Wouldn't it be better to use: What could go wrong? qemu will die either way, I think. Olaf signature.asc Description: PGP signature

[Xen-devel] [PATCH] xen-disk: use g_malloc0 to fix build

2017-07-28 Thread Olaf Hering
g_malloc0_n is available since glib-2.24. To allow build with older glib versions use the generic g_malloc0, which is already used in many other places in the code. Fixes commit 3284fad728 ("xen-disk: add support for multi-page shared rings") Signed-off-by: Olaf Hering <o...@aepfl

Re: [Xen-devel] [PULL 3/3] xen-disk: add support for multi-page shared rings

2017-07-27 Thread Olaf Hering
On Tue, Jun 27, Stefano Stabellini wrote: > From: Paul Durrant > The blkif protocol has had provision for negotiation of multi-page shared > rings for some time now and many guest OS have support in their frontend > drivers. > +++ b/hw/block/xen_disk.c > +domids =

[Xen-devel] [PATCH v3 0/3] docs: convert manpages to pod

2017-07-26 Thread Olaf Hering
forward. Olaf v3: - add NAME/DESCRIPION, minor formating tweaks, whitespace v2: fold each add/remove into a single commit Cc: Ian Jackson <ian.jack...@eu.citrix.com> Cc: Wei Liu <wei.l...@citrix.com> To: xen-devel@lists.xen.org Olaf Hering (3): docs: add pod variant of xen-

[Xen-devel] [PATCH v3 1/3] docs: add pod variant of xen-pv-channel.7

2017-07-26 Thread Olaf Hering
Convert source for xen-pv-channel.7 from markdown to pod. This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xen-pv-channel.markdown.7

[Xen-devel] [PATCH v3 2/3] docs: add pod variant of xl-network-configuration.5

2017-07-26 Thread Olaf Hering
Convert source for xl-network-configuration.5 from markdown to pod. This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- ...n.markdown.5 => xl-network-configurat

[Xen-devel] [PATCH v3 3/3] docs: add pod variant of xl-numa-placement

2017-07-26 Thread Olaf Hering
Convert source for xl-numa-placement.7 from markdown to pod. This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- ...lacement.markdown.7 => xl-numa-placement.pod

[Xen-devel] [PATCH v2 3/3] docs: add pod variant of xl-numa-placement

2017-07-25 Thread Olaf Hering
Convert source for xl-numa-placement.7 from markdown to pod. This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- ...lacement.markdown.7 => xl-numa-placement.pod

[Xen-devel] [PATCH v2 1/3] docs: add pod variant of xen-pv-channel.7

2017-07-25 Thread Olaf Hering
Convert source for xen-pv-channel.7 from markdown to pod. This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xen-pv-channel.markdown.7

[Xen-devel] [PATCH v2 0/6] docs: convert manpages to pod

2017-07-25 Thread Olaf Hering
forward. Olaf v2: fold each add/remove into a single commit Cc: Ian Jackson <ian.jack...@eu.citrix.com> Cc: Wei Liu <wei.l...@citrix.com> To: xen-devel@lists.xen.org Olaf Hering (6): docs: add pod variant of xen-pv-channel.7 docs: add pod variant of xl-network-configuration.5

[Xen-devel] [PATCH v2 2/3] docs: add pod variant of xl-network-configuration.5

2017-07-25 Thread Olaf Hering
Convert source for xl-network-configuration.5 from markdown to pod. This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- ...n.markdown.5 => xl-network-configurat

Re: [Xen-devel] [PATCH 0/6] docs: convert manpages to pod

2017-07-24 Thread Olaf Hering
On Mon, Jul 24, Ian Jackson wrote: > * There are a lot of other documents in docs/misc/ which are in > markdown format. Some of them are internal. I'm pretty sure we don't > want them _all_ converted. So even if you convert the manpages, these > documents will remain. I did not intent to

Re: [Xen-devel] [PATCH 0/6] docs: convert manpages to pod

2017-07-24 Thread Olaf Hering
On Mon, Jul 24, Ian Jackson wrote: > Olaf Hering writes ("[PATCH 0/6] docs: convert manpages to pod"): > > To remove the buildtime dependency to pandoc/ghc some manpages are > > converted from markdown to pod format. This will provide more manpages > > which are r

[Xen-devel] [PATCH 6/6] docs: remove markdown variant of xl-numa-placement.7

2017-07-24 Thread Olaf Hering
A variant in pod format exists now. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xl-numa-placement.markdown.7 | 239 -- 1 file changed, 239 deletions(-) delete mode 100644 docs/man/xl-numa-placement.markdown.7 diff --git a/docs/man/x

[Xen-devel] [PATCH 3/6] docs: add pod variant of xl-numa-placement

2017-07-24 Thread Olaf Hering
Add source in pod format for xl-numa-placement.7 This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xl-numa-placement.pod.7

[Xen-devel] [PATCH 4/6] docs: remove markdown variant of xen-pv-channel.7

2017-07-24 Thread Olaf Hering
A variant in pod format exists now. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xen-pv-channel.markdown.7 | 106 - 1 file changed, 106 deletions(-) delete mode 100644 docs/man/xen-pv-channel.markdown.7 diff --git a/docs/man/

[Xen-devel] [PATCH 5/6] docs: remove markdown variant of xl-network-configuration.5

2017-07-24 Thread Olaf Hering
A variant in pod format exists now. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xl-network-configuration.markdown.5 | 173 --- 1 file changed, 173 deletions(-) delete mode 100644 docs/man/xl-network-configuration.markdown.5 diff --git a/docs/man/xl-n

[Xen-devel] [PATCH 1/6] docs: add pod variant of xen-pv-channel.7

2017-07-24 Thread Olaf Hering
Add source in pod format for xen-pv-channel.7 This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xen-pv-channel.pod.7

[Xen-devel] [PATCH 2/6] docs: add pod variant of xl-network-configuration.5

2017-07-24 Thread Olaf Hering
Add source in pod format for xl-network-configuration.5 This removes the buildtime requirement for pandoc, and subsequently the need for ghc, in the chain for BuildRequires of xen.rpm. Signed-off-by: Olaf Hering <o...@aepfle.de> --- docs/man/xl-network-configuration.pod.5

[Xen-devel] [PATCH 0/6] docs: convert manpages to pod

2017-07-24 Thread Olaf Hering
forward. Olaf Cc: Ian Jackson <ian.jack...@eu.citrix.com> Cc: Wei Liu <wei.l...@citrix.com> To: xen-devel@lists.xen.org Olaf Hering (6): docs: add pod variant of xen-pv-channel.7 docs: add pod variant of xl-network-configuration.5 docs: add pod variant of xl-numa-placement

Re: [Xen-devel] [PATCH for-4.9] docs: replace xm with xl in xen-tscmode [and 1 more messages]

2017-07-24 Thread Olaf Hering
On Thu, May 25, Julien Grall wrote: > Hi Ian, > > On 24/05/2017 12:07, Ian Jackson wrote: > > Olaf Hering writes ("[PATCH] docs: replace xm with xl in xen-tscmode"): > > > Signed-off-by: Olaf Hering <o...@aepfle.de> > > Olaf Hering writes ("[P

Re: [Xen-devel] API to query NUMA node of mfn

2017-07-10 Thread Olaf Hering
On Mon, Jul 10, Konrad Rzeszutek Wilk wrote: > Soo I wrote some code for exactly this for Xen 4.4.4 , along with > creation of a PGM map to see the NUMA nodes locality. Are you planning to prepare that for staging at some point? I have not checked this series is already merged. Olaf

[Xen-devel] API to query NUMA node of mfn

2017-07-10 Thread Olaf Hering
I would like to verify on which NUMA node the PFNs used by a HVM guest are located. Is there an API for that? Something like: foreach (pfn, domid) mfns_per_node[pfn_to_node(pfn)]++ foreach (node) printk("%x %x\n", node, mfns_per_node[node]) Olaf signature.asc Description: PGP

Re: [Xen-devel] time does not move forward in HVM guests

2017-07-05 Thread Olaf Hering
Am Wed, 05 Jul 2017 02:14:23 -0600 schrieb "Jan Beulich" : > Oh, even for HVM. Doesn't that go back to the missing vDSO > support then again, which we had discussed just last week? Yes. This is part of it. With clocksource=tsc there is a performance boost because vdso is

Re: [Xen-devel] time does not move forward in HVM guests

2017-07-05 Thread Olaf Hering
On Wed, Jul 05, Jan Beulich wrote: > > clock_getres(CLOCK_MONOTONIC) indicates a resolution of 1ns. > But what's the implied meaning of resolution here? See below. I have no ide what the returned value is supposed to promise. > Or did you perhaps test with an older version, where the time >

Re: [Xen-devel] valgrind support for xen4.7+

2017-07-05 Thread Olaf Hering
On Wed, Apr 12, Glenn Enright wrote: > Has anyone seen or been working on patches for valgrind for recent versions > of xen? Upstream requires paperwork, via kde.org bugzilla. This is my variant, which is enough to run 'xl create' with valgrind. Olaf --- coregrind/m_syswrap/syswrap-xen.c.orig

[Xen-devel] time does not move forward in HVM guests

2017-07-04 Thread Olaf Hering
In my testing with sysbench in a HVM domU running a linux-4.4 based pvops kernel on a xen-4.7 based dom0 the time does not move forward properly: There (URL below) is basically code like this: clock_gettime(CLOCK_MONOTONIC, a) do_work clock_gettime(CLOCK_MONOTONIC, b) diff_time(a,b) All

  1   2   3   4   5   6   7   8   9   >