Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-25 Thread Jan Beulich 
>>> On 25.08.17 at 14:03,  wrote:
> On 25/08/17 13:58, Jan Beulich wrote:
> On 25.08.17 at 13:40,  wrote:
>>> In the Linux kernel I would then:
>>>
>>> - Re-add grant v2 support
>>> - Add boot parameter for selecting grant v1 or v2
>>> - Use grant v2 if selected via boot parameter, or (pv only:) max. host
>>>   address is above 16TB
>> 
>> Can a PV DomU know?
> 
> XENMEM_maximum_ram_page

Oh, I always forget that we bogusly expose a machine property here.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-25 Thread Juergen Gross 
On 25/08/17 13:58, Jan Beulich wrote:
 On 25.08.17 at 13:40,  wrote:
>> So what about the following idea:
>>
>> - Set the default max number of grant frames to 32 on "small" hosts
>>   (max. physical memory address below 16TB) and to 64 on "large"
>>   hosts
> 
> This looks reasonable to me as long as it's properly documented.
> 
>> - Add a per-domain parameter for the max number of grant frames
>>
>> In the Linux kernel I would then:
>>
>> - Re-add grant v2 support
>> - Add boot parameter for selecting grant v1 or v2
>> - Use grant v2 if selected via boot parameter, or (pv only:) max. host
>>   address is above 16TB
> 
> Can a PV DomU know?

XENMEM_maximum_ram_page

> Plus can you easily switch between
> versions after having been migrated?

Other than console and xenbus there should be no active grant after
migration. So yes, I'm planning to support both, v1->v2 and v2->v1.

> And wouldn't HVM or ARM
> guests need to judge by max guest address?

Yes.


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-25 Thread Jan Beulich 
>>> On 25.08.17 at 13:40,  wrote:
> So what about the following idea:
> 
> - Set the default max number of grant frames to 32 on "small" hosts
>   (max. physical memory address below 16TB) and to 64 on "large"
>   hosts

This looks reasonable to me as long as it's properly documented.

> - Add a per-domain parameter for the max number of grant frames
> 
> In the Linux kernel I would then:
> 
> - Re-add grant v2 support
> - Add boot parameter for selecting grant v1 or v2
> - Use grant v2 if selected via boot parameter, or (pv only:) max. host
>   address is above 16TB

Can a PV DomU know? Plus can you easily switch between
versions after having been migrated? And wouldn't HVM or ARM
guests need to judge by max guest address?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-25 Thread Juergen Gross 
On 24/08/17 17:20, Jan Beulich wrote:
 On 24.08.17 at 17:13,  wrote:
>> On 24/08/17 17:04, Jan Beulich wrote:
>> On 24.08.17 at 16:48,  wrote:
 How would the guest know whether using v2 grants is no disadvantage?
>>>
>>> As said - it's always going to be a disadvantage. Even if controlling
>>> the number of frames per-domain, that same number of frames
>>> will always fit more v1 than v2 entries.
>>
>> And my patches break the assumption "same number of frames".
>>
>>> I don't think the config
>>> setting should directly control the number of active grants.
>>
>> Why not? In the end that number is the one the guest is interested in.
> 
> And the resource use is what the admin is interested in.
> 
>> BTW: the number of needed maptrack frames is depending on the number
>> of grants only, not on v1 or v2. And the default for the max. number
>> of maptrack frames is much higher than the one of the grant frames
>> (1024 vs 32).
> 
> Of course.

So what about the following idea:

- Set the default max number of grant frames to 32 on "small" hosts
  (max. physical memory address below 16TB) and to 64 on "large"
  hosts
- Add a per-domain parameter for the max number of grant frames

In the Linux kernel I would then:

- Re-add grant v2 support
- Add boot parameter for selecting grant v1 or v2
- Use grant v2 if selected via boot parameter, or (pv only:) max. host
  address is above 16TB


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-24 Thread Jan Beulich 
>>> On 24.08.17 at 17:13,  wrote:
> On 24/08/17 17:04, Jan Beulich wrote:
> On 24.08.17 at 16:48,  wrote:
>>> How would the guest know whether using v2 grants is no disadvantage?
>> 
>> As said - it's always going to be a disadvantage. Even if controlling
>> the number of frames per-domain, that same number of frames
>> will always fit more v1 than v2 entries.
> 
> And my patches break the assumption "same number of frames".
> 
>> I don't think the config
>> setting should directly control the number of active grants.
> 
> Why not? In the end that number is the one the guest is interested in.

And the resource use is what the admin is interested in.

> BTW: the number of needed maptrack frames is depending on the number
> of grants only, not on v1 or v2. And the default for the max. number
> of maptrack frames is much higher than the one of the grant frames
> (1024 vs 32).

Of course.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-24 Thread Juergen Gross 
On 24/08/17 17:04, Jan Beulich wrote:
 On 24.08.17 at 16:48,  wrote:
>> On 24/08/17 16:28, Jan Beulich wrote:
>> On 21.08.17 at 20:05,  wrote:
 Today a guest can get the maximum grant table frame number for the
 currently selected grant table interface version (1 or 2) only. Add
 a new grant table operation to get the limits of both variants in
 order to give the guest an opportunity to select the version serving
 its needs best.

 Background for the need for this new hypercall is that e.g. the Linux
 kernel won't use v2 as long as this will allow less active grants as
 v1. As soon as the kernel can detect it can create as many v2 entries
 as for v1, it will have no reason not to use v2. And this will allow
 Xen placing a pv-domain with such a kernel above the current 16TB
 RAM limit.

 For setting up the grant table a kernel needs the following
 information:
 - current version (kexec case)
 - current size (kexec case)
 - max size v1
 - max size v2
 In order not to have to issue 3 hypercalls (GNTTABOP_query_size,
 GNTTABOP_get_version, GNTTABOP_get_v1_and_v2_max), let the new
 hypercall return all the needed information.
>>>
>>> I'm not sure I follow: v2 is always going to allow less active grants
>>> than v1, as the limit is always derived from the number of frames
>>> allowed for a domain.
>>
>> Right. Patch 3 adds support for different number of allowed frames for
>> v1 and v2. So the system can be configured to allow the same max.
>> number of grants for v1 and v2, or even more v2 grants than v1.
>>
>>> I also don't see a problem with issuing multiple
>>> calls - none of this ought to be performance critical. I would,
>>> however, agree that rather than adding a new hypercall to just
>>> return the max sizes adding one like you suggest would be
>>> preferable. I'm simply not convinced yet that returning the max
>>> sizes is actually necessary.
>>
>> How would the guest know whether using v2 grants is no disadvantage?
> 
> As said - it's always going to be a disadvantage. Even if controlling
> the number of frames per-domain, that same number of frames
> will always fit more v1 than v2 entries.

And my patches break the assumption "same number of frames".

> I don't think the config
> setting should directly control the number of active grants.

Why not? In the end that number is the one the guest is interested in.
BTW: the number of needed maptrack frames is depending on the number
of grants only, not on v1 or v2. And the default for the max. number
of maptrack frames is much higher than the one of the grant frames
(1024 vs 32).

> Or if we did it that way, then the answer to your question would be
> "based on hypervisor version".

Yeah, or based on the answer from the hypervisor regarding my new
info call (if the answer is "-ENOSYS" the guest probably would choose
v1 as today).


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-24 Thread Jan Beulich 
>>> On 24.08.17 at 16:48,  wrote:
> On 24/08/17 16:28, Jan Beulich wrote:
> On 21.08.17 at 20:05,  wrote:
>>> Today a guest can get the maximum grant table frame number for the
>>> currently selected grant table interface version (1 or 2) only. Add
>>> a new grant table operation to get the limits of both variants in
>>> order to give the guest an opportunity to select the version serving
>>> its needs best.
>>>
>>> Background for the need for this new hypercall is that e.g. the Linux
>>> kernel won't use v2 as long as this will allow less active grants as
>>> v1. As soon as the kernel can detect it can create as many v2 entries
>>> as for v1, it will have no reason not to use v2. And this will allow
>>> Xen placing a pv-domain with such a kernel above the current 16TB
>>> RAM limit.
>>>
>>> For setting up the grant table a kernel needs the following
>>> information:
>>> - current version (kexec case)
>>> - current size (kexec case)
>>> - max size v1
>>> - max size v2
>>> In order not to have to issue 3 hypercalls (GNTTABOP_query_size,
>>> GNTTABOP_get_version, GNTTABOP_get_v1_and_v2_max), let the new
>>> hypercall return all the needed information.
>> 
>> I'm not sure I follow: v2 is always going to allow less active grants
>> than v1, as the limit is always derived from the number of frames
>> allowed for a domain.
> 
> Right. Patch 3 adds support for different number of allowed frames for
> v1 and v2. So the system can be configured to allow the same max.
> number of grants for v1 and v2, or even more v2 grants than v1.
> 
>> I also don't see a problem with issuing multiple
>> calls - none of this ought to be performance critical. I would,
>> however, agree that rather than adding a new hypercall to just
>> return the max sizes adding one like you suggest would be
>> preferable. I'm simply not convinced yet that returning the max
>> sizes is actually necessary.
> 
> How would the guest know whether using v2 grants is no disadvantage?

As said - it's always going to be a disadvantage. Even if controlling
the number of frames per-domain, that same number of frames
will always fit more v1 than v2 entries. I don't think the config
setting should directly control the number of active grants. Or if
we did it that way, then the answer to your question would be
"based on hypervisor version".

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-24 Thread Juergen Gross 
On 24/08/17 16:48, Juergen Gross wrote:
> On 24/08/17 16:28, Jan Beulich wrote:
> On 21.08.17 at 20:05,  wrote:
>>> Today a guest can get the maximum grant table frame number for the
>>> currently selected grant table interface version (1 or 2) only. Add
>>> a new grant table operation to get the limits of both variants in
>>> order to give the guest an opportunity to select the version serving
>>> its needs best.
>>>
>>> Background for the need for this new hypercall is that e.g. the Linux
>>> kernel won't use v2 as long as this will allow less active grants as
>>> v1. As soon as the kernel can detect it can create as many v2 entries
>>> as for v1, it will have no reason not to use v2. And this will allow
>>> Xen placing a pv-domain with such a kernel above the current 16TB
>>> RAM limit.
>>>
>>> For setting up the grant table a kernel needs the following
>>> information:
>>> - current version (kexec case)
>>> - current size (kexec case)
>>> - max size v1
>>> - max size v2
>>> In order not to have to issue 3 hypercalls (GNTTABOP_query_size,
>>> GNTTABOP_get_version, GNTTABOP_get_v1_and_v2_max), let the new
>>> hypercall return all the needed information.
>>
>> I'm not sure I follow: v2 is always going to allow less active grants
>> than v1, as the limit is always derived from the number of frames
>> allowed for a domain.
> 
> Right. Patch 3 adds support for different number of allowed frames for

This should read "Patch 4", of course

> v1 and v2. So the system can be configured to allow the same max.
> number of grants for v1 and v2, or even more v2 grants than v1.
> 
>> I also don't see a problem with issuing multiple
>> calls - none of this ought to be performance critical. I would,
>> however, agree that rather than adding a new hypercall to just
>> return the max sizes adding one like you suggest would be
>> preferable. I'm simply not convinced yet that returning the max
>> sizes is actually necessary.
> 
> How would the guest know whether using v2 grants is no disadvantage?
> 
> 
> Juergen
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
> 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-24 Thread Juergen Gross 
On 24/08/17 16:28, Jan Beulich wrote:
 On 21.08.17 at 20:05,  wrote:
>> Today a guest can get the maximum grant table frame number for the
>> currently selected grant table interface version (1 or 2) only. Add
>> a new grant table operation to get the limits of both variants in
>> order to give the guest an opportunity to select the version serving
>> its needs best.
>>
>> Background for the need for this new hypercall is that e.g. the Linux
>> kernel won't use v2 as long as this will allow less active grants as
>> v1. As soon as the kernel can detect it can create as many v2 entries
>> as for v1, it will have no reason not to use v2. And this will allow
>> Xen placing a pv-domain with such a kernel above the current 16TB
>> RAM limit.
>>
>> For setting up the grant table a kernel needs the following
>> information:
>> - current version (kexec case)
>> - current size (kexec case)
>> - max size v1
>> - max size v2
>> In order not to have to issue 3 hypercalls (GNTTABOP_query_size,
>> GNTTABOP_get_version, GNTTABOP_get_v1_and_v2_max), let the new
>> hypercall return all the needed information.
> 
> I'm not sure I follow: v2 is always going to allow less active grants
> than v1, as the limit is always derived from the number of frames
> allowed for a domain.

Right. Patch 3 adds support for different number of allowed frames for
v1 and v2. So the system can be configured to allow the same max.
number of grants for v1 and v2, or even more v2 grants than v1.

> I also don't see a problem with issuing multiple
> calls - none of this ought to be performance critical. I would,
> however, agree that rather than adding a new hypercall to just
> return the max sizes adding one like you suggest would be
> preferable. I'm simply not convinced yet that returning the max
> sizes is actually necessary.

How would the guest know whether using v2 grants is no disadvantage?


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-24 Thread Jan Beulich 
>>> On 21.08.17 at 20:05,  wrote:
> Today a guest can get the maximum grant table frame number for the
> currently selected grant table interface version (1 or 2) only. Add
> a new grant table operation to get the limits of both variants in
> order to give the guest an opportunity to select the version serving
> its needs best.
> 
> Background for the need for this new hypercall is that e.g. the Linux
> kernel won't use v2 as long as this will allow less active grants as
> v1. As soon as the kernel can detect it can create as many v2 entries
> as for v1, it will have no reason not to use v2. And this will allow
> Xen placing a pv-domain with such a kernel above the current 16TB
> RAM limit.
> 
> For setting up the grant table a kernel needs the following
> information:
> - current version (kexec case)
> - current size (kexec case)
> - max size v1
> - max size v2
> In order not to have to issue 3 hypercalls (GNTTABOP_query_size,
> GNTTABOP_get_version, GNTTABOP_get_v1_and_v2_max), let the new
> hypercall return all the needed information.

I'm not sure I follow: v2 is always going to allow less active grants
than v1, as the limit is always derived from the number of frames
allowed for a domain. I also don't see a problem with issuing multiple
calls - none of this ought to be performance critical. I would,
however, agree that rather than adding a new hypercall to just
return the max sizes adding one like you suggest would be
preferable. I'm simply not convinced yet that returning the max
sizes is actually necessary.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 5/5] xen: add new hypercall to get grant table limits

2017-08-21 Thread Juergen Gross 
Today a guest can get the maximum grant table frame number for the
currently selected grant table interface version (1 or 2) only. Add
a new grant table operation to get the limits of both variants in
order to give the guest an opportunity to select the version serving
its needs best.

Background for the need for this new hypercall is that e.g. the Linux
kernel won't use v2 as long as this will allow less active grants as
v1. As soon as the kernel can detect it can create as many v2 entries
as for v1, it will have no reason not to use v2. And this will allow
Xen placing a pv-domain with such a kernel above the current 16TB
RAM limit.

For setting up the grant table a kernel needs the following
information:
- current version (kexec case)
- current size (kexec case)
- max size v1
- max size v2
In order not to have to issue 3 hypercalls (GNTTABOP_query_size,
GNTTABOP_get_version, GNTTABOP_get_v1_and_v2_max), let the new
hypercall return all the needed information.

Signed-off-by: Juergen Gross 
---
 xen/common/grant_table.c | 26 ++
 xen/include/public/grant_table.h | 14 ++
 2 files changed, 40 insertions(+)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 33cf1f9ea2..83370a0390 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3041,6 +3041,28 @@ 
gnttab_get_version(XEN_GUEST_HANDLE_PARAM(gnttab_get_version_t) uop)
 return 0;
 }
 
+static long
+gnttab_get_setup_info(XEN_GUEST_HANDLE_PARAM(gnttab_setup_info_t) uop)
+{
+gnttab_setup_info_t op;
+struct domain *d = rcu_lock_current_domain();
+struct grant_table *gt = d->grant_table;
+
+grant_write_lock(gt);
+op.version = gt->gt_version;
+op.nr_frames = nr_grant_frames(gt);
+grant_write_unlock(gt);
+op.max_nr_frames_v1 = max_grant_frames_v1;
+op.max_nr_frames_v2 = max_grant_frames_v2;
+
+rcu_unlock_domain(d);
+
+if ( __copy_to_guest(uop, , 1) )
+return -EFAULT;
+
+return 0;
+}
+
 static s16
 swap_grant_ref(grant_ref_t ref_a, grant_ref_t ref_b)
 {
@@ -3389,6 +3411,10 @@ do_grant_table_op(
 break;
 }
 
+case GNTTABOP_get_setup_info:
+rc = gnttab_get_setup_info(guest_handle_cast(uop, 
gnttab_setup_info_t));
+break;
+
 default:
 rc = -ENOSYS;
 break;
diff --git a/xen/include/public/grant_table.h b/xen/include/public/grant_table.h
index 018036e825..b30f85b2f8 100644
--- a/xen/include/public/grant_table.h
+++ b/xen/include/public/grant_table.h
@@ -312,6 +312,7 @@ typedef uint16_t grant_status_t;
 #define GNTTABOP_get_version  10
 #define GNTTABOP_swap_grant_ref  11
 #define GNTTABOP_cache_flush 12
+#define GNTTABOP_get_setup_info   13
 #endif /* __XEN_INTERFACE_VERSION__ */
 /* ` } */
 
@@ -597,6 +598,19 @@ struct gnttab_cache_flush {
 typedef struct gnttab_cache_flush gnttab_cache_flush_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_cache_flush_t);
 
+/*
+ * GNTTABOP_get_setup_info: Get information needed for grant table setup
+ * of the calling domain.
+ */
+struct gnttab_setup_info {
+uint32_t version;
+uint32_t nr_frames;
+uint32_t max_nr_frames_v1;
+uint32_t max_nr_frames_v2;
+};
+typedef struct gnttab_setup_info gnttab_setup_info_t;
+DEFINE_XEN_GUEST_HANDLE(gnttab_setup_info_t);
+
 #endif /* __XEN_INTERFACE_VERSION__ */
 
 /*
-- 
2.12.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel