Re: [Xen-devel] [RFC PATCH 3/7] xen: psr: reserve an RMID for each core

2015-04-07 Thread Chao Peng
On Sat, Apr 04, 2015 at 04:14:41AM +0200, Dario Faggioli wrote:
 This allows for a new item to be passed as part of the psr=
 boot option: percpu_cmt. If that is specified, Xen tries,
 at boot time, to associate an RMID to each core.
 
 XXX This all looks rather straightforward, if it weren't
 for the fact that it is, apparently, more common than
 I though to run out of RMID. For example, on a dev box
 we have in Cambridge, there are 144 pCPUs and only 71
 RMIDs.
 
 In this preliminary version, nothing particularly smart
 happens if we run out of RMIDs, we just fail attaching
 the remaining cores and that's it. In future, I'd
 probably like to:
  + check whether the operation have any chance to
succeed up front (by comparing number of pCPUs with
available RMIDs)
  + on unexpected failure, rollback everything... it
seems to make more sense to me than just leaving
the system half configured for per-cpu CMT
 
 Thoughts?
 
 XXX Another idea I just have is to allow the user to
 somehow specify a different 'granularity'. Something
 like allowing 'percpu_cmt'|'percore_cmt'|'persocket_cmt'
 with the following meaning:
  + 'percpu_cmt': as in this patch
  + 'percore_cmt': same RMID to hthreads of the same core
  + 'persocket_cmt': same RMID to all cores of the same
 socket.
 
 'percore_cmt' would only allow gathering info on a
 per-core basis... still better than nothing if we
 do not have enough RMIDs for each pCPUs.
 
 'persocket_cmt' would basically only allow to track the
 amount of free L3 on each socket (by subtracting the
 monitored value from the total). Again, still better
 than nothing, would use very few RMIDs, and I could
 think of ways of using this information in a few
 places in the scheduler...
 
 Again, thought?

This even can be extended to the concept of 'cache monitoring group',
which can hold arbitrary cpus into one group. Actually Linux
implementation does this by using the cgoup mechanism to allocate RMID
to a group of threads. Such design can solve the RMID-shortage somehow.

Chao

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] xen/arm64: Use virtual address when setting up early_printk fixmap

2015-04-07 Thread Chen Baozi
From: Chen Baozi baoz...@gmail.com

We have already had the boot pagetable when reaching the point
of early_printk fixmap setup. Thus there is no longer necessary
to caculate physical address of xen_fixmap.

Signed-off-by: Chen Baozi baoz...@gmail.com
---
 xen/arch/arm/arm64/head.S | 1 -
 1 file changed, 1 deletion(-)

diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index 75b7ee1..a2ac2d4 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -520,7 +520,6 @@ paging:
 
 /* Add UART to the fixmap table */
 ldr   x1, =xen_fixmap
-add   x1, x1, x20/* x1 := paddr (xen_fixmap) */
 lsr   x2, x23, #THIRD_SHIFT
 lsl   x2, x2, #THIRD_SHIFT   /* 4K aligned paddr of UART */
 mov   x3, #PT_DEV_L3
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] xen/arm64: Avoid sending SGI when kicking secondary cpus with spin_table

2015-04-07 Thread Chen Baozi
From: Chen Baozi baoz...@gmail.com

On arm64, either firmware or xen's smp_up_cpu gate uses WFE on secondary
cpus to stand-by when booting. Thus, using SEV is enough for the boot
cpu to kick other secondaries. Further more, the current implementation
of cpu_up_send_sgi would pass a NULL cpumask pointer to send_SGI, which
then lead a data fault on GICv3 send_SGI implementation.

Signed-off-by: Chen Baozi baoz...@gmail.com
---
 xen/arch/arm/arm64/smpboot.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/arm64/smpboot.c b/xen/arch/arm/arm64/smpboot.c
index 341cc77..62e6abb 100644
--- a/xen/arch/arm/arm64/smpboot.c
+++ b/xen/arch/arm/arm64/smpboot.c
@@ -38,7 +38,7 @@ static int __init smp_spin_table_cpu_up(int cpu)
 
 sev();
 
-return cpu_up_send_sgi(cpu);
+return 0;
 }
 
 static void __init smp_spin_table_init(int cpu, struct dt_device_node *dn)
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 00/10] PML (Paging Modification Logging) support

2015-04-07 Thread Kai Huang

Hi Jan, Tim, other maintainers,

Do you have comments? Or should I send out the v2 addressing Andrew's 
comments, as it's been more than a week since this patch series were 
sent out?


Thanks,
-Kai

On 03/30/2015 01:50 PM, Kai Huang wrote:



On 03/28/2015 05:26 AM, Andrew Cooper wrote:

On 27/03/15 02:35, Kai Huang wrote:

Hi all,

This patch series adds PML support to Xen. Please kindly help to 
review it.

Overall this looks like a very good series, and it is particularly
helpful given the level of commenting.

Which platforms is/will PML be available for?

Hi Andrew,

Thanks for your quick review. PML will be available from Intel's 
Broadwell server platform.


Thanks,
-Kai


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-4.2-testing test] 50335: regressions - FAIL

2015-04-07 Thread osstest service user
flight 50335 xen-4.2-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50335/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-qemuu-freebsd10-i386  8 guest-start   fail REGR. vs. 36512
 test-i386-i386-pair17 guest-migrate/src_host/dst_host fail REGR. vs. 36512
 test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 36512

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 13 guest-localmigrate/x10 fail pass in 
50318
 test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-localmigrate/x10 fail in 50318 
pass in 50335

Tests which did not succeed, but are not blocking:
 test-i386-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 debian-hvm-install fail never pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 debian-hvm-install  fail never pass
 test-amd64-i386-libvirt  10 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 10 migrate-support-checkfail   never pass
 test-i386-i386-libvirt   10 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-i386-i386-xl-qemut-winxpsp3 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-i386-i386-xl-qemuu-winxpsp3 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-i386-i386-xl-winxpsp3   14 guest-stop   fail   never pass
 test-amd64-i386-xend-winxpsp3 17 leak-check/check fail  never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 build-amd64-rumpuserxen   5 rumpuserxen-buildfail   never pass
 test-amd64-i386-xend-qemut-winxpsp3 17 leak-check/checkfail never pass
 build-i386-rumpuserxen5 rumpuserxen-buildfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop fail in 50318 never pass

version targeted for testing:
 xen  e3bfa4003ceaa2746cdd77655953ab2601acaf9c
baseline version:
 xen  5bec01c19839e150e489dd04376c65f961830c86


People who touched revisions under test:
  Ian Campbell ian.campb...@citrix.com
  Ian Jackson ian.jack...@eu.citrix.com
  Jan Beulich jbeul...@suse.com
  Konrad Rzeszutek Wilk konrad.w...@oracle.com


jobs:
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  fail
 build-i386-rumpuserxen   fail
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-i386-i386-xlpass
 test-amd64-i386-rhel6hvm-amd pass
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64pass
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-qemuu-freebsd10-amd64pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail
 test-amd64-amd64-rumpuserxen-amd64   blocked 
 test-amd64-amd64-xl-qemut-win7-amd64

[Xen-devel] [xen-4.5-testing test] 50334: regressions - FAIL

2015-04-07 Thread osstest service user
flight 50334 xen-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50334/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-winxpsp3  13 guest-localmigrate/x10fail REGR. vs. 50268
 test-amd64-i386-freebsd10-amd64 14 guest-localmigrate/x10 fail in 50317 REGR. 
vs. 50268

Tests which are failing intermittently (not blocking):
 test-amd64-i386-freebsd10-amd64 11 guest-localmigrate   fail pass in 50317
 test-amd64-i386-xl-win7-amd64 13 guest-localmigrate/x10 fail pass in 50317
 test-amd64-i386-freebsd10-i386 11 guest-localmigrate fail in 50317 pass in 
50334
 test-amd64-amd64-xl-winxpsp3 10 guest-localmigrate fail in 50317 pass in 50334

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-freebsd10-i386 14 guest-localmigrate/x10   fail like 50268
 test-armhf-armhf-libvirt  9 guest-start   fail REGR. vs. 50268

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel  9 guest-start  fail  never pass
 test-amd64-i386-libvirt  10 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd   9 guest-start  fail   never pass
 test-armhf-armhf-xl-credit2  10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 10 migrate-support-checkfail never pass
 test-armhf-armhf-xl  10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-sedf 10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 10 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3 14 guest-stopfail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-armhf-armhf-xl-sedf-pin 10 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemut-winxpsp3 14 guest-stopfail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail in 50317 never pass

version targeted for testing:
 xen  0b754fb3ed6b7b6d0f2e1f7c1877d3c0a7da2168
baseline version:
 xen  7fe1c1b28581686aca42361d4fee740c643dde1b


People who touched revisions under test:
  Andrew Cooper andrew.coop...@citrix.com
  denys drozdov denys.droz...@globallogic.com
  Ian Campbell ian.campb...@citrix.com
  Ian Jackson ian.jack...@eu.citrix.com
  Jan Beulich jbeul...@suse.com
  Julien Grall julien.gr...@linaro.org
  Konrad Rzeszutek Wilk konrad.w...@oracle.com
  Stefano Stabellini stefano.stabell...@eu.citrix.com


jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-xl-pvh-amd  fail
 test-amd64-i386-rhel6hvm-amd pass
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 

Re: [Xen-devel] [RFC PATCH 0/7] Intel Cache Monitoring: Current Status and Future Opportunities

2015-04-07 Thread Chao Peng
On Sat, Apr 04, 2015 at 04:14:15AM +0200, Dario Faggioli wrote:
 Hi Everyone,
 
 This RFC series is the outcome of an investigation I've been doing about
 whether we can take better advantage of features like Intel CMT (and of PSR
 features in general). By take better advantage of them I mean, for example,
 use the data obtained from monitoring within the scheduler and/or within
 libxl's automatic NUMA placement algorithm, or similar.
 
 I'm putting here in the cover letter a markdown document I wrote to better
 describe my findings and ideas (sorry if it's a bit long! :-D). You can also
 fetch it at the following links:
 
  * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.pdf
  * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.markdown
 
 See the document itself and the changelog of the various patches for details.

Very good summary and possible usage analysis. Most of the problems do
exist and some of them may be solved partially but some looks
unavoidable.

 
 The series includes one Chao's patch on top, as I found it convenient to build
 on top of it. The series itself is available here:
 
   git://xenbits.xen.org/people/dariof/xen.git  wip/sched/icachemon
   
 http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/wip/sched/icachemon
 
 Thanks a lot to everyone that will read and reply! :-)
 
 Regards,
 Dario
 ---
 
 This is exactly what happens in the current implementation. Result looks as
 follows:
 
 [root@redbrick ~]# xl psr-cmt-attach 0
 [root@redbrick ~]# xl psr-cmt-attach 1
 Total RMID: 71
 NameIDSocket 0
 Socket 1Socket 2Socket 3
 Total L3 Cache Size   46080 KB
 46080 KB46080 KB46080 KB
 Domain-0 0 6768 KB
 0 KB0 KB0 KB
 wheezy64 10 KB  
 144 KB  144 KB0 KB
 
 Let's assume that RMID 1 (RMID 0 is reserved) is used for Domain-0 and RMID 2
 is used for wheezy64. Then:
 
 [root@redbrick ~]# xl psr-cmt-detach 0
 [root@redbrick ~]# xl psr-cmt-detach 1
 
 So now both RMID 1 and 2 are free to be reused. Now, let's issue the following
 commands:
 
 [root@redbrick ~]# xl psr-cmt-attach 1
 [root@redbrick ~]# xl psr-cmt-attach 0
 
 Which means that RMID 1 is now assigned to wheezy64, and RMID 2 is given to
 Domain-0. Here's the effect:
 
 [root@redbrick ~]# xl psr-cmt-show cache_occupancy
 Total RMID: 71
 NameIDSocket 0
 Socket 1Socket 2Socket 3
 Total L3 Cache Size   46080 KB
 46080 KB46080 KB46080 KB
 Domain-0 0  216 KB  
 144 KB  144 KB0 KB
 wheezy64 1 7416 KB
 0 KB 1872 KB0 KB
 
 It looks quite likely that the 144KB occupancy on sockets 1 and 2, now being
 accounted to Domain-0, is really what has been allocated by domain wheezy64,
 before the RMID switch. The same applies to the 7416KB on socket 0 now
 accounted to wheezy64, i.e., most of this is not accurate and was allocated
 there by Domain-0.
 
 This is only a simple example, others have been performed, restricting the
 affinity of the various domains involved in order to control on what socket
 cache load were to be expected, and all confirm the above reasoning.
 
 It is rather easy to appreciate that any kind of 'flushing' mechanism, to be
 triggered when reusing an RMID (if anything like that even exists!) would
 impact system performance (e.g., it is not an option in hot paths), but the
 situation outlined above needs to be fixed, before the mechanism could be
 considered usable and reliable enough to do anything on top of it.

As I know, no such 'flushing' mechanism available at present. One
possible software solution to lighten this issue is rotating the RMIDs
with algorithm like 'use oldest unused RMID first'.

Chao

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Crash of guest with nested vmx with Unknown nested vmexit reason 80000021.

2015-04-07 Thread Li, Liang Z
Hi All,

   I found a way to preproduce the bug very easy by using the 
apic-send_IPI_all(NMI_VECTOR)
 in L2 in a kernel module to trigger MNI. And I have verified that the bug can 
be fixed as Jan's
 suggestion, ' the second half of vmx_idtv_reinject() needs to be done without 
regard to
 nestedhvm_vcpu_in_guestmode(v)', I will send a patch later.

Liang

 -Original Message-
 From: xen-devel-boun...@lists.xen.org [mailto:xen-devel-
 boun...@lists.xen.org] On Behalf Of Jeroen Groenewegen van der Weyden
 Sent: Wednesday, March 04, 2015 6:24 PM
 To: Jan Beulich
 Cc: Tian, Kevin; George Dunlap; Dong, Eddie; xen-devel@lists.xen.org;
 Nakajima, Jun; Zhang, Yang Z
 Subject: Re: [Xen-devel] Crash of guest with nested vmx with Unknown
 nested vmexit reason 8021.
 
 Hi Jan,
 
 I am sorry, But  had the impression you were somehow the owner of these
 feature. My mistake.
 I understand the feature is in preview, But are you feeling intense presure
 when I ask for some status every 2 months? :-))
 
 
 BR,
 Jeroen
 
 Jan Beulich schreef op 27-2-2015 om 09:08:
  On 26.02.15 at 19:56, groen...@grosc.com wrote:
  Anything planned concerning this?
  Thanks for asking, but to be honest I don't understand why you're (not
  the first time iirc) pinging me rather than the VMX maintainers who
  had promised to do that work. All I can say is that I'm being told
  it's still on their todo list, but no-one has found time yet. And
  please remember that all of nested virt is still marked preview, i.e.
  not fully supported - for reasons like the bug here.
 
  Jan
 
  Jan Beulich schreef op 9-12-2014 om 10:17:
  On 09.12.14 at 10:09, groen...@grosc.com wrote:
  Did anyone find the time yet? I'm still more then willing testing
  any patches.
  Just yesterday we were told by Intel that they still can't foresee
  when they will find time.
 
  Jan
 
 
 
 
 
  ___
  Xen-devel mailing list
  Xen-devel@lists.xen.org
  http://lists.xen.org/xen-devel
 
 
 
 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 00/10] PML (Paging Modification Logging) support

2015-04-07 Thread Tim Deegan
Hi,

At 16:30 +0800 on 07 Apr (1428424218), Kai Huang wrote:
 Hi Jan, Tim, other maintainers,
 
 Do you have comments? Or should I send out the v2 addressing Andrew's 
 comments, as it's been more than a week since this patch series were 
 sent out?

I'm sorry, I was away last week so I haven't had a chance to review
these patches.  I'll probably be able to look at them on Thursday.

Cheers,

Tim.

 On 03/30/2015 01:50 PM, Kai Huang wrote:
 
 
  On 03/28/2015 05:26 AM, Andrew Cooper wrote:
  On 27/03/15 02:35, Kai Huang wrote:
  Hi all,
 
  This patch series adds PML support to Xen. Please kindly help to 
  review it.
  Overall this looks like a very good series, and it is particularly
  helpful given the level of commenting.
 
  Which platforms is/will PML be available for?
  Hi Andrew,
 
  Thanks for your quick review. PML will be available from Intel's 
  Broadwell server platform.
 
  Thanks,
  -Kai
 
  ~Andrew
 
  ___
  Xen-devel mailing list
  Xen-devel@lists.xen.org
  http://lists.xen.org/xen-devel
 
 
  ___
  Xen-devel mailing list
  Xen-devel@lists.xen.org
  http://lists.xen.org/xen-devel
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH 3/7] xen: psr: reserve an RMID for each core

2015-04-07 Thread Dario Faggioli
On Tue, 2015-04-07 at 16:24 +0800, Chao Peng wrote:
 On Sat, Apr 04, 2015 at 04:14:41AM +0200, Dario Faggioli wrote:

  'persocket_cmt' would basically only allow to track the
  amount of free L3 on each socket (by subtracting the
  monitored value from the total). Again, still better
  than nothing, would use very few RMIDs, and I could
  think of ways of using this information in a few
  places in the scheduler...
  
  Again, thought?
 
 This even can be extended to the concept of 'cache monitoring group',
 which can hold arbitrary cpus into one group. 

Yes, indeed.

 Actually Linux
 implementation does this by using the cgoup mechanism to allocate RMID
 to a group of threads. 

Does it? I dig the threads of the Linux's patches up to a certain point,
and it seemed that the maintainer disliked the idea of PSR-cgroups. I
have to admit I stopped at some point, and did not check how the
checked-in code actually looks like.

Anyway, yes, I'll explore the 'grouping idea'. In fact, in order to be
able to use CMT (and other PSR features) from inside the scheduler, the
shortage of RMIDs is probably not the biggest issue.

I'm much more concerned about the difficulties in making both static
monitoring (i.e., the per-core/per-group monitoring required by the
scheduler and other Xen components) and dynamic monitoring (i.e., the
per-domain monitoring, happening on user request, via `xl
psr-cmt-attach') play well together, if enabled at the same time.

If you've got time and are interested in providing your view on that, it
would be great.

 Such design can solve the RMID-shortage somehow.
 
Exactly. Actually, I've got a question: I think I remember reading on
Intel's SDM that RMID makes sense per-socket. If that is the case, it
means I can use, say, RMID #18 for both core 4, on socket 0, and core
73, on socket 2, do you confirm that? I'm asking because, as I said, I
think I read it in the manual, but the Xen implementation does not seem
to take advantage of this (perhaps because it wasn't necessary?).

Thanks and Regards,
Dario


signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen: arm: X-Gene Storm check GIC DIST address for EOI quirk

2015-04-07 Thread Stefano Stabellini
On Mon, 6 Apr 2015, Julien Grall wrote:
 Hi Pranav,
 
 Thank you for the patch.
 
 On 06/04/2015 10:54, Pranavkumar Sawargaonkar wrote:
  In old X-Gene Storm firmware and DT, secure mode addresses have been
  mentioned in GICv2 node. In this case maintenance interrupt is used
  instead of EOI HW method.
  
  This patch checks the GIC Distributor Base Address to enable EOI quirk
  for old firmware.
  
  Ref:
  http://lists.xen.org/archives/html/xen-devel/2014-07/msg01263.html
  
  Signed-off-by: Pranavkumar Sawargaonkar pranavku...@linaro.org
  ---
xen/arch/arm/platforms/xgene-storm.c |   37
  +-
1 file changed, 36 insertions(+), 1 deletion(-)
  
  diff --git a/xen/arch/arm/platforms/xgene-storm.c
  b/xen/arch/arm/platforms/xgene-storm.c
  index eee650e..dd7cbfc 100644
  --- a/xen/arch/arm/platforms/xgene-storm.c
  +++ b/xen/arch/arm/platforms/xgene-storm.c
  @@ -22,6 +22,7 @@
#include asm/platform.h
#include xen/stdbool.h
#include xen/vmap.h
  +#include xen/device_tree.h
#include asm/io.h
#include asm/gic.h
  
  @@ -35,9 +36,41 @@ static u64 reset_addr, reset_size;
static u32 reset_mask;
static bool reset_vals_valid = false;
  
  +#define XGENE_SEC_GICV2_DIST_ADDR0x7801
  +static u32 quirk_guest_pirq_need_eoi;
 
 This variable will mostly be read. So, I would add __read_mostly.
 
  +
  +static void xgene_check_pirq_eoi(void)
 
 If I'm not mistaken, this function is only called during Xen initialization.
 So, I would add __init.
 
  +{
  +struct dt_device_node *node;
  +int res;
  +paddr_t dbase;
  +
  +dt_for_each_device_node( dt_host, node )
  +{
 
 It would be better to create a new callback for platform specific GIC
 initialization and use dt_interrupt_controller.
 
 This would avoid to have this loop and rely on there is always only one
 interrupt controller in the DT.

That is true, however we do know that on this SoC there is only one GIC,
so it might be acceptable to rely on this knowledge: this is already
very platform specific code.



  +if ( !dt_get_property(node, interrupt-controller, NULL) )
  +continue;
  +
  +res = dt_device_get_address(node, 0, dbase, NULL);
  +if ( !dbase )
  +panic(%s: Cannot find a valid address for the 
  +distributor, __func__);
  +
  +/*
  + * In old X-Gene Storm firmware and DT, secure mode addresses have
  + * been mentioned in GICv2 node. We have to use maintenance
  interrupt
  + * instead of EOI HW in this case. We check the GIC Distributor
  Base
  + * Address to maintain compatibility with older firmware.
  + */
  + if (dbase == XGENE_SEC_GICV2_DIST_ADDR)
 
 Coding style:
 
 if ( ... )
 
  + quirk_guest_pirq_need_eoi =
  PLATFORM_QUIRK_GUEST_PIRQ_NEED_EOI;
  + else
  + quirk_guest_pirq_need_eoi = 0;
 
 I would print a warning in order to notify the user that his platform would be
 slow/buggy...

Good idea.


  +}
  +}
  +
static uint32_t xgene_storm_quirks(void)
{
  -return
  PLATFORM_QUIRK_GIC_64K_STRIDE|PLATFORM_QUIRK_GUEST_PIRQ_NEED_EOI;
  +return PLATFORM_QUIRK_GIC_64K_STRIDE| quirk_guest_pirq_need_eoi;
 
 This function is called every time Xen injects a physical IRQ to a guest (i.e
 very often). It might be better to create a variable quirks which will contain
 PLATFORM_QUIRK_GIC_64K_STRIDE and, when necessary,
 PLATFORM_QUIRK_GUEST_PIRQ_NEED_EOI.
 
 That would avoid the or at each call.
 
 Regards,
 
 -- 
 Julien Grall
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH 0/7] Intel Cache Monitoring: Current Status and Future Opportunities

2015-04-07 Thread Dario Faggioli
On Tue, 2015-04-07 at 16:19 +0800, Chao Peng wrote:
 On Sat, Apr 04, 2015 at 04:14:15AM +0200, Dario Faggioli wrote:

  I'm putting here in the cover letter a markdown document I wrote to better
  describe my findings and ideas (sorry if it's a bit long! :-D). You can also
  fetch it at the following links:
  
   * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.pdf
   * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.markdown
  
  See the document itself and the changelog of the various patches for 
  details.
 
 Very good summary and possible usage analysis. 

Thanks. :-)

 Most of the problems do
 exist and some of them may be solved partially but some looks
 unavoidable.
 
I see.

  It is rather easy to appreciate that any kind of 'flushing' mechanism, to be
  triggered when reusing an RMID (if anything like that even exists!) would
  impact system performance (e.g., it is not an option in hot paths), but the
  situation outlined above needs to be fixed, before the mechanism could be
  considered usable and reliable enough to do anything on top of it.
 
 As I know, no such 'flushing' mechanism available at present. One
 possible software solution to lighten this issue is rotating the RMIDs
 with algorithm like 'use oldest unused RMID first'.
 
Ok. Yes, that was something I was thinking to as well. It certainly
would make the issue less severe / less likely to happen.

Let's see what others think.

Regards,
Dario


signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 8/8] tools: add tools support for Intel CAT

2015-04-07 Thread Chao Peng
On Thu, Apr 02, 2015 at 11:15:53AM +0800, Chao Peng wrote:
 On Wed, Apr 01, 2015 at 10:23:01AM +0100, Ian Campbell wrote:
  On Wed, 2015-04-01 at 17:06 +0800, Chao Peng wrote:
 If this one returns all sockets but not a specified socket data 
 (which I agreed)
 and not consider legacy cmt code, then I think I can make
 libxl_count_physical_sockets() private and move it to 
 libxl/libxl_psr.c.

What is the legacy cmt code? But otherwise I agree, yes.
   
   In libxl/xl_cmdimpl.c, psr_cmt_show also calculates the socket count
   itself. If we want to refactor it with new libxl_count_physical_sockets
   then libxl_count_physical_sockets should be public, otherwise it can be
   private to libxl_psr.c only. From my side, both directions sound OK.
  
  Ah, so we would want a return a list of all sockets variant of
  libxl_psr_cmt_get_cache_occupancy too? I think that's fine, we need to
  keep the old interface but we could easily add a new one, e.g.
  libxl_psr_cmt_get_all_cache_occupancy (insert the word sockets if you
  like).
 
 The libxl_psr_cmt_get_cache_occupancy is actually not used in xl anymore
 and should be considered as obsolete. Substitute is
 libxl_psr_cmt_get_sample which is the common API for both cache
 occupancy and memory bandwidth(For the latter the routine will be called
 twice). So if we really want to do everything batch in libxl then
 perhaps we should do that for libxl_psr_cmt_get_sample.
 
 The other candidate is libxl_psr_cmt_get_l3_cache_size, which can also
 do batch in libxl. 
 
 While there is also drawback: If both
 libxl_psr_cmt_get_sample/libxl_psr_cmt_get_l3_cache_size do their
 batches in libxl, then libxl_count_physical_sockets will be called twice
 in libxl which may bring performance drop.
 
 Another solution is only do batch for libxl_psr_cmt_get_l3_cache_size.
 It returns all sockets's cache size and meantime the socket number. Once
 we get socket number, other functions(Both libxl_psr_cmt_get_sample for
 CMT/MBM and libxl_psr_cat_get_l3_info/libxl_psr_cat_get_cbm for CAT)
 can be called socket by socket in xl.

Ian, does this solution sounds OK to you?
Chao
  
  So both interfaces would be something like:
 int libxl_psr_(ctx, domid, TYPE **list_r, int *nr);
  
  And on success *list_r points to a newly allocated array and *nr is the
  number of elements in that array. TYPE depends on which op it is, so for
  cache_occupancy it seems a uint32_t.
  
  Is the socket address space always contiguous and starting at zero? If
  not then the array might need to contain (socket,TYPE) structs.
  
  Ian.
 
 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] sudden hang on boot on AMD Notebook

2015-04-07 Thread Olaf Hering
On Mon, Mar 30, Olaf Hering wrote:

 
 Just last week I was running staging to test pvscsi on this HP ProBook
 6555b, since around Wednesday I think. But todays staging fails to boot.
 So I started a bisect in xen/, but every attempt to boot xen.gz results
 in a hang at the same place. Even going back to the state of 4.5.0.

It boots with maxcpus=1, so the Brought up n CPUs line is whats missing.
Even the RELEASE-4.0.0 tag fails to boot, have to check if maxcpus= will
help there as well.

Olaf

 __  __  
 \ \/ /___ _ __  
  \  // _ \ '_ \ 
  /  \  __/ | | |
 /_/\_\___|_| |_|
 
 _  ___  _  __   
| || |  / /_ ___| |_ __ _  __ _(_)_ __   __ _  __  _(_)_ __  
| || |_| '_ \ __/ __| __/ _` |/ _` | | '_ \ / _` | \ \ /\ / / | '_ \ 
|__   _| (_) |__\__ \ || (_| | (_| | | | | | (_| |  \ V  V /| | |_) |
   |_|(_)___/   |___/\__\__,_|\__, |_|_| |_|\__, |___\_/\_/ |_| .__/ 
  |___/ |___/_|   |_|
(XEN) Xen version 4.6-staging_wip (o...@fritz.box) (gcc (SUSE Linux) 4.5.1 
20101208 [gcc-4_5-branch revision 167585]) debug=n Tue Mar 31 13:24:42 CEST 2015
(XEN) Latest ChangeSet: Tue Mar 31 13:12:12 2015 +0200 git:43a6f93
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: loglvl=all guest_loglvl=all conring_size=1M maxcpus=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)   - 0009fc00 (usable)
(XEN)  0009fc00 - 000a (reserved)
(XEN)  000e - 0010 (reserved)
(XEN)  0010 - af3cf000 (usable)
(XEN)  af3cf000 - afacf000 (reserved)
(XEN)  afacf000 - afbcf000 (ACPI NVS)
(XEN)  afbcf000 - afbff000 (ACPI data)
(XEN)  afbff000 - afc0 (usable)
(XEN)  e000 - f000 (reserved)
(XEN)  fec0 - fec01000 (reserved)
(XEN)  fec1 - fec11000 (reserved)
(XEN)  fee0 - fee01000 (reserved)
(XEN)  ffe0 - 0001 (reserved)
(XEN)  0001 - 00024000 (usable)
(XEN) ACPI: RSDP 000F2A70, 0014 (r0 HPQOEM)
(XEN) ACPI: RSDT AFBFE038, 0044 (r1 HPQOEM SLIC-MPC3   113)
(XEN) ACPI: FACP AFBFD000, 0074 (r1 HPQOEM 307E3 HP  1)
(XEN) ACPI Warning (tbfadt-0464): Optional field Pm2ControlBlock has zero 
address or length: /1 [20070126]
(XEN) ACPI: DSDT AFBDD000, 1C447 (r1 HPQOEM   SB700 1 INTL 20060912)
(XEN) ACPI: FACS AFB7E000, 0040
(XEN) ACPI: APIC AFBFC000, 0084 (r1 HPQOEM 307E1 HP  1)
(XEN) ACPI: MCFG AFBFB000, 003C (r1 HPQOEM 307E1 HP  1)
(XEN) ACPI: SLIC AFBFA000, 0176 (r1 HPQOEM SLIC-MPC1 HP  1)
(XEN) ACPI: SSDT AFBDC000, 005F (r1 HPQOEM   HPQNLP1 INTL 20060912)
(XEN) ACPI: TCPA AFBDB000, 0032 (r2 HPQOEM 307E0 HP  1)
(XEN) ACPI: HPET AFBDA000, 0038 (r1 HPQOEM 307E1 HP  1)
(XEN) ACPI: SSDT AFBD9000, 07AD (r1 AMDPOWERNOW1 AMD 1)
(XEN) System RAM: 7923MB (8113596kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at -00024000
(XEN) Domain heap initialised
(XEN) Allocated console ring of 1024 KiB.
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: wakeup_vec[afb7e00c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee0
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 0:5 APIC version 16
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 0:5 APIC version 16
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 0:5 APIC version 16
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x04] address[0xfec0] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 4, version 33, address 0xfec0, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x43538301 base: 0xfed0
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 1 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 184 MSI/MSI-X
(XEN) Using 

Re: [Xen-devel] [PATCH] xen: arm: X-Gene Storm check GIC DIST address for EOI quirk

2015-04-07 Thread Julien Grall

Hi Stefano,

On 07/04/2015 10:40, Stefano Stabellini wrote:

On Mon, 6 Apr 2015, Julien Grall wrote:

Hi Pranav,

Thank you for the patch.

On 06/04/2015 10:54, Pranavkumar Sawargaonkar wrote:

In old X-Gene Storm firmware and DT, secure mode addresses have been
mentioned in GICv2 node. In this case maintenance interrupt is used
instead of EOI HW method.

This patch checks the GIC Distributor Base Address to enable EOI quirk
for old firmware.

Ref:
http://lists.xen.org/archives/html/xen-devel/2014-07/msg01263.html

Signed-off-by: Pranavkumar Sawargaonkar pranavku...@linaro.org
---
   xen/arch/arm/platforms/xgene-storm.c |   37
+-
   1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/platforms/xgene-storm.c
b/xen/arch/arm/platforms/xgene-storm.c
index eee650e..dd7cbfc 100644
--- a/xen/arch/arm/platforms/xgene-storm.c
+++ b/xen/arch/arm/platforms/xgene-storm.c
@@ -22,6 +22,7 @@
   #include asm/platform.h
   #include xen/stdbool.h
   #include xen/vmap.h
+#include xen/device_tree.h
   #include asm/io.h
   #include asm/gic.h

@@ -35,9 +36,41 @@ static u64 reset_addr, reset_size;
   static u32 reset_mask;
   static bool reset_vals_valid = false;

+#define XGENE_SEC_GICV2_DIST_ADDR0x7801
+static u32 quirk_guest_pirq_need_eoi;


This variable will mostly be read. So, I would add __read_mostly.


+
+static void xgene_check_pirq_eoi(void)


If I'm not mistaken, this function is only called during Xen initialization.
So, I would add __init.


+{
+struct dt_device_node *node;
+int res;
+paddr_t dbase;
+
+dt_for_each_device_node( dt_host, node )
+{


It would be better to create a new callback for platform specific GIC
initialization and use dt_interrupt_controller.

This would avoid to have this loop and rely on there is always only one
interrupt controller in the DT.


That is true, however we do know that on this SoC there is only one GIC,
so it might be acceptable to rely on this knowledge: this is already
very platform specific code.


If we keep the loop, I would add a comment on the above the loop 
explaining that we rely on there is always only GIC.


Although, I was wondering if we could simply rely on the node path to 
get the DT node?


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH 2/7] Xen: x86: print max usable RMID during init

2015-04-07 Thread Dario Faggioli
On Mon, 2015-04-06 at 09:48 -0400, Konrad Rzeszutek Wilk wrote:
 On Sat, Apr 04, 2015 at 04:14:33AM +0200, Dario Faggioli wrote:

  diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
  index c902625..0f2a6ce 100644
  --- a/xen/arch/x86/psr.c
  +++ b/xen/arch/x86/psr.c
  @@ -111,7 +111,8 @@ static void __init init_psr_cmt(unsigned int rmid_max)
   for ( rmid = 1; rmid = psr_cmt-rmid_max; rmid++ )
   psr_cmt-rmid_to_dom[rmid] = DOMID_INVALID;
   
  -printk(XENLOG_INFO Cache Monitoring Technology enabled\n);
  +printk(XENLOG_INFO Cache Monitoring Technology enabled, RMIDs: %u\n,
 
 max RMID: ?
 
I considered it, but don't like it.

IMO 'max' tells something about the values one can use as RMIDs, not
about the fact that they are a limited resource.

Perhaps something like available RMIDs: or total nr. of RMIDs ?

Thanks and Regards,
Dario


signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC v3 00/28] libxl: Cancelling asynchronous operations

2015-04-07 Thread Ian Jackson
Ian Jackson writes ([PATCH RFC v3 00/28] libxl: Cancelling asynchronous 
operations):
 This is v3 of my work-in-progress series to support cancellation of
 long-running libxl operations.

These are also here

 http://xenbits.xen.org/gitweb/?p=people/iwj/xen.git;a=summary
 git://xenbits.xen.org/people/iwj/xen.git

 base.ao-cancel.v3..wip.ao-cancel.v3..

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH 0/7] Intel Cache Monitoring: Current Status and Future Opportunities

2015-04-07 Thread Andrew Cooper
On 04/04/2015 03:14, Dario Faggioli wrote:
 Hi Everyone,

 This RFC series is the outcome of an investigation I've been doing about
 whether we can take better advantage of features like Intel CMT (and of PSR
 features in general). By take better advantage of them I mean, for example,
 use the data obtained from monitoring within the scheduler and/or within
 libxl's automatic NUMA placement algorithm, or similar.

 I'm putting here in the cover letter a markdown document I wrote to better
 describe my findings and ideas (sorry if it's a bit long! :-D). You can also
 fetch it at the following links:

  * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.pdf
  * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.markdown

 See the document itself and the changelog of the various patches for details.

 The series includes one Chao's patch on top, as I found it convenient to build
 on top of it. The series itself is available here:

   git://xenbits.xen.org/people/dariof/xen.git  wip/sched/icachemon
   
 http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/wip/sched/icachemon

 Thanks a lot to everyone that will read and reply! :-)

 Regards,
 Dario
 ---

There seem to be several areas of confusion indicated in your document. 
I am unsure whether this is a side effect of the way you have written
it, but here are (hopefully) some words of clarification.  To the best
of my knowledge:

PSR CMT works by tagging cache lines with the currently-active RMID. 
The cache utilisation is a count of the number of lines which are tagged
with a specific RMID.  MBM on the other hand counts the number of cache
line fills and cache line evictions tagged with a specific RMID.

By this nature, the information will never reveal the exact state of
play.  e.g. a core with RMID A which gets a cache line hit against a
line currently tagged with RMID B will not alter any accounting. 
Furthermore, as alterations of the RMID only occur in
__context_switch(), Xen actions such as handling an interrupt will be
accounted against the currently active domain (or other future
granularity of RMID).

max_rmid is a per-socket property.  There is no requirement for it to
be the same for each socket in a system, although it is likely, given a
homogeneous system.  The limit on RMID is based on the size of the
accounting table.

As far as MSRs themselves go, an extra MSR write in the context switch
path is likely to pale into the noise.  However, querying the data is an
indirect MSR read (write to the event select MSR, read from  the data
MSR).  Furthermore there is no way to atomically read all data at once
which means that activity on other cores can interleave with
back-to-back reads in the scheduler.


As far as the plans here go, I have some concerns.  PSR is only
available on server platforms, which will be 2/4 socket systems with
large numbers of cores.  As you have discovered, there insufficient
RMIDs for redbrick pcpus, and on a system that size, XenServer typically
gets 7x vcpus to pcpus.

I think it is unrealistic to expect to use any scheduler scheme which is
per-pcpu or per-vcpu while the RMID limit is as small as it is. 
Depending on workload, even a per-domain scheme might be problematic. 
One of our tests involves running 500xWin7 VMs on that particular box.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 3/3] libxl: datacopier: Avoid theoretical eof/POLLHUP race

2015-04-07 Thread Roger Pau Monné
El 02/04/15 a les 18.29, Ian Jackson ha escrit:
 Ian Jackson writes ([PATCH 3/3] libxl: datacopier: Avoid theoretical 
 eof/POLLHUP race):
 We solve the race with a poll of the reading fd, to double-check, when
 we detect eof via read.  (This is only necessary if the caller has
 specified callback_pollhup, as otherwise POLLHUP|POLLIN - and,
 presumably, POLLIN followed perhaps by POLLHUP|POLLIN, is to be
 treated as eof anyway.)
 
 I have been unable to reproduce this race.  On Linux, the pty master
 returns EGAIN from read, not EOF, when the bootloader exits.
 
 Below is the patch I used to test this.  I think this should repro the
 bug on FreeBSD when doing xl create of a guest which (a) uses pygrub
 (b) autoboots within 15 seconds.
 
 Roger, can you conveniently test this ?  With the hacky patch below
 you should see the bug, which should in turn be fixed by the 3/3 I am
 just replying to.

Hello,

I don't seem to be able to trigger the issue with the debug patch 
applied. AFAICT pygrub is blocked (probably because the output buffer 
is full) and total execution greatly exceeds 15s seconds. Here is the 
full output:

# xl -vvv create debian.cfg
Parsing config from debian.cfg
libxl: debug: libxl_create.c:1512:do_domain_create: ao 0x802834080: create: 
how=0x0 callback=0x0 poller=0x80281f0a0
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk vdev=xvda 
spec.backend=unknown
libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk 
vdev=xvda, using backend phy
libxl: debug: libxl_create.c:915:initiate_domain_create: running bootloader
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk 
vdev=(null) spec.backend=phy
libxl: debug: libxl.c:3020:libxl__device_disk_local_initiate_attach: locally 
attaching PHY disk /dev/zvol/tank/debian
libxl: debug: libxl_bootloader.c:411:bootloader_disk_attached_cb: Config 
bootloader value: pygrub
libxl: debug: libxl_bootloader.c:427:bootloader_disk_attached_cb: Checking for 
bootloader in libexec path: /usr/local/lib/xen/bin/pygrub
libxl: debug: libxl_create.c:1528:do_domain_create: ao 0x802834080: inprogress: 
poller=0x80281f0a0, flags=i
libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch w=0x802836780 
wpath=/local/domain/4 token=3/0: register slotnum=3
libxl: debug: libxl_event.c:1942:libxl__ao_progress_report: ao 0x802834080: 
progress report: ignored
libxl: debug: libxl_bootloader.c:537:bootloader_gotptys: executing bootloader: 
/usr/local/lib/xen/bin/pygrub
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
/usr/local/lib/xen/bin/pygrub
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
--output=/var/run/xen/bootloader.4.out
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
--output-format=simple0
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
--output-directory=/var/run/xen/bootloader.4.d
libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
/dev/zvol/tank/debian
libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x802836780 
wpath=/local/domain/4 token=3/0: event epath=/local/domain/4
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=315 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=71 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=988 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC READABLE
DC READABLE CONTINUING 1
DC READABLE LOOP
DC READABLE READ r=127 Resource temporarily unavailable
DC READABLE LOOP
DC READABLE READ r=-1 Resource temporarily unavailable
DC 

[Xen-devel] [PATCH V2] xen/arm64: Use virtual address when setting up early_printk fixmap

2015-04-07 Thread Chen Baozi
We have already had the boot pagetable when reaching the point
of early_printk fixmap setup. Thus there is no longer necessary
to calculate physical address of xen_fixmap.

Signed-off-by: Chen Baozi baoz...@gmail.com
---
 xen/arch/arm/arm64/head.S | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index 75b7ee1..9ed9a93 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -519,8 +519,7 @@ paging:
 cbnz  x22, 1f
 
 /* Add UART to the fixmap table */
-ldr   x1, =xen_fixmap
-add   x1, x1, x20/* x1 := paddr (xen_fixmap) */
+ldr   x1, =xen_fixmap/* x1 := vaddr (xen_fixmap) */
 lsr   x2, x23, #THIRD_SHIFT
 lsl   x2, x2, #THIRD_SHIFT   /* 4K aligned paddr of UART */
 mov   x3, #PT_DEV_L3
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Nested Virt - Xen 4.4 through 4.6 - Hyper-V; Can't boot after enabling Hyper-V

2015-04-07 Thread George Dunlap
On Tue, Apr 7, 2015 at 2:42 AM, mailing lists theli...@gmail.com wrote:
 Hi --

 I've been trying to get nested virtualization working with Xen so that I
 could boot Windows and use Hyper-V related features, however I have not had
 much success.  Using Windows 8.1 or Windows 2012r2, I'm able to install
 Windows, select and install Hyper-V features, and start rebooting.  However,
 at that point, the Windows VM only partially boots, then drops me to a
 screen stating:

 Your PC needs to restart.
 Please hold down the power button.
 Error Code: 0x001E
 Parameters:
 0xC096
 0xF80315430485
 0x
 0x


 Restarting does not yield any different results.

 I've set up Xen in accordance with the notes for patches and config options
 here:

 http://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

 Trying Xen 4.4.2 stable, 4.5.1 staging, and 4.6 staging.  I applied the
 patch labeled (2/2) from the wiki link above, compiled, and used the three
 options provided for the DomU running Windows (hap, nestedhvm, and cpuid
 mask).  Windows installs and allows me to turn on HyperV features on all
 versions of Xen listed above, however all give the same or similar message
 on reboot... I'm never able to get to a running state.

 I've tried this on two separate systems.  One has an Intel E5-1620 v2, and
 the other is a n E5-1650 (original, v1 I guess).  All the virtualization
 options are enabled in the BIOS.

 If the cpuid mask is removed from the DomU config, Windows boots, however
 I'm unable to start any virtual machines (there was a message in the Windows
 event log about a component not being started in regards to Hyper V).

 Has anyone else run into similar issues?  Any thoughts on next steps?


CC'ing Liang Z to see if he has any ideas.

Liang, I'm pretty sure you guys are testing HyperV internally, right?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxl new bitmap functions

2015-04-07 Thread Wei Liu
Sorry for keeping you waiting. I just get back from vacation.

First I would like to thank Dario and Konrad for their reviews. Their
comments are quite to the point and you should fix your code as
suggested.

In addition to their comments, I have a few comments. See below.

On Thu, Apr 02, 2015 at 11:38:16AM -0600, Linda Jacobson wrote:
 From: Linda lin...@jma3.com
 
 Added bitmap functions for union intersection and difference betweenn two 
 bitmaps
 
 Signed-off-by: Linda lin...@jma3.com
 ---
  tools/libxl/libxl_utils.c | 115 
 ++
  tools/libxl/libxl_utils.h |  10 
  2 files changed, 125 insertions(+)
 
 diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
 index 9053b27..c390ddc 100644
 --- a/tools/libxl/libxl_utils.c
 +++ b/tools/libxl/libxl_utils.c
 @@ -699,6 +699,121 @@ int libxl_bitmap_count_set(const libxl_bitmap *bitmap)
  
  return nr_set_bits;
  }
 +int libxl_bitmap_union(libxl_ctx *ctx, libxl_bitmap *union_bitmap, 
 +libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)
 +{
 +int size;

uint32_t because size field in bitmap type is uint32_t.

 +int rc;
 +
 +GC_INIT(ctx);
 +
 +// if bitmaps aren't the same size, union should be size of larger bit map
 +size = (bitmap1-size  bitmap2-size) ? bitmap1-size : bitmap2-size;
 +
 +libxl_bitmap_init(union_bitmap);

There is no need to call this, because the caller of this function is
supposed to do that.

 +rc = libxl_bitmap_alloc(ctx, union_bitmap, size);
 +if (rc)
 +{
 +// Following the coding standards here.  
 + //First goto I've written in decades.

Don't put in such comments please.

And you can omit the {} if there is only one statement.

 +goto out;
 +}
 +
 +for (int bit = 0; bit  (size * 8); bit++)
 +{
 +// libxl_bitmap_test returns 0 if past end of bitmap
 +// if the bit is set in either bitmap, set it in their union

The above comment is not needed. Not that it is wrong, just that the
code below is self-explanatory.

 +if (libxl_bitmap_test(bitmap1, bit))
 +{
 +libxl_bitmap_set(union_bitmap, bit);
 +}
 +else if (libxl_bitmap_test(bitmap2, bit))
 +{
 +libxl_bitmap_set(union_bitmap, bit);
 +}
 +}
 +
 +out:
 +GC_FREE;
 +return rc;
 +}
 +
 +int libxl_bitmap_intersection (libxl_ctx *ctx, libxl_bitmap 
 +*intersection_bitmap, libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)
 +{
 +int size;
 +int rc;
 +
 +GC_INIT(ctx);
 +
 +// if bitmaps aren't same size, intersection should be size of 
 +// smaller bit map
 +size = (bitmap1-size  bitmap2-size) ? bitmap2-size : bitmap1-size;
 +
 +libxl_bitmap_init(intersection_bitmap);
 +rc = libxl_bitmap_alloc(ctx, intersection_bitmap, size);
 +if (rc)
 +{
 +goto out;
 +}
 +
 +for (int bit = 0; bit  (size * 8); bit++)
 +{
 +// libxl_bitmap_test returns 0 if past end of bitmap
 +// if the bit is set in both bitmaps, set it in their intersection

The above comment is not needed.

 +if (libxl_bitmap_test (bitmap1, bit) 
 +  libxl_bitmap_test (bitmap2, bit) )
 +{
 +libxl_bitmap_set (intersection_bitmap, bit);
 +}
 +}
 +
 +out:
 +GC_FREE;
 +return rc;
 +}

Please have an extra line between functions.

 +int libxl_bitmap_difference(libxl_ctx *ctx, libxl_bitmap *difference_bitmap, 
 +libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)

I think there is confusion on how this function should be implemented
and the semantics of this function.  Since we're short on time I think
it's better to just ignore this for now and polish the other two well
defined functions.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen/arm64: Use virtual address when setting up early_printk fixmap

2015-04-07 Thread Julien Grall

Hi Chen,

On 07/04/2015 08:33, Chen Baozi wrote:

From: Chen Baozi baoz...@gmail.com

We have already had the boot pagetable when reaching the point
of early_printk fixmap setup. Thus there is no longer necessary
to caculate physical address of xen_fixmap.


s/caculate/calculate/


OOI, is it a bug or a clean up?



Signed-off-by: Chen Baozi baoz...@gmail.com
---
  xen/arch/arm/arm64/head.S | 1 -
  1 file changed, 1 deletion(-)

diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index 75b7ee1..a2ac2d4 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -520,7 +520,6 @@ paging:

  /* Add UART to the fixmap table */
  ldr   x1, =xen_fixmap
-add   x1, x1, x20/* x1 := paddr (xen_fixmap) */


You need to add a comment on ldr x1... in order to explain what x1 
contains. i.e /* x1 := vaddr (xen_fixmap) */


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen/arm64: Use virtual address when setting up early_printk fixmap

2015-04-07 Thread Chen Baozi
On Tue, Apr 07, 2015 at 12:07:28PM +0100, Julien Grall wrote:
 Hi Chen,
 
 On 07/04/2015 08:33, Chen Baozi wrote:
 From: Chen Baozi baoz...@gmail.com
 
 We have already had the boot pagetable when reaching the point
 of early_printk fixmap setup. Thus there is no longer necessary
 to caculate physical address of xen_fixmap.
 
 s/caculate/calculate/
 

Oops, ;-)

 
 OOI, is it a bug or a clean up?

I think it should be a clean up. There is 1:1 mapping at early stage.
So it is all right to access the 'physical' address.

 
 
 Signed-off-by: Chen Baozi baoz...@gmail.com
 ---
   xen/arch/arm/arm64/head.S | 1 -
   1 file changed, 1 deletion(-)
 
 diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
 index 75b7ee1..a2ac2d4 100644
 --- a/xen/arch/arm/arm64/head.S
 +++ b/xen/arch/arm/arm64/head.S
 @@ -520,7 +520,6 @@ paging:
 
   /* Add UART to the fixmap table */
   ldr   x1, =xen_fixmap
 -add   x1, x1, x20/* x1 := paddr (xen_fixmap) */
 
 You need to add a comment on ldr x1... in order to explain what x1
 contains. i.e /* x1 := vaddr (xen_fixmap) */

Ok. I'll add it in V2.

Cheers,

Baozi.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Migration v2 and related work for 4.6

2015-04-07 Thread Wei Liu
On Wed, Apr 01, 2015 at 12:03:51PM +0100, Andrew Cooper wrote:
 Hello,
 
 I believe I have CC'd all interested parties.  If I have missed anyone,
 please shout.
 
 There are several large series:
 * Cancellable async operations
 * COLO
 
 and a wish to start experimenting with migration on ARM which all
 interact with my large series for migration v2 support in libxc and libxl.
 
 I am working on the libxl side of things, but realise that I am being a
 blockage to the other areas waiting on migration v2.
 
 The current state of the libxc bit is functionally complete.  It is
 shipping in two versions of XenServer, has not changed substantially in
 9 months, and not changed in a backwards incompatible way since v1.
 
 I propose that the libxc series be accepted independently of the libxl
 series.  The libxc series is still implemented as an API-compatible
 alternative to legacy migration, and is not enabled by default, meaning
 no change to current users.
 
 However, it will unblock development of ARM migration and COLO, and will
 allow people in the wider community to experiment with migration v2.  An
 existing `xl migrate` can be swapped from legacy to migration v2 simply
 by setting an environment variable (for PV guests.  HVM requires the
 libxl changed as the handling of the Qemu save record is changing.)
 
 I feel that this is best interest for the community, to get some testing
 available earlier in the development cycle, rather than attempting to
 splice 3 large series in a related area together towards the code freeze.
 
 Thoughts? (especially from a release management point of view)
 

I've looked at your libxc series. It looks complete. I agree it would be
good to get it in as early as possible.

Wei.

 ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Migration v2 and related work for 4.6

2015-04-07 Thread Wei Liu
On Thu, Apr 02, 2015 at 10:34:34AM +0100, Andrew Cooper wrote:
 On 02/04/15 10:03, Ian Campbell wrote:
  On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
  I propose that the libxc series be accepted independently of the libxl
  series.
  That is most likely a good idea IMHO.
 
  What do you estimate the chances of the libxl bit being done for 4.6 to
  be?
 
 I hope to have everything complete for 4.6, including removal of the
 legacy code.
 

Not sure what legacy code you're referring to, but we definitely want
to support 4.5 - 4.6 migration so the legacy code might need to stay
for 4.6?

Wei.

 Given the current timescales, I would say most likely.
 
 
  Is there an option of actually switching to the new libxc without
  switching libxl and fixing libxl later, or would that be as much work as
  just fixing libxl?
 
 For PV guests, yes.  One can transparently swap the legacy algorithm
 from the v2 algorithm and every works.
 
 For HVM guests, no.  The handling of the Qemu save record and toolstack
 records needs fixing.
 
 In the upgrade case, libxl also needs to take care of piping the legacy
 stream through the conversion script.
 
 ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 3/3] libxl: datacopier: Avoid theoretical eof/POLLHUP race

2015-04-07 Thread Ian Jackson
Roger Pau Monné writes (Re: [PATCH 3/3] libxl: datacopier: Avoid theoretical 
eof/POLLHUP race):
 I don't seem to be able to trigger the issue with the debug patch 
 applied. AFAICT pygrub is blocked (probably because the output buffer 
 is full) and total execution greatly exceeds 15s seconds. Here is the 
 full output:

For reference, here is the relevant bits of the IRC conversation I
just had with Roger:

12:53 Diziet Maybe replace the bootloader with a script which doessleep 
   1; echo hi; sleep 1; exit 0
13:05 royger Diziet: OK, with the dummy script I'm able to trigger the issue: 
   http://pastebin.com/raw.php?i=uUeJviV1
13:05 royger now I'm going to try with your wip series
13:15 royger Diziet: patch 3/3 seems to solve the issue, here's the output of 
   the debug patch appplied on top of 
   wip.datacopier-pollhup-fixes.v1 and with the dummy pygrub 
   script: http://pastebin.com/raw.php?i=cQqsFuf3
13:17 Diziet royger: Marvellous, those are the outputs I'm expecting.  Thank 
   you very much.
13:18 Diziet So, I will repost these with your Tested-by on the final patch, 
   if that's OK.
13:18 royger Diziet: sure :)

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Migration v2 and related work for 4.6

2015-04-07 Thread Andrew Cooper
On 07/04/15 13:00, Wei Liu wrote:
 On Thu, Apr 02, 2015 at 10:34:34AM +0100, Andrew Cooper wrote:
 On 02/04/15 10:03, Ian Campbell wrote:
 On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
 I propose that the libxc series be accepted independently of the libxl
 series.
 That is most likely a good idea IMHO.

 What do you estimate the chances of the libxl bit being done for 4.6 to
 be?
 I hope to have everything complete for 4.6, including removal of the
 legacy code.

 Not sure what legacy code you're referring to, but we definitely want
 to support 4.5 - 4.6 migration so the legacy code might need to stay
 for 4.6?

legacy migration is what currently exists.

Backwards compatibility is provided by a short python script which
converts legacy to v2.

Once the libxl changes are complete, xc_domain_save.c and
xc_domain_restore.c can be deleted, along with other assorted bits and
pieces.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v2 0/2]: libxl: datacopier POLLHUP fixes

2015-04-07 Thread Ian Jackson
7e9ec50b0535 tools/libxl: Adjust datacopiers POLLHUP handling when
the fd is also readable has been reverted by the application of what
was 1/3 of v1 of this mini-series.

(Sorry about the odd reference in the commit message to `later in this
mail', which was a reference to `Avoid [theoretical] eof/POLLHUP race'
and ought to have just read `later'.)

These are the remaining two patches:

[PATCH 1/2] libxl: datacopier: Avoid eof/POLLHUP race

This was 3/3 and was described as theoretical but with a test patch
Roger has been able to reproduce it and verify that this patch fixes
the problem.  This bug is still (I think) quite unlikely, but
this patch is a IMO candidate for backport.

[PATCH 2/2] libxl: Cope with pipes which signal POLLHUP|POLLIN on

An earlier (wrong, I now think) approach to this was 2/3 of the
previous series.  I have removed Ian Campbell's ack.  It would be good
to know whether this revised version actually fixes the bug that
7e9ec50b0535 was aimed at.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 1/2] libxl: datacopier: Avoid eof/POLLHUP race

2015-04-07 Thread Ian Jackson
When the bootloader exits, several things change, all at once:
 (a) The master pty fd (held by libxl) starts to signal POLLHUP
and maybe also POLLIN.
 (b) The child exits (so that the SIGCHLD self-pipe signals POLLIN,
which will be handled by the libxl child process code.
 (c) reads on the master pty fd start to return EOF

From the point of view of the datacopier these might happen in any
order.

(c) can be detected only after a previous POLLIN without POLLHUP and
that previous POLLIN would be associated with data which was read,
which must therefore have ended up in the dc's buffer.  But nothing
stops the dc from writing that data into the output fd and reporting
eof before it calls poll again.

This race is unlikely.  But  nevertheless it should be fixed.

We solve the race with a poll of the reading fd, to double-check, when
we detect eof via read.  (This is only necessary if the caller has
specified callback_pollhup, as otherwise POLLHUP|POLLIN - and,
presumably, POLLIN followed perhaps by POLLHUP|POLLIN, is to be
treated as eof anyway.)

With a testing patch supplied by me, Roger Pau Monné has reproduced
the failure on FreeBSD and verified that this patch fixes the problem.

Signed-off-by: Ian Jackson ian.jack...@eu.citrix.com
Tested-by: Roger Pau Monné roger@citrix.com
Acked-by: Ian Campbell ian.campb...@citrix.com
CC: Andrew Cooper andrew.coop...@citrix.com
CC: Ross Lagerwall ross.lagerw...@citrix.com
CC: Wei Liu wei.l...@citrix.com
---
 tools/libxl/libxl_aoutils.c |   16 
 1 file changed, 16 insertions(+)

diff --git a/tools/libxl/libxl_aoutils.c b/tools/libxl/libxl_aoutils.c
index 3942634..ddbe6ae 100644
--- a/tools/libxl/libxl_aoutils.c
+++ b/tools/libxl/libxl_aoutils.c
@@ -250,6 +250,22 @@ static void datacopier_readable(libxl__egc *egc, 
libxl__ev_fd *ev,
 return;
 }
 if (r == 0) {
+if (dc-callback_pollhup) {
+/* It might be that this eof is actually a HUP.  If
+ * the caller cares about the difference,
+ * double-check using poll(2). */
+struct pollfd hupchk;
+hupchk.fd = ev-fd;
+hupchk.events = POLLIN;
+hupchk.revents = 0;
+r = poll(hupchk, 1, 0);
+if (r  0)
+LIBXL__EVENT_DISASTER(egc,
+ unexpected failure polling fd for datacopier eof hup check,
+  errno, 0);
+if (datacopier_pollhup_handled(egc, dc, hupchk.revents, 0))
+return;
+}
 libxl__ev_fd_deregister(gc, dc-toread);
 break;
 }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [linux-3.16 test] 50336: regressions - FAIL

2015-04-07 Thread osstest service user
flight 50336 linux-3.16 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50336/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-credit2  15 guest-localmigrate/x10fail REGR. vs. 34167

Tests which are failing intermittently (not blocking):
 test-amd64-i386-pair 17 guest-migrate/src_host/dst_host fail pass in 50289

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-libvirt  9 guest-start   fail in 50289 like 34167

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-amd64-xl-pvh-intel  9 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm  9 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm   9 guest-start  fail   never pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-amd64-xl-xsm   9 guest-start  fail   never pass
 test-amd64-i386-libvirt  10 migrate-support-checkfail   never pass
 test-amd64-i386-xl-xsm9 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-amd   9 guest-start  fail   never pass
 test-amd64-amd64-libvirt 10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-amd64-xl-multivcpu 15 guest-localmigrate/x10   fail  never pass
 test-amd64-i386-freebsd10-i386  7 freebsd-install  fail never pass
 test-armhf-armhf-libvirt-xsm  5 xen-boot fail   never pass
 test-armhf-armhf-xl-xsm   5 xen-boot fail   never pass
 test-amd64-amd64-xl-sedf 15 guest-localmigrate/x10   fail   never pass
 test-armhf-armhf-xl-credit2  10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 10 migrate-support-checkfail  never pass
 test-armhf-armhf-xl  10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 10 migrate-support-checkfail never pass
 test-armhf-armhf-xl-sedf 10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-sedf-pin 10 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 10 migrate-support-checkfail   never pass
 test-amd64-i386-freebsd10-amd64  7 freebsd-install fail never pass
 test-amd64-i386-xl-qemut-winxpsp3 14 guest-stopfail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-amd64-xl-sedf-pin 15 guest-localmigrate/x10   fail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-armhf-armhf-xl-arndale  10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-i386-xl-winxpsp3  14 guest-stop   fail   never pass
 test-amd64-i386-xl-qemuu-winxpsp3 14 guest-stopfail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass

version targeted for testing:
 linux0596eeda8839481a3a2d64b07116581c0fc5fa0a
baseline version:
 linux19583ca584d6f574384e17fe7613dfaeadcdc4a6


1187 people touched revisions under test,
not listing them all


jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 

Re: [Xen-devel] [ovmf test] 50324: regressions - trouble: blocked/broken/fail/pass

2015-04-07 Thread Wei Liu
On Tue, Apr 07, 2015 at 02:56:59PM +0100, Ian Jackson wrote:
 osstest service user writes ([ovmf test] 50324: regressions - trouble: 
 blocked/broken/fail/pass):
  flight 50324 ovmf real [real]
  http://logs.test-lab.xenproject.org/osstest/logs/50324/
  
  Regressions :-(
  
  Tests which did not succeed and are blocking,
  including tests which could not be run:
   build-i3865 xen-buildfail REGR. vs. 36764
 
 This seems to be a real problem:
 
 gcc -g -fshort-wchar -fno-strict-aliasing -Wall -Werror -Wno-array-bounds 
 -ffunction-sections -fdata-sections -c -include AutoGen.h 
 -DSTRING_ARRAY_NAME=BasePeCoffLibStrings -m64 -fno-stack-protector 
 -DEFIAPI=__attribute__((ms_abi)) -DNO_BUILTIN_VA_FUNCS -mno-red-zone 
 -Wno-address -mcmodel=large -Wno-address -Wno-unused-but-set-variable 
 -mno-mmx -mno-sse -o 
 /home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/Build/OvmfX64/DEBUG_GCC47/X64/MdePkg/Library/BasePeCoffLib/BasePeCoffLib/OUTPUT/./PeCoffLoaderEx.obj
  
 -I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Library/BasePeCoffLib
  
 -I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/Build/OvmfX64/DEBUG_GCC47/X64/MdePkg/Library/BasePeCoffLib/BasePeCoffLib/DEBUG
  
 -I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg
  
 -I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Include
  -I/home/osstest/build.50324.build-
 i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Include/X64 
/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Library/BasePeCoffLib/PeCoffLoaderEx.c
 In file included from 
 /home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LzmaDecompress.c:15:0:
 /home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LzmaDecompressLibInternal.h:23:33:
  fatal error: Guid/LzmaDecompress.h: No such file or directory
 compilation terminated.
 

This is fixed upstream. The next run shall pass.

Wei.

 Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [ovmf test] 50324: regressions - trouble: blocked/broken/fail/pass

2015-04-07 Thread Ian Jackson
Wei Liu writes (Re: [ovmf test] 50324: regressions - trouble: 
blocked/broken/fail/pass):
 On Tue, Apr 07, 2015 at 02:56:59PM +0100, Ian Jackson wrote:
...
  /home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LzmaDecompressLibInternal.h:23:33:
   fatal error: Guid/LzmaDecompress.h: No such file or directory
  compilation terminated.
 
 This is fixed upstream. The next run shall pass.

Excellent, thanks.

(I'm keeping a bit of a closer eye on osstest right now as it has just
moved to the new test lab colo in Massachusetts.)

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [ovmf test] 50324: regressions - trouble: blocked/broken/fail/pass

2015-04-07 Thread Ian Jackson
osstest service user writes ([ovmf test] 50324: regressions - trouble: 
blocked/broken/fail/pass):
 flight 50324 ovmf real [real]
 http://logs.test-lab.xenproject.org/osstest/logs/50324/
 
 Regressions :-(
 
 Tests which did not succeed and are blocking,
 including tests which could not be run:
  build-i3865 xen-buildfail REGR. vs. 36764

This seems to be a real problem:

gcc -g -fshort-wchar -fno-strict-aliasing -Wall -Werror -Wno-array-bounds 
-ffunction-sections -fdata-sections -c -include AutoGen.h 
-DSTRING_ARRAY_NAME=BasePeCoffLibStrings -m64 -fno-stack-protector 
-DEFIAPI=__attribute__((ms_abi)) -DNO_BUILTIN_VA_FUNCS -mno-red-zone 
-Wno-address -mcmodel=large -Wno-address -Wno-unused-but-set-variable -mno-mmx 
-mno-sse -o 
/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/Build/OvmfX64/DEBUG_GCC47/X64/MdePkg/Library/BasePeCoffLib/BasePeCoffLib/OUTPUT/./PeCoffLoaderEx.obj
 
-I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Library/BasePeCoffLib
 
-I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/Build/OvmfX64/DEBUG_GCC47/X64/MdePkg/Library/BasePeCoffLib/BasePeCoffLib/DEBUG
 
-I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg
 
-I/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Include
 -I/home/osstest/build.50324.build-i3
 86/xen/tools/firmware/ovmf-dir-remote/MdePkg/Include/X64 
/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/MdePkg/Library/BasePeCoffLib/PeCoffLoaderEx.c
In file included from 
/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LzmaDecompress.c:15:0:
/home/osstest/build.50324.build-i386/xen/tools/firmware/ovmf-dir-remote/IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LzmaDecompressLibInternal.h:23:33:
 fatal error: Guid/LzmaDecompress.h: No such file or directory
compilation terminated.

Ian.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 50322: regressions - trouble: broken/fail/pass

2015-04-07 Thread Ian Jackson
osstest service user writes ([xen-unstable test] 50322: regressions - trouble: 
broken/fail/pass):
 flight 50322 xen-unstable real [real]
 http://logs.test-lab.xenproject.org/osstest/logs/50322/
 
 Regressions :-(
 
 Tests which did not succeed and are blocking,
 including tests which could not be run:
  test-amd64-amd64-pair 4 host-install/dst_host(4) broken REGR. vs. 36514

It appears that power cycling baroque1 for 5 seconds didn't cause it
to actually reboot.  The serial log shows it having been left in some
kind of crashed state by the previous test.  baroque1 appears to be
working now so I think this probably means that 5s is marginal for
rebooting.  I have changed the database to specify 15s instead for
this pair of machines.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [linux-next test] 50321: regressions - FAIL

2015-04-07 Thread Ian Jackson
osstest service user writes ([linux-next test] 50321: regressions - FAIL):
 flight 50321 linux-next real [real]
 http://logs.test-lab.xenproject.org/osstest/logs/50321/
 
 Regressions :-(
 
 Tests which did not succeed and are blocking,
 including tests which could not be run:
...
  test-armhf-armhf-xl   5 xen-boot   fail REGR. vs. 50276
...
  test-amd64-i386-xl5 xen-boot   fail REGR. vs. 50276

It appears that linux-next cannot boot as dom0 on i386 (32-bit x86) or
armhf (32-bit ARM).

The bugs don't seem to be related.


http://logs.test-lab.xenproject.org/osstest/logs/50321/test-amd64-i386-xl/info.html

http://logs.test-lab.xenproject.org/osstest/logs/50321/test-amd64-i386-xl/serial-fiano1.log
 (14:12:47 is when osstest timed out and started capturing logs,
  sending debug keys, etc.)


http://logs.test-lab.xenproject.org/osstest/logs/50321/test-armhf-armhf-xl/info.html

http://logs.test-lab.xenproject.org/osstest/logs/50321/test-armhf-armhf-xl/serial-arndale-lakeside.log
 (up to 13:17:35, likewise)

 version targeted for testing:
  linuxb0a12fb5bc87820b12df22c64dd680a96443de00

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxl new bitmap functions

2015-04-07 Thread Julien Grall
Hi Linda,

On 07/04/15 16:45, Linda wrote:
 +size = (bitmap1-size  bitmap2-size) ? bitmap1-size :
 bitmap2-size;
 There might be an 'max' macro somwhere in the code that you could use
 for this?
 Any guesses on where I might find the max macro.  When I grep max, all I
 find are definitions of constants that are the
 max of something.

There is a macro max_t in libxl_internal.h that can be used to get the
maximum of 2 numbers.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 2/5] sysctl: Add sysctl interface for querying PCI topology

2015-04-07 Thread Andrew Cooper
On 06/04/15 23:12, Boris Ostrovsky wrote:
 Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com
 ---

 Changes in v6:
 * Dropped continuations, the sysctl now returns after 64 iteration if 
 necessary

Why? The caller should reasonably expect a single hypercall to complete,
not to have to reissue repeated to get all the data they want.

 * -ENODEV returned if device is not found

You don't actually break on -ENODEV, which causes the loop to continue
to new devices and leave no hint as to which device doesn't exist.

~Andrew

 * sysctl's first_dev is now expected to be used by userspace to continue the 
 query
 * Added XSM hooks

  docs/misc/xsm-flask.txt |1 +
  xen/common/sysctl.c |   58 
 +++
  xen/include/public/sysctl.h |   30 ++
  xen/xsm/flask/hooks.c   |1 +
  xen/xsm/flask/policy/access_vectors |1 +
  5 files changed, 91 insertions(+), 0 deletions(-)

 diff --git a/docs/misc/xsm-flask.txt b/docs/misc/xsm-flask.txt
 index 90a2aef..4e0f14f 100644
 --- a/docs/misc/xsm-flask.txt
 +++ b/docs/misc/xsm-flask.txt
 @@ -121,6 +121,7 @@ __HYPERVISOR_sysctl (xen/include/public/sysctl.h)
   * XEN_SYSCTL_cpupool_op
   * XEN_SYSCTL_scheduler_op
   * XEN_SYSCTL_coverage_op
 + * XEN_SYSCTL_pcitopoinfo
  
  __HYPERVISOR_memory_op (xen/include/public/memory.h)
  
 diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c
 index d75440e..449ff70 100644
 --- a/xen/common/sysctl.c
 +++ b/xen/common/sysctl.c
 @@ -399,6 +399,64 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) 
 u_sysctl)
  break;
  #endif
  
 +#ifdef HAS_PCI
 +case XEN_SYSCTL_pcitopoinfo:
 +{
 +xen_sysctl_pcitopoinfo_t *ti = op-u.pcitopoinfo;
 +unsigned dev_cnt = 0;
 +
 +if ( guest_handle_is_null(ti-devs) ||
 + guest_handle_is_null(ti-nodes) ||
 + (ti-first_dev  ti-num_devs) )
 +{
 +ret = -EINVAL;
 +break;
 +}
 +
 +while ( ti-first_dev  ti-num_devs )
 +{
 +physdev_pci_device_t dev;
 +uint32_t node;
 +struct pci_dev *pdev;
 +
 +if ( copy_from_guest_offset(dev, ti-devs, ti-first_dev, 1) )
 +{
 +ret = -EFAULT;
 +break;
 +}
 +
 +spin_lock(pcidevs_lock);
 +pdev = pci_get_pdev(dev.seg, dev.bus, dev.devfn);
 +if ( !pdev )
 +{
 +ret = -ENODEV;
 +node = XEN_INVALID_NODE_ID;
 +}
 +else if ( pdev-node == NUMA_NO_NODE )
 +node = XEN_INVALID_NODE_ID;
 +else
 +node = pdev-node;
 +spin_unlock(pcidevs_lock);
 +
 +if ( copy_to_guest_offset(ti-nodes, ti-first_dev, node, 1) )
 +{
 +ret = -EFAULT;
 +break;
 +}
 +
 +ti-first_dev++;
 +
 +if ( (++dev_cnt  0x3f)  hypercall_preempt_check() )
 +break;
 +}
 +
 +if ( (ret != -EFAULT) 
 + __copy_field_to_guest(u_sysctl, op, u.pcitopoinfo.first_dev) )
 +ret = -EFAULT;
 +}
 +break;
 +#endif
 +
  default:
  ret = arch_do_sysctl(op, u_sysctl);
  copyback = 0;
 diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
 index 5aa3708..877b661 100644
 --- a/xen/include/public/sysctl.h
 +++ b/xen/include/public/sysctl.h
 @@ -33,6 +33,7 @@
  
  #include xen.h
  #include domctl.h
 +#include physdev.h
  
  #define XEN_SYSCTL_INTERFACE_VERSION 0x000C
  
 @@ -668,6 +669,33 @@ struct xen_sysctl_psr_cmt_op {
  typedef struct xen_sysctl_psr_cmt_op xen_sysctl_psr_cmt_op_t;
  DEFINE_XEN_GUEST_HANDLE(xen_sysctl_psr_cmt_op_t);
  
 +/* XEN_SYSCTL_pcitopoinfo */
 +struct xen_sysctl_pcitopoinfo {
 +/* IN: Number of elements in 'pcitopo' and 'nodes' arrays. */
 +uint32_t num_devs;
 +
 +/*
 + * IN/OUT:
 + *   IN: First element of pcitopo array that needs to be processed by
 + *   the hypervisor.
 + *  OUT: Index of the first still unprocessed element of pcitopo array.
 + */
 +uint32_t first_dev;
 +
 +/* IN: list of devices for which node IDs are requested. */
 +XEN_GUEST_HANDLE_64(physdev_pci_device_t) devs;
 +
 +/*
 + * OUT: node identifier for each device.
 + * If information for a particular device is not avalable then set
 + * to XEN_INVALID_NODE_ID. In addition, if device is not known to the
 + * hypervisor, sysctl will stop further processing and return -ENODEV.
 + */
 +XEN_GUEST_HANDLE_64(uint32) nodes;
 +};
 +typedef struct xen_sysctl_pcitopoinfo xen_sysctl_pcitopoinfo_t;
 +DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
 +
  struct xen_sysctl {
  uint32_t cmd;
  #define XEN_SYSCTL_readconsole1
 @@ -690,12 +718,14 @@ struct xen_sysctl {
  #define XEN_SYSCTL_scheduler_op  

Re: [Xen-devel] [PATCH v6 1/5] sysctl: Make XEN_SYSCTL_numainfo a little more efficient

2015-04-07 Thread Boris Ostrovsky

On 04/07/2015 12:04 PM, Andrew Cooper wrote:

On 06/04/15 23:12, Boris Ostrovsky wrote:

A number of changes to XEN_SYSCTL_numainfo interface:

* Make sysctl NUMA topology query use fewer copies by combining some
   fields into a single structure and copying distances for each node
   in a single copy.
* NULL meminfo and distance handles are a request for maximum number
   of nodes (num_nodes). If those handles are valid and num_nodes is
   is smaller than the number of nodes in the system then -ENOBUFS is
   returned (and correct num_nodes is provided)
* Instead of using max_node_index for passing number of nodes keep this
   value in num_nodes: almost all uses of max_node_index required adding
   or subtracting one to eventually get to number of nodes anyway.
* Replace INVALID_NUMAINFO_ID with XEN_INVALID_MEM_SZ and add
   XEN_INVALID_NODE_DIST.

Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com
Acked-by: Ian Campbell ian.campb...@citrix.com

This subtly changes the behaviour of XEN_SYSCTL_numainfo with regards to
NULL guest handles.

Previously, a caller was able to select which information they wanted by
choosing which guest handles were non-NULL.

With the new semantics, the caller must pass both ni-meminfo and
ni-distance to get either bit of information.  Each
copy_to_guest_offset() should be gated on a !guest_handle_is_null() so a
caller can request meminfo information without distance information.



Currently the caller, in fact, can have either of three pointers 
(node_to_memsize, node_to_memfree or node_to_node_distance) as NULL and 
the hypervisor will fill whichever pointer is valid. Because I put the 
first two together into a struct we are already changing behavior in 
that regard. Not to mention that having all three as NULL now has new 
meaning as well.


I thought that either both pointers should be valid or neither. If 
people disagree I can change this.


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 1/5] sysctl: Make XEN_SYSCTL_numainfo a little more efficient

2015-04-07 Thread Andrew Cooper
On 06/04/15 23:12, Boris Ostrovsky wrote:
 A number of changes to XEN_SYSCTL_numainfo interface:

 * Make sysctl NUMA topology query use fewer copies by combining some
   fields into a single structure and copying distances for each node
   in a single copy.
 * NULL meminfo and distance handles are a request for maximum number
   of nodes (num_nodes). If those handles are valid and num_nodes is
   is smaller than the number of nodes in the system then -ENOBUFS is
   returned (and correct num_nodes is provided)
 * Instead of using max_node_index for passing number of nodes keep this
   value in num_nodes: almost all uses of max_node_index required adding
   or subtracting one to eventually get to number of nodes anyway.
 * Replace INVALID_NUMAINFO_ID with XEN_INVALID_MEM_SZ and add
   XEN_INVALID_NODE_DIST.

 Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com
 Acked-by: Ian Campbell ian.campb...@citrix.com

This subtly changes the behaviour of XEN_SYSCTL_numainfo with regards to
NULL guest handles.

Previously, a caller was able to select which information they wanted by
choosing which guest handles were non-NULL.

With the new semantics, the caller must pass both ni-meminfo and
ni-distance to get either bit of information.  Each
copy_to_guest_offset() should be gated on a !guest_handle_is_null() so a
caller can request meminfo information without distance information.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxl new bitmap functions

2015-04-07 Thread Linda

Hey Konrad,

On 4/2/2015 1:34 PM, Konrad Rzeszutek Wilk wrote:

On Thu, Apr 02, 2015 at 11:38:16AM -0600, Linda Jacobson wrote:

From: Linda lin...@jma3.com

Added bitmap functions for union intersection and difference betweenn two 
bitmaps

Signed-off-by: Linda lin...@jma3.com
---
  tools/libxl/libxl_utils.c | 115 ++
  tools/libxl/libxl_utils.h |  10 
  2 files changed, 125 insertions(+)

diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 9053b27..c390ddc 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -699,6 +699,121 @@ int libxl_bitmap_count_set(const libxl_bitmap *bitmap)
  
  return nr_set_bits;

  }
+int libxl_bitmap_union(libxl_ctx *ctx, libxl_bitmap *union_bitmap,

You have an extra space at the end..

+libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)

And this should really start at the same line length as 'libxl_ctx'

Ditto for the rest of the functions.

+{
+int size;
+int rc;
+
+GC_INIT(ctx);
+
+// if bitmaps aren't the same size, union should be size of larger bit map

The comments are in:

  /* Comment here. */



+size = (bitmap1-size  bitmap2-size) ? bitmap1-size : bitmap2-size;

There might be an 'max' macro somwhere in the code that you could use
for this?
Any guesses on where I might find the max macro.  When I grep max, all I 
find are definitions of constants that are the

max of something.

Thanks.

Linda



+
+libxl_bitmap_init(union_bitmap);
+rc = libxl_bitmap_alloc(ctx, union_bitmap, size);
+if (rc)
+{
+// Following the coding standards here.
+   //First goto I've written in decades.

Hehe.

No need to add the braces, you can just do:

if (rc)
goto out;


+goto out;
+}
+
+for (int bit = 0; bit  (size * 8); bit++)

Please move the 'int bit' to the start of the function.


+{
+// libxl_bitmap_test returns 0 if past end of bitmap
+// if the bit is set in either bitmap, set it in their union

I would change that comment to be:

/* NB. libxl_bitmap_test returns 0 if past the end of bitmap. */


+if (libxl_bitmap_test(bitmap1, bit))
+{
+libxl_bitmap_set(union_bitmap, bit);
+}
+else if (libxl_bitmap_test(bitmap2, bit))
+{
+libxl_bitmap_set(union_bitmap, bit);
+}

You can ditch the braces.


+}
+
+out:
+GC_FREE;
+return rc;
+}
+
+int libxl_bitmap_intersection (libxl_ctx *ctx, libxl_bitmap

There is space :^ - which should not be there.


+*intersection_bitmap, libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)
+{
+int size;
+int rc;
+
+GC_INIT(ctx);
+
+// if bitmaps aren't same size, intersection should be size of
+// smaller bit map

I believe the comments I've above apply to the code below as well.


+size = (bitmap1-size  bitmap2-size) ? bitmap2-size : bitmap1-size;
+
+libxl_bitmap_init(intersection_bitmap);
+rc = libxl_bitmap_alloc(ctx, intersection_bitmap, size);
+if (rc)
+{
+goto out;
+}
+
+for (int bit = 0; bit  (size * 8); bit++)
+{
+// libxl_bitmap_test returns 0 if past end of bitmap
+// if the bit is set in both bitmaps, set it in their intersection
+if (libxl_bitmap_test (bitmap1, bit) 
+libxl_bitmap_test (bitmap2, bit) )

You have an extra space at the end there. Don't think you need that
in libxl code (thought you do for libxc). Yeah, two different
styleguides!


+{
+libxl_bitmap_set (intersection_bitmap, bit);
+}
+}
+
+out:
+GC_FREE;
+return rc;
+}
+int libxl_bitmap_difference(libxl_ctx *ctx, libxl_bitmap *difference_bitmap,
+libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)
+{
+int size;
+int rc;
+
+GC_INIT(ctx);
+
+// if bitmaps aren't the same size, difference should be size of larger
+size = (bitmap1-size  bitmap2-size) ? bitmap1-size : bitmap2-size;
+
+libxl_bitmap_init(difference_bitmap);
+rc = libxl_bitmap_alloc(ctx, difference_bitmap, size);
+if (rc)
+{
+goto out;
+}
+
+for (int bit = 0; bit  (size * 8); bit++)
+{
+/* libxl_bitmap_test returns 0 if past end of bitmap
+ if the bit is set in one bitmap and not the other, set it in
+their difference
+NOTE:  if one bit map is larger, this will result in all bits
+being set past the size of the smaller bitmap;  if this is not
+the desired behavior, please let me know

That would make this a bit strange. Perhaps demand that they
must be the same size? If they are not then just return an error?

+*/
+
+if (libxl_bitmap_test (bitmap1, bit)

You have an extra space at the end there.  ^

+(!libxl_bitmap_test (bitmap2, bit)) )
+{
+libxl_bitmap_set (difference_bitmap, bit);
+}
+}
+
+out:
+GC_FREE;
+return rc;
+}
+
+
+
  
  /* NB. caller is responsible 

Re: [Xen-devel] [PATCH] x86/hvm: Fix the unknown nested vmexit reason 80000021 bug

2015-04-07 Thread Andrew Cooper
On 07/04/15 14:27, Liang Li wrote:
 This bug will be trigged when NMI happen in the L2 guest. The current
 code handles the NMI incorrectly. According to Intel SDM 31.7.1.2
 (Resuming Guest Software after Handling an Exception), If bit 31 of the
 IDT-vectoring information fields is set, and the virtual NMIs VM-execution
 control is 1, while bits 10:8 in the IDT-vectoring information field is
 2, bit 3 in the interruptibility-state field should be cleared to avoid
 the next VM entry fail.

 Signed-off-by: Liang Li liang.z...@intel.com
 ---
  xen/arch/x86/hvm/vmx/vmx.c | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)

 diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
 index e1c55ce..b1f2df8 100644
 --- a/xen/arch/x86/hvm/vmx/vmx.c
 +++ b/xen/arch/x86/hvm/vmx/vmx.c
 @@ -2621,7 +2621,8 @@ static void vmx_idtv_reinject(unsigned long idtv_info)
   * Clear NMI-blocking interruptibility info if an NMI delivery 
 faulted.
   * Re-delivery will re-set it (see SDM 3B 25.7.1.2).
   */
 -if ( (idtv_info  INTR_INFO_INTR_TYPE_MASK) == 
 (X86_EVENTTYPE_NMI8) )
 +if ( cpu_has_vmx_vnmi  ((idtv_info  INTR_INFO_INTR_TYPE_MASK) ==
 + (X86_EVENTTYPE_NMI8)) )

This would be easier to read as

if ( cpu_has_vmx_vnmi 
 (idtv_info  INTR_INFO_INTR_TYPE_MASK) == (X86_EVENTTYPE_NMI8)) )

~Andrew

  {
  unsigned long intr_info;
  
 @@ -2772,8 +2773,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
  hvm_maybe_deassert_evtchn_irq();
  
  __vmread(IDT_VECTORING_INFO, idtv_info);
 -if ( !nestedhvm_vcpu_in_guestmode(v)  
 - exit_reason != EXIT_REASON_TASK_SWITCH )
 +if ( exit_reason != EXIT_REASON_TASK_SWITCH )
  vmx_idtv_reinject(idtv_info);
  
  switch ( exit_reason )


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Nested Virt - Xen 4.4 through 4.6 - Hyper-V; Can't boot after enabling Hyper-V

2015-04-07 Thread Andrew Cooper
On 07/04/15 02:42, mailing lists wrote:
 Hi --

 I've been trying to get nested virtualization working with Xen so that
 I could boot Windows and use Hyper-V related features, however I have
 not had much success.  Using Windows 8.1 or Windows 2012r2, I'm able
 to install Windows, select and install Hyper-V features, and start
 rebooting.  However, at that point, the Windows VM only partially
 boots, then drops me to a screen stating:

 Your PC needs to restart.
 Please hold down the power button.
 Error Code: 0x001E
 Parameters:
 0xC096
 0xF80315430485
 0x
 0x


 Restarting does not yield any different results.

 I've set up Xen in accordance with the notes for patches and config
 options here:

 http://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

 Trying Xen 4.4.2 stable, 4.5.1 staging, and 4.6 staging.  I applied
 the patch labeled (2/2) from the wiki link above, compiled, and used
 the three options provided for the DomU running Windows (hap,
 nestedhvm, and cpuid mask).  Windows installs and allows me to turn on
 HyperV features on all versions of Xen listed above, however all give
 the same or similar message on reboot... I'm never able to get to a
 running state.

 I've tried this on two separate systems.  One has an Intel E5-1620 v2,
 and the other is a n E5-1650 (original, v1 I guess).  All the
 virtualization options are enabled in the BIOS.

 If the cpuid mask is removed from the DomU config, Windows boots,
 however I'm unable to start any virtual machines (there was a message
 in the Windows event log about a component not being started in
 regards to Hyper V).

 Has anyone else run into similar issues?  Any thoughts on next steps?

I am not aware of anyone who has successfully got a setup like this to work.

From
https://msdn.microsoft.com/en-us/library/windows/hardware/ff557408%28v=vs.85%29.aspx

0x1E is KMODE_EXCEPTION_NOT_HANDLED.

http://source.winehq.org/source/include/ntstatus.h suggests that
0xC096 is STATUS_PRIVILEGED_INSTRUCTION.

Your best bet for debugging this is to debug the minidump generated and
see which driver 0xF80315430485 is a part of, and perhaps exactly
what instruction 0xF80315430485 actually is.

~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 2/5] sysctl: Add sysctl interface for querying PCI topology

2015-04-07 Thread Boris Ostrovsky

On 04/07/2015 12:09 PM, Andrew Cooper wrote:

On 06/04/15 23:12, Boris Ostrovsky wrote:

Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com
---

Changes in v6:
* Dropped continuations, the sysctl now returns after 64 iteration if necessary

Why? The caller should reasonably expect a single hypercall to complete,
not to have to reissue repeated to get all the data they want.


The caller is expected to restart from first_dev, not from beginning.




* -ENODEV returned if device is not found

You don't actually break on -ENODEV, which causes the loop to continue
to new devices and leave no hint as to which device doesn't exist.


Doh, I must have lost the break during various merges.

-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Issue With Patch Compilation Fails ( xen/arm: Introduce a generic way to describe device) with HAS_PCI and HAS_PASSTHROUGH.

2015-04-07 Thread Stefano Stabellini
On Tue, 7 Apr 2015, Jaggi, Manish wrote:
 Hi Julien,
 
 
 Following patch generated compiler error when HAS_PCI adn HAS_PASSTHROUGH 
 enabled.
 Please advice how to fix this issue, or you can revert this patch.
 Should I add a device structure in pci_dev or there is another way.

Hello Manish,

we have never really built Xen on ARM with HAS_PCI=y so it is normal
that it won't compile out of the box, it is not just a problem caused by
the commit below.  I imagine that you'll need to do more than setting
HAS_PCI to y in order to get PCI and PCI passthrough working properly
with Xen on ARM.  Feel free to go ahead and propose any changes
necessary.

Cheers,

Stefano


     xen/arm: Introduce a generic way to describe device
     
     Currently, Xen is supporting PCI and Platform device (based on Device 
 Tree).
    
     While Xen only supports Platform device on ARM, Xen will gain support of
     PCI soon.
     
     Some drivers, such as IOMMU drivers, may handle PCI and platform device in
     the same way. Only few lines of code differs.
     
     Rather than requesting to provide 2 set of functions (one for PCI and
     one for platform device), introduce a generic structure device which
     is embedded in each specialized device.
     
     As x86 only supports PCI, introduce a new type device_t which will be an
     alias to pci_dev for this architecture. It will avoid to add a new field
     for this place.
     
     Signed-off-by: Julien Grall julien.gr...@linaro.org
     Acked-by: Jan Beulich jbeul...@suse.com
     Acked-by: Stefano Stabellini stefano.stabell...@eu.citrix.com
     CC: Keir Fraser k...@xen.org
     CC: Andrew Cooper andrew.coop...@citrix.com
 
 
 
 Compilation error
 pci.c: In function ‘iommu_add_device’:
 pci.c:1263:5: error: implicit declaration of function ‘pci_to_dev’ 
 [-Werror=implicit-function-declaration]
 pci.c:1263:5: error: nested extern declaration of ‘pci_to_dev’ 
 [-Werror=nested-externs]
 pci.c:1263:5: error: passing argument 2 of ‘hd-platform_ops-add_device’ 
 makes pointer from integer without a cast [-Werror]
 pci.c:1263:5: note: expected ‘struct device_t *’ but argument is of type ‘int’
 pci.c:1272:9: error: passing argument 2 of ‘hd-platform_ops-add_device’ 
 makes pointer from integer without a cast [-Werror]
 pci.c:1272:9: note: expected ‘struct device_t *’ but argument is of type ‘int’
 
 
 Regards,
 Manish Jaggi  ___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v2 00/29] libxl: Cancelling asynchronous operations

2015-04-07 Thread Euan Harris
Hi,

On Wed, Feb 18, 2015 at 04:10:35PM +, Euan Harris wrote:
 We had a chat about testing these changes, and integrating them into xenopsd.
 We agreed that we each had slightly different expectations of what we were 
 going to do, and when.   I think we came to the following major conclusions:
 
   - I will start work on a simple test framework for cancellation,
 hopefully to have first results in a fortnight or so.
   - Once the test framework is available you will fix whatever bugs it
 unearths, then we will rinse and repeat.
   - You will think some more about the possibility of adding cancellation
 to the xl command line tool, but since this is tricky there is no 
 expectation of when it might happen.

I think the most straightforward way to test the cancellation mechanism in
LibXL will be to adapt the way we test similar functionality in xenopsd:

   * define numbered 'cancellation points' at which cancellable operations
 can be cancelled
   * before testing a cancellable operation, pre-set the cancellation point
 at which cancellation should be attempted
   * when execution reaches the pre-set cancellation point, run the cancellation
 procedure

This approach alone will not allow us to test asynchronous cancellation in
the middle of long-running operations, such as writing a suspend image
to disk - that will require a way to synchronize the test program with
the long-running operation.

My first guess about how this might be done was:

   * add current cancellation point and a trigger point variables to the context
 struct
   * increment the counter and fire the cancellation logic in
 libxl__ao_cancellable_register()

In this way we could write a loop which iterated through all possible
cancellation points.   However you pointed out that we cannot call
libxl_ao_cancel() while holding the context lock, so this idea needs
some refinement.   One possibility would be to tell another thread to try
to do the cancellation immediately after we release the lock;  another
option, if we didn't want to write a multi-thread test driver,
would be to do the cancellation at the top of libxl's event loop.

I think this captures roughly what we talked about.   Please let me know
if I misunderstood or missed out any details.

Thanks,
Euan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] tg3 NIC driver bug in 3.14.x under Xen

2015-04-07 Thread Michael Chan
On Tue, 2015-04-07 at 16:12 +0100, Ian Jackson wrote:
 The symptom is a very high level of packet loss: around 25-30% (as
 seen in `ping').  There don't seem to be any untoward-looking kernel
 messages.  The lost packets get added to the `errors' counter shown in
 ifconfig. 

Please provide the output of ethtool -S which has a better breakdown
of the error counters.  Thanks.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v2 00/29] libxl: Cancelling asynchronous operations

2015-04-07 Thread Ian Jackson
Ian Campbell writes (Re: [Xen-devel] [RFC PATCH v2 00/29] libxl: Cancelling 
asynchronous operations):
 On Tue, 2015-02-10 at 20:09 +, Ian Jackson wrote:
  This is v2 of my work-in-progress series to support cancellation of
  long-running libxl operations.
 [...]
  I wouldn't recommend testing it yet until I've at least smoke tested
  it to see that things still work if you don't cancel them.
 
 Would review of the series be useful and/or appreciated at this stage?

Review of the APIs, and general approach, would be very much
appreciated.  That's probably best done by looking at the tip and
diffing libxl[_internal].h.

 Perhaps the first half dozen or so look like preparatory cleanups which
 I could sensibly look at?

That would also be useful.

  Here's a list of the patches:
  
01/29  libxl: Further fix exit paths from libxl_device_events_handler
02/29  libxl: Comment cleanups
03/29  libxl: suspend: switch_logdirty_done takes rc
04/29  libxl: suspend: common suspend callbacks take rc
05/29  libxl: suspend: Return correct error from callbacks
06/29  libxl: Use libxl__xswait* in libxl__ao_device
07/29  libxl: xswait/devstate: Move xswait to before devstate
08/29  libxl: devstate: Use libxl__xswait*

These first eight are cleanups and could in principle go in right
away.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v2 00/29] libxl: Cancelling asynchronous operations

2015-04-07 Thread Ian Jackson
Euan Harris writes (Re: [RFC PATCH v2 00/29] libxl: Cancelling asynchronous 
operations):
 On Wed, Feb 18, 2015 at 04:10:35PM +, Euan Harris wrote:
 I think the most straightforward way to test the cancellation mechanism in
 LibXL will be to adapt the way we test similar functionality in xenopsd:
 
* define numbered 'cancellation points' at which cancellable operations
  can be cancelled
* before testing a cancellable operation, pre-set the cancellation point
  at which cancellation should be attempted
* when execution reaches the pre-set cancellation point, run the 
 cancellation
  procedure

This seems likely to work.

 This approach alone will not allow us to test asynchronous cancellation in
 the middle of long-running operations, such as writing a suspend image
 to disk - that will require a way to synchronize the test program with
 the long-running operation.

On the contrary, I think many long-running operations, such as suspend
and migrations, involve multiple iterations of the libxl event loop.
Actual suspend/migrate is done in a helper process; the main process
is responsible for progress report handling, coordination, etc.

 My first guess about how this might be done was:
 
* add current cancellation point and a trigger point variables to the 
 context
  struct
* increment the counter and fire the cancellation logic in
  libxl__ao_cancellable_register()
 
 In this way we could write a loop which iterated through all possible
 cancellation points.   However you pointed out that we cannot call
 libxl_ao_cancel() while holding the context lock, so this idea needs
 some refinement.   One possibility would be to tell another thread to try
 to do the cancellation immediately after we release the lock;  another
 option, if we didn't want to write a multi-thread test driver,
 would be to do the cancellation at the top of libxl's event loop.

The relevant function for this latter approach is eventloop_iteration
in libxl_event.c.  This is used by libxl whenever the caller specifies
that a long-running operation is to be done synchronously (ao_how==0),
which is what xl does.

You might also consider whether to add a debug option for
afterpoll_internal to make it return after every callback (ie, after
the call to efd-func() and the call to time_occurs).  That would
allow you to inject cancellation in a slightly more fine-grained
manner.

 I think this captures roughly what we talked about.   Please let me know
 if I misunderstood or missed out any details.

I also mentioned that you counting invocations of
libxl__ao_cancellable_register is less than ideal because it is very
coarse-grained.

Regards,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 6/6] x86/boot: Ensure the BSS is aligned on an 8 byte boundary

2015-04-07 Thread Andrew Cooper
... and zero it more efficiently.

A diff of the resulting binaries shows that the size of the BSS doesn't
actually change from the extra ALIGN()s, but they do guarantee that it is safe
to clear in an more efficient manner than 1 byte at a time.

Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Keir Fraser k...@xen.org
CC: Jan Beulich jbeul...@suse.com
---
 xen/arch/x86/boot/head.S |3 ++-
 xen/arch/x86/xen.lds.S   |2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 2d0e56c..b5a1d45 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -127,7 +127,8 @@ __start:
 mov $sym_phys(__bss_end),%ecx
 sub %edi,%ecx
 xor %eax,%eax
-rep stosb
+shr $2,%ecx
+rep stosl
 
 /* Interrogate CPU extended features via CPUID. */
 mov $0x8000,%eax
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 4699a04..b1926e3 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -163,6 +163,7 @@ SECTIONS
   __init_end = .;
 
   .bss : { /* BSS */
+   . = ALIGN(8);
__bss_start = .;
*(.bss.stack_aligned)
. = ALIGN(PAGE_SIZE);
@@ -175,6 +176,7 @@ SECTIONS
*(.bss.percpu.read_mostly)
. = ALIGN(SMP_CACHE_BYTES);
__per_cpu_data_end = .;
+   . = ALIGN(8);
__bss_end = .;
   } :text
   _end = . ;
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 3/6] x86/smp: Clean up use of memflags in cpu_smpboot_alloc()

2015-04-07 Thread Andrew Cooper
Hoist MEMF_node(cpu_to_node(cpu)) to the start of the function, and avoid
passing (potentially bogus) memflags if node information is not available.

Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Keir Fraser k...@xen.org
CC: Jan Beulich jbeul...@suse.com
---
 xen/arch/x86/smpboot.c |   14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index d3fe116..a009e91 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -635,32 +635,34 @@ static void cpu_smpboot_free(unsigned int cpu)
 
 static int cpu_smpboot_alloc(unsigned int cpu)
 {
-unsigned int order;
+unsigned int order, memflags = 0;
+nodeid_t node = cpu_to_node(cpu);
 struct desc_struct *gdt;
 
+if ( node != NUMA_NO_NODE )
+memflags = MEMF_node(node);
+
 stack_base[cpu] = alloc_xenheap_pages(STACK_ORDER, 0);
 if ( stack_base[cpu] == NULL )
 goto oom;
 memguard_guard_stack(stack_base[cpu]);
 
 order = get_order_from_pages(NR_RESERVED_GDT_PAGES);
-per_cpu(gdt_table, cpu) = gdt =
-alloc_xenheap_pages(order, MEMF_node(cpu_to_node(cpu)));
+per_cpu(gdt_table, cpu) = gdt = alloc_xenheap_pages(order, memflags);
 if ( gdt == NULL )
 goto oom;
 memcpy(gdt, boot_cpu_gdt_table, NR_RESERVED_GDT_PAGES * PAGE_SIZE);
 BUILD_BUG_ON(NR_CPUS  0x1);
 gdt[PER_CPU_GDT_ENTRY - FIRST_RESERVED_GDT_ENTRY].a = cpu;
 
-per_cpu(compat_gdt_table, cpu) = gdt =
-alloc_xenheap_pages(order, MEMF_node(cpu_to_node(cpu)));
+per_cpu(compat_gdt_table, cpu) = gdt = alloc_xenheap_pages(order, 
memflags);
 if ( gdt == NULL )
 goto oom;
 memcpy(gdt, boot_cpu_compat_gdt_table, NR_RESERVED_GDT_PAGES * PAGE_SIZE);
 gdt[PER_CPU_GDT_ENTRY - FIRST_RESERVED_GDT_ENTRY].a = cpu;
 
 order = get_order_from_bytes(IDT_ENTRIES * sizeof(idt_entry_t));
-idt_tables[cpu] = alloc_xenheap_pages(order, MEMF_node(cpu_to_node(cpu)));
+idt_tables[cpu] = alloc_xenheap_pages(order, memflags);
 if ( idt_tables[cpu] == NULL )
 goto oom;
 memcpy(idt_tables[cpu], idt_table, IDT_ENTRIES * sizeof(idt_entry_t));
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] tg3 NIC driver bug in 3.14.x under Xen

2015-04-07 Thread Ian Jackson
Ian Jackson writes (Re: tg3 NIC driver bug in 3.14.x under Xen):
 Evidently on this particular kernel, the error counters are _not_
 increasing, contrary to what I said before.  I confess that I didn't
 keep a record of on which particular machine and kernel I observed the
 error count increasing.
 
 If it would help I could try to check various other machines and/or
 other kernels to see if I can get one of them to display the error
 counter behaviour.

I just realised that I didn't check xenbr0 as well as eth0:

root@bedbug:~# ifconfig xenbr0
xenbr0Link encap:Ethernet  HWaddr 00:13:72:14:c0:51  
  inet addr:10.80.249.102  Bcast:10.80.251.255
  Mask:255.255.252.0
  inet6 addr: fe80::213:72ff:fe14:c051/64 Scope:Link
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:12296 errors:0 dropped:793 overruns:0 frame:0
  TX packets:437 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:0 
  RX bytes:618757 (604.2 KiB)  TX bytes:49084 (47.9 KiB)

root@bedbug:~# ethtool -S xenbr0 | grep -v ': 0$'
no stats available
root@bedbug:~#

The value for dropped increases steadily.  This particular box is on
a network with a lot of other stuff, so it will be constantly
receiving broadcasts of various kinds even when I am not trying to
address it directly.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 4/6] x86/link: Introduce and use __bss_end

2015-04-07 Thread Andrew Cooper
No functional change.

Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Keir Fraser k...@xen.org
CC: Jan Beulich jbeul...@suse.com
---
 xen/arch/x86/boot/head.S |2 +-
 xen/arch/x86/setup.c |4 ++--
 xen/arch/x86/tboot.c |4 ++--
 xen/arch/x86/xen.lds.S   |1 +
 4 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index c99f739..2d0e56c 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -124,7 +124,7 @@ __start:
 
 /* Initialize BSS (no nasty surprises!) */
 mov $sym_phys(__bss_start),%edi
-mov $sym_phys(_end),%ecx
+mov $sym_phys(__bss_end),%ecx
 sub %edi,%ecx
 xor %eax,%eax
 rep stosb
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 89e4827..2b9787a 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -185,7 +185,7 @@ static void free_xen_data(char *s, char *e)
 memguard_guard_range(__va(__pa(s)), e-s);
 }
 
-extern char __init_begin[], __init_end[], __bss_start[];
+extern char __init_begin[], __init_end[], __bss_start[], __bss_end[];
 
 static void __init init_idle_domain(void)
 {
@@ -1497,7 +1497,7 @@ int __hwdom_init xen_in_range(unsigned long mfn)
 xen_regions[region_text].e = __pa(__init_begin);
 /* bss */
 xen_regions[region_bss].s = __pa(__bss_start);
-xen_regions[region_bss].e = __pa(_end);
+xen_regions[region_bss].e = __pa(__bss_end);
 }
 
 start = (paddr_t)mfn  PAGE_SHIFT;
diff --git a/xen/arch/x86/tboot.c b/xen/arch/x86/tboot.c
index 7b95ad3..01b9530 100644
--- a/xen/arch/x86/tboot.c
+++ b/xen/arch/x86/tboot.c
@@ -48,7 +48,7 @@
 #define TXTCR_HEAP_BASE 0x0300
 #define TXTCR_HEAP_SIZE 0x0308
 
-extern char __init_begin[], __bss_start[];
+extern char __init_begin[], __bss_start[], __bss_end[];
 
 #define SHA1_SIZE  20
 typedef uint8_t   sha1_hash_t[SHA1_SIZE];
@@ -374,7 +374,7 @@ void tboot_shutdown(uint32_t shutdown_type)
   __pa(_stext);
 /* bss */
 g_tboot_shared-mac_regions[2].start = (uint64_t)__pa(__bss_start);
-g_tboot_shared-mac_regions[2].size = __pa(_end) - __pa(__bss_start);
+g_tboot_shared-mac_regions[2].size = __pa(__bss_end) - 
__pa(__bss_start);
 
 /*
  * MAC domains and other Xen memory
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index c854181..4699a04 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -175,6 +175,7 @@ SECTIONS
*(.bss.percpu.read_mostly)
. = ALIGN(SMP_CACHE_BYTES);
__per_cpu_data_end = .;
+   __bss_end = .;
   } :text
   _end = . ;
 
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 1/6] x86/link: Discard the alternatives .discard sections

2015-04-07 Thread Andrew Cooper
This appears to have been missed when porting the alternatives framework from
Linux, and saves us a section which is otherwise loaded into memory.

Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Keir Fraser k...@xen.org
CC: Jan Beulich jbeul...@suse.com
---
 xen/arch/x86/xen.lds.S |2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index d4b1f1a..c854181 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -197,6 +197,8 @@ SECTIONS
*(.exit.text)
*(.exit.data)
*(.exitcall.exit)
+   *(.discard)
+   *(.discard.*)
*(.eh_frame)
 #ifdef EFI
*(.comment)
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 2/3] arm, arm64/xen: move Xen initialization earlier

2015-04-07 Thread Stefano Stabellini
Russell,

I think that this patch should go via the Xen tree, however as it
touches arch/arm/kernel/setup.c, I would like your Acked-by before
adding it to the Xen queue.

Thanks,

Stefano


On Wed, 18 Feb 2015, Julien Grall wrote:
 From: Stefano Stabellini stefano.stabell...@eu.citrix.com
 
 Currently, Xen is initialized/discovered in an initcall. This doesn't
 allow us to support earlyprintk or choosing the preferred console when
 running on Xen.
 
 The current function xen_guest_init is now split in 2 parts:
 - xen_early_init: Check if there is a Xen node in the device tree
 and setup domain type
 - xen_guest_init: Retrieve the information from the device node and
 initialize Xen (grant table, shared page...)
 
 The former is called in setup_arch, while the latter is an initcall.
 
 Signed-off-by: Stefano Stabellini stefano.stabell...@eu.citrix.com
 Signed-off-by: Julien Grall julien.gr...@linaro.org
 Acked-by: Ian Campbell ian.campb...@citrix.com
 Cc: Russell King li...@arm.linux.org.uk
 Cc: Catalin Marinas catalin.mari...@arm.com
 Cc: Will Deacon will.dea...@arm.com
 
 ---
 It's based on a patch sent by Stefano nearly 2 years ago [1].
 
 [1] http://lists.xen.org/archives/html/xen-devel/2013-08/msg02960.html
 
 Changes in v2:
 - Add Ian's ack
 ---
  arch/arm/include/asm/xen/hypervisor.h |  8 +
  arch/arm/kernel/setup.c   |  2 ++
  arch/arm/xen/enlighten.c  | 58 
 ---
  arch/arm64/kernel/setup.c |  2 ++
  4 files changed, 46 insertions(+), 24 deletions(-)
 
 diff --git a/arch/arm/include/asm/xen/hypervisor.h 
 b/arch/arm/include/asm/xen/hypervisor.h
 index 1317ee4..04ff8e7 100644
 --- a/arch/arm/include/asm/xen/hypervisor.h
 +++ b/arch/arm/include/asm/xen/hypervisor.h
 @@ -1,6 +1,8 @@
  #ifndef _ASM_ARM_XEN_HYPERVISOR_H
  #define _ASM_ARM_XEN_HYPERVISOR_H
  
 +#include linux/init.h
 +
  extern struct shared_info *HYPERVISOR_shared_info;
  extern struct start_info *xen_start_info;
  
 @@ -18,4 +20,10 @@ static inline enum paravirt_lazy_mode 
 paravirt_get_lazy_mode(void)
  
  extern struct dma_map_ops *xen_dma_ops;
  
 +#ifdef CONFIG_XEN
 +void __init xen_early_init(void);
 +#else
 +static inline void xen_early_init(void) { return; }
 +#endif
 +
  #endif /* _ASM_ARM_XEN_HYPERVISOR_H */
 diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
 index e55408e..8b59d0d 100644
 --- a/arch/arm/kernel/setup.c
 +++ b/arch/arm/kernel/setup.c
 @@ -46,6 +46,7 @@
  #include asm/cacheflush.h
  #include asm/cachetype.h
  #include asm/tlbflush.h
 +#include asm/xen/hypervisor.h
  
  #include asm/prom.h
  #include asm/mach/arch.h
 @@ -936,6 +937,7 @@ void __init setup_arch(char **cmdline_p)
  
   arm_dt_init_cpu_maps();
   psci_init();
 + xen_early_init();
  #ifdef CONFIG_SMP
   if (is_smp()) {
   if (!mdesc-smp_init || !mdesc-smp_init()) {
 diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
 index c8d3a17..1660432 100644
 --- a/arch/arm/xen/enlighten.c
 +++ b/arch/arm/xen/enlighten.c
 @@ -53,6 +53,8 @@ EXPORT_SYMBOL_GPL(xen_platform_pci_unplug);
  
  static __read_mostly unsigned int xen_events_irq;
  
 +static __initdata struct device_node *xen_node;
 +
  /* map fgmfn of domid to lpfn in the current domain */
  static int map_foreign_page(unsigned long lpfn, unsigned long fgmfn,
   unsigned int domid)
 @@ -222,42 +224,28 @@ static irqreturn_t xen_arm_callback(int irq, void *arg)
   * documentation of the Xen Device Tree format.
   */
  #define GRANT_TABLE_PHYSADDR 0
 -static int __init xen_guest_init(void)
 +void __init xen_early_init(void)
  {
 - struct xen_add_to_physmap xatp;
 - static struct shared_info *shared_info_page = 0;
 - struct device_node *node;
   int len;
   const char *s = NULL;
   const char *version = NULL;
   const char *xen_prefix = xen,xen-;
 - struct resource res;
 - phys_addr_t grant_frames;
  
 - node = of_find_compatible_node(NULL, NULL, xen,xen);
 - if (!node) {
 + xen_node = of_find_compatible_node(NULL, NULL, xen,xen);
 + if (!xen_node) {
   pr_debug(No Xen support\n);
 - return 0;
 + return;
   }
 - s = of_get_property(node, compatible, len);
 + s = of_get_property(xen_node, compatible, len);
   if (strlen(xen_prefix) + 3   len 
   !strncmp(xen_prefix, s, strlen(xen_prefix)))
   version = s + strlen(xen_prefix);
   if (version == NULL) {
   pr_debug(Xen version not found\n);
 - return 0;
 - }
 - if (of_address_to_resource(node, GRANT_TABLE_PHYSADDR, res))
 - return 0;
 - grant_frames = res.start;
 - xen_events_irq = irq_of_parse_and_map(node, 0);
 - if (!xen_events_irq) {
 - pr_debug(Xen event channel interrupt not found\n);
 - return -ENODEV;
 + return;
   }
  
 - pr_info(Xen 

Re: [Xen-devel] [PATCH v6 2/5] sysctl: Add sysctl interface for querying PCI topology

2015-04-07 Thread Daniel De Graaf

On 04/06/2015 06:12 PM, Boris Ostrovsky wrote:

Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com.


Acked-by: Daniel De Graaf dgde...@tycho.nsa.gov

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 1/5] sysctl: Make XEN_SYSCTL_numainfo a little more efficient

2015-04-07 Thread Boris Ostrovsky

On 04/07/2015 01:04 PM, Andrew Cooper wrote:

On 07/04/15 17:57, Boris Ostrovsky wrote:

On 04/07/2015 12:04 PM, Andrew Cooper wrote:

On 06/04/15 23:12, Boris Ostrovsky wrote:

A number of changes to XEN_SYSCTL_numainfo interface:

* Make sysctl NUMA topology query use fewer copies by combining some
fields into a single structure and copying distances for each node
in a single copy.
* NULL meminfo and distance handles are a request for maximum number
of nodes (num_nodes). If those handles are valid and num_nodes is
is smaller than the number of nodes in the system then -ENOBUFS is
returned (and correct num_nodes is provided)
* Instead of using max_node_index for passing number of nodes keep this
value in num_nodes: almost all uses of max_node_index required
adding
or subtracting one to eventually get to number of nodes anyway.
* Replace INVALID_NUMAINFO_ID with XEN_INVALID_MEM_SZ and add
XEN_INVALID_NODE_DIST.

Signed-off-by: Boris Ostrovsky boris.ostrov...@oracle.com
Acked-by: Ian Campbell ian.campb...@citrix.com

This subtly changes the behaviour of XEN_SYSCTL_numainfo with regards to
NULL guest handles.

Previously, a caller was able to select which information they wanted by
choosing which guest handles were non-NULL.

With the new semantics, the caller must pass both ni-meminfo and
ni-distance to get either bit of information.  Each
copy_to_guest_offset() should be gated on a !guest_handle_is_null() so a
caller can request meminfo information without distance information.


Currently the caller, in fact, can have either of three pointers
(node_to_memsize, node_to_memfree or node_to_node_distance) as NULL
and the hypervisor will fill whichever pointer is valid. Because I put
the first two together into a struct we are already changing behavior
in that regard. Not to mention that having all three as NULL now has
new meaning as well.

I thought that either both pointers should be valid or neither. If
people disagree I can change this.

I have a usecase (hwloc) which wants meminfo but not distance.  In such
a case, the inner for loop can be skipped which is an advantage for Xen.


Are you planning on calling sysctls directly from your hwloc extension? 
Why not go via libxl which is a stable interface AFAIUI.



-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 2/2] vtpmmgr: execute deep quote in locality 0

2015-04-07 Thread Daniel De Graaf

On 04/07/2015 03:12 AM, Emil Condrea wrote:

On Mon, Apr 6, 2015 at 6:49 PM, Daniel De Graaf dgde...@tycho.nsa.gov
wrote:


On 04/05/2015 07:09 AM, Emil Condrea wrote:


Enables deep quote execution for vtpmmgr which can not be started
using locality 2. The VTPM_ORD_GET_QUOTE command is backwards
compatible. When receives flags=0 from vTPM it extends additional
information into the PCRs as it did before.



Even without extending values into the resettable PCRs, the PCR
selection for the quote should not be ignored: otherwise, the
measurement of the hypervisor (stored in either PCR 4-5 or PCR 18-19)
cannot be included in a deep quote.  Allowing a guest to attest to the
measurement of its hypervisor is one of the major reasons for allowing a
deep quote and is likely to be mandatory in a vTPM provisioning quote.


very good point, I missed that someone would include additional PCRs in the
request.



The backwards compatibility added here has some caveats that need to be
treated carefully.  In the current version, it is not possible to
distinguish between a request that has the new externData structure and
a request that uses flags=0 where the vTPM itself has constructed a
requestData value that looks like a valid externData.

The simplest solution is to always use the structure prepared by the
vTPM manager for externData and remove backwards compatibility from the
series.  Since the new quote format would be introduced with a new
version of the vTPM Manager and (likely) a new hypervisor, any support
software that makes use of the quotes can be updated at the same time.

If backwards compatibility is needed, then there must be a way to
distinguish a new-format quote from an old-format quote.  If the reset
of the modified PCRs is moved to the end of the quote operation (instead
of the beginning, as is currently done), the value of PCR 20 will be
at its reset value (either 00..00 or ff..ff) when the new-format quote
is requested.  Including the value of PCR 20 in all new-format quotes
would then be sufficient evidence that the externData value in the quote
was generated by the vTPM Manager.

In addition, it would be useful to add a command-line switch to the vTPM
manager that disables this backwards compatibility mode if a given
system will not be using it: this would avoid issues if the users of
that system have not read this discussion and decide to use quotes that
do not include PCR 20-23 (which is normally a sensible thing to do).


I think it is reasonable to specify in the documentation the changes and
include only the new version.


That sounds good to me.
 

  Flags are interpreted as a bitmask of:

   * VTPM_QUOTE_FLAGS_HASH_UUID
   * VTPM_QUOTE_FLAGS_VTPM_MEASUREMENTS
   * VTPM_QUOTE_FLAGS_GROUP_INFO
When using flags!=0, vtpmmgr ignores quoteSelect and TPM_Quote is
executed with:
externData = SHA1(
 flags,
 requestData,
 [UUIDs if requested],
 [vTPM measurements if requested],
 [vTPM group update policy if requested]
)



This specification does not match what you actually do below.  There are
some advantages to doing it the way you have coded, but the method for
calculating the hash needs to be fully documented.


I will update the documentation: sha1(UUIDs) instead of UUIDs and so on.


  Signed-off-by: Emil Condrea emilcond...@gmail.com

---


[...]

  diff --git a/stubdom/vtpmmgr/mgmt_authority.c

b/stubdom/vtpmmgr/mgmt_authority.c
index 0526a12..0e4aac7 100644
--- a/stubdom/vtpmmgr/mgmt_authority.c
+++ b/stubdom/vtpmmgr/mgmt_authority.c
@@ -128,6 +128,49 @@ static int do_load_aik(struct mem_group *group,
TPM_HANDLE *handle)
 return TPM_LoadKey(TPM_SRK_KEYHANDLE, key, handle,
(void*)vtpm_globals.srk_auth, vtpm_globals.oiap);
   }

+static void do_vtpminfo_hash(uint32_t extra_info_flags,struct mem_group
*group,
+   const void* uuid, const uint8_t* kern_hash,unsigned char**
calc_hashes)
+{
+   int i;
+   sha1_context ctx;
+   if(extra_info_flags  VTPM_QUOTE_FLAGS_HASH_UUID){
+   printk(hashing for FLAGS_HASH_UUID: );
+   if(uuid){
+   printk(true);
+   sha1_starts(ctx);
+   sha1_update(ctx, (void*)uuid, 16);
+   sha1_finish(ctx, *calc_hashes);
+   *calc_hashes = *calc_hashes + 20;
+   }
+   printk(\n);
+   }
+   if(extra_info_flags  VTPM_QUOTE_FLAGS_VTPM_MEASUREMENTS){
+   printk(hashing for VTPM_QUOTE_FLAGS_VTPM_MEASUREMENTS:
);
+   if(kern_hash){
+   printk(true);
+   sha1_starts(ctx);
+   sha1_update(ctx, (void*)kern_hash, 20);
+   sha1_finish(ctx, *calc_hashes);
+   *calc_hashes = *calc_hashes + 20;
+   }
+   printk(\n);
+   }



If exactly one of (uuid) and (kern_hash) is NULL and both were requested
in extra_info_flags, it is not easy to 

[Xen-devel] [PATCH 5/6] x86/smp: Allocate pcpu stacks on their local numa node

2015-04-07 Thread Andrew Cooper
Previously, all pcpu stacks tended to be allocated on node 0.

Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Keir Fraser k...@xen.org
CC: Jan Beulich jbeul...@suse.com

---
I have not done any performance, but this is obviously an improvement.
---
 xen/arch/x86/smpboot.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index a009e91..116c8f8 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -642,7 +642,7 @@ static int cpu_smpboot_alloc(unsigned int cpu)
 if ( node != NUMA_NO_NODE )
 memflags = MEMF_node(node);
 
-stack_base[cpu] = alloc_xenheap_pages(STACK_ORDER, 0);
+stack_base[cpu] = alloc_xenheap_pages(STACK_ORDER, memflags);
 if ( stack_base[cpu] == NULL )
 goto oom;
 memguard_guard_stack(stack_base[cpu]);
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] tg3 NIC driver bug in 3.14.x under Xen

2015-04-07 Thread Ian Jackson
Michael Chan writes (Re: tg3 NIC driver bug in 3.14.x under Xen):
 On Tue, 2015-04-07 at 16:12 +0100, Ian Jackson wrote:
  The symptom is a very high level of packet loss: around 25-30% (as
  seen in `ping').  There don't seem to be any untoward-looking kernel
  messages.  The lost packets get added to the `errors' counter shown in
  ifconfig. 
 
 Please provide the output of ethtool -S which has a better breakdown
 of the error counters.  Thanks.

root@bedbug:~# ethtool -S eth0 | grep -v ': 0$'
NIC statistics:
 rx_octets: 793954
 rx_ucast_packets: 342
 rx_mcast_packets: 197
 rx_bcast_packets: 11049
 tx_octets: 59093
 tx_ucast_packets: 396
 tx_mcast_packets: 8
 tx_bcast_packets: 4
root@bedbug:~# ifconfig eth0
eth0  Link encap:Ethernet  HWaddr 00:13:72:14:c0:51  
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:16012 errors:0 dropped:0 overruns:0 frame:0
  TX packets:459 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000 
  RX bytes:1090543 (1.0 MiB)  TX bytes:64575 (63.0 KiB)
  Interrupt:17 

root@bedbug:~# 

And:

--- bedbug.cam.xci-test.com ping statistics ---
120 packets transmitted, 90 received, 25% packet loss, time 119354ms
rtt min/avg/max/mdev = 0.294/0.459/0.844/0.073 ms


Evidently on this particular kernel, the error counters are _not_
increasing, contrary to what I said before.  I confess that I didn't
keep a record of on which particular machine and kernel I observed the
error count increasing.

If it would help I could try to check various other machines and/or
other kernels to see if I can get one of them to display the error
counter behaviour.


I'm about to try the experiment suggested by Konrad.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 0/6] xen/x86: Misc x86 improvements

2015-04-07 Thread Andrew Cooper
This series is some preparity cleanup to allow the Xen text/data/bss regions
to be mapped with superpages rather than small pages.

There are more areas needing to be tweaked before superpages can be used (pcpu
0's stack being in the bss, .init being between .data and .bss, probably
more), but these improvements are ready to go.

Andrew Cooper (6):
  xen/x86: Discard the alternatives .discard sections
  x86/numa: Correct the extern of cpu_to_node
  x86/smp: Clean up use of memflags in cpu_smpboot_alloc()
  x86/link: Introduce and use __bss_end
  x86/smp: Allocate pcpu stacks on their local numa node
  x86/boot: Ensure the BSS is aligned on an 8 byte boundary

 xen/arch/x86/boot/head.S   |5 +++--
 xen/arch/x86/setup.c   |4 ++--
 xen/arch/x86/smpboot.c |   16 +---
 xen/arch/x86/tboot.c   |4 ++--
 xen/arch/x86/xen.lds.S |5 +
 xen/include/asm-x86/numa.h |2 +-
 6 files changed, 22 insertions(+), 14 deletions(-)

-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 2/6] x86/numa: Correct the extern of cpu_to_node

2015-04-07 Thread Andrew Cooper
This was missed by c/s 54ce2db x86/numa: adjust datatypes for node and pxm
which changed the array definition in numa.c

Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Keir Fraser k...@xen.org
CC: Jan Beulich jbeul...@suse.com
CC: Boris Ostrovsky boris.ostrov...@oracle.com
---
 xen/include/asm-x86/numa.h |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-x86/numa.h b/xen/include/asm-x86/numa.h
index 7a489d3..0c5e5b4 100644
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -9,7 +9,7 @@
 
 extern int srat_rev;
 
-extern unsigned char cpu_to_node[];
+extern nodeid_t  cpu_to_node[NR_CPUS];
 extern cpumask_t node_to_cpumask[];
 
 #define cpu_to_node(cpu)   (cpu_to_node[cpu])
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Nested Virt - Xen 4.4 through 4.6 - Hyper-V; Can't boot after enabling Hyper-V

2015-04-07 Thread mailing lists
Which portion of this is the unicorn?  Nested virtualization of Hyper-V
under Xen?  Or something else about my setup?

I did try to go down the path of evaluating a memory dump, however while
minidumps are enabled, they do not seem to be getting created on either of
my test systems when the message about needing to reboot is presented.

-- Bill

On Tue, Apr 7, 2015 at 12:19 PM, Andrew Cooper andrew.coop...@citrix.com
wrote:

  On 07/04/15 02:42, mailing lists wrote:

 Hi --

  I've been trying to get nested virtualization working with Xen so that I
 could boot Windows and use Hyper-V related features, however I have not had
 much success.  Using Windows 8.1 or Windows 2012r2, I'm able to install
 Windows, select and install Hyper-V features, and start rebooting.
 However, at that point, the Windows VM only partially boots, then drops me
 to a screen stating:

  Your PC needs to restart.
 Please hold down the power button.
 Error Code: 0x001E
 Parameters:
 0xC096
 0xF80315430485
 0x
 0x


  Restarting does not yield any different results.

  I've set up Xen in accordance with the notes for patches and config
 options here:

  http://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

  Trying Xen 4.4.2 stable, 4.5.1 staging, and 4.6 staging.  I applied the
 patch labeled (2/2) from the wiki link above, compiled, and used the three
 options provided for the DomU running Windows (hap, nestedhvm, and cpuid
 mask).  Windows installs and allows me to turn on HyperV features on all
 versions of Xen listed above, however all give the same or similar message
 on reboot... I'm never able to get to a running state.

  I've tried this on two separate systems.  One has an Intel E5-1620 v2,
 and the other is a n E5-1650 (original, v1 I guess).  All the
 virtualization options are enabled in the BIOS.

  If the cpuid mask is removed from the DomU config, Windows boots,
 however I'm unable to start any virtual machines (there was a message in
 the Windows event log about a component not being started in regards to
 Hyper V).

  Has anyone else run into similar issues?  Any thoughts on next steps?


 I am not aware of anyone who has successfully got a setup like this to
 work.

 From
 https://msdn.microsoft.com/en-us/library/windows/hardware/ff557408%28v=vs.85%29.aspx

 0x1E is KMODE_EXCEPTION_NOT_HANDLED.

 http://source.winehq.org/source/include/ntstatus.h suggests that
 0xC096 is STATUS_PRIVILEGED_INSTRUCTION.

 Your best bet for debugging this is to debug the minidump generated and
 see which driver 0xF80315430485 is a part of, and perhaps exactly what
 instruction 0xF80315430485 actually is.

 ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [rumpuserxen test] 50341: regressions - FAIL

2015-04-07 Thread osstest service user
flight 50341 rumpuserxen real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50341/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-rumpuserxen5 rumpuserxen-build fail REGR. vs. 33866
 build-amd64-rumpuserxen   5 rumpuserxen-build fail REGR. vs. 33866

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a

version targeted for testing:
 rumpuserxen  80294081448af74270ca2dd2ba758f0ed308f46c
baseline version:
 rumpuserxen  30d72f3fc5e35cd53afd82c8179cc0e0b11146ad


People who touched revisions under test:
  Antti Kantee po...@iki.fi
  Ian Jackson ian.jack...@eu.citrix.com
  Martin Lucina mar...@lucina.net
  Wei Liu l...@liuw.name


jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  fail
 build-i386-rumpuserxen   fail
 test-amd64-amd64-rumpuserxen-amd64   blocked 
 test-amd64-i386-rumpuserxen-i386 blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/osstest/pub/logs
images: /home/osstest/pub/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 527 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [ovmf test] 50339: regressions - FAIL

2015-04-07 Thread osstest service user
flight 50339 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50339/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3865 xen-build fail REGR. vs. 36764
 build-amd64-xsm   5 xen-build fail REGR. vs. 36764
 build-i386-xsm5 xen-build fail REGR. vs. 36764
 build-amd64   5 xen-build fail REGR. vs. 36764

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-amd   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-sedf-pin  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-sedf  1 build-check(1)   blocked  n/a
 test-amd64-i386-rhel6hvm-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-rhel6hvm-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-win7-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-winxpsp3   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-winxpsp3-vcpus1  1 build-check(1)   blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a

version targeted for testing:
 ovmf 710585782756d00143b09adde43949cf2d10d4d4
baseline version:
 ovmf 785d183b4eae6ac2749d1ffdc1b67facf92610e6


People who touched revisions under test:
  Ard Biesheuvel ard.biesheu...@linaro.org
  Elvin Li elvin...@intel.com
  Felix Polyudov fel...@ami.com
  Fu Siyuan siyuan...@intel.com
  Gabriel Somlo so...@cmu.edu
  Hao Wu hao.a...@intel.com
  

[Xen-devel] Performance evaluation and Questions: Eliminating Xen (RTDS) scheduler overhead on dedicated CPU

2015-04-07 Thread Meng Xu
Hi George, Dario and Konrad,

I finished a prototype of the RTDS scheduler with the dedicated CPU
feature and did some quick evaluation on this feature. Right now, I
need to refactor the code (because it is kind of messy when I was
exploring different approaches :() and will send out the clean patch
later (this week or next week). But the design follows our discussion
at http://lists.xenproject.org/archives/html/xen-devel/2015-03/msg02854.html.

In a nutshell of the design, when a CPU is marked as dedicated CPU,
the scheduler on that CPU will return the dedicated VCPU on it with a
negative time so that it disable the scheduler timer on that CPU and
other CPUs will no longer send SCHEDULE_SOFTIRQ to the dedicated CPU.
The scheduler on the dedicated CPU may still be invoked when the
dedicated VCPU is blocked/unblocked by the domU. Once this situation
occurs, the schedule go though a fast pass to just return an idle
VCPU/the dedicated VCPU instead of go through the runq.

I did the following evaluation to show the benefits of introducing
this dedicated CPU feature:

I created a simple cpu-intensive task which just do the multiplication
for specified times:
start = rdtsc();
while ( i++  cpu_measurement-multiply_times )
result += i * i;
finish = rdtsc();
latencies[k] = finish - start;

I run this task and measure the execution time of above piece of code
on different environments: native Linux on bare metal, domU on Xen
with RTDS scheduler and domU on Xen with RTDS scheduler with dedicated
CPU feature, domU on Xen with Credit/Credit2 scheduler.

The difference between the execution time in virtualization
environment and the execution time on native linux on bare metal is
the virtualization overhead introduced by Xen.

I want to see that
1) The virtualization overhead decreases a lot after the dedicated CPU
feature is employed for RTDS scheduler (because the execution of the
task will no longer suffer the scheduler overhead any more).
2) The frequency of invoking the scheduler on the dedicated CPU
becomes very low once the dedicated CPU feature is applied.

The result is as follows:
When the cpu-intensive task did the multiplication for 1024 times, the
execution time of the piece of code is:
9264 cycles on native linux on bare metal;
10320 cycles on Xen RTDS scheduler with dedicated CPU feature;
10324 cycles on Xen RTDS scheduler without dedicated CPU feature;

We didn't see the improvement of the dedicated CPU feature here
because the execution time is too short and it may not experience the
scheduler overhead yet.

When the cpu-intensive task did the multiplication for  536870912
times, the execution time of the piece of code is:
4838016028  cycles on native linux on bare metal;
4839649567 cycles on Xen RTDS scheduler with dedicated CPU feature;
4855509977 cycles on Xen RTDS scheduler without dedicated CPU feature;

We can see that the dedicated CPU feature did save time for the
cpu-intensive task. Without the dedicated CPU feature, the hypervisor
scheduler may steal time from the domU and delay the execution of the
task inside domU.

I did vary the number of multiplications of the above piece of code in
cpu-intensive task, and draw a figure to show the relation of the
overhead and the execution time of the task on native linux. The
figure can be found at
http://www.cis.upenn.edu/~mengxu/xen-ml/cpu-base-alone_multiply_0_0_100.virtOhVSwcetnative.pdf.
Please note the x-axis is the log value of the execution time. So
the overhead is actually linear to the execution time of the task.

As to the frequence of invoking the RTDS scheduler with/without the
dedicated CPU feature, I add some code to trace which event triggers
the scheduler on the dedicated cpu and how frequent it is.

Before we apply the dedicated CPU feature to the RTDS scheduler, the
dedicated CPU 3 was invoked once
every 3.5us in average.
(XEN) cpu 3 has invoked 356805936 SCHED_SOFTIRQ (sched) within 1267613845122 ns
(XEN) tasklet_enqueue(0), do_tasklet(0), s_timer_fn(356789129), do_pool(18)
(XEN) vcpu_yield(0), vcpu_block(10483)


After we apply the dedicated CPU feature to the RTDS scheduler, the
dedicated CPU 3 was invoked once every 136ms in average. And the
scheduler was invoked because of vcpu_block/vcpu_unblock event. (We
could modify Linux in domU as Konrad suggests to avoid the hypercall
when vcpu is blocked/unblocked, but I'm unsure if it is better to do
that since it involves the change in domU.)
(XEN) cpu 3 has invoked 5396 SCHED_SOFTIRQ (nooh) within 736973916783 ns
(XEN) tasklet_enqueue(0), do_tasklet(0), s_timer_fn(0), do_pool(0)
(XEN) vcpu_yield(0), vcpu_block(2698)


Here are some conclusions/observation we have:
1) Dedicated CPU feature can save the scheduler overhead from domU and
thus reduce the virtualization overhead.
2) Scheduler overhead of the current RTDS scheduler in the view of
application is higher than the scheduler overhead of the current
credit/credit2 scheduler because the 

[Xen-devel] [qemu-mainline test] 50337: regressions - FAIL

2015-04-07 Thread osstest service user
flight 50337 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50337/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-freebsd10-amd64 14 guest-localmigrate/x10 fail REGR. vs. 36709
 test-amd64-i386-freebsd10-i386 14 guest-localmigrate/x10  fail REGR. vs. 36709

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-pair17 guest-migrate/src_host/dst_host fail like 36709

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-amd64-libvirt 10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel  9 guest-start  fail  never pass
 test-amd64-i386-libvirt-xsm   9 guest-start  fail   never pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-i386-libvirt  10 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 7 debian-hvm-install fail never 
pass
 test-amd64-i386-xl-xsm9 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-amd   9 guest-start  fail   never pass
 test-amd64-amd64-xl-xsm   9 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm  9 guest-start  fail   never pass
 test-armhf-armhf-xl  10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-sedf 10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 10 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-sedf-pin 10 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm   5 xen-boot fail   never pass
 test-armhf-armhf-libvirt-xsm  5 xen-boot fail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-armhf-armhf-libvirt 10 migrate-support-checkfail   never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-armhf-armhf-xl-cubietruck 10 migrate-support-checkfail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-armhf-armhf-xl-arndale  10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-winxpsp3 14 guest-stopfail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3 14 guest-stopfail never pass
 test-amd64-i386-xl-winxpsp3  14 guest-stop   fail   never pass

version targeted for testing:
 qemuuf2155a089600e80cf7bcdc814520ef3304882cc4
baseline version:
 qemuu362ca922eea03240916287a8a6267801ab095d12


People who touched revisions under test:
  Alex Bennée alex.ben...@linaro.org
  Alexander Graf ag...@suse.de
  Alexey Kardashevskiy a...@ozlabs.ru
  Andreas Färber afaer...@suse.de
  Bastian Koppelmann kbast...@mail.uni-paderborn.de
  Christian Borntraeger borntrae...@de.ibm.com
  Christoffer Dall christoffer.d...@linaro.org
  Cole Robinson crobi...@redhat.com
  Cornelia Huck cornelia.h...@de.ibm.com
  Daniel P. Berrange berra...@redhat.com
  David Gibson da...@gibson.dropbear.id.au
  Dirk Mueller dmuel...@suse.com
  Dirk Müller d...@dmllr.de
  Dr. David Alan Gilbert dgilb...@redhat.com
  Ed Maste ema...@freebsd.org
  Eduardo Otubo eduardo.ot...@profitbricks.com
  Fam Zheng f...@redhat.com
  Gabriel L. Somlo so...@cmu.edu
  Gabriel Somlo so...@cmu.edu
  Gerd Hoffmann kra...@redhat.com
  Gonglei arei.gong...@huawei.com
  Hervé Poussineau hpous...@reactos.org
  Igor Mammedov imamm...@redhat.com
  Jason Wang jasow...@redhat.com
  John Snow js...@redhat.com
  Joseph Hindin jhin...@daynix.com
  Juan Quintela quint...@redhat.com
  Leon Alrae leon.al...@imgtec.com
  Lin Ma l...@suse.com
  Marcel Apfelbaum mar...@redhat.com
  Markus Armbruster arm...@redhat.com
  Meghana Cheripady meghana.cherip...@emulex.com
  

Re: [Xen-devel] tg3 NIC driver bug in 3.14.x under Xen

2015-04-07 Thread Michael Chan
On Tue, 2015-04-07 at 19:13 +0100, Ian Jackson wrote:
 root@bedbug:~# ethtool -S xenbr0 | grep -v ': 0$'
 no stats available
 root@bedbug:~# 

Please provide ethtool -S on the tg3 device.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] hvmloader: fix building error

2015-04-07 Thread Wen Congyang
commit b9245b75 introduces a building error:
make[1]: Entering directory `/root/work/xen/tools/firmware/hvmloader'
gcc   -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing 
-std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement   -O0 -g3 
-D__XEN_TOOLS__ -MMD -MF .smbios.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE 
-fno-optimize-sibling-calls -mno-tls-direct-seg-refs  -Werror 
-fno-stack-protector -fno-exceptions -fno-builtin -msoft-float 
-I/root/work/xen/tools/firmware/hvmloader/../../../tools/include 
-DENABLE_ROMBIOS -DENABLE_SEABIOS -D__SMBIOS_DATE__=04/08/2015  -c -o 
smbios.o smbios.c
smbios.c:384:46: error: invalid digit 8 in octal constant
smbios.c:792:46: error: invalid digit 8 in octal constant
make[1]: *** [smbios.o] Error 1

Signed-off-by: Wen Congyang we...@cn.fujitsu.com
---
 tools/firmware/hvmloader/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/firmware/hvmloader/Makefile 
b/tools/firmware/hvmloader/Makefile
index 34008d2..e4830ca 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -91,7 +91,7 @@ all: subdirs-all
$(MAKE) hvmloader
 
 ovmf.o rombios.o seabios.o hvmloader.o: roms.inc
-smbios.o: CFLAGS += -D__SMBIOS_DATE__=$(SMBIOS_REL_DATE)
+smbios.o: CFLAGS += -D__SMBIOS_DATE__=\$(SMBIOS_REL_DATE)\
 
 hvmloader: $(OBJS) acpi/acpi.a
$(LD) $(LDFLAGS_DIRECT) -N -Ttext $(LOADADDR) -o hvmloader.tmp $^
-- 
2.1.0

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v6 1/3] xenbus_client: Extend interface to support multi-page ring

2015-04-07 Thread Juergen Gross

On 04/03/2015 08:44 AM, Bob Liu wrote:

From: Wei Liu wei.l...@citrix.com

Originally Xen PV drivers only use single-page ring to pass along
information. This might limit the throughput between frontend and
backend.

The patch extends Xenbus driver to support multi-page ring, which in
general should improve throughput if ring is the bottleneck. Changes to
various frontend / backend to adapt to the new interface are also
included.

Affected Xen drivers:
* blkfront/back
* netfront/back
* pcifront/back
* scsifront/back

The interface is documented, as before, in xenbus_client.c.

Change in V2:
* allow ring has arbitrary number of pages = XENBUS_MAX_RING_PAGES

Change in V3:
* update function prototypes
* carefully deal with types of different sizes

Change in V4:
* use PAGE_KERNEL instead of PAGE_KERNEL_IO to avoid breakage on Arm

Change in V5:
* fix off-by-one error and other minor glitches spotted by Mathew Daley

Change in V6:
* add pvscsi driver support by Bob Liu

Signed-off-by: Wei Liu wei.l...@citrix.com
Signed-off-by: Paul Durrant paul.durr...@citrix.com
Signed-off-by: Bob Liu bob@oracle.com
Cc: Konrad Wilk konrad.w...@oracle.com
Cc: David Vrabel david.vra...@citrix.com
Cc: Boris Ostrovsky boris.ostrov...@oracle.com


Acked-by: Juergen Gross jgr...@suse.com


Juergen


---
  drivers/block/xen-blkback/xenbus.c |5 +-
  drivers/block/xen-blkfront.c   |5 +-
  drivers/net/xen-netback/netback.c  |4 +-
  drivers/net/xen-netfront.c |9 +-
  drivers/pci/xen-pcifront.c |5 +-
  drivers/scsi/xen-scsifront.c   |5 +-
  drivers/xen/xen-pciback/xenbus.c   |2 +-
  drivers/xen/xen-scsiback.c |2 +-
  drivers/xen/xenbus/xenbus_client.c |  387 ++--
  include/xen/xenbus.h   |   20 +-
  10 files changed, 321 insertions(+), 123 deletions(-)

diff --git a/drivers/block/xen-blkback/xenbus.c 
b/drivers/block/xen-blkback/xenbus.c
index e3afe97..ff30259 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -193,7 +193,7 @@ fail:
return ERR_PTR(-ENOMEM);
  }

-static int xen_blkif_map(struct xen_blkif *blkif, unsigned long shared_page,
+static int xen_blkif_map(struct xen_blkif *blkif, grant_ref_t gref,
 unsigned int evtchn)
  {
int err;
@@ -202,7 +202,8 @@ static int xen_blkif_map(struct xen_blkif *blkif, unsigned 
long shared_page,
if (blkif-irq)
return 0;

-   err = xenbus_map_ring_valloc(blkif-be-dev, shared_page, 
blkif-blk_ring);
+   err = xenbus_map_ring_valloc(blkif-be-dev, gref, 1,
+blkif-blk_ring);
if (err  0)
return err;

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 37779e4..2c61cf8 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -1245,6 +1245,7 @@ static int setup_blkring(struct xenbus_device *dev,
 struct blkfront_info *info)
  {
struct blkif_sring *sring;
+   grant_ref_t gref;
int err;

info-ring_ref = GRANT_INVALID_REF;
@@ -1257,13 +1258,13 @@ static int setup_blkring(struct xenbus_device *dev,
SHARED_RING_INIT(sring);
FRONT_RING_INIT(info-ring, sring, PAGE_SIZE);

-   err = xenbus_grant_ring(dev, virt_to_mfn(info-ring.sring));
+   err = xenbus_grant_ring(dev, info-ring.sring, 1, gref);
if (err  0) {
free_page((unsigned long)sring);
info-ring.sring = NULL;
goto fail;
}
-   info-ring_ref = err;
+   info-ring_ref = gref;

err = xenbus_alloc_evtchn(dev, info-evtchn);
if (err)
diff --git a/drivers/net/xen-netback/netback.c 
b/drivers/net/xen-netback/netback.c
index 997cf09..865203f 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -1782,7 +1782,7 @@ int xenvif_map_frontend_rings(struct xenvif_queue *queue,
int err = -ENOMEM;

err = xenbus_map_ring_valloc(xenvif_to_xenbus_device(queue-vif),
-tx_ring_ref, addr);
+tx_ring_ref, 1, addr);
if (err)
goto err;

@@ -1790,7 +1790,7 @@ int xenvif_map_frontend_rings(struct xenvif_queue *queue,
BACK_RING_INIT(queue-tx, txs, PAGE_SIZE);

err = xenbus_map_ring_valloc(xenvif_to_xenbus_device(queue-vif),
-rx_ring_ref, addr);
+rx_ring_ref, 1, addr);
if (err)
goto err;

diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index e9b960f..13f5e7f 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -1486,6 +1486,7 @@ static int setup_netfront(struct xenbus_device *dev,
  {
struct xen_netif_tx_sring *txs;
struct xen_netif_rx_sring *rxs;
+   grant_ref_t gref;
int 

Re: [Xen-devel] Issue With Patch Compilation Fails ( xen/arm: Introduce a generic way to describe device) with HAS_PCI and HAS_PASSTHROUGH.

2015-04-07 Thread Manish Jaggi


On Tuesday 07 April 2015 10:13 PM, Stefano Stabellini wrote:

On Tue, 7 Apr 2015, Jaggi, Manish wrote:

Hi Julien,


Following patch generated compiler error when HAS_PCI adn HAS_PASSTHROUGH 
enabled.
Please advice how to fix this issue, or you can revert this patch.
Should I add a device structure in pci_dev or there is another way.

Hello Manish,

we have never really built Xen on ARM with HAS_PCI=y so it is normal
that it won't compile out of the box, it is not just a problem caused by
the commit below.
The problem with the patch is it introduces two different structures for 
device for x86 and arm. While x86 device = pci_dev, for ARM there is a 
proper device structure and != pci_dev.

So the compilation failure is by design.

  I imagine that you'll need to do more than setting
HAS_PCI to y in order to get PCI and PCI passthrough working properly
with Xen on ARM.  Feel free to go ahead and propose any changes
necessary.

ok


Cheers,

Stefano



 xen/arm: Introduce a generic way to describe device
 
 Currently, Xen is supporting PCI and Platform device (based on Device Tree).

 While Xen only supports Platform device on ARM, Xen will gain support of

 PCI soon.
 
 Some drivers, such as IOMMU drivers, may handle PCI and platform device in

 the same way. Only few lines of code differs.
 
 Rather than requesting to provide 2 set of functions (one for PCI and

 one for platform device), introduce a generic structure device which
 is embedded in each specialized device.
 
 As x86 only supports PCI, introduce a new type device_t which will be an

 alias to pci_dev for this architecture. It will avoid to add a new field
 for this place.
 
 Signed-off-by: Julien Grall julien.gr...@linaro.org

 Acked-by: Jan Beulich jbeul...@suse.com
 Acked-by: Stefano Stabellini stefano.stabell...@eu.citrix.com
 CC: Keir Fraser k...@xen.org
 CC: Andrew Cooper andrew.coop...@citrix.com



Compilation error
pci.c: In function ‘iommu_add_device’:
pci.c:1263:5: error: implicit declaration of function ‘pci_to_dev’ 
[-Werror=implicit-function-declaration]
pci.c:1263:5: error: nested extern declaration of ‘pci_to_dev’ 
[-Werror=nested-externs]
pci.c:1263:5: error: passing argument 2 of ‘hd-platform_ops-add_device’ makes 
pointer from integer without a cast [-Werror]
pci.c:1263:5: note: expected ‘struct device_t *’ but argument is of type ‘int’
pci.c:1272:9: error: passing argument 2 of ‘hd-platform_ops-add_device’ makes 
pointer from integer without a cast [-Werror]
pci.c:1272:9: note: expected ‘struct device_t *’ but argument is of type ‘int’


Regards,
Manish Jaggi



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] How to set full update mode in QEMU (in regards to display output)

2015-04-07 Thread mailing lists
As a shot in the dark, I went through vga.c for QEMU as distributed with
Xen, located each instance of the full_update variable, and set it to 1.
After recompiling, instead of a blank screen when the host is waiting for
login, I get a flickering message Waiting for display 1  That doesn't
seem to be the right answer...

On Tue, Apr 7, 2015 at 10:44 PM, mailing lists theli...@gmail.com wrote:

 Following the guide for nested virtualization here:

 http://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

 It states that one option for display issues is to force full update
 mode in QEMU.  How is that done?  I can't seem to find any documentation
 on it, and in the QEMU source, full_update isn't a config option.

 Does it require a code patch, or is there another way?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] How to set full update mode in QEMU (in regards to display output)

2015-04-07 Thread mailing lists
Following the guide for nested virtualization here:

http://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

It states that one option for display issues is to force full update mode
in QEMU.  How is that done?  I can't seem to find any documentation on it,
and in the QEMU source, full_update isn't a config option.

Does it require a code patch, or is there another way?
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 00/10] PML (Paging Modification Logging) support

2015-04-07 Thread Kai Huang



On 04/07/2015 05:24 PM, Tim Deegan wrote:

Hi,

At 16:30 +0800 on 07 Apr (1428424218), Kai Huang wrote:

Hi Jan, Tim, other maintainers,

Do you have comments? Or should I send out the v2 addressing Andrew's
comments, as it's been more than a week since this patch series were
sent out?

I'm sorry, I was away last week so I haven't had a chance to review
these patches.  I'll probably be able to look at them on Thursday.

Yeah sure. Thanks Tim!

Thanks,
-Kai


Cheers,

Tim.


On 03/30/2015 01:50 PM, Kai Huang wrote:


On 03/28/2015 05:26 AM, Andrew Cooper wrote:

On 27/03/15 02:35, Kai Huang wrote:

Hi all,

This patch series adds PML support to Xen. Please kindly help to
review it.

Overall this looks like a very good series, and it is particularly
helpful given the level of commenting.

Which platforms is/will PML be available for?

Hi Andrew,

Thanks for your quick review. PML will be available from Intel's
Broadwell server platform.

Thanks,
-Kai

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel