[Xen-devel] [qemu-mainline test] 103992: tolerable FAIL - PUSHED

[osstest service owner]

flight 103992 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103992/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail  like 103980
 test-armhf-armhf-libvirt-qcow2 12 saverestore-support-check   fail like 103980
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 103980
 test-armhf-armhf-libvirt 13 saverestore-support-checkfail  like 103980
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 103980
 test-armhf-armhf-libvirt-raw 12 saverestore-support-checkfail  like 103980
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 103980

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuudbe2b65566e76d3c3a0c3358285c0336ac61e757
baseline version:
 qemuu0f72559fbc9e1ae45aa8ebeed0443fe3a7a388a8

Last test of basis   103980  2016-12-28 00:14:25 Z1 days
Testing same since   103992  2016-12-29 00:42:45 Z0 days1 attempts


People who touched revisions under test:
  Laurent Vivier 
  Peter Maydell 
  Richard Henderson 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 

[Xen-devel] [linux-3.18 baseline-only test] 68284: regressions - FAIL

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 68284 linux-3.18 real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/68284/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-pair   21 guest-migrate/src_host/dst_host fail REGR. vs. 67943
 test-armhf-armhf-xl-credit2  15 guest-start/debian.repeat fail REGR. vs. 67943

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail like 67943

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 build-i386-rumprun6 xen-buildfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 build-amd64-rumprun   6 xen-buildfail   never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail never pass

version targeted for testing:
 linux1e20e732ffa107e21f2089ce56e32d99b9229944
baseline version:
 linuxa6846cfd266b48af1ee7c3c19d5cb60477ca4469

Last test of basis67943  2016-10-27 10:49:16 Z   62 days
Testing same since68284  2016-12-28 18:17:26 Z0 days1 attempts


People who touched revisions under test:
  Adrian Hunter 
  Alan Stern 
  Alex Deucher 
  Alexander Usyskin 
  Andreas Gruenbacher 
  Andreas Gruenbacher 
  Andrew Morton 
  Andy Lutomirski 
  Boris Brezillon 
  Brian Norris 
  Bryan Paluch 
  Ching Huang 
  Chris Mason 
  Dan Carpenter 
  Darrick J. Wong 
  Dave Chinner 
  David Howells 
  David S. Miller 

[Xen-devel] [qemu-mainline baseline-only test] 68281: regressions - FAIL

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 68281 qemu-mainline real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/68281/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-libvirt-vhd  6 xen-boot  fail REGR. vs. 68270

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail blocked 
in 68270
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  9 windows-installfail like 68270

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass

version targeted for testing:
 qemuu0f72559fbc9e1ae45aa8ebeed0443fe3a7a388a8
baseline version:
 qemuua470b33259bf82ef2336bfcd5d07640562d3f63b

Last test of basis68270  2016-12-25 00:44:02 Z4 days
Testing same since68281  2016-12-28 08:14:48 Z0 days1 attempts


People who touched revisions under test:
  Alastair D'Silva 
  Alistair Francis 
  Andrew Gacek 
  Andrew Jones 
  Cédric Le Goater 
  Julian Brown 
  Peter Maydell 
  Richard Henderson 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl

[Xen-devel] [ovmf test] 103993: all pass - PUSHED

[osstest service owner]

flight 103993 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103993/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf d9bb3ca3c2a1bd3f7b78adf27e82e5605f665e16
baseline version:
 ovmf af8ba51aca4e0b41a359fe467fb5c5b9baa75a05

Last test of basis   103986  2016-12-28 11:15:08 Z0 days
Testing same since   103993  2016-12-29 01:46:28 Z0 days1 attempts


People who touched revisions under test:
  Chen A Chen 
  Ruiyu Ni 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=d9bb3ca3c2a1bd3f7b78adf27e82e5605f665e16
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
d9bb3ca3c2a1bd3f7b78adf27e82e5605f665e16
+ branch=ovmf
+ revision=d9bb3ca3c2a1bd3f7b78adf27e82e5605f665e16
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.8-testing
+ '[' xd9bb3ca3c2a1bd3f7b78adf27e82e5605f665e16 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git

[Xen-devel] [linux-4.1 test] 103991: regressions - FAIL

[osstest service owner]

flight 103991 linux-4.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103991/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt 11 guest-start  fail REGR. vs. 101737

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-credit2  15 guest-start/debian.repeatfail  like 101687
 test-armhf-armhf-xl-cubietruck 15 guest-start/debian.repeat   fail like 101715
 test-armhf-armhf-xl-xsm  15 guest-start/debian.repeatfail  like 101715
 test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail  like 101737
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101737
 test-armhf-armhf-xl  15 guest-start/debian.repeatfail  like 101737
 test-armhf-armhf-xl-multivcpu 15 guest-start/debian.repeatfail like 101737
 test-armhf-armhf-libvirt-xsm 15 guest-start/debian.repeatfail  like 101737
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101737
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101737
 test-armhf-armhf-xl-rtds 11 guest-start  fail  like 101737
 test-armhf-armhf-xl-vhd   9 debian-di-installfail  like 101737
 test-armhf-armhf-libvirt-qcow2  9 debian-di-install   fail like 101737
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101737

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass

version targeted for testing:
 linux35327468a79dd9e343eaf7e66cc372f8277b2a84
baseline version:
 linux9ca365c0c8bdd8552ec064f0e696600cf7ea66dd

Last test of basis   101737  2016-10-28 04:21:54 Z   61 days
Failing since102733  2016-11-30 09:50:09 Z   28 days   31 attempts
Testing same since   103978  2016-12-27 22:45:26 Z1 days3 attempts


People who touched revisions under test:
  "Eric W. Biederman" 
  Adrian Hunter 
  Alan Stern 
  Alex Deucher 
  Alexander Polakov 
  Alexander Usyskin 
  Andreas Dilger 
  Andreas Gruenbacher 
  Andreas Gruenbacher 
  Andrew Morton 
  Andy Lutomirski 
  Arve Hjønnevåg 
  Boris Brezillon 
  Brian Norris 
  Bryan Paluch 
  CAI Qian 
  Carlos Santa 
  Carlos Santa  (v1)
  Ching Huang 
  Chris Mason 
  Dan Carpenter 
  Daniel Vetter 
  Darrick J. Wong 
  Dave Airlie 
  Dave Chinner 
  

[Xen-devel] [PATCH RFC 54/59] controller/run: Add RunConfig.NumaDisable

[Ronald Rojas]

From: George Dunlap 

If RunConfig.NumaDisable is set, get each worker's
WorkerConfig.SoftAffinity to the cpumap of the cpupool the VM is going
to be running in.

Signed-off-by: George Dunlap 
---
 benchmark.go |  4 
 run.go   | 35 ---
 2 files changed, 36 insertions(+), 3 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index d5a0ac8..abe2dfb 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -142,6 +142,7 @@ type RunConfig struct {
Scheduler string
Pool string
Cpus []int
+   NumaDisable *bool `json:",omitempty"`
 }
 
 // Propagate unset values from a higher level
@@ -155,6 +156,9 @@ func (l *RunConfig) PropagateFrom(g RunConfig) {
if l.Cpus == nil {
l.Cpus = g.Cpus
}
+   if l.NumaDisable == nil {
+   l.NumaDisable = g.NumaDisable
+   }
 }
 
 type BenchmarkRun struct {
diff --git a/run.go b/run.go
index 2d0db01..d1c5d95 100644
--- a/run.go
+++ b/run.go
@@ -312,14 +312,43 @@ func (run *BenchmarkRun) Prep() (ready bool, why string) {
return 
 }
 
+func (run *BenchmarkRun) GetCpumap() (Cpumap Bitmap) {
+   if run.RunConfig.Pool == "" {
+   fmt.Printf("Run.Prep: No pool set, using 0\n")
+   pool := Ctx.CpupoolInfo(0)
+   Cpumap = pool.Cpumap
+   } else {
+   pool, poolPresent := Ctx.CpupoolFindByName(run.RunConfig.Pool)
+   if poolPresent {
+   Cpumap = pool.Cpumap
+   } else {
+   panic("run.GetCpumap(): Pool "+run.RunConfig.Pool+" not 
found!")
+   }
+   }
+   return
+}
+
 func (run *BenchmarkRun) Run() (err error) {
for wsi := range run.WorkerSets {
-   run.WorkerSets[wsi].Config.PropagateFrom(run.WorkerConfig)
-   if run.WorkerSets[wsi].Config.Pool == "" {
-   run.WorkerSets[wsi].Config.Pool = run.RunConfig.Pool
+   conf := [wsi].Config
+   
+   conf.PropagateFrom(run.WorkerConfig)
+   if conf.Pool == "" {
+   conf.Pool = run.RunConfig.Pool
}
run.WorkerSets[wsi].Params.SetkHZ(CpukHZ)

+   if *run.RunConfig.NumaDisable {
+   if conf.SoftAffinity != "" {
+   err = fmt.Errorf("Cannot disable Numa if 
SoftAffinity is set!")
+   return
+   }
+   // Disable libxl NUMA by setting the soft
+   // affinity to the set of cpus in the cpupool
+   conf.SoftAffinity = run.GetCpumap().String()
+   fmt.Printf("Setting SoftAffinity to %s to disable NUMA 
placement\n",
+   conf.SoftAffinity)
+   }
}

Workers, err := NewWorkerList(run.WorkerSets, WorkerXen)
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 59/59] tools/xenlight: Create interface for xenlight info

[Ronald Rojas]

Create interface to interact with libxl_verions_info
and libxl_physinfo. Also introduce proper error
handling.

Signed-off-by: Ronald Rojas 
---
 tools/golang/xenlight/libxl.go|  720 --
 tools/golang/xenlight/xenlight.go | 1000 +
 2 files changed, 1000 insertions(+), 720 deletions(-)
 delete mode 100644 tools/golang/xenlight/libxl.go
 create mode 100644 tools/golang/xenlight/xenlight.go

diff --git a/tools/golang/xenlight/libxl.go b/tools/golang/xenlight/libxl.go
deleted file mode 100644
index aa5c01c..000
--- a/tools/golang/xenlight/libxl.go
+++ /dev/null
@@ -1,720 +0,0 @@
-/*
- * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; version 2 of the
- * License only.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- * 02110-1301, USA.
- */
-package main
-
-/*
-#cgo LDFLAGS: -lxenlight -lyajl_s -lxengnttab -lxenstore -lxenguest 
-lxentoollog -lxenevtchn -lxenctrl -lblktapctl -lxenforeignmemory -lxencall -lz 
-luuid -lutil
-#include 
-#include 
-*/
-import "C"
-
-/*
- * Other flags that may be needed at some point: 
- *  -lnl-route-3 -lnl-3
- *
- * To get back to simple dynamic linking:
-#cgo LDFLAGS: -lxenlight -lyajl
-*/
-
-import (
-   "unsafe"
-   "fmt"
-   "time"
-)
-
-/*
- * Types: Builtins
- */
-
-type Domid uint32
-
-type MemKB uint64
-
-// typedef struct {
-// uint32_t size;  /* number of bytes in map */
-// uint8_t *map;
-// } libxl_bitmap;
-
-// Implement the Go bitmap type such that the underlying data can
-// easily be copied in and out.  NB that we still have to do copies
-// both directions, because cgo runtime restrictions forbid passing to
-// a C function a pointer to a Go-allocated structure which contains a
-// pointer.
-type Bitmap struct {
-   bitmap []C.uint8_t
-}
-
-type Context struct {
-   ctx *C.libxl_ctx
-}
-
-type Uuid C.libxl_uuid
-
-/*
- * Types: IDL
- * 
- * FIXME: Generate these automatically from the IDL
- */
-type Dominfo struct {
-   Uuid  Uuid
-   Domid Domid
-   Running   bool
-   Blocked   bool
-   Pausedbool
-   Shutdown  bool
-   Dying bool
-   Never_stopbool
-   
-   Shutdown_reason   int32 // FIXME shutdown_reason enumeration
-   Outstanding_memkb MemKB
-   Current_memkb MemKB
-   Shared_memkb  MemKB
-   Paged_memkb   MemKB
-   Max_memkb MemKB
-   Cpu_time  time.Duration
-   Vcpu_max_id   uint32
-   Vcpu_online   uint32
-   Cpupool   uint32
-   Domain_type   int32 //FIXME libxl_domain_type enumeration
-
-}
-
-// # Consistent with values defined in domctl.h
-// # Except unknown which we have made up
-// libxl_scheduler = Enumeration("scheduler", [
-// (0, "unknown"),
-// (4, "sedf"),
-// (5, "credit"),
-// (6, "credit2"),
-// (7, "arinc653"),
-// (8, "rtds"),
-// ])
-type Scheduler int
-var (
-   SchedulerUnknown  Scheduler = C.LIBXL_SCHEDULER_UNKNOWN
-   SchedulerSedf Scheduler = C.LIBXL_SCHEDULER_SEDF
-   SchedulerCredit   Scheduler = C.LIBXL_SCHEDULER_CREDIT
-   SchedulerCredit2  Scheduler = C.LIBXL_SCHEDULER_CREDIT2
-   SchedulerArinc653 Scheduler = C.LIBXL_SCHEDULER_ARINC653
-   SchedulerRTDS Scheduler = C.LIBXL_SCHEDULER_RTDS
-)
-
-// libxl_cpupoolinfo = Struct("cpupoolinfo", [
-// ("poolid",  uint32),
-// ("pool_name",   string),
-// ("sched",   libxl_scheduler),
-// ("n_dom",   uint32),
-// ("cpumap",  libxl_bitmap)
-// ], dir=DIR_OUT)
-
-type CpupoolInfo struct {
-   Poolid uint32
-   PoolName string
-   Scheduler Scheduler
-   DomainCount int
-   Cpumap Bitmap
-}
-
-/*
- * Context
- */
-var Ctx Context
-
-func (Ctx *Context) IsOpen() bool {
-   return Ctx.ctx != nil
-}
-
-func (Ctx *Context) Open() (err error) {
-   if Ctx.ctx != nil {
-   return
-   }
-   
-   ret := C.libxl_ctx_alloc(unsafe.Pointer(), C.LIBXL_VERSION, 0, 
nil)
-
-   if ret != 0 {
-   err = fmt.Errorf("Allocating libxl context: %d", ret)
-   }
-   return
-}
-
-func (Ctx *Context) Close() (err error) {
-   ret := C.libxl_ctx_free(unsafe.Pointer(Ctx.ctx))
-   Ctx.ctx = nil
-
-   if ret != 

[Xen-devel] [PATCH RFC 56/59] controller/plan: Add NumaDisable to SimpleMatrix

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 plan.go | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/plan.go b/plan.go
index 736d9f3..b8e0c6b 100644
--- a/plan.go
+++ b/plan.go
@@ -26,6 +26,7 @@ type PlanSimpleMatrix struct {
Schedulers []string
Workers []string
Count []int
+   NumaDisable []bool
 }
 
 type PlanInput struct {
@@ -127,6 +128,27 @@ func (plan *BenchmarkPlan) ExpandInput() (err error) {
b = nil
}
 
+   // ...and NumaDisable
+   if len(plan.Input.SimpleMatrix.NumaDisable) > 0 {
+   for _, base := range a {
+   for _, d := range plan.Input.SimpleMatrix.NumaDisable {
+   run := base
+   // Need to make a copy of this so that
+   // we have a pointer to use as a tristate
+   run.RunConfig.NumaDisable = new(bool)
+   *run.RunConfig.NumaDisable = d
+   if d {
+   run.Label = run.Label+" NumaOff"
+   } else {
+   run.Label = run.Label+" NumaOn "
+   }
+   b = append(b, run)
+   }
+   }
+   a = b
+   b = nil
+   }
+
for i := range a {
fmt.Printf("%s\n", a[i].Label)
}
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 48/59] libxl: Add Ctx.CheckOpen

[Ronald Rojas]

From: George Dunlap 

To standardize the error in the case that Ctx isn't open.  Add checks to all 
functions
which call libxl functions directly.

Signed-off-by: George Dunlap 
---
 libxl.go | 48 +---
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/libxl.go b/libxl.go
index e73c06b..8f16b1d 100644
--- a/libxl.go
+++ b/libxl.go
@@ -164,9 +164,16 @@ func (Ctx *Context) Close() (err error) {
return
 }
 
-func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err error) {
+func (Ctx *Context) CheckOpen() (err error) {
if Ctx.ctx == nil {
err = fmt.Errorf("Context not opened")
+   }
+   return
+}
+
+func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err error) {
+   err = Ctx.CheckOpen()
+   if err != nil {
return
}
 
@@ -209,8 +216,8 @@ func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err 
error) {
 }
 
 func (Ctx *Context) DomainUnpause(Id Domid) (err error) {
-   if Ctx.ctx == nil {
-   err = fmt.Errorf("Context not opened")
+   err = Ctx.CheckOpen()
+   if err != nil {
return
}
 
@@ -359,6 +366,11 @@ func SchedulerFromString(name string) (s Scheduler, err 
error) {
 // libxl_cpupoolinfo * libxl_list_cpupool(libxl_ctx*, int *nb_pool_out);
 // void libxl_cpupoolinfo_list_free(libxl_cpupoolinfo *list, int nb_pool);
 func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
+   err := Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
var nbPool C.int
 
c_cpupool_list := C.libxl_list_cpupool(Ctx.ctx, )
@@ -394,6 +406,11 @@ func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
 // FIXME: uuid
 // FIXME: Setting poolid
 func (Ctx *Context) CpupoolCreate(Name string, Scheduler Scheduler, Cpumap 
Bitmap) (err error, Poolid uint32) {
+   err = Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
poolid := C.uint32_t(0)
name := C.CString(Name)
defer C.free(unsafe.Pointer(name))
@@ -420,6 +437,11 @@ func (Ctx *Context) CpupoolCreate(Name string, Scheduler 
Scheduler, Cpumap Bitma
 
 // int libxl_cpupool_destroy(libxl_ctx *ctx, uint32_t poolid);
 func (Ctx *Context) CpupoolDestroy(Poolid uint32) (err error) {
+   err = Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
ret := C.libxl_cpupool_destroy(Ctx.ctx, C.uint32_t(Poolid))
// FIXME: Proper error
if ret != 0 {
@@ -432,6 +454,11 @@ func (Ctx *Context) CpupoolDestroy(Poolid uint32) (err 
error) {
 
 // int libxl_cpupool_cpuadd(libxl_ctx *ctx, uint32_t poolid, int cpu);
 func (Ctx *Context) CpupoolCpuadd(Poolid uint32, Cpu int) (err error) {
+   err = Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
ret := C.libxl_cpupool_cpuadd(Ctx.ctx, C.uint32_t(Poolid), C.int(Cpu))
// FIXME: Proper error
if ret != 0 {
@@ -445,6 +472,11 @@ func (Ctx *Context) CpupoolCpuadd(Poolid uint32, Cpu int) 
(err error) {
 // int libxl_cpupool_cpuadd_cpumap(libxl_ctx *ctx, uint32_t poolid,
 // const libxl_bitmap *cpumap);
 func (Ctx *Context) CpupoolCpuaddCpumap(Poolid uint32, Cpumap Bitmap) (err 
error) {
+   err = Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
cbm := bitmapGotoC(Cpumap)
defer C.libxl_bitmap_dispose()

@@ -460,6 +492,11 @@ func (Ctx *Context) CpupoolCpuaddCpumap(Poolid uint32, 
Cpumap Bitmap) (err error
 
 // int libxl_cpupool_cpuremove(libxl_ctx *ctx, uint32_t poolid, int cpu);
 func (Ctx *Context) CpupoolCpuremove(Poolid uint32, Cpu int) (err error) {
+   err = Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
ret := C.libxl_cpupool_cpuremove(Ctx.ctx, C.uint32_t(Poolid), 
C.int(Cpu))
// FIXME: Proper error
if ret != 0 {
@@ -473,6 +510,11 @@ func (Ctx *Context) CpupoolCpuremove(Poolid uint32, Cpu 
int) (err error) {
 // int libxl_cpupool_cpuremove_cpumap(libxl_ctx *ctx, uint32_t poolid,
 //const libxl_bitmap *cpumap);
 func (Ctx *Context) CpupoolCpuremoveCpumap(Poolid uint32, Cpumap Bitmap) (err 
error) {
+   err = Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
cbm := bitmapGotoC(Cpumap)
defer C.libxl_bitmap_dispose()

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 39/59] libxl.go: Put common link flags in libxl.go

[Ronald Rojas]

From: George Dunlap 

...so that the only things added in the makefile are extra flags
required when linking against non-installed builds.

Add in the text of the GPL while we're at it.

Signed-off-by: George Dunlap 
---
 Makefile |  3 +--
 libxl.go | 19 +++
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 0e0b231..0265dc8 100644
--- a/Makefile
+++ b/Makefile
@@ -7,11 +7,10 @@ all: $(BIN)
 
 
 CGO_CFLAGS = -I/build/hg/xen.git/dist/install/usr/local/include
-CGO_LIBS = -lyajl -lxenlight
 
 # FIXME
 XENLIB_PATH ?= /build/hg/xen.git/dist/install/usr/local/lib/
-CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH) $(CGO_LIBS)
+CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH)
 
 schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go plan.go
CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
diff --git a/libxl.go b/libxl.go
index 9477bca..6621974 100644
--- a/libxl.go
+++ b/libxl.go
@@ -1,6 +1,25 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
 package main
 
 /*
+#cgo LDFLAGS: -lyajl -lxenlight
 #include 
 #include 
 */
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 49/59] libxl: Implement libxl_cpupool_info and Scheduler.FromString()

[Ronald Rojas]

From: George Dunlap 

Unify the libxl_cpupoolinfo -> CpupoolInfo translation in a function.

Also make bitmapCToGo pass-by-value rather than pass-by-reference,
since it only has two elements and doesn't need to be modified.

Signed-off-by: George Dunlap 
---
 libxl.go | 64 ++--
 1 file changed, 54 insertions(+), 10 deletions(-)

diff --git a/libxl.go b/libxl.go
index 8f16b1d..dfe4f40 100644
--- a/libxl.go
+++ b/libxl.go
@@ -234,7 +234,7 @@ func (Ctx *Context) DomainUnpause(Id Domid) (err error) {
  */
 
 // Return a Go bitmap which is a copy of the referred C bitmap.
-func bitmapCToGo(cbm *C.libxl_bitmap) (gbm Bitmap) {
+func bitmapCToGo(cbm C.libxl_bitmap) (gbm Bitmap) {
// Alloc a Go slice for the bytes
size := int(cbm.size)
gbm.bitmap = make([]C.uint8_t, size)
@@ -338,7 +338,6 @@ func (a Bitmap) And(b Bitmap) (c Bitmap) {
 }
 
 // const char *libxl_scheduler_to_string(libxl_scheduler p);
-// int libxl_scheduler_from_string(const char *s, libxl_scheduler *e);
 func (s Scheduler) String() (string) {
cs := C.libxl_scheduler_to_string(C.libxl_scheduler(s))
// No need to free const return value
@@ -346,6 +345,32 @@ func (s Scheduler) String() (string) {
return C.GoString(cs)
 }
 
+// int libxl_scheduler_from_string(const char *s, libxl_scheduler *e);
+func (s *Scheduler) FromString(gstr string) (err error) {
+   cstr := C.CString(gstr)
+   defer C.free(unsafe.Pointer(cstr))
+
+   var cs C.libxl_scheduler
+   ret := C.libxl_scheduler_from_string(cstr, )
+   if ret != 0 {
+   err = fmt.Errorf("libxl_scheduler_from_string: %d\n", ret)
+   return
+   }
+
+   *s = Scheduler(cs)
+   return
+}
+
+func translateCpupoolInfoCToGo(cci C.libxl_cpupoolinfo) (gci CpupoolInfo) {
+   gci.Poolid = uint32(cci.poolid)
+   gci.PoolName = C.GoString(cci.pool_name)
+   gci.Scheduler = Scheduler(cci.sched)
+   gci.DomainCount = int(cci.n_dom)
+   gci.Cpumap = bitmapCToGo(cci.cpumap)
+
+   return
+}
+
 func SchedulerFromString(name string) (s Scheduler, err error) {
cname := C.CString(name)
defer C.free(unsafe.Pointer(cname))
@@ -385,20 +410,37 @@ func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
cpupoolListSlice := (*[1 << 
30]C.libxl_cpupoolinfo)(unsafe.Pointer(c_cpupool_list))[:nbPool:nbPool]
 
for i := range cpupoolListSlice {
-   var info CpupoolInfo
+   info := translateCpupoolInfoCToGo(cpupoolListSlice[i])

-   info.Poolid = uint32(cpupoolListSlice[i].poolid)
-   info.PoolName = C.GoString(cpupoolListSlice[i].pool_name)
-   info.Scheduler = Scheduler(cpupoolListSlice[i].sched)
-   info.DomainCount = int(cpupoolListSlice[i].n_dom)
-   info.Cpumap = bitmapCToGo([i].cpumap)
-
list = append(list, info)
}
 
return
 }
 
+// int libxl_cpupool_info(libxl_ctx *ctx, libxl_cpupoolinfo *info, uint32_t 
poolid);
+func (Ctx *Context) CpupoolInfo(Poolid uint32) (pool CpupoolInfo) {
+   err := Ctx.CheckOpen()
+   if err != nil {
+   return
+   }
+
+   var c_cpupool C.libxl_cpupoolinfo
+   
+   ret := C.libxl_cpupool_info(Ctx.ctx, _cpupool, C.uint32_t(Poolid))
+   if ret != 0 {
+   err = fmt.Errorf("libxl_cpupool_info failed: %d", ret)
+   return
+   }
+   defer C.libxl_cpupoolinfo_dispose(_cpupool)
+
+   pool = translateCpupoolInfoCToGo(c_cpupool)
+
+   return
+}
+
+
+
 // int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
 //  libxl_scheduler sched,
 //  libxl_bitmap cpumap, libxl_uuid *uuid,
@@ -532,7 +574,6 @@ func (Ctx *Context) CpupoolCpuremoveCpumap(Poolid uint32, 
Cpumap Bitmap) (err er
 // int libxl_cpupool_cpuadd_node(libxl_ctx *ctx, uint32_t poolid, int node, 
int *cpus);
 // int libxl_cpupool_cpuremove_node(libxl_ctx *ctx, uint32_t poolid, int node, 
int *cpus);
 // int libxl_cpupool_movedomain(libxl_ctx *ctx, uint32_t poolid, uint32_t 
domid);
-// int libxl_cpupool_info(libxl_ctx *ctx, libxl_cpupoolinfo *info, uint32_t 
poolid);
 
 //
 // Utility functions
@@ -615,5 +656,8 @@ func XlTest(Args []string) {
fmt.Printf("Pool id: %d\n", Poolid)
}
 
+   pool = Ctx.CpupoolInfo(0)
+   fmt.Printf("Cpupool 0 info: %v\n", pool)
+   
Ctx.Close()
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 29/59] report: Add basic html report

[Ronald Rojas]

From: George Dunlap 

Use Google's chartapi to create a (mostly) self-contained html file.
Start with just scatterplots of the raw data for proof-of-concept.

Signed-off-by: George Dunlap 
---
 Makefile  |   2 +-
 benchmark.go  |   1 +
 htmlreport.go | 233 ++
 main.go   |  12 +++
 4 files changed, 247 insertions(+), 1 deletion(-)
 create mode 100644 htmlreport.go

diff --git a/Makefile b/Makefile
index 54f2ce8..c1b9ee4 100644
--- a/Makefile
+++ b/Makefile
@@ -13,7 +13,7 @@ CGO_LIBS = -lyajl -lxenlight
 XENLIB_PATH ?= /build/hg/xen.git/dist/install/usr/local/lib/
 CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH) $(CGO_LIBS)
 
-schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go
+schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go
CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
 
 .PHONY: clean
diff --git a/benchmark.go b/benchmark.go
index de1f650..8be00a0 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -438,3 +438,4 @@ func (plan *BenchmarkPlan) TextReport(level int) (err 
error) {
 
return
 }
+
diff --git a/htmlreport.go b/htmlreport.go
new file mode 100644
index 000..6f61998
--- /dev/null
+++ b/htmlreport.go
@@ -0,0 +1,233 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
+package main
+
+import (
+   "fmt"
+   "os"
+   "io"
+   "encoding/json"
+)
+
+type OptionAxis struct {
+   Title string `json:"title"`
+   MinValue float64 `json:"minValue"`
+   MaxValue float64 `json:"maxValue"`
+}
+
+type Options struct {
+   Title string `json:"title"`
+   HAxis OptionAxis `json:"hAxis"`
+   VAxis OptionAxis `json:"vAxis"`
+   Legend string`json:"legend"`
+}
+
+type Point struct {
+   x float64
+   y float64
+}
+
+type RunRaw struct {
+   Label string
+   Points [][]Point
+}
+
+func (options *Options) OutputJavascript(w io.Writer, id int) (err error) {
+   var optionsJson []byte
+   optionsJson, err = json.Marshal(options)
+   if err != nil {
+   return
+   }
+
+   fmt.Fprintf(w, "var sp%dopt = ", id)
+   fmt.Fprint(w, string(optionsJson))
+   fmt.Fprintln(w, ";")
+
+   return
+}
+
+func (p *Point) OutputJson(w io.Writer, id int, max int) (err error) {
+   fmt.Fprintf(w, "[%f", p.x)
+   for i := 0; i < max; i++ {
+   if i == id {
+   fmt.Fprintf(w, ", %f", p.y)
+   } else {
+   fmt.Fprintf(w, ", null")
+   }
+   }
+   fmt.Fprint(w, "],\n")
+   return
+}
+
+func (d *RunRaw) OutputHTML(w io.Writer, run int) (err error) {
+   fmt.Fprintf(w, "\n", run)
+   return
+}
+
+func (d *RunRaw) OutputJavascript(w io.Writer, run int) (err error) {
+   var options Options
+
+   options.Title = fmt.Sprintf("Run %s (%d) Individual Throughput", 
d.Label, run)
+   options.HAxis.Title = "Time"
+   options.VAxis.Title = "Throughput"
+
+   var xmm MinMax
+   var ymm MinMax
+   for i := range d.Points {
+   for j := range d.Points[i] {
+   xmm.Update(d.Points[i][j].x)
+   ymm.Update(d.Points[i][j].y)
+   }
+   }
+
+   options.HAxis.MaxValue = xmm.Max
+   options.VAxis.MaxValue = ymm.Max
+
+   err = options.OutputJavascript(w, run)
+   if err != nil {
+   return
+   }
+
+   fmt.Printf("var sp%ddata = new 
google.visualization.DataTable();\n", run)
+   fmt.Printf("sp%ddata.addColumn('number', 'Time');\n", run)
+   for i := range d.Points {
+   fmt.Printf("sp%ddata.addColumn('number', 'Worker 
%d');\n", run, i)
+   }
+   fmt.Printf("sp%ddata.addRows([\n", run)
+
+   // Can't use json here because we need to be able to use 'null' for 
non-existent values
+   for i := range d.Points {
+   for j := range d.Points[i] {
+   err = d.Points[i][j].OutputJson(w, i, len(d.Points))
+   if err != nil {
+

[Xen-devel] [PATCH RFC 41/59] libxl.go: Link statically rather than dynamically

[Ronald Rojas]

From: George Dunlap 

Link statically rather than dynamically, because Go prefers that.

This also gets rid of the need for schedbench-report, so disable that
from the build (although keep the rule and the dummy file around just
in case.

Also update README.md.

Signed-off-by: George Dunlap 
---
 Makefile | 11 +++
 libxl.go | 10 +-
 2 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 0265dc8..699cc53 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,4 @@
-BIN = schedbench schedbench-report
+BIN = schedbench
 BINALL = $(BIN)
 
 .PHONY: all
@@ -10,12 +10,15 @@ CGO_CFLAGS = 
-I/build/hg/xen.git/dist/install/usr/local/include
 
 # FIXME
 XENLIB_PATH ?= /build/hg/xen.git/dist/install/usr/local/lib/
-CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH)
+CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH) 
 
 schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go plan.go
-   CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
+   CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build 
-ldflags '-linkmode external -extldflags "-static"' -o $@ $^
 
-# FIXME: Do with dlopen instead
+# If we use a statically linked binary we don't need this; the same
+# binary can be used on any system.  Keep this version (without any
+# run support) support) around for now in case we want to go back to
+# it.
 schedbench-report: main.go benchmark.go stubs.go htmlreport.go plan.go
go build -o $@ $^
 
diff --git a/libxl.go b/libxl.go
index 6621974..27e7766 100644
--- a/libxl.go
+++ b/libxl.go
@@ -19,12 +19,20 @@
 package main
 
 /*
-#cgo LDFLAGS: -lyajl -lxenlight
+#cgo LDFLAGS: -lxenlight -lyajl_s -lxengnttab -lxenstore -lxenguest 
-lxentoollog -lxenevtchn -lxenctrl -lblktapctl -lxenforeignmemory -lxencall -lz 
-luuid -lutil
 #include 
 #include 
 */
 import "C"
 
+/*
+ * Other flags that may be needed at some point: 
+ *  -lnl-route-3 -lnl-3
+ *
+ * To get back to simple dynamic linking:
+#cgo LDFLAGS: -lxenlight -lyajl
+*/
+
 import (
"unsafe"
"fmt"
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 46/59] libxl: Reorganize bitmapGotoC

[Ronald Rojas]

From: George Dunlap 

No need to "allocate" the structure elsewhere; passing as a return
value makes usage a lot cleaner.

Signed-off-by: George Dunlap 
---
 libxl.go | 15 +++
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/libxl.go b/libxl.go
index 92064ca..16f4645 100644
--- a/libxl.go
+++ b/libxl.go
@@ -191,8 +191,8 @@ func bitmapCToGo(cbm *C.libxl_bitmap) (gbm Bitmap) {
 }
 
 // Must be C.libxl_bitmap_dispose'd of afterwards
-func bitmapGotoC(gbm Bitmap, cbm *C.libxl_bitmap) {
-   C.libxl_bitmap_init(cbm)
+func bitmapGotoC(gbm Bitmap) (cbm C.libxl_bitmap) {
+   C.libxl_bitmap_init()
 
size := len(gbm.bitmap)
cbm._map = (*C.uint8_t)(C.malloc(C.size_t(size)))
@@ -206,6 +206,8 @@ func bitmapGotoC(gbm Bitmap, cbm *C.libxl_bitmap) {
 
// And copy the Go array into the C array
copy(mapslice, gbm.bitmap)
+
+   return
 }
 
 func (bm *Bitmap) Test(bit int) (bool) {
@@ -385,8 +387,7 @@ func (Ctx *Context) CpupoolCreate(Name string, Scheduler 
Scheduler, Cpumap Bitma
var uuid C.libxl_uuid
C.libxl_uuid_generate()
 
-   var cbm C.libxl_bitmap
-   bitmapGotoC(Cpumap, )
+   cbm := bitmapGotoC(Cpumap)
defer C.libxl_bitmap_dispose()

ret := C.libxl_cpupool_create(Ctx.ctx, name, 
C.libxl_scheduler(Scheduler),
@@ -429,8 +430,7 @@ func (Ctx *Context) CpupoolCpuadd(Poolid uint32, Cpu int) 
(err error) {
 // int libxl_cpupool_cpuadd_cpumap(libxl_ctx *ctx, uint32_t poolid,
 // const libxl_bitmap *cpumap);
 func (Ctx *Context) CpupoolCpuaddCpumap(Poolid uint32, Cpumap Bitmap) (err 
error) {
-   var cbm C.libxl_bitmap
-   bitmapGotoC(Cpumap, )
+   cbm := bitmapGotoC(Cpumap)
defer C.libxl_bitmap_dispose()

ret := C.libxl_cpupool_cpuadd_cpumap(Ctx.ctx, C.uint32_t(Poolid), )
@@ -458,8 +458,7 @@ func (Ctx *Context) CpupoolCpuremove(Poolid uint32, Cpu 
int) (err error) {
 // int libxl_cpupool_cpuremove_cpumap(libxl_ctx *ctx, uint32_t poolid,
 //const libxl_bitmap *cpumap);
 func (Ctx *Context) CpupoolCpuremoveCpumap(Poolid uint32, Cpumap Bitmap) (err 
error) {
-   var cbm C.libxl_bitmap
-   bitmapGotoC(Cpumap, )
+   cbm := bitmapGotoC(Cpumap)
defer C.libxl_bitmap_dispose()

ret := C.libxl_cpupool_cpuremove_cpumap(Ctx.ctx, C.uint32_t(Poolid), 
)
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 34/59] controller: Make a useful config file

[Ronald Rojas]

From: George Dunlap 

Allow a simple "skeleton" config file which can be adapted for the
particular use case.  Have 'plan' expand this with the real benchmark
plan.

Include "sample.bench" as a template to modify.  Also update the
README.md accordingly.

Signed-off-by: George Dunlap 
---
 Makefile |  4 +--
 benchmark.go |  9 ---
 main.go  | 48 ++--
 plan.go  | 79 
 4 files changed, 93 insertions(+), 47 deletions(-)
 create mode 100644 plan.go

diff --git a/Makefile b/Makefile
index 6dee499..af55e0a 100644
--- a/Makefile
+++ b/Makefile
@@ -13,11 +13,11 @@ CGO_LIBS = -lyajl -lxenlight
 XENLIB_PATH ?= /build/hg/xen.git/dist/install/usr/local/lib/
 CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH) $(CGO_LIBS)
 
-schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go
+schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go plan.go
CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
 
 # FIXME: Do with dlopen instead
-schedbench-report: main.go processworker.go xenworker_dummy.go benchmark.go 
run.go htmlreport.go
+schedbench-report: main.go processworker.go xenworker_dummy.go benchmark.go 
run.go htmlreport.go plan.go
go build -o $@ $^
 
 .PHONY: clean
diff --git a/benchmark.go b/benchmark.go
index 8be00a0..31b3711 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -148,12 +148,13 @@ type BenchmarkRun struct {
 }
 
 type BenchmarkPlan struct {
-   filename string
-   WorkerType int
+   Input *PlanInput `json:",omitempty"`
+   filename string  `json:",omitempty"`
+   WorkerType int   `json:",omitempty"`
// Global options for workers that will be over-ridden by Run
// and WorkerSet config options
-   WorkerConfig
-   Runs []BenchmarkRun
+   WorkerConfig `json:",omitempty"`
+   Runs []BenchmarkRun  `json:",omitempty"`
 }
 
 func (run *BenchmarkRun) checkSummary() (done bool, err error) {
diff --git a/main.go b/main.go
index 2aa8bae..059a54a 100644
--- a/main.go
+++ b/main.go
@@ -49,49 +49,15 @@ func main() {
verbosity, _ = strconv.Atoi(Args[1])
Args = Args[2:]
case "plan":
-   workerA := []string{"burnwait", "70", "20"}
-   //workerB := []string{"burnwait", "10", "2000"}
-   workerB := []string{"burnwait", "10", "30",
-   "burnwait", "20", "30",
-   "burnwait", "10", "30",
-   "burnwait", "10", "30",
-   "burnwait", "10", "30",
-   "burnwait", "10", "30",
-   "burnwait", "30", "30",
-   }
-   
-   
-   plan :=  BenchmarkPlan{
-   WorkerType:WorkerXen,
-   WorkerConfig:WorkerConfig{Pool:"schedbench"},
-   filename:filename,
-   Runs:[]BenchmarkRun{
-   {Label:"baseline-a",
-   WorkerSets:[]WorkerSet{
-   
{Params:WorkerParams{workerA},
-   Count:1}},
-   RuntimeSeconds:10,},
-   {Label:"baseline-b",
-   WorkerSets:[]WorkerSet{
-   
{Params:WorkerParams{workerB},
-   Count:1}},
-   RuntimeSeconds:10,},
-   }}
-   
-   for i := 1; i <= 16 ; i *= 2 {
-   label := fmt.Sprintf("%da+%db", i, i)
-   run := BenchmarkRun{
-   Label:label,
-   WorkerSets:[]WorkerSet{
-   {Params:WorkerParams{workerA},
-   Count:i},
-   {Params:WorkerParams{workerB},
-   Count:i}},
-   RuntimeSeconds:10}
-   plan.Runs = append(plan.Runs, run)
+   plan, err := LoadBenchmark(filename)
+   if err != nil {
+   

[Xen-devel] [PATCH RFC 42/59] plan: Allow "templating" from other runs

[Ronald Rojas]

From: George Dunlap 

Allow 'plan' to use an existing run as a "template": load the template,
discard the runs, and save the result in the new file.

Use like this:

./schedbench -t 18.bench -f 20.bench plan

This will make 20 with an empty plan identical to that in 18.

Signed-off-by: George Dunlap 
---
 main.go | 38 +-
 plan.go | 32 ++--
 2 files changed, 63 insertions(+), 7 deletions(-)

diff --git a/main.go b/main.go
index 8cbd708..ead6ab0 100644
--- a/main.go
+++ b/main.go
@@ -30,6 +30,7 @@ func main() {
 
Args = Args[1:]
filename := "test.bench"
+   template := ""
verbosity := 0
 
for len(Args) > 0 {
@@ -41,6 +42,13 @@ func main() {
}
filename = Args[1]
Args = Args[2:]
+   case "-t":
+   if len(Args) < 2 {
+   fmt.Println("Need arg for -t")
+   os.Exit(1)
+   }
+   template = Args[1]
+   Args = Args[2:]
case "-v":
if len(Args) < 2 {
fmt.Println("Need arg for -v")
@@ -49,20 +57,40 @@ func main() {
verbosity, _ = strconv.Atoi(Args[1])
Args = Args[2:]
case "plan":
-   plan, err := LoadBenchmark(filename)
+   // Load either the template benchmark or the filename
+   loadfile := filename
+   if template != "" {
+   loadfile = template
+   }
+   plan, err := LoadBenchmark(loadfile)
if err != nil {
-   fmt.Println("Loading benchmark ", filename, " 
", err)
+   fmt.Printf("Loading benchmark %s: %v\n",
+   loadfile, err)
os.Exit(1)
}
+
+   if template != "" {
+   plan.filename = filename
+   err = plan.ClearRuns()
+   if err != nil {
+   fmt.Printf("Clearing runs: %v\n",
+   err)
+   os.Exit(1)
+   }
+   }

-   plan.ExpandInput()
+   err = plan.ExpandInput()
+   if err != nil {
+   fmt.Printf("Expanding plan: %v\n", err)
+   os.Exit(1)
+   }
 
err = plan.Save()
if err != nil {
-   fmt.Println("Saving plan ", filename, " ", err)
+   fmt.Printf("Saving plan %s: %v\n", filename, 
err)
os.Exit(1)
}
-   fmt.Println("Created plan in ", filename)
+   fmt.Printf("Created plan in %s\n", filename)
Args = Args[1:]
case "run":
plan, err := LoadBenchmark(filename)
diff --git a/plan.go b/plan.go
index 24c6bd4..e535603 100644
--- a/plan.go
+++ b/plan.go
@@ -1,3 +1,21 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
 package main
 
 import (
@@ -19,15 +37,25 @@ var WorkerPresets = map[string]WorkerParams{
"P001":WorkerParams{[]string{"burnwait", "70", "20"}},
 }
 
+func (plan *BenchmarkPlan) ClearRuns() (err error) {
+   plan.Runs = nil
+
+   return
+}
 
 func (plan *BenchmarkPlan) ExpandInput() (err error) {
if plan.Runs != nil {
-   fmt.Printf("plan.Expand: Runs non-empty, not doing anything\n");
+   err = fmt.Errorf("Runs non-empty, not doing anything\n");
+   return
+   }
+
+   if plan.Input == nil {
+   err = fmt.Errorf("Input nil, 

[Xen-devel] [PATCH RFC 47/59] libxl: Reorganize code

[Ronald Rojas]

From: George Dunlap 

Put all type and enumeration definitions at the top of the file,
separated into "builtins" and "idl-generated".

Also define Uuid as C.libxl_uud.  (Not sure if this gives us the
functionality we need without having to manually copy things in.)

Finally, get rid of Bitmap.Alloc().  It's redundant now that
Bitmap.Set() automatically extends the array.

Signed-off-by: George Dunlap 
---
 libxl.go | 189 ++-
 1 file changed, 102 insertions(+), 87 deletions(-)

diff --git a/libxl.go b/libxl.go
index 16f4645..e73c06b 100644
--- a/libxl.go
+++ b/libxl.go
@@ -39,10 +39,102 @@ import (
"time"
 )
 
+/*
+ * Types: Builtins
+ */
+
+type Domid uint32
+
+type MemKB uint64
+
+// typedef struct {
+// uint32_t size;  /* number of bytes in map */
+// uint8_t *map;
+// } libxl_bitmap;
+
+// Implement the Go bitmap type such that the underlying data can
+// easily be copied in and out.  NB that we still have to do copies
+// both directions, because cgo runtime restrictions forbid passing to
+// a C function a pointer to a Go-allocated structure which contains a
+// pointer.
+type Bitmap struct {
+   bitmap []C.uint8_t
+}
+
 type Context struct {
ctx *C.libxl_ctx
 }
 
+type Uuid C.libxl_uuid
+
+/*
+ * Types: IDL
+ * 
+ * FIXME: Generate these automatically from the IDL
+ */
+type Dominfo struct {
+   Uuid  Uuid
+   Domid Domid
+   Running   bool
+   Blocked   bool
+   Pausedbool
+   Shutdown  bool
+   Dying bool
+   Never_stopbool
+   
+   Shutdown_reason   int32 // FIXME shutdown_reason enumeration
+   Outstanding_memkb MemKB
+   Current_memkb MemKB
+   Shared_memkb  MemKB
+   Paged_memkb   MemKB
+   Max_memkb MemKB
+   Cpu_time  time.Duration
+   Vcpu_max_id   uint32
+   Vcpu_online   uint32
+   Cpupool   uint32
+   Domain_type   int32 //FIXME libxl_domain_type enumeration
+
+}
+
+// # Consistent with values defined in domctl.h
+// # Except unknown which we have made up
+// libxl_scheduler = Enumeration("scheduler", [
+// (0, "unknown"),
+// (4, "sedf"),
+// (5, "credit"),
+// (6, "credit2"),
+// (7, "arinc653"),
+// (8, "rtds"),
+// ])
+type Scheduler int
+var (
+   SchedulerUnknown  Scheduler = C.LIBXL_SCHEDULER_UNKNOWN
+   SchedulerSedf Scheduler = C.LIBXL_SCHEDULER_SEDF
+   SchedulerCredit   Scheduler = C.LIBXL_SCHEDULER_CREDIT
+   SchedulerCredit2  Scheduler = C.LIBXL_SCHEDULER_CREDIT2
+   SchedulerArinc653 Scheduler = C.LIBXL_SCHEDULER_ARINC653
+   SchedulerRTDS Scheduler = C.LIBXL_SCHEDULER_RTDS
+)
+
+// libxl_cpupoolinfo = Struct("cpupoolinfo", [
+// ("poolid",  uint32),
+// ("pool_name",   string),
+// ("sched",   libxl_scheduler),
+// ("n_dom",   uint32),
+// ("cpumap",  libxl_bitmap)
+// ], dir=DIR_OUT)
+
+type CpupoolInfo struct {
+   Poolid uint32
+   PoolName string
+   Scheduler Scheduler
+   DomainCount int
+   Cpumap Bitmap
+}
+
+/*
+ * Context
+ */
 var Ctx Context
 
 func (Ctx *Context) IsOpen() bool {
@@ -72,43 +164,12 @@ func (Ctx *Context) Close() (err error) {
return
 }
 
-// Builtins
-type Domid uint32
-
-type MemKB uint64
-
-// FIXME: Use the idl to generate types
-type Dominfo struct {
-   // FIXME: uuid
-   Domid Domid
-   Running   bool
-   Blocked   bool
-   Pausedbool
-   Shutdown  bool
-   Dying bool
-   Never_stopbool
-   
-   Shutdown_reason   int32 // FIXME shutdown_reason enumeration
-   Outstanding_memkb MemKB
-   Current_memkb MemKB
-   Shared_memkb  MemKB
-   Paged_memkb   MemKB
-   Max_memkb MemKB
-   Cpu_time  time.Duration
-   Vcpu_max_id   uint32
-   Vcpu_online   uint32
-   Cpupool   uint32
-   Domain_type   int32 //FIXME libxl_domain_type enumeration
-
-}
-
 func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err error) {
if Ctx.ctx == nil {
err = fmt.Errorf("Context not opened")
return
}
 
-   
var cdi C.libxl_dominfo
 
ret := C.libxl_domain_info(Ctx.ctx, unsafe.Pointer(), 
C.uint32_t(Id))
@@ -119,8 +180,12 @@ func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err 
error) {
return
}
 
-   // FIXME -- use introspection to make this more robust
+   // We could consider having this boilerplate generated by the
+   // idl, in a function like this:
+   //
+   // di = translateCdomaininfoToGoDomaininfo(cdi)
di = {}
+   di.Uuid = Uuid(cdi.uuid)
di.Domid 

[Xen-devel] [PATCH RFC 28/59] controller: Handle worker early death

[Ronald Rojas]

From: George Dunlap 

Log raw worker output.  In the event of an unexpected worker death,
dump the output and stop further processing.

Also fix an assert that caused workers to die if the timer was too
exact.

Signed-off-by: George Dunlap 
---
 benchmark.go |  7 ---
 processworker.go | 18 --
 run.go   | 24 ++--
 xenworker.go | 21 +++--
 4 files changed, 53 insertions(+), 17 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 5e35997..de1f650 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -74,13 +74,6 @@ type WorkerSet struct {
Count int
 }
 
-type Worker interface {
-   SetId(WorkerId)
-   Init(WorkerParams, WorkerConfig) error
-   Shutdown()
-   Process(chan WorkerReport, chan bool)
-}
-
 const (
USEC = 1000
MSEC = USEC * 1000
diff --git a/processworker.go b/processworker.go
index f517321..999e76a 100644
--- a/processworker.go
+++ b/processworker.go
@@ -32,6 +32,7 @@ type ProcessWorker struct {
c *exec.Cmd
stdout io.ReadCloser
jsonStarted bool
+   Log []string
 }
 
 func (w *ProcessWorker) SetId(i WorkerId) {
@@ -54,7 +55,19 @@ func (w *ProcessWorker) Shutdown() {
w.c.Process.Kill()
 }
 
-func (w *ProcessWorker) Process(report chan WorkerReport, done chan bool) {
+func (w *ProcessWorker) DumpLog(f io.Writer) (err error) {
+   b := bufio.NewWriter(f)
+   defer b.Flush()
+   for _, line := range w.Log {
+   _, err = fmt.Println(b, line)
+   if err != nil {
+   return
+   }
+   }
+   return
+}
+
+func (w *ProcessWorker) Process(report chan WorkerReport, done chan WorkerId) {
w.c.Start()
 
scanner := bufio.NewScanner(w.stdout)
@@ -63,6 +76,7 @@ func (w *ProcessWorker) Process(report chan WorkerReport, 
done chan bool) {
s := scanner.Text()

//fmt.Println("Got these bytes: ", s);
+   w.Log = append(w.Log, s)
 
if w.jsonStarted {
var r WorkerReport
@@ -77,7 +91,7 @@ func (w *ProcessWorker) Process(report chan WorkerReport, 
done chan bool) {
}
}
 
-   done <- true
+   done <- w.id
 
w.c.Wait()
 }
diff --git a/run.go b/run.go
index ed1957b..1b39730 100644
--- a/run.go
+++ b/run.go
@@ -26,6 +26,7 @@ import (
"regexp"
"strconv"
"bufio"
+   "io"
 )
 
 type WorkerState struct {
@@ -33,6 +34,14 @@ type WorkerState struct {
LastReport WorkerReport
 }
 
+type Worker interface {
+   SetId(WorkerId)
+   Init(WorkerParams, WorkerConfig) error
+   Shutdown()
+   Process(chan WorkerReport, chan WorkerId)
+   DumpLog(io.Writer) error
+}
+
 func Report(ws *WorkerState, r WorkerReport) {
//fmt.Println(r)
 
@@ -57,7 +66,7 @@ func Report(ws *WorkerState, r WorkerReport) {
 
 type WorkerList map[WorkerId]*WorkerState
 
-func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) (i int) {
+func (ws *WorkerList) Start(report chan WorkerReport, done chan WorkerId) (i 
int) {
i = 0
for j := range *ws {
go (*ws)[j].w.Process(report, done)
@@ -160,7 +169,7 @@ func (run *BenchmarkRun) Run() (err error) {
}

report := make(chan WorkerReport)
-   done := make(chan bool)
+   done := make(chan WorkerId)
signals := make(chan os.Signal, 1)
 
signal.Notify(signals, os.Interrupt)
@@ -179,12 +188,16 @@ func (run *BenchmarkRun) Run() (err error) {
run.Results.Raw = append(run.Results.Raw, r)
Report(Workers[r.Id], r)
}
-   case <-done:
+   case did := <-done:
if ! stopped {
-   fmt.Println("WARNING: Worker left early")
+   fmt.Println("WARNING: Worker", did, "left 
early, shutting down workers")
+   Workers.Stop()
+   stopped = true
+   err = fmt.Errorf("Worker %v exited early", did)
+   Workers[did].w.DumpLog(os.Stdout)
}
i--;
-   fmt.Println(i, "workers left");
+   fmt.Printf("Worker %v exited; %d workers left\n", did, 
i);
case <-timeout:
if ! stopped {
Workers.Stop()
@@ -201,7 +214,6 @@ func (run *BenchmarkRun) Run() (err error) {
}
err = fmt.Errorf("Interrupted")
} else {
-   err = fmt.Errorf("Interrupted")
fmt.Println("SIGINT received after stop, 

[Xen-devel] [PATCH RFC 30/59] htmlreport: Include utilization scatterplots

[Ronald Rojas]

From: George Dunlap 

Reformat RunRaw so that it's a more generic scatterplot, with the
caller specifying the javascript tag, title, and axes.  Keep them as a
straight list to be displayed rather than defining them on a per-run
basis.

Collect a slice of utilization points at the same time as collecting
the throughput metrics, add them both at the same time.

While we're here, add "omitempty" to some of the options so no json is
generated if nothing is specified.  Leave "MinValue" so that we are
able to specify a zero origin (which will also be the default).

Finally, don't bother finding minimum and maximum points; the chart
plotter will do that anyway.

Signed-off-by: George Dunlap 
---
 htmlreport.go | 89 +++
 1 file changed, 47 insertions(+), 42 deletions(-)

diff --git a/htmlreport.go b/htmlreport.go
index 6f61998..545582d 100644
--- a/htmlreport.go
+++ b/htmlreport.go
@@ -26,16 +26,17 @@ import (
 )
 
 type OptionAxis struct {
-   Title string `json:"title"`
+   Title string `json:"title,omitempty"`
+   // Always include this one so that we can start graphs at 0
MinValue float64 `json:"minValue"`
-   MaxValue float64 `json:"maxValue"`
+   MaxValue float64 `json:"maxValue,omitempty"`
 }
 
 type Options struct {
-   Title string `json:"title"`
+   Title string `json:"title,omitempty"`
HAxis OptionAxis `json:"hAxis"`
VAxis OptionAxis `json:"vAxis"`
-   Legend string`json:"legend"`
+   Legend string`json:"legend,omitempty"`
 }
 
 type Point struct {
@@ -44,18 +45,21 @@ type Point struct {
 }
 
 type RunRaw struct {
-   Label string
+   Tag string
+   Title string
+   hTitle string
+   vTitle string
Points [][]Point
 }
 
-func (options *Options) OutputJavascript(w io.Writer, id int) (err error) {
+func (options *Options) OutputJavascript(w io.Writer, tag string) (err error) {
var optionsJson []byte
optionsJson, err = json.Marshal(options)
if err != nil {
return
}
 
-   fmt.Fprintf(w, "var sp%dopt = ", id)
+   fmt.Fprintf(w, "var %sopt = ", tag)
fmt.Fprint(w, string(optionsJson))
fmt.Fprintln(w, ";")
 
@@ -75,41 +79,29 @@ func (p *Point) OutputJson(w io.Writer, id int, max int) 
(err error) {
return
 }
 
-func (d *RunRaw) OutputHTML(w io.Writer, run int) (err error) {
-   fmt.Fprintf(w, "\n", run)
+func (d *RunRaw) OutputHTML(w io.Writer) (err error) {
+   fmt.Fprintf(w, "\n", d.Tag)
return
 }
 
-func (d *RunRaw) OutputJavascript(w io.Writer, run int) (err error) {
+func (d *RunRaw) OutputJavascript(w io.Writer) (err error) {
var options Options
 
-   options.Title = fmt.Sprintf("Run %s (%d) Individual Throughput", 
d.Label, run)
-   options.HAxis.Title = "Time"
-   options.VAxis.Title = "Throughput"
+   options.Title = d.Title
+   options.HAxis.Title = d.hTitle
+   options.VAxis.Title = d.vTitle
 
-   var xmm MinMax
-   var ymm MinMax
-   for i := range d.Points {
-   for j := range d.Points[i] {
-   xmm.Update(d.Points[i][j].x)
-   ymm.Update(d.Points[i][j].y)
-   }
-   }
-
-   options.HAxis.MaxValue = xmm.Max
-   options.VAxis.MaxValue = ymm.Max
-
-   err = options.OutputJavascript(w, run)
+   err = options.OutputJavascript(w, d.Tag)
if err != nil {
return
}
 
-   fmt.Printf("var sp%ddata = new 
google.visualization.DataTable();\n", run)
-   fmt.Printf("sp%ddata.addColumn('number', 'Time');\n", run)
+   fmt.Printf("var %sdata = new 
google.visualization.DataTable();\n", d.Tag)
+   fmt.Printf("%sdata.addColumn('number', 'Time');\n", d.Tag)
for i := range d.Points {
-   fmt.Printf("sp%ddata.addColumn('number', 'Worker 
%d');\n", run, i)
+   fmt.Printf("%sdata.addColumn('number', 'Worker 
%d');\n", d.Tag, i)
}
-   fmt.Printf("sp%ddata.addRows([\n", run)
+   fmt.Printf("%sdata.addRows([\n", d.Tag)
 
// Can't use json here because we need to be able to use 'null' for 
non-existent values
for i := range d.Points {
@@ -122,8 +114,8 @@ func (d *RunRaw) OutputJavascript(w io.Writer, run int) 
(err error) {
}
fmt.Print("  ]);\n")

-   fmt.Printf("var sp%dchart = new 
google.visualization.ScatterChart(document.getElementById('scatterplot%d'));\n",
 run, run);
-   fmt.Printf("sp%dchart.draw(sp%ddata, sp%dopt);\n\n", run, run, 
run)
+   fmt.Printf("var %schart = new 
google.visualization.ScatterChart(document.getElementById('scatterplot%s'));\n",
 d.Tag, d.Tag);
+   fmt.Printf("%schart.draw(%sdata, %sopt);\n\n", 

[Xen-devel] [PATCH RFC 55/59] plan: Make the matrix generation more programmatic

[Ronald Rojas]

From: George Dunlap 

Rather than have nested loops, keep a running list of all the configs
we've created so far.  Then for each existing new element of the
matrix, combine it with all the existing columns.

Signed-off-by: George Dunlap 
---
 plan.go | 50 +++---
 1 file changed, 31 insertions(+), 19 deletions(-)

diff --git a/plan.go b/plan.go
index e535603..736d9f3 100644
--- a/plan.go
+++ b/plan.go
@@ -72,7 +72,9 @@ func (plan *BenchmarkPlan) ExpandInput() (err error) {
schedulers = append(schedulers, "")
}
 
-   // Always do the baselines
+   // Start by making a slice with baselines and each of the counts
+   var a, b []BenchmarkRun
+   
for _, wn := range plan.Input.SimpleMatrix.Workers {
wp := WorkerPresets[wn]

@@ -85,39 +87,49 @@ func (plan *BenchmarkPlan) ExpandInput() (err error) {
WorkerSets:[]WorkerSet{{Params:wp, Count:1}},
RuntimeSeconds:10,
}
-   
-   for _, s := range schedulers {
-   fmt.Printf("Making baseline %s run, sched %s\n", wn, s)
-   run.RunConfig.Scheduler = s
-   run.Label = wn+" baseline "+s
-   plan.Runs = append(plan.Runs, run)
-   }
+
+   run.Label = wn+" baseline"
+   a = append(a, run)
}
-   
+
+
for _, c := range plan.Input.SimpleMatrix.Count {
run := BenchmarkRun{
RuntimeSeconds:10,
}

-   var label string
for _, wn := range plan.Input.SimpleMatrix.Workers {
wp := WorkerPresets[wn]

-   if label != "" {
-   label = label+" + "
+   if run.Label != "" {
+   run.Label = run.Label+" + "
}
-   label = fmt.Sprintf("%s%s %d ", label, wn, c)
-   
+   run.Label = fmt.Sprintf("%s%s %d", run.Label, wn, c)
+
ws := WorkerSet{Params:wp, Count:c}
run.WorkerSets = append(run.WorkerSets, ws)
}
-   for _, s := range schedulers {
-   fmt.Printf("Making count %d run, sched %s\n", c, s)
-   run.RunConfig.Scheduler = s
-   run.Label = label+s
-   plan.Runs = append(plan.Runs, run)
+
+   a = append(a, run)
+   }
+
+   // ...then cross it by schedulers
+   if len(schedulers) > 0 {
+   for _, base := range a {
+   for _, s := range schedulers {
+   run := base
+   run.RunConfig.Scheduler = s
+   run.Label = run.Label+" "+s
+   b = append(b, run)
+   }
}
+   a = b
+   b = nil
}
 
+   for i := range a {
+   fmt.Printf("%s\n", a[i].Label)
+   }
+   plan.Runs = a;
return
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 50/59] libxl: Fix Bitmap.Max(), make Test() / Clear() more robust

[Ronald Rojas]

From: George Dunlap 

Bitmap.Max() should return the largest currently addressable bit; at the
moment it returns the total number of bits (which is one greater than
the largest addressable bit).

This also implicitly fixes a bug in Test() and Clear() which would cause
an out-of-range error when testing a bit equal to Max().

While we're here, allow Test() and Clear() to handle null bitmaps, the
same way Set() can.

Signed-off-by: George Dunlap 
---
 libxl.go | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libxl.go b/libxl.go
index dfe4f40..bfcaf0d 100644
--- a/libxl.go
+++ b/libxl.go
@@ -270,7 +270,7 @@ func bitmapGotoC(gbm Bitmap) (cbm C.libxl_bitmap) {
 
 func (bm *Bitmap) Test(bit int) (bool) {
ubit := uint(bit)
-   if (bit > bm.Max()) {
+   if (bit > bm.Max() || bm.bitmap == nil) {
return false
}

@@ -294,7 +294,7 @@ func (bm *Bitmap) SetRange(start int, end int) {
 
 func (bm *Bitmap) Clear(bit int) {
ubit := uint(bit)
-   if (bit > bm.Max()) {
+   if (bit > bm.Max() || bm.bitmap == nil) {
return
}

@@ -308,7 +308,7 @@ func (bm *Bitmap) ClearRange(start int, end int) {
 }
 
 func (bm *Bitmap) Max() (int) {
-   return len(bm.bitmap) * 8
+   return len(bm.bitmap) * 8 - 1
 }
 
 func (bm *Bitmap) IsEmpty() (bool) {
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 16/59] controller: Rename an element in BenchmarkRun to be more accurate

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 benchmark.go | 6 +++---
 main.go  | 6 +++---
 run.go   | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 2e03fe5..aaa4c98 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -84,7 +84,7 @@ type BenchmarkRunData struct {
 
 type BenchmarkRun struct {
Label string
-   Workers []WorkerSet
+   WorkerSets []WorkerSet
RuntimeSeconds int
Completed bool
Results BenchmarkRunData 
@@ -168,8 +168,8 @@ func (run *BenchmarkRun) TextReport() (err error) {
fmt.Printf("== RUN %s ==", run.Label)
 
wStart := 0
-   for i := range run.Workers {
-   ws := [i]
+   for i := range run.WorkerSets {
+   ws := [i]
n := ws.Count
params := ""
for _, s := range ws.Params.Args {
diff --git a/main.go b/main.go
index ebf990f..f58068f 100644
--- a/main.go
+++ b/main.go
@@ -46,12 +46,12 @@ func main() {
filename:filename,
Runs:[]BenchmarkRun{
{Label:"baseline-a",
-   Workers:[]WorkerSet{
+   WorkerSets:[]WorkerSet{
{Params:WorkerParams{workerA},
Count:1}},
RuntimeSeconds:10,},
{Label:"baseline-b",
-   Workers:[]WorkerSet{
+   WorkerSets:[]WorkerSet{
{Params:WorkerParams{workerB},
Count:1}},
RuntimeSeconds:10,},
@@ -62,7 +62,7 @@ func main() {
label := fmt.Sprintf("%da+%db", i, i)
run := BenchmarkRun{
Label:label,
-   Workers:[]WorkerSet{
+   WorkerSets:[]WorkerSet{
{Params:WorkerParams{workerA},
Count:i},
{Params:WorkerParams{workerB},
diff --git a/run.go b/run.go
index 7b5ef0a..762b408 100644
--- a/run.go
+++ b/run.go
@@ -105,7 +105,7 @@ func NewWorkerList(WorkerSets []WorkerSet, workerType int) 
(wl WorkerList, err e
 }
 
 func (run *BenchmarkRun) Run() (err error) {
-   Workers, err := NewWorkerList(run.Workers, WorkerXen)
+   Workers, err := NewWorkerList(run.WorkerSets, WorkerXen)
if err != nil {
fmt.Println("Error creating workers: %v", err)
return
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 31/59] Make a binary that can run reports on a system without libxl

[Ronald Rojas]

From: George Dunlap 

To simplify report-generation and iteration, make a binary that
doesn't link against libxl.  Create a "dummy" Xen library that just
returns errors and warnings.

A better way to do this is probably to do some dlopen hackery (so we
can use the same binary), but that's for another day.

Signed-off-by: George Dunlap 
---
 Makefile   |  6 +-
 xenworker_dummy.go | 54 ++
 2 files changed, 59 insertions(+), 1 deletion(-)
 create mode 100644 xenworker_dummy.go

diff --git a/Makefile b/Makefile
index c1b9ee4..6dee499 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,4 @@
-BIN = schedbench
+BIN = schedbench schedbench-report
 BINALL = $(BIN)
 
 .PHONY: all
@@ -16,6 +16,10 @@ CGO_LDFLAGS = -L$(XENLIB_PATH) 
-Wl,-rpath-link=$(XENLIB_PATH) $(CGO_LIBS)
 schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go
CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
 
+# FIXME: Do with dlopen instead
+schedbench-report: main.go processworker.go xenworker_dummy.go benchmark.go 
run.go htmlreport.go
+   go build -o $@ $^
+
 .PHONY: clean
 clean:
rm -f $(BINALL)
diff --git a/xenworker_dummy.go b/xenworker_dummy.go
new file mode 100644
index 000..02e90ce
--- /dev/null
+++ b/xenworker_dummy.go
@@ -0,0 +1,54 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
+package main
+
+import (
+   "fmt"
+   "io"
+)
+
+type XenWorker struct {
+}
+
+func (w *XenWorker) SetId(i WorkerId) {
+}
+
+func (w *XenWorker) Init(p WorkerParams, g WorkerConfig) (err error) {
+   err = fmt.Errorf("Xen functionality not implemented");
+   return
+}
+
+// FIXME: Return an error
+func (w *XenWorker) Shutdown() {
+   
+   return
+}
+
+func (w *XenWorker) DumpLog(f io.Writer) (err error) {
+   err = fmt.Errorf("Xen functionality not implemented");
+   return
+}
+
+
+
+// FIXME: Return an error
+func (w *XenWorker) Process(report chan WorkerReport, done chan WorkerId) {
+   return;
+}
+
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 58/59] remove irrelevant files from old repository

[Ronald Rojas]

Signed-off-by: Ronald Rojas 
---
 Makefile   |  31 --
 benchmark.go   | 467 --
 htmlreport.go  | 238 --
 libxl.go   | 720 -
 main.go| 146 -
 plan.go| 157 -
 processworker.go   |  98 --
 run.go | 456 --
 stubs.go   |  34 --
 tools/golang/xenlight/libxl.go | 720 +
 xenworker.go   | 278 
 11 files changed, 720 insertions(+), 2625 deletions(-)
 delete mode 100644 Makefile
 delete mode 100644 benchmark.go
 delete mode 100644 htmlreport.go
 delete mode 100644 libxl.go
 delete mode 100644 main.go
 delete mode 100644 plan.go
 delete mode 100644 processworker.go
 delete mode 100644 run.go
 delete mode 100644 stubs.go
 create mode 100644 tools/golang/xenlight/libxl.go
 delete mode 100644 xenworker.go

diff --git a/Makefile b/Makefile
deleted file mode 100644
index 699cc53..000
--- a/Makefile
+++ /dev/null
@@ -1,31 +0,0 @@
-BIN = schedbench
-BINALL = $(BIN)
-
-.PHONY: all
-all: $(BIN)
-
-
-
-CGO_CFLAGS = -I/build/hg/xen.git/dist/install/usr/local/include
-
-# FIXME
-XENLIB_PATH ?= /build/hg/xen.git/dist/install/usr/local/lib/
-CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH) 
-
-schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go 
htmlreport.go plan.go
-   CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build 
-ldflags '-linkmode external -extldflags "-static"' -o $@ $^
-
-# If we use a statically linked binary we don't need this; the same
-# binary can be used on any system.  Keep this version (without any
-# run support) support) around for now in case we want to go back to
-# it.
-schedbench-report: main.go benchmark.go stubs.go htmlreport.go plan.go
-   go build -o $@ $^
-
-.PHONY: clean
-clean:
-   rm -f $(BINALL)
-
-.PHONY: dist
-dist:
-   cp $(BIN) $(DISTDIR)
diff --git a/benchmark.go b/benchmark.go
deleted file mode 100644
index abe2dfb..000
--- a/benchmark.go
+++ /dev/null
@@ -1,467 +0,0 @@
-/*
- * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; version 2 of the
- * License only.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- * 02110-1301, USA.
- */
-package main
-
-import (
-   "fmt"
-   "os"
-   "io/ioutil"
-   "encoding/json"
-   "math"
-   "time"
-)
-
-type WorkerId struct {
-   Set int
-   Id int
-}
-
-func (wid WorkerId) String() (string) {
-   return fmt.Sprintf("%d:%d", wid.Set, wid.Id)
-}
-
-type WorkerReport struct {
-   Id WorkerId
-   Now int
-   Kops int
-   MaxDelta int
-   Cputime time.Duration
-}
-
-type WorkerParams struct {
-   Args []string
-}
-
-func (l *WorkerParams) SetkHZ(kHZ uint64) {
-   if l.Args[0] == "kHZ" {
-   l.Args[1] = fmt.Sprintf("%d", kHZ)
-   } else {
-   l.Args = append([]string{"kHZ", fmt.Sprintf("%d", kHZ)}, 
l.Args...)
-   }
-}
-
-type WorkerConfig struct {
-   Pool string
-   SoftAffinity string
-}
-
-// Propagate unset values from a higher level
-func (l *WorkerConfig) PropagateFrom(g WorkerConfig) {
-   if l.Pool == "" {
-   l.Pool = g.Pool
-   }
-}
-
-type WorkerSet struct {
-   Params WorkerParams
-   Config WorkerConfig
-   Count int
-}
-
-const (
-   USEC = 1000
-   MSEC = USEC * 1000
-   SEC = MSEC * 1000
-)
-
-func Throughput(lt int, lm int, t int, m int) (tput float64) {
-   time := float64(t - lt) / SEC
-   kops := m - lm
-   
-   tput = float64(kops) / time
-   return
-}
-
-func Utilization(lt int, lct time.Duration, t int, ct time.Duration) (util 
float64) {
-   util = float64(ct - lct) / float64(t - lt)
-   return
-}
-
-type MinMax struct {
-   Min float64
-   Max float64
-}
-
-func (mm *MinMax) Update(x float64) {
-   if x > mm.Max {
-   mm.Max = x
-   }
-   if x < mm.Min || mm.Min == 0 {
-   mm.Min = x
-   }
-}
-
-type WorkerSummary struct {
-   Raw []WorkerReport
-   MinMaxTput MinMax
-   MinMaxUtil MinMax
-   TotalTput int
-   TotalTime time.Duration
-   

[Xen-devel] [PATCH RFC 43/59] libxl: Add bitmap support

[Ronald Rojas]

From: George Dunlap 

Make the structure compatible so that we can just copy things out or
in.

Use it to complete the Cpupool structure.

Signed-off-by: George Dunlap 
---
 libxl.go | 104 +++
 1 file changed, 98 insertions(+), 6 deletions(-)

diff --git a/libxl.go b/libxl.go
index 27e7766..3f6ba01 100644
--- a/libxl.go
+++ b/libxl.go
@@ -20,8 +20,8 @@ package main
 
 /*
 #cgo LDFLAGS: -lxenlight -lyajl_s -lxengnttab -lxenstore -lxenguest 
-lxentoollog -lxenevtchn -lxenctrl -lblktapctl -lxenforeignmemory -lxencall -lz 
-luuid -lutil
-#include 
 #include 
+#include 
 */
 import "C"
 
@@ -72,6 +72,7 @@ func (Ctx *Context) Close() (err error) {
return
 }
 
+// Builtins
 type Domid uint32
 
 type MemKB uint64
@@ -160,8 +161,66 @@ func (Ctx *Context) DomainUnpause(Id Domid) (err error) {
 // uint32_t size;  /* number of bytes in map */
 // uint8_t *map;
 // } libxl_bitmap;
-// void libxl_bitmap_init(libxl_bitmap *map);
-// void libxl_bitmap_dispose(libxl_bitmap *map);
+
+// Implement the Go bitmap type such that the underlying data can
+// easily be copied in and out.  NB that we still have to do copies
+// both directions, because cgo runtime restrictions forbid passing to
+// a C function a pointer to a Go-allocated structure which contains a
+// pointer.
+type Bitmap struct {
+   bitmap []C.uint8_t
+}
+
+func (bm *Bitmap) Alloc(max int) {
+   bm.bitmap = make([]C.uint8_t, (max + 7) / 8)
+}
+
+// Return a Go bitmap which is a copy of the referred C bitmap.
+func bitmapCToGo(cbm *C.libxl_bitmap) (bm Bitmap) {
+   // Alloc a Go slice for the bytes
+   size := int(cbm.size)
+   bm.Alloc(size*8)
+
+   // Make a slice pointing to the C array
+   mapslice := (*[1 << 30]C.uint8_t)(unsafe.Pointer(cbm._map))[:size:size]
+
+   // And copy the C array into the Go array
+   copy(bm.bitmap, mapslice)
+
+   return
+}
+
+func (bm *Bitmap) Test(bit int) (bool) {
+   ubit := uint(bit)
+   if (bit > bm.Max()) {
+   return false
+   }
+   
+   return (bm.bitmap[bit / 8] & (1 << (ubit & 7))) != 0
+}
+
+// FIXME: Do we really just want to silently fail here?
+func (bm *Bitmap) Set(bit int) {
+   ubit := uint(bit)
+   if (bit > bm.Max()) {
+   return
+   }
+   
+   bm.bitmap[bit / 8] |= 1 << (ubit & 7)
+}
+
+func (bm *Bitmap) Clear(bit int) {
+   ubit := uint(bit)
+   if (bit > bm.Max()) {
+   return
+   }
+   
+   bm.bitmap[bit / 8] &= ^(1 << (ubit & 7))
+}
+
+func (bm *Bitmap) Max() (int) {
+   return len(bm.bitmap) * 8
+}
 
 // # Consistent with values defined in domctl.h
 // # Except unknown which we have made up
@@ -222,7 +281,7 @@ type CpupoolInfo struct {
PoolName string
Scheduler Scheduler
DomainCount int
-   // Punt on cpumap for now
+   CpuMap Bitmap
 }
 
 // libxl_cpupoolinfo * libxl_list_cpupool(libxl_ctx*, int *nb_pool_out);
@@ -248,6 +307,7 @@ func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
info.PoolName = C.GoString(cpupoolListSlice[i].pool_name)
info.Scheduler = Scheduler(cpupoolListSlice[i].sched)
info.DomainCount = int(cpupoolListSlice[i].n_dom)
+   info.CpuMap = bitmapCToGo([i].cpumap)
 
list = append(list, info)
}
@@ -268,6 +328,24 @@ func (Ctx *Context) CpupoolFindByName(name string) (info 
CpupoolInfo, found bool
return
 }
 
+// int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
+//  libxl_scheduler sched,
+//  libxl_bitmap cpumap, libxl_uuid *uuid,
+//  uint32_t *poolid);
+// int libxl_cpupool_destroy(libxl_ctx *ctx, uint32_t poolid);
+// int libxl_cpupool_rename(libxl_ctx *ctx, const char *name, uint32_t poolid);
+// int libxl_cpupool_cpuadd(libxl_ctx *ctx, uint32_t poolid, int cpu);
+// int libxl_cpupool_cpuadd_node(libxl_ctx *ctx, uint32_t poolid, int node, 
int *cpus);
+// int libxl_cpupool_cpuadd_cpumap(libxl_ctx *ctx, uint32_t poolid,
+// const libxl_bitmap *cpumap);
+// int libxl_cpupool_cpuremove(libxl_ctx *ctx, uint32_t poolid, int cpu);
+// int libxl_cpupool_cpuremove_node(libxl_ctx *ctx, uint32_t poolid, int node, 
int *cpus);
+// int libxl_cpupool_cpuremove_cpumap(libxl_ctx *ctx, uint32_t poolid,
+//const libxl_bitmap *cpumap);
+// int libxl_cpupool_movedomain(libxl_ctx *ctx, uint32_t poolid, uint32_t 
domid);
+// int libxl_cpupool_info(libxl_ctx *ctx, libxl_cpupoolinfo *info, uint32_t 
poolid);
+
+   
 func XlTest(Args []string) {
var Ctx Context
 
@@ -290,7 +368,21 @@ func XlTest(Args []string) {
fmt.Printf("Error: %v\n", err)
}
 
-   fmt.Printf("a: %d b: %s c: %d\n", a, b, int(c)) 
+ 

[Xen-devel] [PATCH RFC 14/59] benchmark: Store data in terms of worker sets and worker ids

[Ronald Rojas]

From: George Dunlap 

...rather than having a single global index, in preparation for
summarizing sets of workers as well as individual workers.

To do this:

- Define a WorkerId type which consists of the worker set + worker id
  within that set.  Pass this to the workers and use this in the
  report values.

- Use maps based on this WorkerId for currently running processes,
  accumulated data, and (for now) the results summary.

In a future patch, we'll break down the results summary by worker sets
as well.

Signed-off-by: George Dunlap 
---
 benchmark.go | 120 +++
 processworker.go |   4 +-
 xenworker.go |   6 +--
 3 files changed, 63 insertions(+), 67 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 2a78d26..4354a47 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -33,8 +33,17 @@ type WorkerSummary struct {
MinTput float64
 }
 
-type WorkerReport struct {
+type WorkerId struct {
+   Set int
Id int
+}
+
+func (wid WorkerId) String() (string) {
+   return fmt.Sprintf("%d:%d", wid.Set, wid.Id)
+}
+
+type WorkerReport struct {
+   Id WorkerId
Now int
Mops int
MaxDelta int
@@ -50,7 +59,7 @@ type WorkerSet struct {
 }
 
 type Worker interface {
-   SetId(int)
+   SetId(WorkerId)
Init(WorkerParams) error
Shutdown()
Process(chan WorkerReport, chan bool)
@@ -86,13 +95,13 @@ func Report(ws *WorkerState, r WorkerReport) {
 
tput := Throughput(lr.Now, lr.Mops, r.Now, r.Mops)

-   fmt.Printf("%d Time: %2.3f Mops: %d Tput: %4.2f\n", r.Id, time, 
mops, tput);
+   fmt.Printf("%v Time: %2.3f Mops: %d Tput: %4.2f\n", r.Id, time, 
mops, tput);
}
 
ws.LastReport = r
 }
 
-type WorkerList []WorkerState
+type WorkerList map[WorkerId]*WorkerState
 
 func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) (i int) {
i = 0
@@ -114,41 +123,44 @@ const (
WorkerXen = iota
 )
 
-func NewWorkerList(workers []WorkerSet, workerType int) (ws WorkerList, err 
error) {
-   count := 0
+func NewWorkerList(workers []WorkerSet, workerType int) (wl WorkerList, err 
error) {
+   wl = WorkerList(make(map[WorkerId]*WorkerState))
 
-   // wsi: WorkerSet index
for wsi := range workers {
-   count += workers[wsi].Count
-   }
+   for i := 0; i < workers[wsi].Count; i = i+1 {
+   Id := WorkerId{Set:wsi,Id:i}
 
-   fmt.Println("Making ", count, " total workers")
-   ws = WorkerList(make([]WorkerState, count))
+   ws := wl[Id]
 
-   // wli: WorkerList index
-   wli := 0
-   for wsi := range workers {
-   for i := 0; i < workers[wsi].Count; i, wli = i+1, wli+1 {
+   if ws != nil {
+   panic("Duplicate worker for id!")
+   }
+   
+   ws = {}
+   
switch workerType {
case WorkerProcess:
-   ws[wli].w = {}
+   ws.w = {}
case WorkerXen:
-   ws[wli].w = {}
+   ws.w = {}
default:
err = fmt.Errorf("Unknown type: %d", workerType)
+   return
}
-   ws[wli].w.SetId(wli)
+   
+   ws.w.SetId(Id)

-   ws[wli].w.Init(workers[wsi].Params)
+   ws.w.Init(workers[wsi].Params)
+
+   wl[Id] = ws
}
}
return
 }
 
 type BenchmarkRunData struct {
-   WorkerCount int
Raw []WorkerReport   `json:",omitempty"`
-   Summary []WorkerSummary  `json:",omitempty"`
+   Summary map[WorkerId]*WorkerSummary  `json:",omitempty"`
 }
 
 type BenchmarkRun struct {
@@ -174,8 +186,6 @@ func (run *BenchmarkRun) Run() (err error) {

i := Workers.Start(report, done)
 
-   run.Results.WorkerCount = i
-
// FIXME:
// 1. Make a zero timeout mean "never"
// 2. Make the signals / timeout thing a bit more rational; signal then 
timeout shouldn't hard kill
@@ -185,7 +195,7 @@ func (run *BenchmarkRun) Run() (err error) {
select {
case r := <-report:
run.Results.Raw = append(run.Results.Raw, r)
-   Report([r.Id], r)
+   Report(Workers[r.Id], r)
case <-done:
i--;
fmt.Println(i, "workers left");
@@ -215,22 +225,11 @@ func (run *BenchmarkRun) Run() (err error) {
 }
 
 func (run 

[Xen-devel] [PATCH RFC 51/59] controller: Make and/or modify cpupools when possible

[Ronald Rojas]

From: George Dunlap 

Make it possible for schedbench to modify cpupools in order to satisfy
run constraints.

Make a "RunConfig" option which contains the name of the pool, the
scheduler, and the number of cpus.  As with WorkerConfig, make these
automatically inherited from larger levels to smaller levels.  This
makes it straightforward to allow arbitrary pool configurations in the
same plan.

Modify sample.bench to have a RunConfig entry at the top level
(instead of the WorkerConfig option).  SimplePlan already sets
RunConfig.Scheduler, so the effect will be for each run to have all
three options set.  (Punt on SimplePlan generating more complex
options for now.)

Change BenchmarkRun.Ready() to BenchmarkRun.Prep(), which will cause
the configuration in RunConfig to become true if possible.  Empty
'Pool' element means the default pool (Pool-0).  Always create the
cpupool if it's non-pool-0, destroying the current one if it exists.

If it is pool 0, just check if it matches, and skip if not.

(We could in theory modify cpupool 0, but push that for future work.)

Signed-off-by: George Dunlap 
---
 benchmark.go |  17 +-
 run.go   | 176 +++
 2 files changed, 169 insertions(+), 24 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index aecb574..bd513fd 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -67,7 +67,6 @@ func (l *WorkerConfig) PropagateFrom(g WorkerConfig) {
}
 }
 
-
 type WorkerSet struct {
Params WorkerParams
Config WorkerConfig
@@ -140,6 +139,21 @@ type BenchmarkRunData struct {
 
 type RunConfig struct {
Scheduler string
+   Pool string
+   Cpus []int
+}
+
+// Propagate unset values from a higher level
+func (l *RunConfig) PropagateFrom(g RunConfig) {
+   if l.Pool == "" {
+   l.Pool = g.Pool
+   }
+   if l.Scheduler == "" {
+   l.Scheduler = g.Scheduler
+   }
+   if l.Cpus == nil {
+   l.Cpus = g.Cpus
+   }
 }
 
 type BenchmarkRun struct {
@@ -159,6 +173,7 @@ type BenchmarkPlan struct {
// Global options for workers that will be over-ridden by Run
// and WorkerSet config options
WorkerConfig `json:",omitempty"`
+   RunConfig RunConfig   `json:",omitempty"`
Runs []BenchmarkRun  `json:",omitempty"`
 }
 
diff --git a/run.go b/run.go
index 1a753bc..2d0db01 100644
--- a/run.go
+++ b/run.go
@@ -156,34 +156,158 @@ func getCpuHz() (err error) {
return
 }
 
-func (run *BenchmarkRun) Ready() (ready bool, why string) {
-   // FIXME: Check WorkerType
-   // Skip this run if it's not the scheduler we want
-   if run.RunConfig.Scheduler != "" {
-   var pool CpupoolInfo
-   if run.WorkerConfig.Pool != "" {
-   var found bool
-   pool, found = 
Ctx.CpupoolFindByName(run.WorkerConfig.Pool)
-   if !found {
-   why = "cpupool error"
-   return
-   }
-   } else {
-   // xl defaults to cpupool 0
-   plist := Ctx.ListCpupool()
-   if len(plist) > 0 {
-   pool = plist[0]
+// If the pool is specified, use that pool; otherwise assume pool 0.
+//
+// Unspecified schedulers match any pool; unspecifiend cpu lists match
+// any pool.
+//
+// If the pool exists and the scheduler and cpu lists match the pool,
+// carry on.  (This is running the VMs in a pre-configured pool.)
+//
+// If the pool exists and either the scheduler or the cpus don't match
+// the pool, and this is pool 0, skip.
+//
+// TODO: If the scheduler matches but the cpus don't, modify the pool
+// by adding or removing cpus.  (This can be done for Pool-0 as well.)
+//
+// If the pool is not Pool-0, and the scheduler doesn't match or the
+// pool doesn't exist, but there are no cpus, skip (because we don't
+// have enough information to create the pool).
+//
+// If the pool is not Pool-0, and either the scheduler or the cpus
+// don't match, and the cpus are specified, create the pool.
+func (run *BenchmarkRun) Prep() (ready bool, why string) {
+   var pool CpupoolInfo
+   poolPresent := false
+   
+   // Generate the requested cpumap
+   var Cpumap Bitmap
+   if run.RunConfig.Cpus != nil {
+   fmt.Print("Run.Prep: Cpus: ")
+   printed := false
+   for _, i := range run.RunConfig.Cpus {
+   if printed {
+   fmt.Printf(",%d", i)
} else {
-   why = "cpupool error"
-   return
+   printed = true
+   fmt.Printf("%d", i)
+   }   
+

[Xen-devel] [PATCH RFC 32/59] controller: Allow specification of an input file

[Ronald Rojas]

From: George Dunlap 

So that we can analyze data later if we want.

Signed-off-by: George Dunlap 
---
 main.go | 194 +++-
 1 file changed, 106 insertions(+), 88 deletions(-)

diff --git a/main.go b/main.go
index f8d77cf..ddb768c 100644
--- a/main.go
+++ b/main.go
@@ -26,101 +26,119 @@ import (
 )
 
 func main() {
-   filename := "test.bench"
-
-   switch(os.Args[1]) {
-   case "plan":
-   workerA := []string{"burnwait", "70", "20"}
-   //workerB := []string{"burnwait", "10", "2000"}
-   workerB := []string{"burnwait", "10", "30",
-   "burnwait", "20", "30",
-   "burnwait", "10", "30",
-   "burnwait", "10", "30",
-   "burnwait", "10", "30",
-   "burnwait", "10", "30",
-   "burnwait", "30", "30",
-   }
+   Args := os.Args
 
+   Args = Args[1:]
+   filename := "test.bench"
 
-   plan :=  BenchmarkPlan{
-   WorkerType:WorkerXen,
-   WorkerConfig:WorkerConfig{Pool:"schedbench"},
-   filename:filename,
-   Runs:[]BenchmarkRun{
-   {Label:"baseline-a",
+   for len(Args) > 0 {
+   switch(Args[0]) {
+   case "-f":
+   if len(Args) < 2 {
+   fmt.Println("Need arg for -f")
+   os.Exit(1)
+   }
+   filename = Args[1]
+   Args = Args[2:]
+   case "plan":
+   workerA := []string{"burnwait", "70", "20"}
+   //workerB := []string{"burnwait", "10", "2000"}
+   workerB := []string{"burnwait", "10", "30",
+   "burnwait", "20", "30",
+   "burnwait", "10", "30",
+   "burnwait", "10", "30",
+   "burnwait", "10", "30",
+   "burnwait", "10", "30",
+   "burnwait", "30", "30",
+   }
+   
+   
+   plan :=  BenchmarkPlan{
+   WorkerType:WorkerXen,
+   WorkerConfig:WorkerConfig{Pool:"schedbench"},
+   filename:filename,
+   Runs:[]BenchmarkRun{
+   {Label:"baseline-a",
+   WorkerSets:[]WorkerSet{
+   
{Params:WorkerParams{workerA},
+   Count:1}},
+   RuntimeSeconds:10,},
+   {Label:"baseline-b",
+   WorkerSets:[]WorkerSet{
+   
{Params:WorkerParams{workerB},
+   Count:1}},
+   RuntimeSeconds:10,},
+   }}
+   
+   for i := 1; i <= 16 ; i *= 2 {
+   label := fmt.Sprintf("%da+%db", i, i)
+   run := BenchmarkRun{
+   Label:label,
WorkerSets:[]WorkerSet{
{Params:WorkerParams{workerA},
-   Count:1}},
-   RuntimeSeconds:10,},
-   {Label:"baseline-b",
-   WorkerSets:[]WorkerSet{
+   Count:i},
{Params:WorkerParams{workerB},
-   Count:1}},
-   RuntimeSeconds:10,},
-   }}
-
-   for i := 1; i <= 16 ; i *= 2 {
-   label := fmt.Sprintf("%da+%db", i, i)
-   run := BenchmarkRun{
-   Label:label,
-   WorkerSets:[]WorkerSet{
-   {Params:WorkerParams{workerA},
-   Count:i},
-   {Params:WorkerParams{workerB},
-   Count:i}},
-   

[Xen-devel] [PATCH RFC 36/59] controller: Make 'dummy' at the level of 'run' rather than xenworker

[Ronald Rojas]

From: George Dunlap 

Get rid of xenworker_dummy.go and use stubs.go instead.  This is much
cleaner (as evidenced by the size of the file), and allows us to use
libxl-based things in run.go without having to make a load of dummy
libxl stub functions.

Signed-off-by: George Dunlap 
---
 Makefile   |  2 +-
 stubs.go   | 34 
 xenworker_dummy.go | 58 --
 3 files changed, 35 insertions(+), 59 deletions(-)
 create mode 100644 stubs.go
 delete mode 100644 xenworker_dummy.go

diff --git a/Makefile b/Makefile
index af55e0a..0e0b231 100644
--- a/Makefile
+++ b/Makefile
@@ -17,7 +17,7 @@ schedbench: main.go processworker.go xenworker.go 
benchmark.go run.go libxl.go h
CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
 
 # FIXME: Do with dlopen instead
-schedbench-report: main.go processworker.go xenworker_dummy.go benchmark.go 
run.go htmlreport.go plan.go
+schedbench-report: main.go benchmark.go stubs.go htmlreport.go plan.go
go build -o $@ $^
 
 .PHONY: clean
diff --git a/stubs.go b/stubs.go
new file mode 100644
index 000..78987ad
--- /dev/null
+++ b/stubs.go
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
+package main
+
+import (
+   "fmt"
+)
+
+func (plan *BenchmarkPlan) Run() (err error) {
+   err = fmt.Errorf("Not implemented")
+
+   return
+}
+
+func XlTest(Args []string) {
+   return
+}
+
diff --git a/xenworker_dummy.go b/xenworker_dummy.go
deleted file mode 100644
index e2dbdae..000
--- a/xenworker_dummy.go
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
- * License only.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License for more details.
- * 
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- * 02110-1301, USA.
- */
-package main
-
-import (
-   "fmt"
-   "io"
-)
-
-type XenWorker struct {
-}
-
-func XlTest(Args []string) {
-   return
-}
-
-func (w *XenWorker) SetId(i WorkerId) {
-}
-
-func (w *XenWorker) Init(p WorkerParams, g WorkerConfig) (err error) {
-   err = fmt.Errorf("Xen functionality not implemented");
-   return
-}
-
-// FIXME: Return an error
-func (w *XenWorker) Shutdown() {
-   
-   return
-}
-
-func (w *XenWorker) DumpLog(f io.Writer) (err error) {
-   err = fmt.Errorf("Xen functionality not implemented");
-   return
-}
-
-
-
-// FIXME: Return an error
-func (w *XenWorker) Process(report chan WorkerReport, done chan WorkerId) {
-   return;
-}
-
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 33/59] controller: Add verbosity argument and update README with new instructions

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 main.go | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/main.go b/main.go
index ddb768c..2aa8bae 100644
--- a/main.go
+++ b/main.go
@@ -30,6 +30,7 @@ func main() {
 
Args = Args[1:]
filename := "test.bench"
+   verbosity := 0
 
for len(Args) > 0 {
switch(Args[0]) {
@@ -40,6 +41,13 @@ func main() {
}
filename = Args[1]
Args = Args[2:]
+   case "-v":
+   if len(Args) < 2 {
+   fmt.Println("Need arg for -v")
+   os.Exit(1)
+   }
+   verbosity, _ = strconv.Atoi(Args[1])
+   Args = Args[2:]
case "plan":
workerA := []string{"burnwait", "70", "20"}
//workerB := []string{"burnwait", "10", "2000"}
@@ -105,12 +113,7 @@ func main() {
Args = Args[1:]

case "report":
-   verbosity := 0
Args = Args[1:]
-   if len(Args) > 0 {
-   verbosity, _ = strconv.Atoi(os.Args[0])
-   Args = Args[1:]
-   }
plan, err := LoadBenchmark(filename)
if err != nil {
fmt.Println("Loading benchmark ", filename, " 
", err)
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 27/59] report: Allow report verbosity to be specified

[Ronald Rojas]

From: George Dunlap 

Take a third argument to control the level of verbosity.

Also add helpful deltas when reporting the raw results.

Signed-off-by: George Dunlap 
---
 benchmark.go | 17 +++--
 main.go  | 11 ---
 2 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index ffecb82..5e35997 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -342,10 +342,23 @@ func (run *BenchmarkRun) TextReport(level int) (err 
error) {
s.AvgUtil, s.MinMaxUtil.Min, 
s.MinMaxUtil.Max)
 
if level >= 2 {
+   var le WorkerReport
for _, e := range s.Raw {
+   var dtime float64
+   var dCputime time.Duration
+   var dKops int
time := float64(e.Now) / SEC
-   fmt.Printf ("   [%8.3f] %8.3f 
%8d %12d\n", time,
-   e.Cputime.Seconds(), 
e.Kops, e.MaxDelta)
+   if e.Now > le.Now {
+   dtime = float64(e.Now - 
le.Now) / SEC
+   dCputime = e.Cputime - 
le.Cputime
+   dKops = e.Kops - le.Kops
+
+   }
+   fmt.Printf ("   [%8.3f] (%8.3f) 
%8.3f (%8.3f) %8d (%8d) %12d\n",
+   time, dtime,
+   e.Cputime.Seconds(), 
dCputime.Seconds(),
+   e.Kops, dKops, 
e.MaxDelta)
+   le = e
}
}
 
diff --git a/main.go b/main.go
index 6eaa39e..13230a7 100644
--- a/main.go
+++ b/main.go
@@ -22,6 +22,7 @@ package main
 import (
"fmt"
"os"
+   "strconv"
 )
 
 func main() {
@@ -29,9 +30,9 @@ func main() {
 
switch(os.Args[1]) {
case "plan":
-   workerA := []string{"burnwait", "7", "2"}
+   workerA := []string{"burnwait", "70", "20"}
//workerB := []string{"burnwait", "10", "2000"}
-   workerB := []string{"burnwait", "100", "300",
+   workerB := []string{"burnwait", "10", "30",
"burnwait", "20", "30",
"burnwait", "10", "30",
"burnwait", "10", "30",
@@ -91,13 +92,17 @@ func main() {
}

case "report":
+   verbosity := 0
+   if len(os.Args) > 2 {
+   verbosity, _ = strconv.Atoi(os.Args[2])
+   }
plan, err := LoadBenchmark(filename)
if err != nil {
fmt.Println("Loading benchmark ", filename, " ", err)
os.Exit(1)
}

-   err = plan.TextReport(0)
+   err = plan.TextReport(verbosity)
if err != nil {
fmt.Println("Running benchmark run:", err)
os.Exit(1)
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 35/59] libxl: Add ListCpupool

[Ronald Rojas]

From: George Dunlap 

Add libxl bindings for libxl_list_cpupool().

Define the Scheduler type (based on libxl_schedler) with appropriate
constants and string functions.

Punt on the cpumap part of CpupoolInfo for now.

Also create "xltest" command for testing libxl bindings

Signed-off-by: George Dunlap 
---
 libxl.go   | 152 +
 main.go|   5 ++
 xenworker_dummy.go |   4 ++
 3 files changed, 161 insertions(+)

diff --git a/libxl.go b/libxl.go
index 415de7f..5bdabb1 100644
--- a/libxl.go
+++ b/libxl.go
@@ -2,6 +2,7 @@ package main
 
 /*
 #include 
+#include 
 */
 import "C"
 
@@ -128,3 +129,154 @@ func (Ctx *Context) DomainUnpause(Id Domid) (err error) {
}
return
 }
+
+
+// typedef struct {
+// uint32_t size;  /* number of bytes in map */
+// uint8_t *map;
+// } libxl_bitmap;
+// void libxl_bitmap_init(libxl_bitmap *map);
+// void libxl_bitmap_dispose(libxl_bitmap *map);
+
+// # Consistent with values defined in domctl.h
+// # Except unknown which we have made up
+// libxl_scheduler = Enumeration("scheduler", [
+// (0, "unknown"),
+// (4, "sedf"),
+// (5, "credit"),
+// (6, "credit2"),
+// (7, "arinc653"),
+// (8, "rtds"),
+// ])
+type Scheduler int
+var (
+   SchedulerUnknown  Scheduler = 0
+   SchedulerSedf Scheduler = 4
+   SchedulerCredit   Scheduler = 5
+   SchedulerCredit2  Scheduler = 6
+   SchedulerArinc653 Scheduler = 7
+   SchedulerRTDS Scheduler = 8
+)
+
+// const char *libxl_scheduler_to_string(libxl_scheduler p);
+// int libxl_scheduler_from_string(const char *s, libxl_scheduler *e);
+func (s Scheduler) String() (string) {
+   cs := C.libxl_scheduler_to_string(C.libxl_scheduler(s))
+   // No need to free const return value
+
+   return C.GoString(cs)
+}
+
+func SchedulerFromString(name string) (s Scheduler, err error) {
+   cname := C.CString(name)
+   defer C.free(unsafe.Pointer(cname))
+
+   var cs C.libxl_scheduler
+
+   ret := C.libxl_scheduler_from_string(cname, )
+   if ret != 0 {
+   err = fmt.Errorf("libxl_scheduler_from_string failed: %d", ret)
+   return
+   }
+
+   s = Scheduler(cs)
+
+   return
+}
+
+// libxl_cpupoolinfo = Struct("cpupoolinfo", [
+// ("poolid",  uint32),
+// ("pool_name",   string),
+// ("sched",   libxl_scheduler),
+// ("n_dom",   uint32),
+// ("cpumap",  libxl_bitmap)
+// ], dir=DIR_OUT)
+
+type CpupoolInfo struct {
+   PoolId uint32
+   PoolName string
+   Scheduler Scheduler
+   DomainCount int
+   // Punt on cpumap for now
+}
+
+// libxl_cpupoolinfo * libxl_list_cpupool(libxl_ctx*, int *nb_pool_out);
+// void libxl_cpupoolinfo_list_free(libxl_cpupoolinfo *list, int nb_pool);
+func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
+   var nbPool C.int
+
+   c_cpupool_list := C.libxl_list_cpupool(Ctx.ctx, )
+
+   defer C.libxl_cpupoolinfo_list_free(c_cpupool_list, nbPool)
+
+   if int(nbPool) == 0 {
+   return
+   }
+
+   // Magic
+   cpupoolListSlice := (*[1 << 
30]C.libxl_cpupoolinfo)(unsafe.Pointer(c_cpupool_list))[:nbPool:nbPool]
+
+   for i := range cpupoolListSlice {
+   var info CpupoolInfo
+   
+   info.PoolId = uint32(cpupoolListSlice[i].poolid)
+   info.PoolName = C.GoString(cpupoolListSlice[i].pool_name)
+   info.Scheduler = Scheduler(cpupoolListSlice[i].sched)
+   info.DomainCount = int(cpupoolListSlice[i].n_dom)
+
+   list = append(list, info)
+   }
+
+   return
+}
+
+func (Ctx *Context) CpupoolFindByName(name string) (info CpupoolInfo, found 
bool) {
+   plist := Ctx.ListCpupool()
+
+   for i := range plist {
+   if plist[i].PoolName == name {
+   found = true
+   info = plist[i]
+   return
+   }
+   }
+   return
+}
+
+func XlTest(Args []string) {
+   var Ctx Context
+
+   err := Ctx.Open()
+   if err != nil {
+   fmt.Printf("Opening context: %v\n", err)
+   return
+   }
+
+   pool, found := Ctx.CpupoolFindByName("schedbench")
+
+   if found {
+   fmt.Printf("%v\n", pool)
+
+   a := int(pool.Scheduler)
+   b := pool.Scheduler.String()
+   c, err  := SchedulerFromString(b)
+
+   if err != nil {
+   fmt.Printf("Error: %v\n", err)
+   }
+
+   fmt.Printf("a: %d b: %s c: %d\n", a, b, int(c)) 
+   } else {
+   fmt.Printf("schedbench not found")
+   }
+
+   pool, found = Ctx.CpupoolFindByName("schedbnch")
+
+   if found {
+   fmt.Printf("%v\n", pool)
+   } else {
+   

[Xen-devel] [PATCH RFC 37/59] libxl.go: Provide a single global context by default

[Ronald Rojas]

From: George Dunlap 

Provide a single global context by default, Ctx.  Allow this to be
called with Ctx.Open() multiple times (but Close() only once).

Move xenworker.go to use this global context instead; and open it when
starting the benchmark (rather than opening and closing it for every
run, which is effectively what happens now).

Signed-off-by: George Dunlap 
---
 libxl.go | 12 +---
 run.go   |  7 +++
 xenworker.go | 25 +
 3 files changed, 17 insertions(+), 27 deletions(-)

diff --git a/libxl.go b/libxl.go
index 5bdabb1..9477bca 100644
--- a/libxl.go
+++ b/libxl.go
@@ -16,19 +16,17 @@ type Context struct {
ctx *C.libxl_ctx
 }
 
-func NewContext() (Ctx *Context, err error) {
-   Ctx = {}
-   
-   err = Ctx.Open()
-
-   return
-}
+var Ctx Context
 
 func (Ctx *Context) IsOpen() bool {
return Ctx.ctx != nil
 }
 
 func (Ctx *Context) Open() (err error) {
+   if Ctx.ctx != nil {
+   return
+   }
+   
ret := C.libxl_ctx_alloc(unsafe.Pointer(), C.LIBXL_VERSION, 0, 
nil)
 
if ret != 0 {
diff --git a/run.go b/run.go
index 1b39730..08a43ff 100644
--- a/run.go
+++ b/run.go
@@ -228,6 +228,13 @@ func (plan *BenchmarkPlan) Run() (err error) {
if err != nil {
return
}
+
+   if plan.WorkerType == WorkerXen {
+   err = Ctx.Open()
+   if err != nil {
+   return
+   }
+   }

for i := range plan.Runs {
r := [i];
diff --git a/xenworker.go b/xenworker.go
index 45e0876..2649392 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -27,13 +27,6 @@ import (
"io"
 )
 
-type xenGlobal struct {
-   Ctx Context
-   count int
-}
-
-var xg xenGlobal
-
 type XenWorker struct {
id WorkerId
Ctx Context
@@ -68,13 +61,9 @@ func (w *XenWorker) SetId(i WorkerId) {
 }
 
 func (w *XenWorker) Init(p WorkerParams, g WorkerConfig) (err error) {
-   if xg.count == 0 {
-   err = xg.Ctx.Open()
-   if err != nil {
-   return
-   }
+   if err != nil {
+   return
}
-   xg.count++

mock := false

@@ -220,11 +209,6 @@ func (w *XenWorker) Shutdown() {
e.Stdout = os.Stdout
e.Stderr = os.Stderr
 
-   xg.count--
-   if xg.count == 0 {
-   defer xg.Ctx.Close()
-   }
-
err := e.Run()
if err != nil {
fmt.Printf("Error destroying domain: %v\n", err)
@@ -249,7 +233,8 @@ func (w *XenWorker) DumpLog(f io.Writer) (err error) {
 // FIXME: Return an error
 func (w *XenWorker) Process(report chan WorkerReport, done chan WorkerId) {
// // xl unpause [vmname]
-   err := xg.Ctx.DomainUnpause(Domid(w.domid))
+   //err := xg.Ctx.DomainUnpause(Domid(w.domid))
+   err := Ctx.DomainUnpause(Domid(w.domid))
if err != nil {
fmt.Printf("Error unpausing domain: %v\n", err)
return
@@ -268,7 +253,7 @@ func (w *XenWorker) Process(report chan WorkerReport, done 
chan WorkerId) {
var r WorkerReport
json.Unmarshal([]byte(s), )
r.Id = w.id
-   di, err := xg.Ctx.DomainInfo(Domid(w.domid))
+   di, err := Ctx.DomainInfo(Domid(w.domid))
// Ignore errors for now
if err == nil {
r.Cputime = di.Cpu_time
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 40/59] controller: Add / update GPL text

[Ronald Rojas]

From: George Dunlap 

Add it to files which are missing it; clarify the wording regarding
version 2 only in others.

Signed-off-by: George Dunlap 
---
 benchmark.go | 2 +-
 main.go  | 2 +-
 processworker.go | 2 +-
 xenworker.go | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index b77b66b..aecb574 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -3,7 +3,7 @@
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
+ * published by the Free Software Foundation; version 2 of the
  * License only.
  *
  * This program is distributed in the hope that it will be useful, but
diff --git a/main.go b/main.go
index 2f68969..8cbd708 100644
--- a/main.go
+++ b/main.go
@@ -3,7 +3,7 @@
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
+ * published by the Free Software Foundation; version 2 of the
  * License only.
  *
  * This program is distributed in the hope that it will be useful, but
diff --git a/processworker.go b/processworker.go
index 999e76a..8c27f15 100644
--- a/processworker.go
+++ b/processworker.go
@@ -3,7 +3,7 @@
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
+ * published by the Free Software Foundation; version 2 of the
  * License only.
  *
  * This program is distributed in the hope that it will be useful, but
diff --git a/xenworker.go b/xenworker.go
index 2649392..10817b5 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -3,7 +3,7 @@
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
+ * published by the Free Software Foundation; version 2 of the
  * License only.
  *
  * This program is distributed in the hope that it will be useful, but
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 38/59] controller: Allow multiple schedulers in the same benchmark file

[Ronald Rojas]

From: George Dunlap 

First, add a RunConfig element to BenchmarkRun, which contains a
(string) name of the scheduler for this run.

Add run.Ready(), which if RunConfig.Scheduler is non-null, will check
to see if the cpupool in which the VMs will run (either
WorkerConfig.Pool or PoolID 0) has the specified scheduler.

Finally, when expanding the SimpleMatrix plan, if
SimpleMatrix.Schedulers is non-emply, add it to the matrix of runs.
Put them next to each other so that it's easy to compare similar runs
on different schedulers.

Refactor schedbench.sh to account for this, and also update the
README.md.

Signed-off-by: George Dunlap 
---
 benchmark.go |  5 +
 plan.go  | 38 +++---
 run.go   | 59 +--
 3 files changed, 81 insertions(+), 21 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 31b3711..b77b66b 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -138,10 +138,15 @@ type BenchmarkRunData struct {
Summary []WorkerSetSummary  `json:",omitempty"`
 }
 
+type RunConfig struct {
+   Scheduler string
+}
+
 type BenchmarkRun struct {
Label string
WorkerSets []WorkerSet
WorkerConfig
+   RunConfig
RuntimeSeconds int
Completed bool
Results BenchmarkRunData 
diff --git a/plan.go b/plan.go
index 2983e78..24c6bd4 100644
--- a/plan.go
+++ b/plan.go
@@ -35,29 +35,42 @@ func (plan *BenchmarkPlan) ExpandInput() (err error) {
WorkerPresets[k] = plan.Input.WorkerPresets[k];
}
 
+   // Use named schedulers, or default to "" (which will use the
+   // current one)
+   var schedulers []string
+   if plan.Input.SimpleMatrix.Schedulers != nil {
+   schedulers = plan.Input.SimpleMatrix.Schedulers
+   } else {
+   schedulers = append(schedulers, "")
+   }
+
// Always do the baselines
for _, wn := range plan.Input.SimpleMatrix.Workers {
wp := WorkerPresets[wn]
-
+   
if wp.Args == nil {
err = fmt.Errorf("Invalid worker preset: %s", wn)
return
}
-
+   
run := BenchmarkRun{
-   Label:wn+" baseline",
WorkerSets:[]WorkerSet{{Params:wp, Count:1}},
RuntimeSeconds:10,
}
-
-   plan.Runs = append(plan.Runs, run)
+   
+   for _, s := range schedulers {
+   fmt.Printf("Making baseline %s run, sched %s\n", wn, s)
+   run.RunConfig.Scheduler = s
+   run.Label = wn+" baseline "+s
+   plan.Runs = append(plan.Runs, run)
+   }
}
-
+   
for _, c := range plan.Input.SimpleMatrix.Count {
run := BenchmarkRun{
RuntimeSeconds:10,
}
-
+   
var label string
for _, wn := range plan.Input.SimpleMatrix.Workers {
wp := WorkerPresets[wn]
@@ -65,14 +78,17 @@ func (plan *BenchmarkPlan) ExpandInput() (err error) {
if label != "" {
label = label+" + "
}
-   label = fmt.Sprintf("%s%s %d", label, wn, c)
+   label = fmt.Sprintf("%s%s %d ", label, wn, c)

ws := WorkerSet{Params:wp, Count:c}
run.WorkerSets = append(run.WorkerSets, ws)
}
-   run.Label = label
-
-   plan.Runs = append(plan.Runs, run)
+   for _, s := range schedulers {
+   fmt.Printf("Making count %d run, sched %s\n", c, s)
+   run.RunConfig.Scheduler = s
+   run.Label = label+s
+   plan.Runs = append(plan.Runs, run)
+   }
}
 
return
diff --git a/run.go b/run.go
index 08a43ff..1a753bc 100644
--- a/run.go
+++ b/run.go
@@ -3,7 +3,7 @@
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
+ * published by the Free Software Foundation; version 2 of the
  * License only.
  *
  * This program is distributed in the hope that it will be useful, but
@@ -156,6 +156,38 @@ func getCpuHz() (err error) {
return
 }
 
+func (run *BenchmarkRun) Ready() (ready bool, why string) {
+   // FIXME: Check WorkerType
+   // Skip this run if it's not the scheduler we want
+   if run.RunConfig.Scheduler != "" {
+   var pool CpupoolInfo
+   if run.WorkerConfig.Pool != "" {
+   

[Xen-devel] [PATCH RFC 53/59] controller: Add WorkerConfig.SoftAffinity

[Ronald Rojas]

From: George Dunlap 

If the SoftAffinity is present, pass it as "cpus_soft = ..." in the
config file.

Signed-off-by: George Dunlap 
---
 benchmark.go | 1 +
 xenworker.go | 4 
 2 files changed, 5 insertions(+)

diff --git a/benchmark.go b/benchmark.go
index bd513fd..d5a0ac8 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -58,6 +58,7 @@ func (l *WorkerParams) SetkHZ(kHZ uint64) {
 
 type WorkerConfig struct {
Pool string
+   SoftAffinity string
 }
 
 // Propagate unset values from a higher level
diff --git a/xenworker.go b/xenworker.go
index 10817b5..f2f316f 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -91,6 +91,10 @@ func (w *XenWorker) Init(p WorkerParams, g WorkerConfig) 
(err error) {
fmt.Fprintf(cfg, "pool = '%s'\n", g.Pool)
}
 
+   if g.SoftAffinity != "" {
+   fmt.Fprintf(cfg, "cpus_soft = '%s'\n", g.SoftAffinity)
+   }
+

// xl create -p [filename]
{
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 57/59] tools/blktap2: remove unused inclusion of sys/sysctl.l

[Ronald Rojas]

From: Alistair Francis 

That header file is not used. Removing it would avoid build error with
musl libc, which doesn't have that header file.

Signed-off-by: Alistair Francis 
Reviewed-by: Doug Goldstein 
[ wei: rewrote commit message ]
Acked-by: Wei Liu 
---
 tools/blktap2/drivers/block-remus.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tools/blktap2/drivers/block-remus.c 
b/tools/blktap2/drivers/block-remus.c
index 079588d..7401800 100644
--- a/tools/blktap2/drivers/block-remus.c
+++ b/tools/blktap2/drivers/block-remus.c
@@ -54,7 +54,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 52/59] libxl: Implement Bitmap.String()

[Ronald Rojas]

From: George Dunlap 

Implement Bitmap.String, which will convert a bitmap into a string
parse-able by libxl.

Signed-off-by: George Dunlap 
---
 libxl.go | 59 ++-
 1 file changed, 58 insertions(+), 1 deletion(-)

diff --git a/libxl.go b/libxl.go
index bfcaf0d..aa5c01c 100644
--- a/libxl.go
+++ b/libxl.go
@@ -337,6 +337,51 @@ func (a Bitmap) And(b Bitmap) (c Bitmap) {
return
 }
 
+func (bm Bitmap) String() (s string) {
+   lastOnline := false
+   crange := false
+   printed := false
+   var i int
+   /// --x-x-x -> 2,4-8,10
+   /// --x-xxx -> 2,4-10
+   for i = 0; i <= bm.Max(); i++ {
+   if bm.Test(i) {
+   if !lastOnline {
+   // Switching offline -> online, print this cpu
+   if printed {
+   s += ","
+   }
+   s += fmt.Sprintf("%d", i)
+   printed = true
+   } else if !crange {
+   // last was online, but we're not in a range; 
print -
+   crange = true
+   s += "-"
+   } else {
+   // last was online, we're in a range,  nothing 
else to do
+   }
+   lastOnline = true
+   } else {
+   if lastOnline {
+   // Switching online->offline; do we need to end 
a range?
+   if crange {
+   s += fmt.Sprintf("%d", i-1)
+   }
+   }
+   lastOnline = false
+   crange = false
+   }
+   }
+   if lastOnline {
+   // Switching online->offline; do we need to end a range?
+   if crange {
+   s += fmt.Sprintf("%d", i-1)
+   }
+   }
+
+   return
+}
+
 // const char *libxl_scheduler_to_string(libxl_scheduler p);
 func (s Scheduler) String() (string) {
cs := C.libxl_scheduler_to_string(C.libxl_scheduler(s))
@@ -608,6 +653,18 @@ func (Ctx *Context) CpupoolMakeFree(Cpumap Bitmap) (err 
error) {
 }
 
 func XlTest(Args []string) {
+   var Cpumap Bitmap
+
+   Cpumap.Set(2)
+   Cpumap.SetRange(4, 8)
+   Cpumap.Set(10)
+
+   fmt.Printf("Cpumap: %v\n", Cpumap)
+
+   Cpumap.Set(9)
+
+   fmt.Printf("Cpumap: %v\n", Cpumap)
+
var Ctx Context
 
err := Ctx.Open()
@@ -635,7 +692,7 @@ func XlTest(Args []string) {
}
}
 
-   var Cpumap Bitmap
+   Cpumap = Bitmap{}
 
Cpumap.SetRange(12, 15)
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 11/59] Refactor to move towards benchmark "plans" and data analysis

[Ronald Rojas]

From: George Dunlap 

A bit of a roll-up of lots of bits.

Signed-off-by: George Dunlap 
---
 Makefile |   4 +-
 benchmark.go | 256 +++
 main.go  | 197 ++---
 3 files changed, 302 insertions(+), 155 deletions(-)
 create mode 100644 benchmark.go

diff --git a/Makefile b/Makefile
index 16af528..7a33cfb 100644
--- a/Makefile
+++ b/Makefile
@@ -1,10 +1,10 @@
-BIN = controller
+BIN = schedbench
 BINALL = $(BIN)
 
 .PHONY: all
 all: $(BIN)
 
-controller: main.go processworker.go xenworker.go
+schedbench: main.go processworker.go xenworker.go benchmark.go
go build -o $@ $^
 
 .PHONY: clean
diff --git a/benchmark.go b/benchmark.go
new file mode 100644
index 000..b2b2399
--- /dev/null
+++ b/benchmark.go
@@ -0,0 +1,256 @@
+package main
+
+import (
+   "fmt"
+   "os"
+   "os/signal"
+   "time"
+   "io/ioutil"
+   "encoding/json"
+)
+
+type WorkerReport struct {
+   Id int
+   Now int
+   Mops int
+   MaxDelta int
+}
+
+type WorkerParams struct {
+   Args []string
+}
+
+type WorkerSet struct {
+   Params WorkerParams
+   Count int
+}
+
+type Worker interface {
+   SetId(int)
+   Init(WorkerParams) error
+   Shutdown()
+   Process(chan WorkerReport, chan bool)
+}
+
+const (
+   USEC = 1000
+   MSEC = USEC * 1000
+   SEC = MSEC * 1000
+)
+
+type WorkerState struct {
+   w Worker
+   LastReport WorkerReport
+}
+
+func Report(ws *WorkerState, r WorkerReport) {
+   //fmt.Println(r)
+
+   lr := ws.LastReport
+
+   if (lr.Now > 0) {
+   time := float64(r.Now - lr.Now) / SEC
+   mops := r.Mops - lr.Mops
+
+   tput := float64(mops) / time
+
+   fmt.Printf("%d Time: %2.3f Mops: %d Tput: %4.2f\n", r.Id, time, 
mops, tput);
+   }
+
+   ws.LastReport = r
+}
+
+type WorkerList []WorkerState
+
+func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) (i int) {
+   i = 0
+   for j := range *ws {
+   go (*ws)[j].w.Process(report, done)
+   i++
+   }
+   return
+}
+
+func (ws *WorkerList) Stop() {
+   for i := range *ws {
+   (*ws)[i].w.Shutdown()
+   }
+}
+
+const (
+   WorkerProcess = iota
+   WorkerXen = iota
+)
+
+func NewWorkerList(workers []WorkerSet, workerType int) (ws WorkerList, err 
error) {
+   count := 0
+
+   // wsi: WorkerSet index
+   for wsi := range workers {
+   count += workers[wsi].Count
+   }
+
+   fmt.Println("Making ", count, " total workers")
+   ws = WorkerList(make([]WorkerState, count))
+
+   // wli: WorkerList index
+   wli := 0
+   for wsi := range workers {
+   for i := 0; i < workers[wsi].Count; i, wli = i+1, wli+1 {
+   switch workerType {
+   case WorkerProcess:
+   ws[wli].w = {}
+   case WorkerXen:
+   ws[wli].w = {}
+   default:
+   err = fmt.Errorf("Unknown type: %d", workerType)
+   }
+   ws[wli].w.SetId(wli)
+   
+   ws[wli].w.Init(workers[wsi].Params)
+   }
+   }
+   return
+}
+
+type BenchmarkRunData struct {
+   Raw []WorkerReport
+}
+
+type BenchmarkRun struct {
+   Completed bool
+   Label string
+   Workers []WorkerSet
+   RuntimeSeconds int
+   Results BenchmarkRunData 
+}
+
+func (run *BenchmarkRun) Run() (err error) {
+   Workers, err := NewWorkerList(run.Workers, WorkerProcess)
+   if err != nil {
+   fmt.Println("Error creating workers: %v", err)
+   return
+   }
+   
+   report := make(chan WorkerReport)
+   done := make(chan bool)
+   signals := make(chan os.Signal, 1)
+
+   signal.Notify(signals, os.Interrupt)
+   
+   i := Workers.Start(report, done)
+
+   // FIXME:
+   // 1. Make a zero timeout mean "never"
+   // 2. Make the signals / timeout thing a bit more rational; signal then 
timeout shouldn't hard kill
+   timeout := time.After(time.Duration(run.RuntimeSeconds) * time.Second);
+   stopped := false
+   for i > 0 {
+   select {
+   case r := <-report:
+   run.Results.Raw = append(run.Results.Raw, r)
+   Report([r.Id], r)
+   case <-done:
+   i--;
+   fmt.Println(i, "workers left");
+   case <-timeout:
+   if ! stopped {
+   Workers.Stop()
+   stopped = true
+   run.Completed = true
+   }
+   case 

[Xen-devel] [PATCH RFC 12/59] Basic 'report' functionality

[Ronald Rojas]

From: George Dunlap 

Go through the raw data and re-calculate throughput for each
timeframe.  Record for each worker the min and max of these; then take
an average over the whole run.

Have the report go through each and show the worker setup, as well as
the avg, min, and max for each worker.

Also default to running Xen again (rather than processes), and clean
up some chattiness wrt starting Xen processes.

Finally, make the "plan" slightly more programmatic, and test a more
"pipelined" testcase.

Signed-off-by: George Dunlap 
---
 benchmark.go | 163 +--
 main.go  |  56 
 xenworker.go |   4 +-
 3 files changed, 205 insertions(+), 18 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index b2b2399..ec62c3d 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -9,6 +9,12 @@ import (
"encoding/json"
 )
 
+type WorkerSummary struct {
+   MaxTput float64
+   AvgTput float64
+   MinTput float64
+}
+
 type WorkerReport struct {
Id int
Now int
@@ -43,6 +49,14 @@ type WorkerState struct {
LastReport WorkerReport
 }
 
+func Throughput(lt int, lm int, t int, m int) (tput float64) {
+   time := float64(t - lt) / SEC
+   mops := m - lm
+   
+   tput = float64(mops) / time
+   return
+}
+
 func Report(ws *WorkerState, r WorkerReport) {
//fmt.Println(r)
 
@@ -52,8 +66,8 @@ func Report(ws *WorkerState, r WorkerReport) {
time := float64(r.Now - lr.Now) / SEC
mops := r.Mops - lr.Mops
 
-   tput := float64(mops) / time
-
+   tput := Throughput(lr.Now, lr.Mops, r.Now, r.Mops)
+   
fmt.Printf("%d Time: %2.3f Mops: %d Tput: %4.2f\n", r.Id, time, 
mops, tput);
}
 
@@ -114,19 +128,21 @@ func NewWorkerList(workers []WorkerSet, workerType int) 
(ws WorkerList, err erro
 }
 
 type BenchmarkRunData struct {
-   Raw []WorkerReport
+   WorkerCount int
+   Raw []WorkerReport   `json:",omitempty"`
+   Summary []WorkerSummary  `json:",omitempty"`
 }
 
 type BenchmarkRun struct {
-   Completed bool
Label string
Workers []WorkerSet
RuntimeSeconds int
+   Completed bool
Results BenchmarkRunData 
 }
 
 func (run *BenchmarkRun) Run() (err error) {
-   Workers, err := NewWorkerList(run.Workers, WorkerProcess)
+   Workers, err := NewWorkerList(run.Workers, WorkerXen)
if err != nil {
fmt.Println("Error creating workers: %v", err)
return
@@ -140,6 +156,8 @@ func (run *BenchmarkRun) Run() (err error) {

i := Workers.Start(report, done)
 
+   run.Results.WorkerCount = i
+
// FIXME:
// 1. Make a zero timeout mean "never"
// 2. Make the signals / timeout thing a bit more rational; signal then 
timeout shouldn't hard kill
@@ -178,8 +196,121 @@ func (run *BenchmarkRun) Run() (err error) {
return
 }
 
+func (run *BenchmarkRun) checkSummary() (done bool, err error) {
+   if run.Results.WorkerCount == 0 {
+   err = fmt.Errorf("Internal error: WorkerCount 0!")
+   return
+   }
+   
+   if len(run.Results.Summary) == run.Results.WorkerCount {
+   done = true
+   return 
+   }
+   
+   if len(run.Results.Summary) != 0 {
+   err = fmt.Errorf("Internal error: len(Summary) %d, len(Workers) 
%d!\n",
+   len(run.Results.Summary), run.Results.WorkerCount)
+   return
+   }
+
+   return
+}
+
+func (run *BenchmarkRun) Process() (err error) {
+   done, err := run.checkSummary()
+   if done || err != nil {
+   return
+   }
+   
+   wcount := run.Results.WorkerCount
+
+   if len(run.Results.Summary) != 0 {
+   err = fmt.Errorf("Internal error: len(Summary) %d, len(Workers) 
%d!\n",
+   len(run.Results.Summary), wcount)
+   return
+   }
+
+   run.Results.Summary = make([]WorkerSummary, wcount)
+
+   // FIXME: Filter out results which started before all have started
+   
+   data := make([]struct{
+   startTime int
+   lastTime int
+   lastMops int}, wcount)
+
+   for i := range run.Results.Raw {
+   e := run.Results.Raw[i]
+   if e.Id > wcount {
+   err = fmt.Errorf("Internal error: id %d > wcount %d", 
e.Id, wcount)
+   return
+   }
+   
+   d := [e.Id]
+   s := [e.Id]
+
+   if d.startTime == 0 {
+   d.startTime = e.Now
+   } else {
+   tput := Throughput(d.lastTime, d.lastMops, e.Now, 
e.Mops)
+   
+   if tput > s.MaxTput {
+   

[Xen-devel] [PATCH RFC 21/59] Report utilization statistics

[Ronald Rojas]

From: George Dunlap 

Refactor the min/max into a separate type.

Signed-off-by: George Dunlap 
---
 benchmark.go | 119 +--
 run.go   |   4 +-
 2 files changed, 78 insertions(+), 45 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 4b2d805..b0b55c7 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -87,21 +87,45 @@ func Throughput(lt int, lm int, t int, m int) (tput 
float64) {
return
 }
 
+func Utilization(lt int, lct time.Duration, t int, ct time.Duration) (util 
float64) {
+   util = float64(ct - lct) / float64(t - lt)
+   return
+}
+
+type MinMax struct {
+   Min float64
+   Max float64
+}
+
+func (mm *MinMax) Update(x float64) {
+   if x > mm.Max {
+   mm.Max = x
+   }
+   if x < mm.Min || mm.Min == 0 {
+   mm.Min = x
+   }
+}
+
 type WorkerSummary struct {
-   MaxTput float64
+   MinMaxTput MinMax
+   MinMaxUtil MinMax
AvgTput float64
-   MinTput float64
+   AvgUtil float64
 }
 
 type WorkerSetSummary struct {
Workers[]WorkerSummary
TotalTput float64
-   MaxTput   float64
AvgAvgTputfloat64
+   MinMaxTputMinMax
+   MinMaxAvgTput MinMax
AvgStdDevTput float64
-   AvgMaxTputfloat64
-   AvgMinTputfloat64
-   MinTput   float64
+
+   TotalUtil float64
+   MinMaxUtilMinMax
+   MinMaxAvgUtil MinMax
+   AvgAvgUtilfloat64
+   AvgStdDevUtil float64
 }
 
 type BenchmarkRunData struct {
@@ -146,8 +170,10 @@ func (run *BenchmarkRun) Process() (err error) {
 
type Data struct{
startTime int
+   startCputime time.Duration
lastTime int
lastMops int
+   lastCputime time.Duration
}

data := make(map[WorkerId]*Data)
@@ -183,24 +209,19 @@ func (run *BenchmarkRun) Process() (err error) {

if d.startTime == 0 {
d.startTime = e.Now
+   d.startCputime = e.Cputime
} else {
tput := Throughput(d.lastTime, d.lastMops, e.Now, 
e.Mops)
-   
-   if tput > s.MaxTput {
-   s.MaxTput = tput
-   }
-   if tput < s.MinTput || s.MinTput == 0 {
-   s.MinTput = tput
-   }
-   if tput > ws.MaxTput {
-   ws.MaxTput = tput
-   }
-   if tput < ws.MinTput || ws.MinTput == 0 {
-   ws.MinTput = tput
-   }
+   util := Utilization(d.lastTime, d.lastCputime, e.Now, 
e.Cputime)
+
+   s.MinMaxTput.Update(tput)
+   s.MinMaxUtil.Update(util)
+   ws.MinMaxTput.Update(tput)
+   ws.MinMaxUtil.Update(util)
}
d.lastTime = e.Now
d.lastMops = e.Mops
+   d.lastCputime = e.Cputime
}
 
for Id, d := range data {
@@ -208,45 +229,51 @@ func (run *BenchmarkRun) Process() (err error) {
s := [Id.Id]
 
s.AvgTput = Throughput(d.startTime, 0, d.lastTime, d.lastMops)
-   if s.AvgTput > ws.AvgMaxTput {
-   ws.AvgMaxTput = s.AvgTput
-   }
-   if s.AvgTput < ws.AvgMinTput || ws.AvgMinTput == 0 {
-   ws.AvgMinTput = s.AvgTput
-   }
-   
+   s.AvgUtil = Utilization(d.startTime, d.startCputime, 
d.lastTime, d.lastCputime)
+
+   ws.MinMaxAvgTput.Update(s.AvgTput)
+   ws.MinMaxAvgUtil.Update(s.AvgUtil)
}
 
// Calculate the average-of-averages for each set
for set := range run.Results.Summary {
ws := [set]

-   var total float64
+   var totalTput float64
+   var totalUtil float64
var count int
for id := range ws.Workers {
-   total += ws.Workers[id].AvgTput
+   totalTput += ws.Workers[id].AvgTput
+   totalUtil += ws.Workers[id].AvgUtil
count++
}
 
// FIXME -- Is this legit?
-   ws.TotalTput = total
-   ws.AvgAvgTput = total / float64(count)
+   ws.TotalTput = totalTput
+   ws.TotalUtil = totalUtil
+   ws.AvgAvgTput = totalTput / float64(count)
+   ws.AvgAvgUtil = totalUtil / float64(count)
}
 
// Then calculate the standard deviation
for set := range run.Results.Summary {

[Xen-devel] [PATCH RFC 22/59] Use tsc for time rather than rumpkernel clock_gettime()

[Ronald Rojas]

From: George Dunlap 

The rumpkernel clock_gettime() drifts drastically when the system is
overloaded.  Switch to using a TSC timesource instead:

* Use tsc_mode='native' in guest config
* Read the host tsc from /proc/cpuinfo when starting a run
* Pass it as an argument to the worker
* Implement now() with rdtsc

The results appear to continue to be accurate even with heavy
overcommitment.

Signed-off-by: George Dunlap 
---
 benchmark.go |  8 
 run.go   | 45 +
 xenworker.go |  1 +
 3 files changed, 54 insertions(+)

diff --git a/benchmark.go b/benchmark.go
index b0b55c7..7ea9aaa 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -48,6 +48,14 @@ type WorkerParams struct {
Args []string
 }
 
+func (l *WorkerParams) SetkHZ(kHZ uint64) {
+   if l.Args[0] == "kHZ" {
+   l.Args[1] = fmt.Sprintf("%d", kHZ)
+   } else {
+   l.Args = append([]string{"kHZ", fmt.Sprintf("%d", kHZ)}, 
l.Args...)
+   }
+}
+
 type WorkerConfig struct {
Pool string
 }
diff --git a/run.go b/run.go
index 2a93405..4222001 100644
--- a/run.go
+++ b/run.go
@@ -23,6 +23,9 @@ import (
"os"
"os/signal"
"time"
+   "regexp"
+   "strconv"
+   "bufio"
 )
 
 type WorkerState struct {
@@ -106,9 +109,45 @@ func NewWorkerList(WorkerSets []WorkerSet, workerType int) 
(wl WorkerList, err e
return
 }
 
+var CpukHZ uint64
+
+func getCpuHz() (err error) {
+   if CpukHZ == 0 {
+   var cpuinfo *os.File
+   cpuinfo, err = os.Open("/proc/cpuinfo")
+   if err != nil {
+   return
+   }
+   re := regexp.MustCompile("^cpu MHz\\s*: ([0-9.]+)$")
+   scanner := bufio.NewScanner(cpuinfo)
+   for scanner.Scan() {
+   s := scanner.Text()
+   m := re.FindStringSubmatch(s)
+   if m != nil {
+   var MHZ float64
+   MHZ, err = strconv.ParseFloat(m[1], 64)
+   if err != nil {
+   return
+   }
+   CpukHZ = uint64(MHZ*1000)
+   break
+   }
+   }
+   if CpukHZ == 0 {
+   err = fmt.Errorf("Couldn't find cpu MHz")
+   return
+   } else {
+   fmt.Println("CpukHZ: ", CpukHZ)
+   
+   }
+   }
+   return
+}
+
 func (run *BenchmarkRun) Run() (err error) {
for wsi := range run.WorkerSets {
run.WorkerSets[wsi].Config.PropagateFrom(run.WorkerConfig)
+   run.WorkerSets[wsi].Params.SetkHZ(CpukHZ)
}

Workers, err := NewWorkerList(run.WorkerSets, WorkerXen)
@@ -164,6 +203,12 @@ func (run *BenchmarkRun) Run() (err error) {
 }
 
 func (plan *BenchmarkPlan) Run() (err error) {
+
+   err = getCpuHz()
+   if err != nil {
+   return
+   }
+   
for i := range plan.Runs {
r := [i];
if ! r.Completed {
diff --git a/xenworker.go b/xenworker.go
index 4077e77..e98c970 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -95,6 +95,7 @@ func (w *XenWorker) Init(p WorkerParams, g WorkerConfig) (err 
error) {
fmt.Fprintf(cfg, "memory = 32\n")
fmt.Fprintf(cfg, "vcpus = 1\n")
fmt.Fprintf(cfg, "on_crash = 'destroy'\n")
+   fmt.Fprintf(cfg, "tsc_mode = 'native'\n")
 
if g.Pool != "" {
fmt.Fprintf(cfg, "pool = '%s'\n", g.Pool)
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 13/59] Add GPL headers / COPYING file (v2 only)

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 benchmark.go | 18 ++
 main.go  | 19 +++
 processworker.go | 18 ++
 xenworker.go | 18 ++
 4 files changed, 73 insertions(+)

diff --git a/benchmark.go b/benchmark.go
index ec62c3d..2a78d26 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -1,3 +1,21 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
 package main
 
 import (
diff --git a/main.go b/main.go
index 2696810..ebf990f 100644
--- a/main.go
+++ b/main.go
@@ -1,3 +1,22 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
+
 package main
 
 import (
diff --git a/processworker.go b/processworker.go
index cca6c3b..6f70ce1 100644
--- a/processworker.go
+++ b/processworker.go
@@ -1,3 +1,21 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
 package main
 
 import (
diff --git a/xenworker.go b/xenworker.go
index 1ed5cf0..7e85032 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -1,3 +1,21 @@
+/*
+ * Copyright (C) 2016 George W. Dunlap, Citrix Systems UK Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License only.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ * 
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
 package main
 
 import (
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 09/59] controller: Refactor creation and stopping of workers into WorkerList methods

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 main.go | 59 ++-
 1 file changed, 42 insertions(+), 17 deletions(-)

diff --git a/main.go b/main.go
index dbd6276..89e9cf3 100644
--- a/main.go
+++ b/main.go
@@ -55,10 +55,43 @@ func Report(ws *WorkerState, r WorkerReport) {
 
 type WorkerList []WorkerState
 
-func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) {
+func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) (i int) {
+   i = 0
+   for j := range *ws {
+   go (*ws)[j].w.Process(report, done)
+   i++
+   }
+   return
+}
+
+func (ws *WorkerList) Stop() {
for i := range *ws {
-   go (*ws)[i].w.Process(report, done)
+   (*ws)[i].w.Shutdown()
+   }
+}
+
+const (
+   WorkerProcess = iota
+   WorkerXen = iota
+)
+
+func NewWorkerList(count int, workerType int) (ws WorkerList, err error) {
+   ws = WorkerList(make([]WorkerState, count))
+
+   for i := 0; i< count; i++ {
+   switch workerType {
+   case WorkerProcess:
+   ws[i].w = {}
+   case WorkerXen:
+   ws[i].w = {}
+   default:
+   err = fmt.Errorf("Unknown type: %d", workerType)
+   }
+   ws[i].w.SetId(i)
+   
+   ws[i].w.Init(WorkerParams{[]string{"burnwait", "20", 
"2000"}})
}
+   return
 }
 
 func main() {
@@ -72,19 +105,13 @@ func main() {
 
signal.Notify(signals, os.Interrupt)

-   i := 0
-
-   Workers := WorkerList(make([]WorkerState, count))
-   
-   for i = 0; i< count; i++ {
-   //Workers[i].w = {}
-   Workers[i].w = {}
-   Workers[i].w.SetId(i)
-   
-   Workers[i].w.Init(WorkerParams{[]string{"burnwait", "20", 
"2000"}})
+   Workers, err := NewWorkerList(count, WorkerProcess)
+   if err != nil {
+   fmt.Println("Error creating workers: %v", err)
+   return
}
-
-   Workers.Start(report, done)
+   
+   i := Workers.Start(report, done)
 
for i > 0 {
select {
@@ -96,9 +123,7 @@ func main() {
case <-signals:
if ! killed {
fmt.Println("SIGINT receieved, shutting down 
workers")
-   for j := range Workers {
-   Workers[j].w.Shutdown()
-   }
+   Workers.Stop()
killed = true
} else {
fmt.Println("Second SIGINT received, exiting 
without cleaning up")
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 17/59] controller: Collect and display statistics on WorkerSets

[Ronald Rojas]

From: George Dunlap 

Introduce WorkerSetSummary, to contain WorkerSummary's and collect
data for the set.

For now "if false"-out reporting of individual worker averages; we can
add this back in as an option later.

Signed-off-by: George Dunlap 
---
 benchmark.go | 156 +--
 1 file changed, 121 insertions(+), 35 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index aaa4c98..4cc9d61 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -23,14 +23,9 @@ import (
"os"
"io/ioutil"
"encoding/json"
+   "math"
 )
 
-type WorkerSummary struct {
-   MaxTput float64
-   AvgTput float64
-   MinTput float64
-}
-
 type WorkerId struct {
Set int
Id int
@@ -77,9 +72,26 @@ func Throughput(lt int, lm int, t int, m int) (tput float64) 
{
return
 }
 
+type WorkerSummary struct {
+   MaxTput float64
+   AvgTput float64
+   MinTput float64
+}
+
+type WorkerSetSummary struct {
+   Workers[]WorkerSummary
+   TotalTput float64
+   MaxTput   float64
+   AvgAvgTputfloat64
+   AvgStdDevTput float64
+   AvgMaxTputfloat64
+   AvgMinTputfloat64
+   MinTput   float64
+}
+
 type BenchmarkRunData struct {
Raw []WorkerReport   `json:",omitempty"`
-   Summary map[WorkerId]*WorkerSummary  `json:",omitempty"`
+   Summary []WorkerSetSummary  `json:",omitempty"`
 }
 
 type BenchmarkRun struct {
@@ -90,6 +102,12 @@ type BenchmarkRun struct {
Results BenchmarkRunData 
 }
 
+type BenchmarkPlan struct {
+   filename string
+   WorkerType int
+   Runs []BenchmarkRun
+}
+
 func (run *BenchmarkRun) checkSummary() (done bool, err error) {
if run.Results.Summary != nil {
done = true
@@ -105,7 +123,7 @@ func (run *BenchmarkRun) Process() (err error) {
return
}
 
-   run.Results.Summary = make(map[WorkerId]*WorkerSummary)
+   run.Results.Summary = make([]WorkerSetSummary, len(run.WorkerSets))
 
type Data struct{
startTime int
@@ -118,18 +136,32 @@ func (run *BenchmarkRun) Process() (err error) {
// FIXME: Filter out results which started before all have started
for i := range run.Results.Raw {
e := run.Results.Raw[i]
+
+   if e.Id.Set > len(run.Results.Summary) {
+   return fmt.Errorf("Internal error: e.Id.Set %d > 
len(Results.Summary) %d\n",
+   e.Id.Set, len(run.Results.Summary))
+   }

+   if run.Results.Summary[e.Id.Set].Workers == nil {
+   run.Results.Summary[e.Id.Set].Workers = 
make([]WorkerSummary,
+   run.WorkerSets[e.Id.Set].Count)
+   }
+
+   ws := [e.Id.Set]
+   
+   if e.Id.Id > len(ws.Workers) {
+   return fmt.Errorf("Internal error: e.Id.Id %d > 
len(Results.Summary[].Workers) %d\n",
+   e.Id.Id, len(ws.Workers))
+   }
+
+   s := [e.Id.Id]
+
d := data[e.Id]
if d == nil {
d = {}
data[e.Id] = d
}
-   s := run.Results.Summary[e.Id]
-   if s == nil {
-   s = {}
-   run.Results.Summary[e.Id] = s
-   }
-
+   
if d.startTime == 0 {
d.startTime = e.Now
} else {
@@ -141,16 +173,63 @@ func (run *BenchmarkRun) Process() (err error) {
if tput < s.MinTput || s.MinTput == 0 {
s.MinTput = tput
}
+   if tput > ws.MaxTput {
+   ws.MaxTput = tput
+   }
+   if tput < ws.MinTput || ws.MinTput == 0 {
+   ws.MinTput = tput
+   }
}
d.lastTime = e.Now
d.lastMops = e.Mops
}
 
-   for Id := range data {
-   run.Results.Summary[Id].AvgTput = Throughput(data[Id].startTime,
-   0, data[Id].lastTime, data[Id].lastMops)
+   for Id, d := range data {
+   ws := [Id.Set]
+   s := [Id.Id]
+
+   s.AvgTput = Throughput(d.startTime, 0, d.lastTime, d.lastMops)
+   if s.AvgTput > ws.AvgMaxTput {
+   ws.AvgMaxTput = s.AvgTput
+   }
+   if s.AvgTput < ws.AvgMinTput || ws.AvgMinTput == 0 {
+   ws.AvgMinTput = s.AvgTput
+   }
+   
}
-   
+
+   // Calculate the average-of-averages for each set
+   for set := range 

[Xen-devel] [PATCH RFC 03/59] controller: Initial attempt to generalize process / vm creation

[Ronald Rojas]

From: George Dunlap 

Create a "Worker" interface, and have ProcessWorker implement that
interface.

Rename "Worker" to "WorkerReport" to make room for the interface.

Signed-off-by: George Dunlap 
---
 main.go | 45 +++--
 1 file changed, 27 insertions(+), 18 deletions(-)

diff --git a/main.go b/main.go
index 0cb9f51..034e2f2 100644
--- a/main.go
+++ b/main.go
@@ -11,24 +11,32 @@ import (

 )
 
-type Worker struct {
+type WorkerReport struct {
Id int
-   
-   c *exec.Cmd
+   Now int
+   Mops int
+   MaxDelta int
+}
+
+type Worker interface {
+   SetId(int)
+   Init() error
+   Shutdown()
+   Process(chan WorkerReport, chan bool)
+}
 
+type ProcessWorker struct {
+   id int
+   c *exec.Cmd
stdout io.ReadCloser
-   
jsonStarted bool
 }
 
-type WorkerReport struct {
-   Id int
-   Now int
-   Mops int
-   MaxDelta int
+func (w *ProcessWorker) SetId(i int) {
+   w.id = i
 }
 
-func (w *Worker) Init() (err error) {
+func (w *ProcessWorker) Init() (err error) {
w.c = exec.Command("../worker/worker-proc", "burnwait", "20", 
"2000")
 
w.stdout, err = w.c.StdoutPipe()
@@ -40,11 +48,11 @@ func (w *Worker) Init() (err error) {
return
 }
 
-func (w *Worker) Shutdown() {
+func (w *ProcessWorker) Shutdown() {
w.c.Process.Kill()
 }
 
-func (w *Worker) Process(report chan WorkerReport, done chan bool) {
+func (w *ProcessWorker) Process(report chan WorkerReport, done chan bool) {
w.c.Start()
 
scanner := bufio.NewScanner(w.stdout)
@@ -57,7 +65,7 @@ func (w *Worker) Process(report chan WorkerReport, done chan 
bool) {
if w.jsonStarted {
var r WorkerReport
json.Unmarshal([]byte(s), )
-   r.Id = w.Id
+   r.Id = w.id
report <- r
} else {
if s == "START JSON" {
@@ -79,7 +87,7 @@ const (
 )
 
 type WorkerState struct {
-   Worker
+   w Worker
LastReport WorkerReport
 }
 
@@ -114,11 +122,12 @@ func main() {
Workers := make([]WorkerState, count)

for i = 0; i< count; i++ {
-   Workers[i].Id = i
+   Workers[i].w = {}
+   Workers[i].w.SetId(i)

-   Workers[i].Init()
+   Workers[i].w.Init()

-   go Workers[i].Process(report, done)
+   go Workers[i].w.Process(report, done)
}
 
for i > 0 {
@@ -131,7 +140,7 @@ func main() {
case <-signals:
fmt.Println("SIGINT receieved, shutting down workers")
for j := range Workers {
-   Workers[j].Shutdown()
+   Workers[j].w.Shutdown()
}
}
}
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 19/59] Add basic libxl framework, get domain cpu_time

[Ronald Rojas]

From: George Dunlap 

Introduce libxl "Context" class with open, close, and dominfo.

Create a global variable in xenworker.go to hold the context; open on
first worker creation, close on last worker destruction.

Add a new element to WorkerReport, Cputime, and print it out.

For now, include hard-coded link to local Xen libraries.  This should
be sorted out at some point.

Signed-off-by: George Dunlap 
---
 Makefile |  13 +--
 benchmark.go |   2 ++
 libxl.go | 116 +++
 run.go   |   2 +-
 xenworker.go |  26 ++
 5 files changed, 156 insertions(+), 3 deletions(-)
 create mode 100644 libxl.go

diff --git a/Makefile b/Makefile
index 2e06f87..54f2ce8 100644
--- a/Makefile
+++ b/Makefile
@@ -4,8 +4,17 @@ BINALL = $(BIN)
 .PHONY: all
 all: $(BIN)
 
-schedbench: main.go processworker.go xenworker.go benchmark.go run.go
-   go build -o $@ $^
+
+
+CGO_CFLAGS = -I/build/hg/xen.git/dist/install/usr/local/include
+CGO_LIBS = -lyajl -lxenlight
+
+# FIXME
+XENLIB_PATH ?= /build/hg/xen.git/dist/install/usr/local/lib/
+CGO_LDFLAGS = -L$(XENLIB_PATH) -Wl,-rpath-link=$(XENLIB_PATH) $(CGO_LIBS)
+
+schedbench: main.go processworker.go xenworker.go benchmark.go run.go libxl.go
+   CGO_LDFLAGS="$(CGO_LDFLAGS)" CGO_CFLAGS="$(CGO_CFLAGS)" go build -o $@ 
$^
 
 .PHONY: clean
 clean:
diff --git a/benchmark.go b/benchmark.go
index 7fa83d2..4b2d805 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -24,6 +24,7 @@ import (
"io/ioutil"
"encoding/json"
"math"
+   "time"
 )
 
 type WorkerId struct {
@@ -40,6 +41,7 @@ type WorkerReport struct {
Now int
Mops int
MaxDelta int
+   Cputime time.Duration
 }
 
 type WorkerParams struct {
diff --git a/libxl.go b/libxl.go
new file mode 100644
index 000..39e47ab
--- /dev/null
+++ b/libxl.go
@@ -0,0 +1,116 @@
+package main
+
+/*
+#include 
+*/
+import "C"
+
+import (
+   "unsafe"
+   "fmt"
+   "time"
+)
+
+type Context struct {
+   ctx *C.libxl_ctx
+}
+
+func NewContext() (Ctx *Context, err error) {
+   Ctx = {}
+   
+   err = Ctx.Open()
+
+   return
+}
+
+func (Ctx *Context) IsOpen() bool {
+   return Ctx.ctx != nil
+}
+
+func (Ctx *Context) Open() (err error) {
+   ret := C.libxl_ctx_alloc(unsafe.Pointer(), C.LIBXL_VERSION, 0, 
nil)
+
+   if ret != 0 {
+   err = fmt.Errorf("Allocating libxl context: %d", ret)
+   }
+   return
+}
+
+func (Ctx *Context) Close() (err error) {
+   ret := C.libxl_ctx_free(unsafe.Pointer(Ctx.ctx))
+   Ctx.ctx = nil
+
+   if ret != 0 {
+   err = fmt.Errorf("Freeing libxl context: %d", ret)
+   }
+   return
+}
+
+type Domid uint32
+
+type MemKB uint64
+
+// FIXME: Use the idl to generate types
+type Dominfo struct {
+   // FIXME: uuid
+   Domid Domid
+   Running   bool
+   Blocked   bool
+   Pausedbool
+   Shutdown  bool
+   Dying bool
+   Never_stopbool
+   
+   Shutdown_reason   int32 // FIXME shutdown_reason enumeration
+   Outstanding_memkb MemKB
+   Current_memkb MemKB
+   Shared_memkb  MemKB
+   Paged_memkb   MemKB
+   Max_memkb MemKB
+   Cpu_time  time.Duration
+   Vcpu_max_id   uint32
+   Vcpu_online   uint32
+   Cpupool   uint32
+   Domain_type   int32 //FIXME libxl_domain_type enumeration
+
+}
+
+func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err error) {
+   if Ctx.ctx == nil {
+   err = fmt.Errorf("Context not opened")
+   return
+   }
+
+   
+   var cdi C.libxl_dominfo
+
+   ret := C.libxl_domain_info(Ctx.ctx, unsafe.Pointer(), 
C.uint32_t(Id))
+
+   // FIXME: IsDomainNotPresentError
+   if ret != 0 {
+   err = fmt.Errorf("libxl_domain_info failed: %d", ret)
+   return
+   }
+
+   // FIXME -- use introspection to make this more robust
+   di = {}
+   di.Domid = Domid(cdi.domid)
+   di.Running = bool(cdi.running)
+   di.Blocked = bool(cdi.blocked)
+   di.Paused = bool(cdi.paused)
+   di.Shutdown = bool(cdi.shutdown)
+   di.Dying = bool(cdi.dying)
+   di.Never_stop = bool(cdi.never_stop)
+   di.Shutdown_reason = int32(cdi.shutdown_reason)
+   di.Outstanding_memkb = MemKB(cdi.outstanding_memkb)
+   di.Current_memkb = MemKB(cdi.current_memkb)
+   di.Shared_memkb = MemKB(cdi.shared_memkb)
+   di.Paged_memkb = MemKB(cdi.paged_memkb)
+   di.Max_memkb = MemKB(cdi.max_memkb)
+   di.Cpu_time = time.Duration(cdi.cpu_time)
+   di.Vcpu_max_id = uint32(cdi.vcpu_max_id)
+   di.Vcpu_online = uint32(cdi.vcpu_online)
+   di.Cpupool = uint32(cdi.cpupool)
+   di.Domain_type = int32(cdi.domain_type)
+   return
+}
diff 

[Xen-devel] [PATCH RFC 26/59] Use kops rather than mops

[Ronald Rojas]

From: George Dunlap 

1 million operations on my test box takes about 3ms -- meaning the
minimum granularity for how long to burn cpu is far longer than
typical.  Make this kops instead, giving us a minimum granularity of 3us.

Update most of the default workers to have similar patterns but on a
1/100 (nb not 1/1000) scale; with the exception of worker A (1/1000
scale) and the first worker in worker B (1/10 scale).

Also, actually fix the name of the scheduler in the README.

Signed-off-by: George Dunlap 
---
 benchmark.go | 18 +-
 main.go  | 16 
 run.go   |  8 
 3 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 73c5441..ffecb82 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -39,7 +39,7 @@ func (wid WorkerId) String() (string) {
 type WorkerReport struct {
Id WorkerId
Now int
-   Mops int
+   Kops int
MaxDelta int
Cputime time.Duration
 }
@@ -89,9 +89,9 @@ const (
 
 func Throughput(lt int, lm int, t int, m int) (tput float64) {
time := float64(t - lt) / SEC
-   mops := m - lm
+   kops := m - lm

-   tput = float64(mops) / time
+   tput = float64(kops) / time
return
 }
 
@@ -184,7 +184,7 @@ func (run *BenchmarkRun) Process() (err error) {
startTime int
startCputime time.Duration
lastTime int
-   lastMops int
+   lastKops int
lastCputime time.Duration
}

@@ -225,7 +225,7 @@ func (run *BenchmarkRun) Process() (err error) {
d.startTime = e.Now
d.startCputime = e.Cputime
} else {
-   tput := Throughput(d.lastTime, d.lastMops, e.Now, 
e.Mops)
+   tput := Throughput(d.lastTime, d.lastKops, e.Now, 
e.Kops)
util := Utilization(d.lastTime, d.lastCputime, e.Now, 
e.Cputime)
 
s.MinMaxTput.Update(tput)
@@ -234,7 +234,7 @@ func (run *BenchmarkRun) Process() (err error) {
ws.MinMaxUtil.Update(util)
}
d.lastTime = e.Now
-   d.lastMops = e.Mops
+   d.lastKops = e.Kops
d.lastCputime = e.Cputime
}
 
@@ -242,11 +242,11 @@ func (run *BenchmarkRun) Process() (err error) {
ws := [Id.Set]
s := [Id.Id]
 
-   s.TotalTput = d.lastMops
+   s.TotalTput = d.lastKops
s.TotalTime = time.Duration(d.lastTime - d.startTime)
s.TotalCputime = d.lastCputime - d.startCputime

-   s.AvgTput = Throughput(d.startTime, 0, d.lastTime, d.lastMops)
+   s.AvgTput = Throughput(d.startTime, 0, d.lastTime, d.lastKops)
s.AvgUtil = Utilization(d.startTime, d.startCputime, 
d.lastTime, d.lastCputime)
 
ws.MinMaxAvgTput.Update(s.AvgTput)
@@ -345,7 +345,7 @@ func (run *BenchmarkRun) TextReport(level int) (err error) {
for _, e := range s.Raw {
time := float64(e.Now) / SEC
fmt.Printf ("   [%8.3f] %8.3f 
%8d %12d\n", time,
-   e.Cputime.Seconds(), 
e.Mops, e.MaxDelta)
+   e.Cputime.Seconds(), 
e.Kops, e.MaxDelta)
}
}
 
diff --git a/main.go b/main.go
index 61d3949..6eaa39e 100644
--- a/main.go
+++ b/main.go
@@ -29,15 +29,15 @@ func main() {
 
switch(os.Args[1]) {
case "plan":
-   workerA := []string{"burnwait", "1", "2000"}
+   workerA := []string{"burnwait", "7", "2"}
//workerB := []string{"burnwait", "10", "2000"}
-   workerB := []string{"burnwait", "1", "3000",
-   "burnwait", "2", "3000",
-   "burnwait", "1", "3000",
-   "burnwait", "1", "3000",
-   "burnwait", "1", "3000",
-   "burnwait", "1", "3000",
-   "burnwait", "3", "3000",
+   workerB := []string{"burnwait", "100", "300",
+   "burnwait", "20", "30",
+   "burnwait", "10", "30",
+   "burnwait", "10", "30",
+   "burnwait", "10", "30",
+   "burnwait", "10", "30",
+   "burnwait", "30", "30",
}
 
 
diff --git a/run.go b/run.go
index 259f427..ed1957b 100644
--- a/run.go
+++ b/run.go
@@ -41,15 +41,15 @@ func Report(ws *WorkerState, r WorkerReport) {
if (lr.Now 

[Xen-devel] [PATCH RFC 24/59] report: Lots of changes

[Ronald Rojas]

From: George Dunlap 

Store raw worker reports in the worker summary for easy printing.

Add "level" argument to textreport to easly switch between different
levels of granularity.

Report total throughput, time, and cputime for workers.

When running, print plain time and aggregate cputime in addition to
deltas.

Signed-off-by: George Dunlap 
---
 benchmark.go | 34 +++---
 main.go  |  2 +-
 run.go   |  7 +--
 3 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 7ea9aaa..73c5441 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -115,8 +115,12 @@ func (mm *MinMax) Update(x float64) {
 }
 
 type WorkerSummary struct {
+   Raw []WorkerReport
MinMaxTput MinMax
MinMaxUtil MinMax
+   TotalTput int
+   TotalTime time.Duration
+   TotalCputime time.Duration
AvgTput float64
AvgUtil float64
 }
@@ -201,7 +205,7 @@ func (run *BenchmarkRun) Process() (err error) {
}
 
ws := [e.Id.Set]
-   
+
if e.Id.Id > len(ws.Workers) {
return fmt.Errorf("Internal error: e.Id.Id %d > 
len(Results.Summary[].Workers) %d\n",
e.Id.Id, len(ws.Workers))
@@ -209,6 +213,8 @@ func (run *BenchmarkRun) Process() (err error) {
 
s := [e.Id.Id]
 
+   s.Raw = append(s.Raw, e)
+   
d := data[e.Id]
if d == nil {
d = {}
@@ -236,6 +242,10 @@ func (run *BenchmarkRun) Process() (err error) {
ws := [Id.Set]
s := [Id.Id]
 
+   s.TotalTput = d.lastMops
+   s.TotalTime = time.Duration(d.lastTime - d.startTime)
+   s.TotalCputime = d.lastCputime - d.startCputime
+   
s.AvgTput = Throughput(d.startTime, 0, d.lastTime, d.lastMops)
s.AvgUtil = Utilization(d.startTime, d.startCputime, 
d.lastTime, d.lastCputime)
 
@@ -287,7 +297,7 @@ func (run *BenchmarkRun) Process() (err error) {
return
 }
 
-func (run *BenchmarkRun) TextReport() (err error) {
+func (run *BenchmarkRun) TextReport(level int) (err error) {
var done bool
done, err = run.checkSummary()
if err != nil {
@@ -320,15 +330,25 @@ func (run *BenchmarkRun) TextReport() (err error) {
ws.MinMaxAvgUtil.Min, ws.MinMaxUtil.Max, 
ws.MinMaxUtil.Min)
}
 
-   if true {
-   fmt.Printf("\n%8s %8s %8s %8s %8s %8s %8s\n", "workerid", 
"tavg", "tmin", "tmax", "uavg", "umin", "umax")
+   if level >= 1 {
+   fmt.Printf("\n%8s %8s %8s %8s %8s %8s %8s %8s %8s %8s\n", 
"workerid", "toput", "time", "cpu", "tavg", "tmin", "tmax", "uavg", "umin", 
"umax")
for set := range run.Results.Summary {
for id := range run.Results.Summary[set].Workers {
s := run.Results.Summary[set].Workers[id]
-   fmt.Printf("%2d:%2d%8.2f %8.2f %8.2f %8.2f 
%8.2f %8.2f\n",
+   fmt.Printf("%2d:%2d%10d %8.2f %8.2f %8.2f 
%8.2f %8.2f %8.2f %8.2f %8.2f\n",
set, id,
+   s.TotalTput, s.TotalTime.Seconds(), 
s.TotalCputime.Seconds(),
s.AvgTput, s.MinMaxTput.Min, 
s.MinMaxTput.Max,
s.AvgUtil, s.MinMaxUtil.Min, 
s.MinMaxUtil.Max)
+
+   if level >= 2 {
+   for _, e := range s.Raw {
+   time := float64(e.Now) / SEC
+   fmt.Printf ("   [%8.3f] %8.3f 
%8d %12d\n", time,
+   e.Cputime.Seconds(), 
e.Mops, e.MaxDelta)
+   }
+   }
+
}
}
}
@@ -391,7 +411,7 @@ func (plan *BenchmarkPlan) Save() (err error) {
return
 }
 
-func (plan *BenchmarkPlan) TextReport() (err error) {
+func (plan *BenchmarkPlan) TextReport(level int) (err error) {
for i := range plan.Runs {
r := [i]
if ! r.Completed {
@@ -404,7 +424,7 @@ func (plan *BenchmarkPlan) TextReport() (err error) {
return
}
 
-   err = r.TextReport()
+   err = r.TextReport(level)
if err != nil {
return
}
diff --git a/main.go b/main.go
index a931567..bb46dbc 100644
--- a/main.go
+++ b/main.go
@@ -98,7 +98,7 @@ func main() {
os.Exit(1)
}

-   err = plan.TextReport()
+   err = plan.TextReport(2)

[Xen-devel] [PATCH RFC 23/59] run: Don't collect results reported after command to stop guests is issued

[Ronald Rojas]

From: George Dunlap 

Particularly on heavily-loaded systems, this can lead to skew as some
guests have already stopped processing.

Signed-off-by: George Dunlap 
---
 run.go | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/run.go b/run.go
index 4222001..41408dd 100644
--- a/run.go
+++ b/run.go
@@ -172,9 +172,14 @@ func (run *BenchmarkRun) Run() (err error) {
for i > 0 {
select {
case r := <-report:
-   run.Results.Raw = append(run.Results.Raw, r)
-   Report(Workers[r.Id], r)
+   if ! stopped {
+   run.Results.Raw = append(run.Results.Raw, r)
+   Report(Workers[r.Id], r)
+   }
case <-done:
+   if ! stopped {
+   fmt.Println("WARNING: Worker left early")
+   }
i--;
fmt.Println(i, "workers left");
case <-timeout:
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 18/59] controller: Add cpupool global config

[Ronald Rojas]

From: George Dunlap 

Define WorkerConfig for worker configuration, add "Pool" as a pool in
which to place the worker (currently implemented in Xen only).

Add WorkerConfig options at the WorkerSet, BenchmarkRun, and
BenchmarkPlan levels; allow local levels to "override" global config
options.

The cpupool must already be defined.

Signed-off-by: George Dunlap 
---
 benchmark.go | 19 ++-
 main.go  |  1 +
 processworker.go |  2 +-
 run.go   | 16 +++-
 xenworker.go |  6 +-
 5 files changed, 36 insertions(+), 8 deletions(-)

diff --git a/benchmark.go b/benchmark.go
index 4cc9d61..7fa83d2 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -46,14 +46,27 @@ type WorkerParams struct {
Args []string
 }
 
+type WorkerConfig struct {
+   Pool string
+}
+
+// Propagate unset values from a higher level
+func (l *WorkerConfig) PropagateFrom(g WorkerConfig) {
+   if l.Pool == "" {
+   l.Pool = g.Pool
+   }
+}
+
+
 type WorkerSet struct {
Params WorkerParams
+   Config WorkerConfig
Count int
 }
 
 type Worker interface {
SetId(WorkerId)
-   Init(WorkerParams) error
+   Init(WorkerParams, WorkerConfig) error
Shutdown()
Process(chan WorkerReport, chan bool)
 }
@@ -97,6 +110,7 @@ type BenchmarkRunData struct {
 type BenchmarkRun struct {
Label string
WorkerSets []WorkerSet
+   WorkerConfig
RuntimeSeconds int
Completed bool
Results BenchmarkRunData 
@@ -105,6 +119,9 @@ type BenchmarkRun struct {
 type BenchmarkPlan struct {
filename string
WorkerType int
+   // Global options for workers that will be over-ridden by Run
+   // and WorkerSet config options
+   WorkerConfig
Runs []BenchmarkRun
 }
 
diff --git a/main.go b/main.go
index f58068f..a931567 100644
--- a/main.go
+++ b/main.go
@@ -43,6 +43,7 @@ func main() {
 
plan :=  BenchmarkPlan{
WorkerType:WorkerXen,
+   WorkerConfig:WorkerConfig{Pool:"schedbench"},
filename:filename,
Runs:[]BenchmarkRun{
{Label:"baseline-a",
diff --git a/processworker.go b/processworker.go
index 5e26d81..f517321 100644
--- a/processworker.go
+++ b/processworker.go
@@ -38,7 +38,7 @@ func (w *ProcessWorker) SetId(i WorkerId) {
w.id = i
 }
 
-func (w *ProcessWorker) Init(p WorkerParams) (err error) {
+func (w *ProcessWorker) Init(p WorkerParams, g WorkerConfig) (err error) {
w.c = exec.Command("./worker-proc", p.Args...)
 
w.stdout, err = w.c.StdoutPipe()
diff --git a/run.go b/run.go
index 762b408..9f1edcf 100644
--- a/run.go
+++ b/run.go
@@ -96,7 +96,7 @@ func NewWorkerList(WorkerSets []WorkerSet, workerType int) 
(wl WorkerList, err e

ws.w.SetId(Id)

-   ws.w.Init(WorkerSets[wsi].Params)
+   ws.w.Init(WorkerSets[wsi].Params, 
WorkerSets[wsi].Config)
 
wl[Id] = ws
}
@@ -105,6 +105,10 @@ func NewWorkerList(WorkerSets []WorkerSet, workerType int) 
(wl WorkerList, err e
 }
 
 func (run *BenchmarkRun) Run() (err error) {
+   for wsi := range run.WorkerSets {
+   run.WorkerSets[wsi].Config.PropagateFrom(run.WorkerConfig)
+   }
+   
Workers, err := NewWorkerList(run.WorkerSets, WorkerXen)
if err != nil {
fmt.Println("Error creating workers: %v", err)
@@ -159,14 +163,16 @@ func (run *BenchmarkRun) Run() (err error) {
 
 func (plan *BenchmarkPlan) Run() (err error) {
for i := range plan.Runs {
-   if ! plan.Runs[i].Completed {
-   fmt.Printf("Running test [%d] %s\n", i, 
plan.Runs[i].Label)
-   err = plan.Runs[i].Run()
+   r := [i];
+   if ! r.Completed {
+   r.WorkerConfig.PropagateFrom(plan.WorkerConfig)
+   fmt.Printf("Running test [%d] %s\n", i, r.Label)
+   err = r.Run()
if err != nil {
return
}
}
-   fmt.Printf("Test [%d] %s completed\n", i, plan.Runs[i].Label)
+   fmt.Printf("Test [%d] %s completed\n", i, r.Label)
err = plan.Save()
if err != nil {
fmt.Println("Error saving: ", err)
diff --git a/xenworker.go b/xenworker.go
index e14676c..4d42e5e 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -58,7 +58,7 @@ func (w *XenWorker) SetId(i WorkerId) {
w.domid = -1 // INVALID DOMID
 }
 
-func (w *XenWorker) Init(p WorkerParams) (err error) {
+func (w *XenWorker) Init(p WorkerParams, g WorkerConfig) (err error) {
mock := false

// Make xl 

[Xen-devel] [PATCH RFC 44/59] libxl: Implement CpupoolCreate

[Ronald Rojas]

From: George Dunlap 

Implement CpupoolCreate.

For now, follow the lead of "xl cpupool-create", by automatically
generating a new uuid each time a new cpupool is created, and by not
allowing the user to attempt to set the poolid.

This requires us to implement bytmapGotoC, which C-allocates the
appropriate array and copies the Go byte map into it.  This must be
C-allocated because the runtime restricts how Go-allocated structures
can be passed into C function calls.

While we're here, reorganize functions slightly, and change the Bitmap
implementation to automatically extend the array if the size of the
byte array is too small.

Signed-off-by: George Dunlap 
---
 libxl.go | 119 +--
 1 file changed, 85 insertions(+), 34 deletions(-)

diff --git a/libxl.go b/libxl.go
index 3f6ba01..6e17b53 100644
--- a/libxl.go
+++ b/libxl.go
@@ -176,20 +176,38 @@ func (bm *Bitmap) Alloc(max int) {
 }
 
 // Return a Go bitmap which is a copy of the referred C bitmap.
-func bitmapCToGo(cbm *C.libxl_bitmap) (bm Bitmap) {
+func bitmapCToGo(cbm *C.libxl_bitmap) (gbm Bitmap) {
// Alloc a Go slice for the bytes
size := int(cbm.size)
-   bm.Alloc(size*8)
+   gbm.Alloc(size*8)
 
// Make a slice pointing to the C array
mapslice := (*[1 << 30]C.uint8_t)(unsafe.Pointer(cbm._map))[:size:size]
 
// And copy the C array into the Go array
-   copy(bm.bitmap, mapslice)
+   copy(gbm.bitmap, mapslice)
 
return
 }
 
+// Must be C.libxl_bitmap_dispose'd of afterwards
+func bitmapGotoC(gbm Bitmap, cbm *C.libxl_bitmap) {
+   C.libxl_bitmap_init(cbm)
+
+   size := len(gbm.bitmap)
+   cbm._map = (*C.uint8_t)(C.malloc(C.size_t(size)))
+   cbm.size = C.uint32_t(size)
+   if cbm._map == nil {
+   panic("C.calloc failed!")
+   }
+
+   // Make a slice pointing to the C array
+   mapslice := (*[1 << 30]C.uint8_t)(unsafe.Pointer(cbm._map))[:size:size]
+
+   // And copy the Go array into the C array
+   copy(mapslice, gbm.bitmap)
+}
+
 func (bm *Bitmap) Test(bit int) (bool) {
ubit := uint(bit)
if (bit > bm.Max()) {
@@ -199,14 +217,13 @@ func (bm *Bitmap) Test(bit int) (bool) {
return (bm.bitmap[bit / 8] & (1 << (ubit & 7))) != 0
 }
 
-// FIXME: Do we really just want to silently fail here?
 func (bm *Bitmap) Set(bit int) {
-   ubit := uint(bit)
-   if (bit > bm.Max()) {
-   return
+   ibit := bit / 8;
+   if (ibit + 1 > len(bm.bitmap)) {
+   bm.bitmap = append(bm.bitmap, make([]C.uint8_t, 
ibit+1-len(bm.bitmap))...)
}

-   bm.bitmap[bit / 8] |= 1 << (ubit & 7)
+   bm.bitmap[ibit] |= 1 << (uint(bit) & 7)
 }
 
 func (bm *Bitmap) Clear(bit int) {
@@ -234,12 +251,12 @@ func (bm *Bitmap) Max() (int) {
 // ])
 type Scheduler int
 var (
-   SchedulerUnknown  Scheduler = 0
-   SchedulerSedf Scheduler = 4
-   SchedulerCredit   Scheduler = 5
-   SchedulerCredit2  Scheduler = 6
-   SchedulerArinc653 Scheduler = 7
-   SchedulerRTDS Scheduler = 8
+   SchedulerUnknown  Scheduler = C.LIBXL_SCHEDULER_UNKNOWN
+   SchedulerSedf Scheduler = C.LIBXL_SCHEDULER_SEDF
+   SchedulerCredit   Scheduler = C.LIBXL_SCHEDULER_CREDIT
+   SchedulerCredit2  Scheduler = C.LIBXL_SCHEDULER_CREDIT2
+   SchedulerArinc653 Scheduler = C.LIBXL_SCHEDULER_ARINC653
+   SchedulerRTDS Scheduler = C.LIBXL_SCHEDULER_RTDS
 )
 
 // const char *libxl_scheduler_to_string(libxl_scheduler p);
@@ -315,23 +332,38 @@ func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
return
 }
 
-func (Ctx *Context) CpupoolFindByName(name string) (info CpupoolInfo, found 
bool) {
-   plist := Ctx.ListCpupool()
+// int libxl_cpupool_create(libxl_ctx *ctx, const char *name,
+//  libxl_scheduler sched,
+//  libxl_bitmap cpumap, libxl_uuid *uuid,
+//  uint32_t *poolid);
+// FIXME: uuid
+// FIXME: Setting poolid
+func (Ctx *Context) CpupoolCreate(Name string, Scheduler Scheduler, Cpumap 
Bitmap) (err error, Poolid uint32) {
+   poolid := C.uint32_t(0)
+   name := C.CString(Name)
+   defer C.free(unsafe.Pointer(name))
+   
+   // For now, just do what xl does, and make a new uuid every time we 
create the pool
+   var uuid C.libxl_uuid
+   C.libxl_uuid_generate()
 
-   for i := range plist {
-   if plist[i].PoolName == name {
-   found = true
-   info = plist[i]
-   return
-   }
+   var cbm C.libxl_bitmap
+   bitmapGotoC(Cpumap, )
+   defer C.libxl_bitmap_dispose()
+   
+   ret := C.libxl_cpupool_create(Ctx.ctx, name, 
C.libxl_scheduler(Scheduler),
+   cbm, , )
+   // FIXME: Proper error
+   if ret != 0 {
+  

[Xen-devel] [PATCH RFC 45/59] libxl: Implement Destroy, Add/Remove operations

[Ronald Rojas]

From: George Dunlap 

Also:

* Implement useful Bitmap operations: SetRange(), ClearRange(),
IsEmpty(), and And().

* Normalize CpupoolInfo element naming.

* Implement CpupoolMakeFree, which will scan through cpupools freeing
  the cpus in the map.

Signed-off-by: George Dunlap 
---
 libxl.go | 187 ++-
 1 file changed, 150 insertions(+), 37 deletions(-)

diff --git a/libxl.go b/libxl.go
index 6e17b53..92064ca 100644
--- a/libxl.go
+++ b/libxl.go
@@ -226,6 +226,12 @@ func (bm *Bitmap) Set(bit int) {
bm.bitmap[ibit] |= 1 << (uint(bit) & 7)
 }
 
+func (bm *Bitmap) SetRange(start int, end int) {
+   for i := start; i <= end; i++ {
+   bm.Set(i)
+   }
+}
+
 func (bm *Bitmap) Clear(bit int) {
ubit := uint(bit)
if (bit > bm.Max()) {
@@ -235,10 +241,42 @@ func (bm *Bitmap) Clear(bit int) {
bm.bitmap[bit / 8] &= ^(1 << (ubit & 7))
 }
 
+func (bm *Bitmap) ClearRange(start int, end int) {
+   for i := start; i <= end; i++ {
+   bm.Clear(i)
+   }
+}
+
 func (bm *Bitmap) Max() (int) {
return len(bm.bitmap) * 8
 }
 
+func (bm *Bitmap) IsEmpty() (bool) {
+   for i:=0; i len(b.bitmap) {
+   max = len(a.bitmap)
+   min = len(b.bitmap)
+   } else {
+   max = len(b.bitmap)
+   min = len(a.bitmap)
+   }
+   c.bitmap = make([]C.uint8_t, max)
+
+   for i := 0; i < min; i++ {
+   c.bitmap[i] = a.bitmap[i] & b.bitmap[i]
+   }
+   return
+}
+
 // # Consistent with values defined in domctl.h
 // # Except unknown which we have made up
 // libxl_scheduler = Enumeration("scheduler", [
@@ -294,11 +332,11 @@ func SchedulerFromString(name string) (s Scheduler, err 
error) {
 // ], dir=DIR_OUT)
 
 type CpupoolInfo struct {
-   PoolId uint32
+   Poolid uint32
PoolName string
Scheduler Scheduler
DomainCount int
-   CpuMap Bitmap
+   Cpumap Bitmap
 }
 
 // libxl_cpupoolinfo * libxl_list_cpupool(libxl_ctx*, int *nb_pool_out);
@@ -320,11 +358,11 @@ func (Ctx *Context) ListCpupool() (list []CpupoolInfo) {
for i := range cpupoolListSlice {
var info CpupoolInfo

-   info.PoolId = uint32(cpupoolListSlice[i].poolid)
+   info.Poolid = uint32(cpupoolListSlice[i].poolid)
info.PoolName = C.GoString(cpupoolListSlice[i].pool_name)
info.Scheduler = Scheduler(cpupoolListSlice[i].sched)
info.DomainCount = int(cpupoolListSlice[i].n_dom)
-   info.CpuMap = bitmapCToGo([i].cpumap)
+   info.Cpumap = bitmapCToGo([i].cpumap)
 
list = append(list, info)
}
@@ -365,15 +403,78 @@ func (Ctx *Context) CpupoolCreate(Name string, Scheduler 
Scheduler, Cpumap Bitma
 }
 
 // int libxl_cpupool_destroy(libxl_ctx *ctx, uint32_t poolid);
-// int libxl_cpupool_rename(libxl_ctx *ctx, const char *name, uint32_t poolid);
+func (Ctx *Context) CpupoolDestroy(Poolid uint32) (err error) {
+   ret := C.libxl_cpupool_destroy(Ctx.ctx, C.uint32_t(Poolid))
+   // FIXME: Proper error
+   if ret != 0 {
+   err = fmt.Errorf("libxl_cpupool_destroy failed: %d", ret)
+   return
+   }
+
+   return
+}
+
 // int libxl_cpupool_cpuadd(libxl_ctx *ctx, uint32_t poolid, int cpu);
-// int libxl_cpupool_cpuadd_node(libxl_ctx *ctx, uint32_t poolid, int node, 
int *cpus);
+func (Ctx *Context) CpupoolCpuadd(Poolid uint32, Cpu int) (err error) {
+   ret := C.libxl_cpupool_cpuadd(Ctx.ctx, C.uint32_t(Poolid), C.int(Cpu))
+   // FIXME: Proper error
+   if ret != 0 {
+   err = fmt.Errorf("libxl_cpupool_cpuadd failed: %d", ret)
+   return
+   }
+
+   return
+}
+
 // int libxl_cpupool_cpuadd_cpumap(libxl_ctx *ctx, uint32_t poolid,
 // const libxl_bitmap *cpumap);
+func (Ctx *Context) CpupoolCpuaddCpumap(Poolid uint32, Cpumap Bitmap) (err 
error) {
+   var cbm C.libxl_bitmap
+   bitmapGotoC(Cpumap, )
+   defer C.libxl_bitmap_dispose()
+   
+   ret := C.libxl_cpupool_cpuadd_cpumap(Ctx.ctx, C.uint32_t(Poolid), )
+   // FIXME: Proper error
+   if ret != 0 {
+   err = fmt.Errorf("libxl_cpupool_cpuadd_cpumap failed: %d", ret)
+   return
+   }
+
+   return
+}
+
 // int libxl_cpupool_cpuremove(libxl_ctx *ctx, uint32_t poolid, int cpu);
-// int libxl_cpupool_cpuremove_node(libxl_ctx *ctx, uint32_t poolid, int node, 
int *cpus);
+func (Ctx *Context) CpupoolCpuremove(Poolid uint32, Cpu int) (err error) {
+   ret := 

[Xen-devel] [PATCH RFC 25/59] main: Change default workload to something a bit more extreme

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 main.go | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/main.go b/main.go
index bb46dbc..61d3949 100644
--- a/main.go
+++ b/main.go
@@ -29,15 +29,15 @@ func main() {
 
switch(os.Args[1]) {
case "plan":
-   workerA := []string{"burnwait", "20", "2000"}
+   workerA := []string{"burnwait", "1", "2000"}
//workerB := []string{"burnwait", "10", "2000"}
-   workerB := []string{"burnwait", "1", "2000",
-   "burnwait", "2", "2000",
-   "burnwait", "1", "2000",
-   "burnwait", "1", "2000",
-   "burnwait", "1", "2000",
-   "burnwait", "1", "2000",
-   "burnwait", "3", "2000",
+   workerB := []string{"burnwait", "1", "3000",
+   "burnwait", "2", "3000",
+   "burnwait", "1", "3000",
+   "burnwait", "1", "3000",
+   "burnwait", "1", "3000",
+   "burnwait", "1", "3000",
+   "burnwait", "3", "3000",
}
 
 
@@ -58,7 +58,6 @@ func main() {
RuntimeSeconds:10,},
}}
 
-
for i := 1; i <= 16 ; i *= 2 {
label := fmt.Sprintf("%da+%db", i, i)
run := BenchmarkRun{
@@ -98,7 +97,7 @@ func main() {
os.Exit(1)
}

-   err = plan.TextReport(2)
+   err = plan.TextReport(0)
if err != nil {
fmt.Println("Running benchmark run:", err)
os.Exit(1)
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 10/59] controller: First cut at BenchmarkParams

[Ronald Rojas]

From: George Dunlap 

Struct for configurable worker sets, as well as a set runtime.

Signed-off-by: George Dunlap 
---
 main.go | 75 ++---
 1 file changed, 53 insertions(+), 22 deletions(-)

diff --git a/main.go b/main.go
index 89e9cf3..7fbb60a 100644
--- a/main.go
+++ b/main.go
@@ -4,7 +4,7 @@ import (
"fmt"
"os"
"os/signal"
-   
+   "time"
 )
 
 type WorkerReport struct {
@@ -18,6 +18,16 @@ type WorkerParams struct {
Args []string
 }
 
+type WorkerSet struct {
+   Params WorkerParams
+   Count int
+}
+
+type BenchmarkParams struct {
+   Workers []WorkerSet
+   RuntimeSeconds int
+}
+
 type Worker interface {
SetId(int)
Init(WorkerParams) error
@@ -75,29 +85,53 @@ const (
WorkerXen = iota
 )
 
-func NewWorkerList(count int, workerType int) (ws WorkerList, err error) {
+func NewWorkerList(workers []WorkerSet, workerType int) (ws WorkerList, err 
error) {
+   count := 0
+
+   // wsi: WorkerSet index
+   for wsi := range workers {
+   count += workers[wsi].Count
+   }
+
+   fmt.Println("Making ", count, " total workers")
ws = WorkerList(make([]WorkerState, count))
 
-   for i := 0; i< count; i++ {
-   switch workerType {
-   case WorkerProcess:
-   ws[i].w = {}
-   case WorkerXen:
-   ws[i].w = {}
-   default:
-   err = fmt.Errorf("Unknown type: %d", workerType)
-   }
-   ws[i].w.SetId(i)
+   // wli: WorkerList index
+   wli := 0
+   for wsi := range workers {
+   for i := 0; i < workers[wsi].Count; i, wli = i+1, wli+1 {
+   switch workerType {
+   case WorkerProcess:
+   ws[wli].w = {}
+   case WorkerXen:
+   ws[wli].w = {}
+   default:
+   err = fmt.Errorf("Unknown type: %d", workerType)
+   }
+   ws[wli].w.SetId(wli)

-   ws[i].w.Init(WorkerParams{[]string{"burnwait", "20", 
"2000"}})
+   ws[wli].w.Init(workers[wsi].Params)
+   }
}
return
 }
 
 func main() {
-   killed := false
-   
-   count := 2
+   bp :=  BenchmarkParams{
+   Workers:[]WorkerSet{
+   {Params:WorkerParams{[]string{"burnwait", "20", 
"2000"}},
+   Count:2},
+   {Params:WorkerParams{[]string{"burnwait", "10", 
"3000"}},
+   Count:3},
+   },
+   RuntimeSeconds:5,
+   }
+
+   Workers, err := NewWorkerList(bp.Workers, WorkerProcess)
+   if err != nil {
+   fmt.Println("Error creating workers: %v", err)
+   return
+   }

report := make(chan WorkerReport)
done := make(chan bool)
@@ -105,14 +139,10 @@ func main() {
 
signal.Notify(signals, os.Interrupt)

-   Workers, err := NewWorkerList(count, WorkerProcess)
-   if err != nil {
-   fmt.Println("Error creating workers: %v", err)
-   return
-   }
-   
i := Workers.Start(report, done)
 
+   timeout := time.After(time.Duration(bp.RuntimeSeconds) * time.Second);
+   killed := false
for i > 0 {
select {
case r := <-report:
@@ -121,6 +151,7 @@ func main() {
i--;
fmt.Println(i, "workers left");
case <-signals:
+   case <-timeout:
if ! killed {
fmt.Println("SIGINT receieved, shutting down 
workers")
Workers.Stop()
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 15/59] controller: Move "running" code to a separate file

[Ronald Rojas]

From: George Dunlap 

Signed-off-by: George Dunlap 
---
 Makefile |   2 +-
 benchmark.go | 153 --
 run.go   | 178 +++
 3 files changed, 179 insertions(+), 154 deletions(-)
 create mode 100644 run.go

diff --git a/Makefile b/Makefile
index 7a33cfb..2e06f87 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ BINALL = $(BIN)
 .PHONY: all
 all: $(BIN)
 
-schedbench: main.go processworker.go xenworker.go benchmark.go
+schedbench: main.go processworker.go xenworker.go benchmark.go run.go
go build -o $@ $^
 
 .PHONY: clean
diff --git a/benchmark.go b/benchmark.go
index 4354a47..2e03fe5 100644
--- a/benchmark.go
+++ b/benchmark.go
@@ -21,8 +21,6 @@ package main
 import (
"fmt"
"os"
-   "os/signal"
-   "time"
"io/ioutil"
"encoding/json"
 )
@@ -71,11 +69,6 @@ const (
SEC = MSEC * 1000
 )
 
-type WorkerState struct {
-   w Worker
-   LastReport WorkerReport
-}
-
 func Throughput(lt int, lm int, t int, m int) (tput float64) {
time := float64(t - lt) / SEC
mops := m - lm
@@ -84,80 +77,6 @@ func Throughput(lt int, lm int, t int, m int) (tput float64) 
{
return
 }
 
-func Report(ws *WorkerState, r WorkerReport) {
-   //fmt.Println(r)
-
-   lr := ws.LastReport
-
-   if (lr.Now > 0) {
-   time := float64(r.Now - lr.Now) / SEC
-   mops := r.Mops - lr.Mops
-
-   tput := Throughput(lr.Now, lr.Mops, r.Now, r.Mops)
-   
-   fmt.Printf("%v Time: %2.3f Mops: %d Tput: %4.2f\n", r.Id, time, 
mops, tput);
-   }
-
-   ws.LastReport = r
-}
-
-type WorkerList map[WorkerId]*WorkerState
-
-func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) (i int) {
-   i = 0
-   for j := range *ws {
-   go (*ws)[j].w.Process(report, done)
-   i++
-   }
-   return
-}
-
-func (ws *WorkerList) Stop() {
-   for i := range *ws {
-   (*ws)[i].w.Shutdown()
-   }
-}
-
-const (
-   WorkerProcess = iota
-   WorkerXen = iota
-)
-
-func NewWorkerList(workers []WorkerSet, workerType int) (wl WorkerList, err 
error) {
-   wl = WorkerList(make(map[WorkerId]*WorkerState))
-
-   for wsi := range workers {
-   for i := 0; i < workers[wsi].Count; i = i+1 {
-   Id := WorkerId{Set:wsi,Id:i}
-
-   ws := wl[Id]
-
-   if ws != nil {
-   panic("Duplicate worker for id!")
-   }
-   
-   ws = {}
-   
-   switch workerType {
-   case WorkerProcess:
-   ws.w = {}
-   case WorkerXen:
-   ws.w = {}
-   default:
-   err = fmt.Errorf("Unknown type: %d", workerType)
-   return
-   }
-   
-   ws.w.SetId(Id)
-   
-   ws.w.Init(workers[wsi].Params)
-
-   wl[Id] = ws
-   }
-   }
-   return
-}
-
 type BenchmarkRunData struct {
Raw []WorkerReport   `json:",omitempty"`
Summary map[WorkerId]*WorkerSummary  `json:",omitempty"`
@@ -171,59 +90,6 @@ type BenchmarkRun struct {
Results BenchmarkRunData 
 }
 
-func (run *BenchmarkRun) Run() (err error) {
-   Workers, err := NewWorkerList(run.Workers, WorkerXen)
-   if err != nil {
-   fmt.Println("Error creating workers: %v", err)
-   return
-   }
-   
-   report := make(chan WorkerReport)
-   done := make(chan bool)
-   signals := make(chan os.Signal, 1)
-
-   signal.Notify(signals, os.Interrupt)
-   
-   i := Workers.Start(report, done)
-
-   // FIXME:
-   // 1. Make a zero timeout mean "never"
-   // 2. Make the signals / timeout thing a bit more rational; signal then 
timeout shouldn't hard kill
-   timeout := time.After(time.Duration(run.RuntimeSeconds) * time.Second);
-   stopped := false
-   for i > 0 {
-   select {
-   case r := <-report:
-   run.Results.Raw = append(run.Results.Raw, r)
-   Report(Workers[r.Id], r)
-   case <-done:
-   i--;
-   fmt.Println(i, "workers left");
-   case <-timeout:
-   if ! stopped {
-   Workers.Stop()
-   stopped = true
-   run.Completed = true
-   }
-   case <-signals:
-   if ! stopped {
-

[Xen-devel] [PATCH RFC 20/59] xenworker: Use libxl_domain_unpause rather than forking xl

[Ronald Rojas]

From: George Dunlap 

Unpause is a time-sensitive operation; use libxl to unpause directly
rather than forking and execing xl for each VM.

Signed-off-by: George Dunlap 
---
 libxl.go | 14 ++
 xenworker.go | 12 ++--
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/libxl.go b/libxl.go
index 39e47ab..415de7f 100644
--- a/libxl.go
+++ b/libxl.go
@@ -114,3 +114,17 @@ func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err 
error) {
di.Domain_type = int32(cdi.domain_type)
return
 }
+
+func (Ctx *Context) DomainUnpause(Id Domid) (err error) {
+   if Ctx.ctx == nil {
+   err = fmt.Errorf("Context not opened")
+   return
+   }
+
+   ret := C.libxl_domain_unpause(Ctx.ctx, C.uint32_t(Id))
+
+   if ret != 0 {
+   err = fmt.Errorf("libxl_domain_unpause failed: %d", ret)
+   }
+   return
+}
diff --git a/xenworker.go b/xenworker.go
index 31af35f..4077e77 100644
--- a/xenworker.go
+++ b/xenworker.go
@@ -232,16 +232,8 @@ func (w *XenWorker) Shutdown() {
 
 // FIXME: Return an error
 func (w *XenWorker) Process(report chan WorkerReport, done chan bool) {
-   mock := false
-   
-   // xl unpause [vmname]
-   args := []string{"xl", "unpause", w.vmname}
-   if mock {
-   args = append([]string{"echo"}, args...)
-   }
-   e := exec.Command(args[0], args[1:]...)
-
-   err := e.Run()
+   // // xl unpause [vmname]
+   err := xg.Ctx.DomainUnpause(Domid(w.domid))
if err != nil {
fmt.Printf("Error unpausing domain: %v\n", err)
return
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 07/59] controller: Introduce basic Xen functionality

[Ronald Rojas]

From: George Dunlap 

Rough-and-ready execution of xl commands, but they work, amazingly.

Signed-off-by: George Dunlap 
---
 Makefile |   2 +-
 main.go  |   3 +-
 xenworker.go | 231 +++
 3 files changed, 234 insertions(+), 2 deletions(-)
 create mode 100644 xenworker.go

diff --git a/Makefile b/Makefile
index 332c8d9..16af528 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ BINALL = $(BIN)
 .PHONY: all
 all: $(BIN)
 
-controller: main.go processworker.go
+controller: main.go processworker.go xenworker.go
go build -o $@ $^
 
 .PHONY: clean
diff --git a/main.go b/main.go
index 90388dd..04c8467 100644
--- a/main.go
+++ b/main.go
@@ -75,7 +75,8 @@ func main() {
Workers := WorkerList(make([]WorkerState, count))

for i = 0; i< count; i++ {
-   Workers[i].w = {}
+   //Workers[i].w = {}
+   Workers[i].w = {}
Workers[i].w.SetId(i)

Workers[i].w.Init(WorkerParams{[]string{"burnwait", "20", 
"2000"}})
diff --git a/xenworker.go b/xenworker.go
new file mode 100644
index 000..6023c50
--- /dev/null
+++ b/xenworker.go
@@ -0,0 +1,231 @@
+package main
+
+import (
+   "fmt"
+   "os"
+   "os/exec"
+   "encoding/json"
+   "bufio"
+   "io"
+)
+
+type XenWorker struct {
+   id int
+   vmname string
+   domid int
+   consoleCmd *exec.Cmd
+   console io.ReadCloser
+   jsonStarted bool
+}
+
+// We have to capitalize the element names so that the json class can
+// get access to it; so annotate the elements so they come out lower
+// case
+type RumpRunConfigBlk struct {
+   Source string `json:"source"`
+   Path string   `json:"path"`
+   Fstype string `json:"fstype"` 
+   Mountpoint string `json:"mountpoint"`
+}
+
+type RumpRunConfig struct {
+   Blk RumpRunConfigBlk `json:"blk"`
+   Cmdline string   `json:"cmdline"`
+   Hostname string  `json:"hostname"`
+}
+
+func (w *XenWorker) SetId(i int) {
+   w.id = i
+   w.vmname = fmt.Sprintf("worker-%d", i)
+   w.domid = -1 // INVALID DOMID
+}
+
+func (w *XenWorker) Init(p WorkerParams) (err error) {
+   mock := false
+   
+   // Make xl config file
+   //  name=worker-$(id)
+
+   cfgName := os.TempDir()+"/schedbench-"+w.vmname+".cfg"
+
+   cfg, err := os.Create(cfgName)
+   //defer os.Remove(cfgName)
+
+   if err != nil {
+   fmt.Printf("Error creating configfile %s: %v\n", cfgName, err)
+   return
+   }
+
+   fmt.Fprintf(cfg, "name = '%s'\n", w.vmname)
+   fmt.Fprintf(cfg, "kernel = 'worker-xen.img'\n")
+   fmt.Fprintf(cfg, "memory = 32\n")
+   fmt.Fprintf(cfg, "vcpus = 1\n")
+   fmt.Fprintf(cfg, "on_crash = 'destroy'\n")
+
+   
+   // xl create -p [filename]
+   {
+   args := []string{"xl", "create", "-p", cfgName}
+   if mock {
+   args = append([]string{"echo"}, args...)
+   }
+   e := exec.Command(args[0], args[1:]...)
+   
+   e.Stdout = os.Stdout
+   e.Stderr = os.Stderr
+
+   err = e.Run()
+   if err != nil {
+   fmt.Printf("Error creating domain: %v\n", err)
+   return
+   }
+   }
+
+   // Get domid
+   {
+   var domidString []byte
+   var args []string
+   
+   if mock {
+   args = []string{"echo", "232"}
+   } else {
+   args = []string{"xl", "domid", w.vmname}
+   }
+   e := exec.Command(args[0], args[1:]...)
+
+   domidString, err = e.Output()
+   if err != nil {
+   fmt.Printf("Error getting domid: %v\n", err)
+   return
+   }
+
+   _, err = fmt.Sscanf(string(domidString), "%d\n", )
+   if err != nil {
+   fmt.Printf("Error converting domid: %v\n", err)
+   return
+   }
+
+   fmt.Printf(" %s domid %d\n", w.vmname, w.domid)
+   }
+   
+   // Set xenstore config
+   {
+   rcfg := RumpRunConfig{
+   Blk:RumpRunConfigBlk{Source:"dev",
+   Path:"virtual",
+   Fstype:"kernfs",
+   Mountpoint:"/kern"},
+   Hostname:w.vmname}
+   
+   rcfg.Cmdline = "worker-xen.img"
+   for _, a := range p.Args {
+   rcfg.Cmdline += fmt.Sprintf(" %s", a)
+   }
+
+   var rcfgBytes []byte
+   
+   rcfgBytes, err = json.Marshal(rcfg)
+   

[Xen-devel] [PATCH RFC 05/59] controller: Add WorkerParams argument to Init in Worker interface

[Ronald Rojas]

From: George Dunlap 

For now just make it a plain []string; later make it more abstrcted.

Signed-off-by: George Dunlap 
---
 main.go  | 22 +-
 processworker.go |  4 ++--
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/main.go b/main.go
index a4986cf..90388dd 100644
--- a/main.go
+++ b/main.go
@@ -14,9 +14,13 @@ type WorkerReport struct {
MaxDelta int
 }
 
+type WorkerParams struct {
+   Args []string
+}
+
 type Worker interface {
SetId(int)
-   Init() error
+   Init(WorkerParams) error
Shutdown()
Process(chan WorkerReport, chan bool)
 }
@@ -49,6 +53,14 @@ func Report(ws *WorkerState, r WorkerReport) {
ws.LastReport = r
 }
 
+type WorkerList []WorkerState
+
+func (ws *WorkerList) Start(report chan WorkerReport, done chan bool) {
+   for i := range *ws {
+   go (*ws)[i].w.Process(report, done)
+   }
+}
+
 func main() {
count := 2

@@ -60,17 +72,17 @@ func main() {

i := 0
 
-   Workers := make([]WorkerState, count)
+   Workers := WorkerList(make([]WorkerState, count))

for i = 0; i< count; i++ {
Workers[i].w = {}
Workers[i].w.SetId(i)

-   Workers[i].w.Init()
-   
-   go Workers[i].w.Process(report, done)
+   Workers[i].w.Init(WorkerParams{[]string{"burnwait", "20", 
"2000"}})
}
 
+   Workers.Start(report, done)
+
for i > 0 {
select {
case r := <-report:
diff --git a/processworker.go b/processworker.go
index c1d1fd5..806e4d7 100644
--- a/processworker.go
+++ b/processworker.go
@@ -20,8 +20,8 @@ func (w *ProcessWorker) SetId(i int) {
w.id = i
 }
 
-func (w *ProcessWorker) Init() (err error) {
-   w.c = exec.Command("../worker/worker-proc", "burnwait", "20", 
"2000")
+func (w *ProcessWorker) Init(p WorkerParams) (err error) {
+   w.c = exec.Command("../worker/worker-proc", p.Args...)
 
w.stdout, err = w.c.StdoutPipe()
if err != nil {
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 04/59] Controller: Move process worker into its own file

[Ronald Rojas]

From: George Dunlap 

Also fix Makefile rule to use correct automatic variable.

Signed-off-by: George Dunlap 
---
 Makefile |  4 ++--
 main.go  | 59 --
 processworker.go | 66 
 3 files changed, 68 insertions(+), 61 deletions(-)
 create mode 100644 processworker.go

diff --git a/Makefile b/Makefile
index 736d959..00f732a 100644
--- a/Makefile
+++ b/Makefile
@@ -4,8 +4,8 @@ BINALL = $(BIN)
 .PHONY: all
 all: $(BIN)
 
-controller: main.go
-   go build -o $@ $<
+controller: main.go processworker.go
+   go build -o $@ $^
 
 .PHONY: clean
 clean:
diff --git a/main.go b/main.go
index 034e2f2..a4986cf 100644
--- a/main.go
+++ b/main.go
@@ -3,11 +3,7 @@ package main
 import (
"fmt"
"os"
-   "os/exec"
"os/signal"
-   "encoding/json"
-   "bufio"
-   "io"

 )
 
@@ -25,61 +21,6 @@ type Worker interface {
Process(chan WorkerReport, chan bool)
 }
 
-type ProcessWorker struct {
-   id int
-   c *exec.Cmd
-   stdout io.ReadCloser
-   jsonStarted bool
-}
-
-func (w *ProcessWorker) SetId(i int) {
-   w.id = i
-}
-
-func (w *ProcessWorker) Init() (err error) {
-   w.c = exec.Command("../worker/worker-proc", "burnwait", "20", 
"2000")
-
-   w.stdout, err = w.c.StdoutPipe()
-   if err != nil {
-   fmt.Print("Conneting to stdout: ", err)
-   return
-   }
-
-   return
-}
-
-func (w *ProcessWorker) Shutdown() {
-   w.c.Process.Kill()
-}
-
-func (w *ProcessWorker) Process(report chan WorkerReport, done chan bool) {
-   w.c.Start()
-
-   scanner := bufio.NewScanner(w.stdout)
-
-   for scanner.Scan() {
-   s := scanner.Text()
-   
-   //fmt.Println("Got these bytes: ", s);
-
-   if w.jsonStarted {
-   var r WorkerReport
-   json.Unmarshal([]byte(s), )
-   r.Id = w.id
-   report <- r
-   } else {
-   if s == "START JSON" {
-   //fmt.Println("Got token to start parsing json")
-   w.jsonStarted = true
-   }
-   }
-   }
-
-   done <- true
-
-   w.c.Wait()
-}
-
 const (
USEC = 1000
MSEC = USEC * 1000
diff --git a/processworker.go b/processworker.go
new file mode 100644
index 000..c1d1fd5
--- /dev/null
+++ b/processworker.go
@@ -0,0 +1,66 @@
+package main
+
+import (
+   "fmt"
+   "os/exec"
+   "encoding/json"
+   "bufio"
+   "io"
+   
+)
+
+type ProcessWorker struct {
+   id int
+   c *exec.Cmd
+   stdout io.ReadCloser
+   jsonStarted bool
+}
+
+func (w *ProcessWorker) SetId(i int) {
+   w.id = i
+}
+
+func (w *ProcessWorker) Init() (err error) {
+   w.c = exec.Command("../worker/worker-proc", "burnwait", "20", 
"2000")
+
+   w.stdout, err = w.c.StdoutPipe()
+   if err != nil {
+   fmt.Print("Conneting to stdout: ", err)
+   return
+   }
+
+   return
+}
+
+func (w *ProcessWorker) Shutdown() {
+   w.c.Process.Kill()
+}
+
+func (w *ProcessWorker) Process(report chan WorkerReport, done chan bool) {
+   w.c.Start()
+
+   scanner := bufio.NewScanner(w.stdout)
+
+   for scanner.Scan() {
+   s := scanner.Text()
+   
+   //fmt.Println("Got these bytes: ", s);
+
+   if w.jsonStarted {
+   var r WorkerReport
+   json.Unmarshal([]byte(s), )
+   r.Id = w.id
+   report <- r
+   } else {
+   if s == "START JSON" {
+   //fmt.Println("Got token to start parsing json")
+   w.jsonStarted = true
+   }
+   }
+   }
+
+   done <- true
+
+   w.c.Wait()
+}
+
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 08/59] controller: Exit after second SIGINT

[Ronald Rojas]

From: George Dunlap 

Otherwise you have to kill -9 the process.

Signed-off-by: George Dunlap 
---
 main.go | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/main.go b/main.go
index 04c8467..dbd6276 100644
--- a/main.go
+++ b/main.go
@@ -62,6 +62,8 @@ func (ws *WorkerList) Start(report chan WorkerReport, done 
chan bool) {
 }
 
 func main() {
+   killed := false
+   
count := 2

report := make(chan WorkerReport)
@@ -92,9 +94,15 @@ func main() {
i--;
fmt.Println(i, "workers left");
case <-signals:
-   fmt.Println("SIGINT receieved, shutting down workers")
-   for j := range Workers {
-   Workers[j].w.Shutdown()
+   if ! killed {
+   fmt.Println("SIGINT receieved, shutting down 
workers")
+   for j := range Workers {
+   Workers[j].w.Shutdown()
+   }
+   killed = true
+   } else {
+   fmt.Println("Second SIGINT received, exiting 
without cleaning up")
+   os.Exit(1)
}
}
}
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 02/59] controller: Revamp communication structure

[Ronald Rojas]

From: George Dunlap 

Two general-purporse channels rather than one per worker.

Set up two workers, gather and collate the information at the central
station.

Have the worker print out a report at start-of-day, so we get timing
information for the first actual report.

Catch SIGINT as a shorthand testing way of managing tear-down
gracefully.

Signed-off-by: George Dunlap 
---
 main.go | 96 -
 1 file changed, 78 insertions(+), 18 deletions(-)

diff --git a/main.go b/main.go
index 6e90754..0cb9f51 100644
--- a/main.go
+++ b/main.go
@@ -2,13 +2,18 @@ package main
 
 import (
"fmt"
+   "os"
"os/exec"
+   "os/signal"
"encoding/json"
"bufio"
"io"
+   
 )
 
 type Worker struct {
+   Id int
+   
c *exec.Cmd
 
stdout io.ReadCloser
@@ -17,14 +22,14 @@ type Worker struct {
 }
 
 type WorkerReport struct {
+   Id int
Now int
Mops int
MaxDelta int
 }
 
-func (w *Worker) Start() (err error) {
+func (w *Worker) Init() (err error) {
w.c = exec.Command("../worker/worker-proc", "burnwait", "20", 
"2000")
-   
 
w.stdout, err = w.c.StdoutPipe()
if err != nil {
@@ -32,47 +37,102 @@ func (w *Worker) Start() (err error) {
return
}
 
-   w.c.Start()
-
-   b, err := json.Marshal(WorkerReport{5,6,7})
-   fmt.Print("Example json: ", string(b))
-   
return
 }
 
-func (w *Worker) Wait() {
-   w.c.Wait()
+func (w *Worker) Shutdown() {
+   w.c.Process.Kill()
 }
 
-func (w *Worker) Process() {
+func (w *Worker) Process(report chan WorkerReport, done chan bool) {
+   w.c.Start()
+
scanner := bufio.NewScanner(w.stdout)
 
for scanner.Scan() {
s := scanner.Text()

-   fmt.Println("Got these bytes: ", s);
+   //fmt.Println("Got these bytes: ", s);
 
if w.jsonStarted {
var r WorkerReport
-   
json.Unmarshal([]byte(s), )
-   fmt.Println(r)
+   r.Id = w.Id
+   report <- r
} else {
if s == "START JSON" {
-   fmt.Println("Got token to start parsing json")
+   //fmt.Println("Got token to start parsing json")
w.jsonStarted = true
}
}
}
+
+   done <- true
+
+   w.c.Wait()
+}
+
+const (
+   USEC = 1000
+   MSEC = USEC * 1000
+   SEC = MSEC * 1000
+)
+
+type WorkerState struct {
+   Worker
+   LastReport WorkerReport
+}
+
+func Report(ws *WorkerState, r WorkerReport) {
+   //fmt.Println(r)
+
+   lr := ws.LastReport
+
+   if (lr.Now > 0) {
+   time := float64(r.Now - lr.Now) / SEC
+   mops := r.Mops - lr.Mops
+
+   tput := float64(mops) / time
+
+   fmt.Printf("%d Time: %2.3f Mops: %d Tput: %4.2f\n", r.Id, time, 
mops, tput);
+   }
+
+   ws.LastReport = r
 }
 
 func main() {
+   count := 2
+   
+   report := make(chan WorkerReport)
+   done := make(chan bool)
+   signals := make(chan os.Signal, 1)
 
-   w:=Worker{}
+   signal.Notify(signals, os.Interrupt)

-   w.Start()
+   i := 0
 
-   w.Process()
+   Workers := make([]WorkerState, count)
+   
+   for i = 0; i< count; i++ {
+   Workers[i].Id = i
+   
+   Workers[i].Init()
+   
+   go Workers[i].Process(report, done)
+   }
 
-   w.Wait()
+   for i > 0 {
+   select {
+   case r := <-report:
+   Report([r.Id], r)
+   case <-done:
+   i--;
+   fmt.Println(i, "workers left");
+   case <-signals:
+   fmt.Println("SIGINT receieved, shutting down workers")
+   for j := range Workers {
+   Workers[j].Shutdown()
+   }
+   }
+   }
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 06/59] Reorganize to enable "Dist" directory

[Ronald Rojas]

From: George Dunlap 

Make a toplevel Makefile which defaults to 'make dist'.  Add "make
dist" targets in all subdirs, which copy the distribution files into
$PWD/dist.

Modify processworker.go such that it looks for worker-proc in the same
directory.

Signed-off-by: George Dunlap 
---
 Makefile | 4 
 processworker.go | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 00f732a..332c8d9 100644
--- a/Makefile
+++ b/Makefile
@@ -10,3 +10,7 @@ controller: main.go processworker.go
 .PHONY: clean
 clean:
rm -f $(BINALL)
+
+.PHONY: dist
+dist:
+   cp $(BIN) $(DISTDIR)
diff --git a/processworker.go b/processworker.go
index 806e4d7..cca6c3b 100644
--- a/processworker.go
+++ b/processworker.go
@@ -21,7 +21,7 @@ func (w *ProcessWorker) SetId(i int) {
 }
 
 func (w *ProcessWorker) Init(p WorkerParams) (err error) {
-   w.c = exec.Command("../worker/worker-proc", p.Args...)
+   w.c = exec.Command("./worker-proc", p.Args...)
 
w.stdout, err = w.c.StdoutPipe()
if err != nil {
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 01/59] Initial controller framework

[Ronald Rojas]

From: George Dunlap 

Go code to execute a worker and parse its output.

The go json marshaller needs capitalized structure elements; make the
output match.

Also add fflush()-es to make sure that output actually gets to the controller
in a timely manner.

Signed-off-by: George Dunlap 
---
 Makefile | 12 ++
 main.go  | 78 
 2 files changed, 90 insertions(+)
 create mode 100644 Makefile
 create mode 100644 main.go

diff --git a/Makefile b/Makefile
new file mode 100644
index 000..736d959
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,12 @@
+BIN = controller
+BINALL = $(BIN)
+
+.PHONY: all
+all: $(BIN)
+
+controller: main.go
+   go build -o $@ $<
+
+.PHONY: clean
+clean:
+   rm -f $(BINALL)
diff --git a/main.go b/main.go
new file mode 100644
index 000..6e90754
--- /dev/null
+++ b/main.go
@@ -0,0 +1,78 @@
+package main
+
+import (
+   "fmt"
+   "os/exec"
+   "encoding/json"
+   "bufio"
+   "io"
+)
+
+type Worker struct {
+   c *exec.Cmd
+
+   stdout io.ReadCloser
+   
+   jsonStarted bool
+}
+
+type WorkerReport struct {
+   Now int
+   Mops int
+   MaxDelta int
+}
+
+func (w *Worker) Start() (err error) {
+   w.c = exec.Command("../worker/worker-proc", "burnwait", "20", 
"2000")
+   
+
+   w.stdout, err = w.c.StdoutPipe()
+   if err != nil {
+   fmt.Print("Conneting to stdout: ", err)
+   return
+   }
+
+   w.c.Start()
+
+   b, err := json.Marshal(WorkerReport{5,6,7})
+   fmt.Print("Example json: ", string(b))
+   
+   return
+}
+
+func (w *Worker) Wait() {
+   w.c.Wait()
+}
+
+func (w *Worker) Process() {
+   scanner := bufio.NewScanner(w.stdout)
+
+   for scanner.Scan() {
+   s := scanner.Text()
+   
+   fmt.Println("Got these bytes: ", s);
+
+   if w.jsonStarted {
+   var r WorkerReport
+   
+   json.Unmarshal([]byte(s), )
+   fmt.Println(r)
+   } else {
+   if s == "START JSON" {
+   fmt.Println("Got token to start parsing json")
+   w.jsonStarted = true
+   }
+   }
+   }
+}
+
+func main() {
+
+   w:=Worker{}
+   
+   w.Start()
+
+   w.Process()
+
+   w.Wait()
+}
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] (no subject)

[Ronald Rojas]

The first 57 commits are merged from previous work done by 
George Dunlap at (https://github.com/gwd/schedbench) and 
implement manipulating Cpu pool. The last 2 commits merge 
his work onto the Xen tree and implement finding system 
information and throwing errors.

CC: xen-devel 
CC: Ian Jackson 
CC: Wei Liu 
CC: George Dunlap 
CC: George Dunlap 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-4.1 test] 103988: regressions - FAIL

[osstest service owner]

flight 103988 linux-4.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103988/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt 11 guest-start  fail REGR. vs. 101737

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 12 guest-saverestore fail in 103978 
pass in 103988
 test-armhf-armhf-xl-multivcpu 11 guest-start   fail pass in 103978

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-multivcpu 15 guest-start/debian.repeat fail in 103978 like 
101737
 test-armhf-armhf-xl-credit2  15 guest-start/debian.repeatfail  like 101687
 test-armhf-armhf-xl-cubietruck 15 guest-start/debian.repeat   fail like 101715
 test-armhf-armhf-xl-xsm  15 guest-start/debian.repeatfail  like 101715
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeatfail  like 101715
 test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail  like 101737
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101737
 test-armhf-armhf-xl  15 guest-start/debian.repeatfail  like 101737
 test-armhf-armhf-libvirt-xsm 15 guest-start/debian.repeatfail  like 101737
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101737
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101737
 test-armhf-armhf-xl-vhd   9 debian-di-installfail  like 101737
 test-armhf-armhf-libvirt-qcow2  9 debian-di-install   fail like 101737
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101737

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail in 103978 never 
pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-check fail in 103978 
never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass

version targeted for testing:
 linux35327468a79dd9e343eaf7e66cc372f8277b2a84
baseline version:
 linux9ca365c0c8bdd8552ec064f0e696600cf7ea66dd

Last test of basis   101737  2016-10-28 04:21:54 Z   61 days
Failing since102733  2016-11-30 09:50:09 Z   28 days   30 attempts
Testing same since   103978  2016-12-27 22:45:26 Z0 days2 attempts


People who touched revisions under test:
  "Eric W. Biederman" 
  Adrian Hunter 
  Alan Stern 
  Alex Deucher 
  Alexander Polakov 
  Alexander Usyskin 
  Andreas Dilger 
  Andreas Gruenbacher 
  Andreas Gruenbacher 
  Andrew Morton 
  Andy Lutomirski 
  Arve Hjønnevåg 
  Boris Brezillon 
  Brian Norris 
  Bryan Paluch 

Re: [Xen-devel] Xen: ARM: Support for mapping ECAM PCIe Config Space Specified In Static ACPI Table

[Julien Grall]

(CC Andrew and Jan)

Hi Stefano,

On 20/12/16 22:33, Stefano Stabellini wrote:

On Tue, 20 Dec 2016, Julien Grall wrote:

Hi Stefano,

On 20/12/2016 00:54, Stefano Stabellini wrote:

On Mon, 19 Dec 2016, Julien Grall wrote:

On 16/12/2016 15:49, Julien Grall wrote:

On 14/12/16 08:00, Jiandi An wrote:

Xen currently doesn't map ECAM space specified in static ACPI table.
Seeking opinion on how this should be handled properly.
Each root complex ECAM region takes up 64K 4K pages (256MB).
For some platforms there might be multiple root complexes.
Is the plan to map all at once?Julien has mentioned support
for mapping ECAM may come when support for PCI passthrough is
added, is that right? What mechanism will it be? Will Xen or
dom0 be the one that parses the staic ACPI tables and map the ECAM
space?


For performance reason, each ECAM region would need to be mapped at
once, so the stage-2 page table could take advantage of superpage (it
will mostly be 2MB).

Now, I don't think Xen should map the ECAM region in stage-2 before
hand. All the regions may not be described in the MCFG and I would like
to see a generic solution.

Looking at the code (see pci_create_ecam_create in drivers/pci/ecam.c),
ioremap is used. I believe the problem is the same for the 2 other
threads you sent ( [1] and [2]).

So it might be good to look at hooking up a call to
XENMEM_add_to_physmap_range in ioremap.

Any opinions?


I thought a bit more about it and I realized we need to be cautious on how
to
proceed here.

DOM0 will have a mix of real devices and emulated devices (e.g some part
of
the GIC). For the emulated devices, DOM0 should not call
XENMEM_add_to_physmap_range. However, DOM0 is not aware what is emulated
or
not, so even the current approach (hooking up in platform device) seems
fragile. We rely on Xen to say "this region cannot be mapped".


You are right that Dom0 doesn't and shouldn't be able to tell the
difference between emulated and real devices. But I don't think that
should be a problem because Xen knows exactly if an MMIO region belongs
to an emulated device thanks to the 1:1 mapping. When
XENMEM_add_to_physmap_range is called, Xen can check whether the region
belongs to an emulated device or a real device, mapping memory only if
it belongs to a real device. No need to report errors: from Dom0 point
of view the operation succeeded either way.


By no need to report errors, did you mean Xen failing, or DOM0 failing?


Sorry I haven't been clear. I meant that if Dom0 asks Xen to map a
region which belongs to an emulated device, of course Xen won't actually
do any mappings, but it is not an error condition. Xen shouldn't return
error for mappings that hasn't performed because the region belongs to
an emulated device.


I disagree here. You make the assumption that DOM0 will always issue the 
hypercall with MFN == GFN.


Today we only check whether the region is allowed in iomem. If it is not 
allowed we will ignore the request and report as "succeeded". But it 
does not mean there will be an emulation behind nor DOM0 decided to map 
the region with MFN != GFN.


So DOM0 expects the region to be mapped, but actually it may not be done 
by the hypervisor. It will be a pain to debug it because the error may 
come up much later.


The description of the hypercall is "the region is mapped in Stage-2 
using the memory attribute Device-nGnRE". Nowhere it is stated the 
region will not be mapped if emulated nor must be called MFN == GFN.


Now, we have two hypercalls XEN_DOMCTL_memory_mapping and 
XENMEM_add_to_physmap_range having two distinct behavior when mapping an 
MMIO into a guest.


We should at least return an error, even if DOM0 decides to ignore it.

I am open to any other suggestion. But I don't think the hypercall 
should silently ignore a request as it is done today.




Of course, Xen should report errors for any genuine error conditions.



I looked at the code (map_dev_mmio_region), we check whether DOM0 is allowed
to access the iomem. If a part of the region is not accessible, it will map
nothing and return as it has succeeded.

This behavior is fairly odd because it means that a domain will never know
whether a region has been mapped correctly. It may fail later on because, for
instance, the domain was trying to map the device with MFN != GFN. Thankfully
we only map one page at the time (see the caller xenmem_add_to_physmap_one)
but still a domain will expect to know what's going on.

So I think we need to associate a specific errno to tell the domain this
region is not accessible.




Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-3.18 test] 103983: tolerable FAIL - PUSHED

[osstest service owner]

flight 103983 linux-3.18 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103983/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101675
 test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail  like 101675
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101675
 test-armhf-armhf-libvirt-raw 12 saverestore-support-checkfail  like 101675
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101675
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 101675
 test-armhf-armhf-libvirt 13 saverestore-support-checkfail  like 101675
 test-armhf-armhf-libvirt-qcow2 12 saverestore-support-check   fail like 101675
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101675

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 linux1e20e732ffa107e21f2089ce56e32d99b9229944
baseline version:
 linuxa6846cfd266b48af1ee7c3c19d5cb60477ca4469

Last test of basis   101675  2016-10-26 03:40:44 Z   63 days
Failing since102732  2016-11-30 09:50:09 Z   28 days   30 attempts
Testing same since   103983  2016-12-28 03:30:38 Z0 days1 attempts


People who touched revisions under test:
  Adrian Hunter 
  Alan Stern 
  Alex Deucher 
  Alexander Usyskin 
  Andreas Gruenbacher 
  Andreas Gruenbacher 
  Andrew Morton 
  Andy Lutomirski 
  Boris Brezillon 
  Brian Norris 
  Bryan Paluch 
  Ching Huang 
  Chris Mason 
  Dan Carpenter 
  Darrick J. Wong 
  Dave Chinner 
  David Howells 
  David S. Miller 
  Dinesh Israni 
  Dmitry Torokhov 
  Dmitry Vyukov 
  Eric Dumazet 
  Ewan D. Milne 
  Felipe Balbi 
  Frank Haverkamp 
  Frederic Barrat 
  Gerald Schaefer 

Re: [Xen-devel] Xen ARM - Exposing a PL011 to the guest

[Julien Grall]

On 28/12/16 17:49, Julien Grall wrote:

On 21/12/16 22:12, Stefano Stabellini wrote:

On Wed, 21 Dec 2016, Julien Grall wrote:

On 20/12/2016 20:53, Stefano Stabellini wrote:

On Tue, 20 Dec 2016, Julien Grall wrote:

On 19/12/2016 21:24, Stefano Stabellini wrote:

On Mon, 19 Dec 2016, Christoffer Dall wrote:

On Fri, Dec 16, 2016 at 05:03:13PM +, Julien Grall wrote:

If we use hvm_params for this, we need two new hvm_params and Xen
needs
to unmap the pfn from the guest immediately, because we don't want
the
guest to have access to it.


If you unmap the pfn, the PV backend will not be able to request
the page
because there will be no translation available.

So what you want to do is preventing the guest to at least write into
region
(not sure if it is worth to restrict read)


That's a good idea.



and unmap the page via the hypercall XENMEM_decrease_reservation.


That would be issued by the guest itself, right? To save address space?


Correct. The main use case today is ballooning, but guest could call
it on any
other RAM baked page.

I was thinking about more about the protection needed. Technically
the data in
the ring are not trusted. So if the guest is messing up with it, it
would not
be a big issue. Or did I miss anything here?


I understand that a guest would be smart to call
XENMEM_decrease_reservation on the PV console page for pl011, but it
cannot be a security measure, because, in fact, it needs to be called by
the guest.  Of course, a malicious guest can simply not call
XENMEM_decrease_reservation for it.


Sorry I was not clear. I was not suggested the guest to call
XENMEM_decrease_reservation on ring for security but a malicious guest
issuing the hypercall on the ring protected and replacing by another page.

This is the exact same problem as the one I mentioned on the ITS thread.
The page live in guest memory but contains data that will only be
touched by Xen.

If you remove those page from stage-2, the translation IPA -> MFN will
be lost unless you store somewhere else. You would have to do it
per-page as the buffer will use contiguous IPA but potentially
noncontiguous MFN.

In the case of ITS the memory is provisioned by the guest. So there are
not much to do there except adding protection in stage-2 such as write
protection and preventing the guest to unmap it. However for the pl011
ring, as Andrew pointed on IRC, what we need to do is accounting this
page to the domain memory. No mapping is necessary in stage-2.


I forgot to mention. The that backend of PV console is already 
considering the data in the ring untrusted. So I think in this case, 
protection is not strictly necessary. The worst thing that could happen 
is the guest is corrupting one of its console. Not really a big deal.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen ARM - Exposing a PL011 to the guest

[Julien Grall]

Hi Stefano,

On 21/12/16 22:12, Stefano Stabellini wrote:

On Wed, 21 Dec 2016, Julien Grall wrote:

On 20/12/2016 20:53, Stefano Stabellini wrote:

On Tue, 20 Dec 2016, Julien Grall wrote:

On 19/12/2016 21:24, Stefano Stabellini wrote:

On Mon, 19 Dec 2016, Christoffer Dall wrote:

On Fri, Dec 16, 2016 at 05:03:13PM +, Julien Grall wrote:

If we use hvm_params for this, we need two new hvm_params and Xen needs
to unmap the pfn from the guest immediately, because we don't want the
guest to have access to it.


If you unmap the pfn, the PV backend will not be able to request the page
because there will be no translation available.

So what you want to do is preventing the guest to at least write into
region
(not sure if it is worth to restrict read)


That's a good idea.



and unmap the page via the hypercall XENMEM_decrease_reservation.


That would be issued by the guest itself, right? To save address space?


Correct. The main use case today is ballooning, but guest could call it on any
other RAM baked page.

I was thinking about more about the protection needed. Technically the data in
the ring are not trusted. So if the guest is messing up with it, it would not
be a big issue. Or did I miss anything here?


I understand that a guest would be smart to call
XENMEM_decrease_reservation on the PV console page for pl011, but it
cannot be a security measure, because, in fact, it needs to be called by
the guest.  Of course, a malicious guest can simply not call
XENMEM_decrease_reservation for it.


Sorry I was not clear. I was not suggested the guest to call 
XENMEM_decrease_reservation on ring for security but a malicious guest 
issuing the hypercall on the ring protected and replacing by another page.


This is the exact same problem as the one I mentioned on the ITS thread. 
The page live in guest memory but contains data that will only be 
touched by Xen.


If you remove those page from stage-2, the translation IPA -> MFN will 
be lost unless you store somewhere else. You would have to do it 
per-page as the buffer will use contiguous IPA but potentially 
noncontiguous MFN.


In the case of ITS the memory is provisioned by the guest. So there are 
not much to do there except adding protection in stage-2 such as write 
protection and preventing the guest to unmap it. However for the pl011 
ring, as Andrew pointed on IRC, what we need to do is accounting this 
page to the domain memory. No mapping is necessary in stage-2.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 1/4] xen/arm: fix GIC_INVALID_LR

[Julien Grall]

Hi Stefano,

On 22/12/16 02:15, Stefano Stabellini wrote:

GIC_INVALID_LR should be 0xff, but actually, defined as ~(uint8_t)0, is
0x. Fix the problem by placing the ~ operator before the cast.

Signed-off-by: Stefano Stabellini 


Reviewed-by: Julien Grall 

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/4] The locking order is: first rank lock, then vgic lock. The order is respected everywhere, except for gic_update_one_lr.

[Julien Grall]

Hi Stefano,

On 22/12/16 02:15, Stefano Stabellini wrote:

gic_update_one_lr is called with the vgic lock held, but it calls
vgic_get_target_vcpu, which tries to obtain the rank lock. This can
cause deadlocks.

We already have a version of vgic_get_target_vcpu that doesn't take the
rank lock: __vgic_get_target_vcpu.

Solve the lock inversion problem, by not taking the rank lock in
gic_update_one_lr (calling __vgic_get_target_vcpu instead of
vgic_get_target_vcpu).  This is safe, because vcpu target modifications
are protected by the same vgic vcpu lock.


I maintain what I said on the first version of this patch. All this 
patch could be simplified by moving the call to irq_set_affinity in 
vgic_irq_migrate.


There are no restriction to do that and no need to have any lock taken 
but the rank lock.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf baseline-only test] 68283: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 68283 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/68283/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf af8ba51aca4e0b41a359fe467fb5c5b9baa75a05
baseline version:
 ovmf 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4

Last test of basis68282  2016-12-28 11:17:00 Z0 days
Testing same since68283  2016-12-28 14:48:10 Z0 days1 attempts


People who touched revisions under test:
  Jeff Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit af8ba51aca4e0b41a359fe467fb5c5b9baa75a05
Author: Jeff Fan 
Date:   Mon Dec 26 19:16:23 2016 +0800

UefiCpuPkg/MpInitLib: Set new AP to idle state after switching BSP

We need to set new AP to idle state if it is expected to be waken up by new 
BSP
later.

Cc: Feng Tian 
Cc: Kinney Michael D 
Cc: Ruiyu Ni 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit b6e45716c3ebccba9c576b1c39502dfa8e62c9df
Author: Jeff Fan 
Date:   Mon Dec 26 19:13:44 2016 +0800

UefiCpuPkg/MpInitLib: Disable and restore system timer interrupt

We need to disable system timer interrup to avoid generating the pending
interrupt on the old BSP.

Cc: Feng Tian 
Cc: Kinney Michael D 
Cc: Ruiyu Ni 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit 26b43433c1367f893ce3ae962761141a93b1d25a
Author: Jeff Fan 
Date:   Mon Dec 26 16:55:12 2016 +0800

UefiCpuPkg/MpInitLib: Swap local APIC timer interrupt state

We need to swap local APIC timer interrupt state between old BSP and new 
BSP.

Cc: Feng Tian 
Cc: Kinney Michael D 
Cc: Ruiyu Ni 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit a8d75a1802df015a79d929f45972de98b9a0a2d9
Author: Jeff Fan 
Date:   Mon Dec 26 16:52:48 2016 +0800

UefiCpuPkg/MpInitLib: Move save/restore interrupt to SwitchBSPWorker()

During switching BSP phase, we need to disable CPU interruput to prevent 
stack
crashed by Timer interrupt handle. But when we enabled source debugging 
feature,
debug timer interrupt handler (existing on both PEI and DXE) also could 
crash
the stack used during switching BSP. So,we need to move save/restore 
interrupt
to SwitchBSPWorker().

Cc: Feng Tian 
Cc: Kinney Michael D 
Cc: Ruiyu Ni 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit ffab244280fd934c9a02142765e40f70fbc061a1
Author: Jeff Fan 
Date:   Mon Dec 26 16:44:24 2016 +0800

UefiCpuPkg/MpInitLib: Sync BSP's local APIC timer settings to APs

If APs are waken up by INIT-SIPI-SIPI command, they will lose original local
APIC timer setting. As a result, the timer library instance based on local 
APIC
timer cannot work on APs function.

This fix is to save BSP's local APIC timer settings before waking up APs and
to sync to APs when APs wakeup by INIT-SIPI-SIPI command.

 

Re: [Xen-devel] [PATCH v2 3/4] arm, vgic_migrate_irq: take the right vgic lock

[Julien Grall]

Hi Stefano,

On 22/12/16 02:15, Stefano Stabellini wrote:

Always take the vgic lock of the old vcpu. When more than one irq
migration is requested before the first one completes, take the vgic
lock of the oldest vcpu.

Write the new vcpu id into the rank from vgic_migrate_irq, protected by
the oldest vgic vcpu lock.

Use barriers to ensure proper ordering between clearing inflight and
MIGRATING and setting vcpu to GIC_INVALID_VCPU.


The field p->status was designed to be accessed without any lock using 
*_bit helpers.


Currently vgic_migrate_irq is protected by the rank lock (an ASSERT 
would probably useful for documentation) and can only be called once at 
the time.


Let's take the following example to describe the problem:
1) IRQ has been injected into vCPU A (e.g present in the LR)
2) IRQ is migrated to vCPU B
3) IRQ is migrated to vCPU C
4) IRQ is EOIed by the guest

When vgic_irq_migrate_irq is called for the first time (step 2), the 
vCPU A vgic lock will be taken and GIC_IRQ_GUEST_MIGRATED will be set at 
the end. The second time (step 3), GIC_IRQ_GUEST_MIGRATING is already 
set, so the function will return directly no lock needed.


So, I think the vgic lock taken is already correct.

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: fix rank/vgic locks inversion bug

[Julien Grall]

Hi Stefano,

On 20/12/16 19:38, Stefano Stabellini wrote:

On Tue, 20 Dec 2016, Julien Grall wrote:

On 20/12/2016 00:22, Stefano Stabellini wrote:

On Mon, 19 Dec 2016, Julien Grall wrote:

On 19/12/2016 23:30, Stefano Stabellini wrote:

On Mon, 19 Dec 2016, Julien Grall wrote:

2) We run gic_update_one_lr and vgic_store_itargetsr in parallel
safely
and locklessly. There might be a way to do it, but it is not easy
I
haven't found it yet.


Correct me if I am wrong. There are no restriction to write into
IROUTER/ITARGETSR while an IRQ is pending. So the irq_set_affinity
could
be
called once at the beginning of vgic_irq_migrate.

We may receive the interrupt on the wrong physical CPU at the
beginning.
But
it would be the same when writing into IROUTER/ITARGETSR.

This would remove the need to get the rank lock in gic_update_one_lr.

Did I miss anything?


I am not sure if we can do that: the virtual interrupt might not be
EOI'd yet at that time. The guest EOI is used to deactivate the
corresponding physical interrupt. Migrating the physical IRQ at that
time, could have unintended consequences? I am not sure what the spec
says about this, but even if it is correct on paper, I would prefer not
to put it to the test: I bet that not many interrupt controllers have
been heavily tested in this scenario. What do you think?


I don't think this is an issue. An interrupt could have the priority drop
happening on pCPU A and be deactivated on pCPU B if the vCPU A is been
migrated when the interrupt is inflight. So if it is fine here, why would
not
it be when the guest is specifically requesting the routing?


That is true, but this is not exactly the same.


My example was to show you that an IRQ can have its priority dropped in pCPU A
and been deactivated to pCPU B. Another example is when only the IRQ is been
migrated. The spec does not promise you to receive the next interrupt on the
CPU you asked because it may take time to update the GIC state. So the
priority drop and deactivation could be done on separate physical CPU here
too.


This is changing the
physical irq affinity while both physical and virtual irqs are still
active.


Physical IRQ state and virtual IRQ state are completely dissociated in the
GIC. The only interaction possible is the virtual interface to send a
deactivate request to the distributor when the virtual interrupt has been
deactivated and correspond to a hardware interrupt.


As I wrote, usually operating systems only change affinity after
deactivating an irq, so I thought it would be wise in Xen to wait at
least for the EOI.


I looked at the Linux code and did not see a such requirement when setting the
affinity (see irq_set_affinity) of an IRQ.


If we tried to inject the same virtual interrupt on a
different vcpu, while the interrupt is still inflight, we could get in
troubles. But we could avoid that by testing for GIC_IRQ_GUEST_MIGRATING
in vgic_vcpu_inject_irq. Maybe I am worrying about nothing.


The only interrupt that can be routed to a guest in Xen are SPI which are
shared between all CPUs. The active bit is handled by the distributor. It is
not possible to receive the same SPI until it has been deactivated.


True, but keep in mind that we don't get any interruptions when the vcpu
issues an EOI. We lazily clean the data structures the first time we get
back to Xen. So there is a window, where the interrupt has already been
EOI'ed on the first vcpu, but struct pending_irq still shows the
interrupt as inflight and would mess up today's checks in
vgic_vcpu_inject_irq on other vcpus.


I am aware that LRs are cleaned lazily. However, you continue to assume 
that the GIC will fire the next IRQ on the right pCPU as soon as 
ITARGETSR and IROUTER will be written.


In case of GICv2, gicv2_irq_set_affinity there is no system barrier (dsb 
sy). So nothing ensure that the GIC has received the write when the 
function returns because it takes time to propagate the information.


So no matter the lock you are trying to take. You will have window where 
the IRQ is received on the wrong physical vCPU.


That's why the field status in pending_irq is accessible with any lock.
gic_update_one_lr is working only on p->status so it is fine here.

The only place where it might be an issue is list_empty(>inflight) in 
vgic_vcpu_inject_irq. But, I think it is perfectly fine because if you 
the IRQ has not been received on the correct vCPU then, it would be 
taken care later. For instance when the vCPU has been kicked (see at the 
end of vgic_vcpu_inject_irq).



Also they wouldn't be protected by
the right vgic lock either. Maybe this is the real reason why I didn't
go for this route originally. Sorry I didn't bring this up earlier, the
irq migration stuff is extremely difficult to get right.


I don't think it is difficult to get the core right. The problem is 
there are no documentation of this code, so we have to try to figure out 
why it was written like that before.


All this code was meant to 

[Xen-devel] Processed: Re: [xl restore and migrating problem - hardware compatibility]

[xen]

Processing commands for x...@bugs.xenproject.org:

> create ^
Created new bug #56 rooted at 
`'
Title: `Re: [Xen-devel] [xl restore and migrating problem - hardware 
compatibility]'
> severity it blocker
Change severity for #56 to `blocker'
> affects it 4.6, 4.7, 4.8
Bug #56 affects `4.6, 4.7, 4.8'
Command failed: Unable to find tag affects:4.6, 4.7, 4.8 at 
/srv/xen-devel-bugs/share/perl/5.14.2/Emesinae/Bug.pm line 127,  line 3.
Stop processing here.

Modified/created Bugs:
 - 56: http://bugs.xenproject.org/xen/bug/56 (new)

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on 
reporting bugs
Contact xen-bugs-ow...@bugs.xenproject.org with any infrastructure issues

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xl restore and migrating problem - hardware compatibility]

[Julien Grall]

create ^
severity it blocker
affects it 4.6, 4.7, 4.8

thanks

On 23/12/16 17:00, Andrew Cooper wrote:

On 23/12/16 16:32, Ing. Ricardo Brisighelli wrote:

Hi, i'm gentoo user and try with xen 4.6.3 and 4.7.1 in both version
have the
same problem.

My cpu is AMD A10-7860K


This issue has been reported before ("[Xen-devel] "X86_PV_VCPU_MSRS
record truncated" during domain restore").

I submitted patches to fix it ("Fix issues with zero-length records in
migration v2" in July even), which made no progress.  As the author of
migration v2, and unfortunately of this bug, I stand by v1 of my fix
without any further modification.

You can find the patches here:
https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/tools-fix-zero-length-records


Julien: As 4.9 RM, please mark this as a release blocker.  It is very
poor that we as a community had this reported and fixed 6 months ago,
and yet it still didn't make it into 4.8

~Andrew


--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xl restore and migrating problem - hardware compatibility]

[Julien Grall]

(Adding Ian and Wei)

Hi Andrew,

On 23/12/16 17:00, Andrew Cooper wrote:

On 23/12/16 16:32, Ing. Ricardo Brisighelli wrote:

Hi, i'm gentoo user and try with xen 4.6.3 and 4.7.1 in both version
have the
same problem.

My cpu is AMD A10-7860K


This issue has been reported before ("[Xen-devel] "X86_PV_VCPU_MSRS
record truncated" during domain restore").

I submitted patches to fix it ("Fix issues with zero-length records in
migration v2" in July even), which made no progress.  As the author of
migration v2, and unfortunately of this bug, I stand by v1 of my fix
without any further modification.

You can find the patches here:
https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/tools-fix-zero-length-records


Julien: As 4.9 RM, please mark this as a release blocker.  It is very
poor that we as a community had this reported and fixed 6 months ago,
and yet it still didn't make it into 4.8


Sounds sensible. I will create an entry in the bug tracker.

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 103986: all pass - PUSHED

[osstest service owner]

flight 103986 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103986/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf af8ba51aca4e0b41a359fe467fb5c5b9baa75a05
baseline version:
 ovmf 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4

Last test of basis   103984  2016-12-28 07:15:07 Z0 days
Testing same since   103986  2016-12-28 11:15:08 Z0 days1 attempts


People who touched revisions under test:
  Jeff Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=af8ba51aca4e0b41a359fe467fb5c5b9baa75a05
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
af8ba51aca4e0b41a359fe467fb5c5b9baa75a05
+ branch=ovmf
+ revision=af8ba51aca4e0b41a359fe467fb5c5b9baa75a05
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.8-testing
+ '[' xaf8ba51aca4e0b41a359fe467fb5c5b9baa75a05 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : 

[Xen-devel] [ovmf baseline-only test] 68282: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 68282 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/68282/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4
baseline version:
 ovmf 1f20b2988cb223907e64a4736a3eb6030aa715b8

Last test of basis68276  2016-12-26 19:16:00 Z1 days
Testing same since68282  2016-12-28 11:17:00 Z0 days1 attempts


People who touched revisions under test:
  Jiewen Yao 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4
Author: Jiewen Yao 
Date:   Tue Dec 27 14:21:16 2016 +0800

UefiCpuPkg/MicrocodeUpdate: Add MP support.

Support the case that BSP and AP are using different Microcode.
The previous logic validates new MCU on BSP only.
The enhanced logic will validate MCU on every BSP and AP.
As long as one processor loads the MCU successfully, it will be updated.

Cc: Jeff Fan 
Cc: Star Zeng 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao 
Reviewed-by: Jeff Fan 

commit 2ed658240c1f3ca10aa65dbbf74f4cf297145d8c
Author: Jiewen Yao 
Date:   Mon Dec 26 23:35:41 2016 +0800

UefiCpuPkg/MicrocodeUpdate: enhance flash write logic

The patch updated MicrocodeWrite() to move the Microcode replacement logic
to a standalone function -  UpdateMicrocodeFlashRegion().
More detail description is added in UpdateMicrocodeFlashRegion()
to improve readability.

The Microcode information is collected in InitializeMicrocodeDescriptor(),
so that FmpGetImage() can get the info directly.
MicrocodeRead() is not needed any more.

Cc: Jeff Fan 
Cc: Star Zeng 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao 
Reviewed-by: Jeff Fan 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3] xen: sched: removal of redundant check in Credit

[Dario Faggioli]

On Wed, 2016-12-28 at 09:02 +, George Dunlap wrote:
> On 17/12/16 01:46, Dario Faggioli wrote:
> > On Sat, 2016-12-17 at 00:53 +0530, Praveen Kumar wrote:
> > > Signed-off-by: Praveen Kumar 
> > > Acked-by: Dario Faggioli 
> > > 
> > Better than before. But still, if I:
> >  - save this mail as mbox
> >  - try to import it in git
> > it fails.
> > 
> > OTOH, if I:
> >  - save this mail as mbox
> >  - run dos2unix on the mbox file
> >  - try to import it in git
> > it works!
> 
> FWIW the script I use runs dos2unix on everything I get via e-mail
> (either saved as an mbox or as an attachment) as a matter of course.
> Are you actually sometimes able to git am mbox'd files *without*
> running
> dos2unix?
> 
Not really, but main reason is I don't `git am' at all! I use `stg
import', but not very often, and I also end up pre-filtering with
dos2unix.

I mentioned it to Praveen as a potential issue, out of _my_ ignorance
of your workflow as committers. It turns out it's normal.. So, thanks
for letting me know, and sorry for the noise. :-)

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/emulate: Don't assume that addr_size == 32 implies protected mode

[Jan Beulich]

>>> George Dunlap  12/28/16 2:34 PM >>>
>On 16/12/16 10:34, Jan Beulich wrote:
> On 16.12.16 at 10:55,  wrote:
>>> Callers of x86_emulate() generally define addr_size based on the code
>>> segment.  In vm86 mode, the code segment is set by the hardware to be
>>> 16-bits; but it is entirely possible to enable protected mode, set the
>>> CS to 32-bits, and then disable protected mode.  (This is commonly
>>> called "unreal mode".)
>> 
>> To better match this description I think it would be preferable ...
>> 
>>> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
>>> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
>>> @@ -2149,11 +2149,8 @@ x86_decode(
>>>  default:
>>>  BUG(); /* Shouldn't be possible. */
>>>  case 2:
>>> -if ( in_realmode(ctxt, ops) || (state->regs->eflags & 
>>> EFLG_VM) )
>>> -break;
>>> -/* fall through */
>>>  case 4:
>>> -if ( modrm_mod != 3 )
>>> +if ( modrm_mod != 3 || !in_protmode(ctxt, ops) )
>>>  break;
>> 
>> ... to keep the EFLAGS.VM in case 2, and check in_realmode()
>> in case 4. Otoh what you have now is the more compact form,
>> resulting in fewer branches ...
>
>You're not giving me a very clear picture of what you'd like me to do
>here. :-)  Did you mean "even though" instead of "OTOH"?  ("On the other
>hand" usually indicates a change of mind.)

My first thought was to ask you to make that code change. But then I
realized that the code as is has its own advantage, in which case it
would seem desirable to make the description better match that change
(as said, it looks to me as if it rather describes your change including
the suggested adjustment, since as it is still shown above it's basically
also allowing 32-bit VM86 mode).

Bottom line: I'd be fine either way, as long as description and code
change match up.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 02/10] x86/guest-walk: use unambiguous register names

[Jan Beulich]

>>> George Dunlap  12/28/16 12:18 PM >>>
>On Tue, Dec 20, 2016 at 10:36 AM, Jan Beulich  wrote:
>> This is in preparation of eliminating the mis-naming of 64-bit fields
>> with 32-bit register names (eflags instead of rflags etc). Use the
>> guaranteed 32-bit underscore prefixed names for now where appropriate.
>>
>> Signed-off-by: Jan Beulich 
>
>Any particular reason to use _eflags rather than just using rflags here?

32-bit accesses are shorter instruction byte wise on average (not requiring 
REX.W).
And the ultimate goal is to switch to eflags (without the underscore here), 
just that
this can't be easily done in a single step (hence the "part I" in the title of 
the
overview mail). And using _eflags now allows us to easily find such uses (once
that field name goes away), whereas rflags is to stay (and hence would need to 
be
grep-ed for instead of the compiler pointing out any leftover uses).

>Either way:
>
>Acked-by: George Dunlap 

Thanks.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/emulate: Don't assume that addr_size == 32 implies protected mode

[George Dunlap]

On 16/12/16 10:34, Jan Beulich wrote:
 On 16.12.16 at 10:55,  wrote:
>> Callers of x86_emulate() generally define addr_size based on the code
>> segment.  In vm86 mode, the code segment is set by the hardware to be
>> 16-bits; but it is entirely possible to enable protected mode, set the
>> CS to 32-bits, and then disable protected mode.  (This is commonly
>> called "unreal mode".)
> 
> To better match this description I think it would be preferable ...
> 
>> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
>> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
>> @@ -2149,11 +2149,8 @@ x86_decode(
>>  default:
>>  BUG(); /* Shouldn't be possible. */
>>  case 2:
>> -if ( in_realmode(ctxt, ops) || (state->regs->eflags & 
>> EFLG_VM) )
>> -break;
>> -/* fall through */
>>  case 4:
>> -if ( modrm_mod != 3 )
>> +if ( modrm_mod != 3 || !in_protmode(ctxt, ops) )
>>  break;
> 
> ... to keep the EFLAGS.VM in case 2, and check in_realmode()
> in case 4. Otoh what you have now is the more compact form,
> resulting in fewer branches ...

You're not giving me a very clear picture of what you'd like me to do
here. :-)  Did you mean "even though" instead of "OTOH"?  ("On the other
hand" usually indicates a change of mind.)

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-4.1 test] 103978: regressions - FAIL

[osstest service owner]

flight 103978 linux-4.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103978/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt 11 guest-start  fail REGR. vs. 101737
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 12 guest-saverestore fail REGR. vs. 
101737

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-credit2  15 guest-start/debian.repeatfail  like 101687
 test-armhf-armhf-xl-cubietruck 15 guest-start/debian.repeat   fail like 101715
 test-armhf-armhf-xl-xsm  15 guest-start/debian.repeatfail  like 101715
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeatfail  like 101715
 test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail  like 101737
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 101737
 test-armhf-armhf-xl  15 guest-start/debian.repeatfail  like 101737
 test-armhf-armhf-xl-multivcpu 15 guest-start/debian.repeatfail like 101737
 test-armhf-armhf-libvirt-xsm 15 guest-start/debian.repeatfail  like 101737
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 101737
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101737
 test-armhf-armhf-xl-vhd   9 debian-di-installfail  like 101737
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101737
 test-armhf-armhf-libvirt-qcow2  9 debian-di-install   fail like 101737

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass

version targeted for testing:
 linux35327468a79dd9e343eaf7e66cc372f8277b2a84
baseline version:
 linux9ca365c0c8bdd8552ec064f0e696600cf7ea66dd

Last test of basis   101737  2016-10-28 04:21:54 Z   61 days
Failing since102733  2016-11-30 09:50:09 Z   28 days   29 attempts
Testing same since   103978  2016-12-27 22:45:26 Z0 days1 attempts


People who touched revisions under test:
  "Eric W. Biederman" 
  Adrian Hunter 
  Alan Stern 
  Alex Deucher 
  Alexander Polakov 
  Alexander Usyskin 
  Andreas Dilger 
  Andreas Gruenbacher 
  Andreas Gruenbacher 
  Andrew Morton 
  Andy Lutomirski 
  Arve Hjønnevåg 
  Boris Brezillon 
  Brian Norris 
  Bryan Paluch 
  CAI Qian 
  Carlos Santa 
  Carlos Santa  (v1)
  Ching Huang 

Re: [Xen-devel] [RFC] Xen PV Drivers Lifecycle

[George Dunlap]

On Tue, Dec 20, 2016 at 11:09 PM, Stefano Stabellini
 wrote:
> On Tue, 20 Dec 2016, Jan Beulich wrote:
>> >>> On 20.12.16 at 01:47,  wrote:
>> > ## Design Phase
>> >
>> > The first step toward acceptance of a new PV protocol is to write a
>> > design document and send it to xen-devel. It should cover the xenstore
>> > handshake mechanism, the ABI, how the protocol works and anything else
>> > which is required to write an implementation of it. The usage of C-like
>> > structs to describe language and platform agnostic protocols is
>> > discouraged.
>> >
>> > An attempt should be made for the protocol ABI to be backward compatible
>> > and OS agnostic, but, realistically, backward and cross-platform
>> > compatibility are not fully expected at this stage.
>>
>> How does backward compatibility matter for a new protocol? Is
>> this perhaps rather about forward compatibility provisions
>> (like requiring reserved fields to be zero to allow future use)?
>
> By "backward compatibility" I mean promising that, in 5 years time, a
> new frontend will still be able to connect to a backend written in 2016.
>
> This level of support requires an understanding of the protocol and its
> subtleties which usually only comes from experience. Hence, I am
> suggesting to make this kind of promises only after the code has lived
> in-tree for a while and has been subject to wider testing.
>
> Maybe I should call it cross-versions compatibility?

Right, so I think you mean that you design the (new) protocol such
that *future* versions can be backwards-compatible.

What if you rephrase it something like the following?

"An attempt should be made to design the ABI such that it will be OS
agnostic, that future versions will not need to introduce
backward-incompatible changes, and so on; but these are not yet hard
requirements."  (Or perhaps, "not yet first-order goals.")

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 02/10] x86/guest-walk: use unambiguous register names

[George Dunlap]

On Tue, Dec 20, 2016 at 10:36 AM, Jan Beulich  wrote:
> This is in preparation of eliminating the mis-naming of 64-bit fields
> with 32-bit register names (eflags instead of rflags etc). Use the
> guaranteed 32-bit underscore prefixed names for now where appropriate.
>
> Signed-off-by: Jan Beulich 

Any particular reason to use _eflags rather than just using rflags here?

Either way:

Acked-by: George Dunlap 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 103984: all pass - PUSHED

[osstest service owner]

flight 103984 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103984/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4
baseline version:
 ovmf 1f20b2988cb223907e64a4736a3eb6030aa715b8

Last test of basis   103957  2016-12-26 13:44:57 Z1 days
Testing same since   103984  2016-12-28 07:15:07 Z0 days1 attempts


People who touched revisions under test:
  Jiewen Yao 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=31d060d94e65c41ccca2b68f7908c0c3b4ac3df4
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
31d060d94e65c41ccca2b68f7908c0c3b4ac3df4
+ branch=ovmf
+ revision=31d060d94e65c41ccca2b68f7908c0c3b4ac3df4
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.8-testing
+ '[' x31d060d94e65c41ccca2b68f7908c0c3b4ac3df4 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : 

[Xen-devel] [distros-debian-squeeze test] 68280: tolerable FAIL

[Platform Team regression test user]

flight 68280 distros-debian-squeeze real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/68280/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-amd64-squeeze-netboot-pygrub 9 debian-di-install fail like 
68252
 test-amd64-amd64-i386-squeeze-netboot-pygrub 9 debian-di-install fail like 
68252
 test-amd64-i386-amd64-squeeze-netboot-pygrub 9 debian-di-install fail like 
68252
 test-amd64-i386-i386-squeeze-netboot-pygrub 9 debian-di-install fail like 68252

baseline version:
 flight   68252

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-squeeze-netboot-pygrubfail
 test-amd64-i386-amd64-squeeze-netboot-pygrub fail
 test-amd64-amd64-i386-squeeze-netboot-pygrub fail
 test-amd64-i386-i386-squeeze-netboot-pygrub  fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-coverity test] 103985: all pass - PUSHED

[osstest service owner]

flight 103985 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103985/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen  ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5
baseline version:
 xen  ad891b1389cbafce2d31ef3f0c060c781689e2e0

Last test of basis   103922  2016-12-25 09:18:44 Z3 days
Testing same since   103985  2016-12-28 09:19:12 Z0 days1 attempts


People who touched revisions under test:
  Alistair Francis 
  Wei Liu 
  Zhang Chen 

jobs:
 coverity-amd64   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-coverity
+ revision=ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push 
xen-unstable-coverity ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5
+ branch=xen-unstable-coverity
+ revision=ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-coverity
+ qemuubranch=qemu-upstream-unstable-coverity
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-coverity
+ prevxenbranch=xen-4.8-testing
+ '[' xee524f2bfa681ad116b8ae925fa8f3f18ee12ba5 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' xgit://xenbits.xen.org/linux-pvops.git = x ']'
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : daily-cron.xen-unstable-coverity
++ : 

Re: [Xen-devel] [PATCH] arm64: Do not include linux/uaccess.h from assembler files

[Thomas Petazzoni]

Hello,

On Mon, 26 Dec 2016 02:31:08 -0800, Guenter Roeck wrote:
> Including linux/uaccess.h from assembler files in arm64 builds results
> in the following build errors.
> 
> In file included from arm64/include/asm/asm-offsets.h:1:0,
>  from arch/arm64/include/asm/assembler.h:26,
>from arch/arm64/include/asm/alternative.h:68,
>from arch/arm64/kernel/entry.S
> 
> include/linux/sched/prio.h: Assembler messages:
> include/linux/sched/prio.h:47: Error:
>   unknown mnemonic `static' --
>   `static inline long nice_to_rlimit(long nice)'
> build/include/linux/sched/prio.h:48: Error:
>   junk at end of line, first unrecognized character is `{'
> 
> [and many more]
> 
> If asm/uaccess.h is not included, many of the affected files fail to build
> with errors such as the following.
> 
> arch/arm64/lib/copy_to_user.S: Assembler messages:
> arch/arm64/lib/copy_to_user.S:66: Error:
>   unknown mnemonic `uaccess_enable_not_uao' --
>   `uaccess_enable_not_uao x3,x4'
> arch/arm64/lib/copy_template.S:71: Error:
>   unknown mnemonic `uao_user_alternative' --
>   `uao_user_alternative 9998f,strb,st trb,tmp1w,dst,#1'
> 
> Either drop the include if unnecessary or, if needed, replace with
> asm/uaccess.h.
> 
> Fixes: 7c0f6ba682b9 ("Replace  with  
> globally")
> Cc: Linus Torvalds 
> Cc: Al Viro 
> Signed-off-by: Guenter Roeck 

Tested-by: Thomas Petazzoni 
 (fixes the arm64 build, result boot tested on real HW)

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 103982: tolerable FAIL

[osstest service owner]

flight 103982 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/103982/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail  like 103975
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 103975
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 103975
 test-armhf-armhf-libvirt 13 saverestore-support-checkfail  like 103975
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 103975
 test-armhf-armhf-libvirt-raw 12 saverestore-support-checkfail  like 103975
 test-armhf-armhf-libvirt-qcow2 12 saverestore-support-check   fail like 103975
 test-armhf-armhf-xl-credit2  15 guest-start/debian.repeatfail  like 103975
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 103975
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 103975

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5
baseline version:
 xen  ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5

Last test of basis   103982  2016-12-28 01:57:22 Z0 days
Testing same since0  1970-01-01 00:00:00 Z 17163 days0 attempts

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-oldkern  pass
 build-i386-oldkern   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 

Re: [Xen-devel] [PATCH v3] xen: sched: removal of redundant check in Credit

[George Dunlap]

On 17/12/16 01:46, Dario Faggioli wrote:
> On Sat, 2016-12-17 at 00:53 +0530, Praveen Kumar wrote:
>> The patch gets rid of a redundant check in csched_vcpu_acct. In fact,
>> the function is only called from csched_tick, which already checks
>> that current is not the idle vcpu. The patch also adds an ASSERT to
>> the same effect, in order to make assumption ( i.e., no calling this
>> on idle vcpus) even more clear and as a guard for future mis-use.
>>
>> Signed-off-by: Praveen Kumar 
>> Acked-by: Dario Faggioli 
>>
> Better than before. But still, if I:
>  - save this mail as mbox
>  - try to import it in git
> it fails.
> 
> OTOH, if I:
>  - save this mail as mbox
>  - run dos2unix on the mbox file
>  - try to import it in git
> it works!

FWIW the script I use runs dos2unix on everything I get via e-mail
(either saved as an mbox or as an attachment) as a matter of course.
Are you actually sometimes able to git am mbox'd files *without* running
dos2unix?

 -George


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel