Re: [Xen-devel] [RFC v1 07/15] vt-d: Add API to update IRTE when VT-d PI is used

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 1:34 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 07/15] vt-d: Add API to update IRTE when VT-d PI is used
 
  From: Wu, Feng
  Sent: Wednesday, March 25, 2015 8:32 PM
 
  This patch adds an API which is used to update the IRTE
  for posted-interrupt when guest changes MSI/MSI-X information.
 
  Signed-off-by: Feng Wu feng...@intel.com
  ---
   xen/drivers/passthrough/vtd/intremap.c | 83
  ++
   xen/drivers/passthrough/vtd/iommu.h|  3 ++
   xen/include/asm-x86/iommu.h|  2 +
   3 files changed, 88 insertions(+)
 
  diff --git a/xen/drivers/passthrough/vtd/intremap.c
  b/xen/drivers/passthrough/vtd/intremap.c
  index 0333686..f44e74d 100644
  --- a/xen/drivers/passthrough/vtd/intremap.c
  +++ b/xen/drivers/passthrough/vtd/intremap.c
  @@ -898,3 +898,86 @@ void iommu_disable_x2apic_IR(void)
   for_each_drhd_unit ( drhd )
   disable_qinval(drhd-iommu);
   }
  +
  +/*
  + * This function is used to update the IRTE for posted-interrupt
  + * when guest changes MSI/MSI-X information
  + */
  +int pi_update_irte(struct vcpu *v, struct pirq *pirq, uint32_t gvec )
  +{
  +struct irq_desc *desc;
  +struct msi_desc *msi_desc;
  +int remap_index, rc = -1;
  +struct pci_dev *pci_dev;
  +struct acpi_drhd_unit *drhd;
  +struct iommu *iommu;
  +struct ir_ctrl *ir_ctrl;
  +struct iremap_entry *iremap_entries = NULL, *p = NULL;
  +struct iremap_entry new_ire;
  +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc;
  +unsigned long flags;
  +
  +desc = pirq_spin_lock_irq_desc(pirq, NULL);
  +if ( !desc )
  +return -1;
  +
  +msi_desc = desc-msi_desc;
  +if ( !msi_desc )
  +goto unlock_out;
  +
  +remap_index = msi_desc-remap_index;
  +pci_dev = msi_desc-dev;
  +if ( !pci_dev )
  +goto unlock_out;
  +
  +drhd = acpi_find_matched_drhd_unit(pci_dev);
  +if (!drhd)
  +{
  +dprintk(XENLOG_INFO VTDPREFIX, failed to get drhd!\n);
  +goto unlock_out;
  +}
  +
  +iommu = drhd-iommu;
  +ir_ctrl = iommu_ir_ctrl(iommu);
  +if ( !ir_ctrl )
  +{
  +dprintk(XENLOG_INFO VTDPREFIX, failed to get ir_ctrl!\n);
  +goto unlock_out;
  +}
  +
  +spin_lock_irqsave(ir_ctrl-iremap_lock, flags);
  +
  +GET_IREMAP_ENTRY(ir_ctrl-iremap_maddr, remap_index,
  iremap_entries, p);
  +
  +memcpy(new_ire, p, sizeof(struct iremap_entry));
  +
  +/* Setup/Update interrupt remapping table entry */
  +new_ire.lo_intpost.urg = 0;
  +new_ire.lo_intpost.vector = gvec;
  +new_ire.lo_intpost.pda_l = (((u64)virt_to_maddr(pi_desc)) 
  +(32 - PDA_LOW_BIT))  ~(-1UL 
  PDA_LOW_BIT);
  +new_ire.hi_intpost.pda_h = (((u64)virt_to_maddr(pi_desc))   32) 
  +~(-1UL  PDA_HIGH_BIT);
  +
  +new_ire.lo_intpost.res_1 = 0;
  +new_ire.lo_intpost.res_2 = 0;
  +new_ire.lo_intpost.res_3 = 0;
  +new_ire.hi_intpost.res_1 = 0;
  +
  +new_ire.lo_intpost.im = 1;
  +
 
 Is above code for creating a new entry or updating an existing entry?
 suppose two purposes would require different steps but here I didn't
 see a difference... or if you mean every update is equal to what's
 required for creating a new one, better add a comment for that.

It just updates an existing IRTE here to a posted format.

Thanks,
Feng

 
 Thanks
 Kevin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 13/15] Update Posted-Interrupts Descriptor during vCPU scheduling

2015-04-02 Thread Tian, Kevin
 From: Wu, Feng
 Sent: Wednesday, March 25, 2015 8:32 PM
 
 The basic idea here is:
 1. When vCPU's state is RUNSTATE_running,
 - set 'NV' to 'Notification Vector'.
 - Clear 'SN' to accpet PI.
 - set 'NDST' to the right pCPU.
 2. When vCPU's state is RUNSTATE_blocked,
 - set 'NV' to 'Wake-up Vector', so we can wake up the
   related vCPU when posted-interrupt happens for it.
 - Clear 'SN' to accpet PI.
 3. When vCPU's state is RUNSTATE_runnable/RUNSTATE_offline,
 - Set 'SN' to suppress non-urgent interrupts.
   (Current, we only support non-urgent interrupts)
 - Set 'NV' back to 'Notification Vector' if needed.
 
 Signed-off-by: Feng Wu feng...@intel.com
 ---
  xen/arch/x86/hvm/vmx/vmx.c | 108
 +
  xen/common/schedule.c  |   3 ++
  2 files changed, 111 insertions(+)
 
 diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
 index b30392c..6323bd6 100644
 --- a/xen/arch/x86/hvm/vmx/vmx.c
 +++ b/xen/arch/x86/hvm/vmx/vmx.c
 @@ -1710,6 +1710,113 @@ static void vmx_handle_eoi(u8 vector)
  __vmwrite(GUEST_INTR_STATUS, status);
  }
 
 +static void vmx_pi_desc_update(struct vcpu *v, int new_state)
 +{
 +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc;
 +struct pi_desc old, new;
 +int old_state = v-runstate.state;
 +unsigned long flags;
 +
 +if ( !iommu_intpost )
 +return;
 +
 +switch ( new_state )
 +{
 +case RUNSTATE_runnable:
 +case RUNSTATE_offline:
 +/*
 + * We don't need to send notification event to a non-running
 + * vcpu, the interrupt information will be delivered to it before
 + * VM-ENTRY when the vcpu is scheduled to run next time.
 + */
 +pi_set_sn(pi_desc);
 +
 +/*
 + * If the state is transferred from RUNSTATE_blocked,
 + * we should set 'NV' feild back to posted_intr_vector,
 + * so the Posted-Interrupts can be delivered to the vCPU
 + * by VT-d HW after it is scheduled to run.
 + */
 +if ( old_state == RUNSTATE_blocked )
 +{
 +do
 +{
 +old.control = new.control = pi_desc-control;
 +new.nv = posted_intr_vector;
 +}
 +while ( cmpxchg(pi_desc-control, old.control, new.control)
 +!= old.control );
 +
 +   /*
 +* Delete the vCPU from the related wakeup queue
 +* if we are resuming from blocked state
 +*/
 +   spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock,
 + v-processor), flags);
 +   list_del(v-blocked_vcpu_list);
 +   spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock,
 +  v-processor), flags);
 +}
 +break;
 +
 +case RUNSTATE_blocked:
 +/*
 + * The vCPU is blocked on the wait queue.
 + * Store the blocked vCPU on the list of the
 + * vcpu-wakeup_cpu, which is the destination
 + * of the wake-up notification event.
 + */
 +spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock,
 +  v-processor), flags);
 +list_add_tail(v-blocked_vcpu_list,
 +  per_cpu(blocked_vcpu_on_cpu, v-processor));
 +spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock,
 +   v-processor), flags);
 +
 +do
 +{
 +old.control = new.control = pi_desc-control;
 +
 +/*
 + * We should not block the vCPU if
 + * an interrupt is posted for it.
 + */
 +
 +if ( pi_test_on(old) == 1 )
 +{
 +tasklet_schedule(v-vcpu_wakeup_tasklet);
 +return;
 +}

so you also need to remove the vcpu from the blocked list, right?

and how do you handle ON is set after above check? looks this is better
handled behind cmpxchg loop...

 +
 +pi_clear_sn(new);
 +new.nv = pi_wakeup_vector;
 +}
 +while ( cmpxchg(pi_desc-control, old.control, new.control)
 +!= old.control );
 +break;
 +
 +case RUNSTATE_running:
 +ASSERT( pi_test_sn(pi_desc) == 1 );
 +
 +do
 +{
 +old.control = new.control = pi_desc-control;
 +if ( x2apic_enabled )
 +new.ndst = cpu_physical_id(v-processor);
 +else
 +new.ndst = (cpu_physical_id(v-processor)  8) 
 0xFF00;
 +
 +pi_clear_sn(new);
 +}
 +while ( cmpxchg(pi_desc-control, old.control, new.control)
 +!= old.control );
 +break;
 +
 +default:
 +break;
 +}
 +}
 +
  void vmx_hypervisor_cpuid_leaf(uint32_t sub_idx,
 uint32_t *eax, uint32_t *ebx,
 

Re: [Xen-devel] [PATCH] x86/xen: Warn when there's no mfn for pfn

2015-04-02 Thread David Vrabel
On 02/04/15 08:28, Ross Lagerwall wrote:
 I recently had to debug dom0 taking a fatal page fault and it would have
 been useful if it warned when creating a non-present pte if there's no
 mfn for a pfn.

This will trigger for ever PTE covering memory hotplugged by the balloon
driver, so I don't think we can take this.

David

 --- a/arch/x86/xen/mmu.c
 +++ b/arch/x86/xen/mmu.c
 @@ -397,6 +397,7 @@ static pteval_t pte_pfn_to_mfn(pteval_t val)
* pte_mfn_to_pfn is asymmetric.
*/
   if (unlikely(mfn == INVALID_P2M_ENTRY)) {
 + printk(KERN_WARNING No mfn for pfn %lx, creating 
 non-present pte\n, pfn);
   mfn = 0;
   flags = 0;
   } else
 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight

2015-04-02 Thread Ian Campbell
Hi,

It's been a long time since v3 of this series, which was
http://article.gmane.org/gmane.comp.emulators.xen.devel/224415. That
contains the background which I won't repeat here.

Since then I have done the big rebase, addressed the review feed back
from last time (I think!) and added a new feature which is a bit
unrelated but uses the new ability to do d-i based installs to test
qcow, vhd and raw file images in the main flight.

There are some patches in here which I think will be useful to the Intel
folks doing the nested virt testing, specifically the refactoring of how
overlays and ssh host keys are done will be useful for installing a
guest to be treated as the L1 host. LongTao, I've CCd you only on the 3
patches which I think will be of interest instead of the full 24 patch
series.

For the benefit of that work it would be good to get at least the
precursor cleanup/refactoring patches into good shape and into the tree
sooner rather than later.

I've run this as an adhoc job on the Cambridge instance, the results of
which are at:
http://www.chiark.greenend.org.uk/~xensrcts/logs/36995/

The failures mainly seem to be some sort of proxy issue returning old
data (similar to what used to happen to mg-debian-installer-update), so
we boot an older d-i kernel+initrd and end up unable to find any kernel
modules. I've tried to workaround this with a patch near the end
ts-debian-di-install: Use ftp.debian.org directly, the latter I'm not
sure how to handle other than by bashing the local proxy over the head.

On a small number I see The installer failed to download a file from
the mirror. messages, which might be the cache/proxy or might be an
actual Debian issue.

On the one armhf attempt which didn't suffer from the first problem
(daily netboot based) it seems the kernel didn't actual halt when
shutdown, which is likely to be a real Debian kernel bug.

I've also run an adhoc xen-unstable test of the new jobs there, results
at http://www.chiark.greenend.org.uk/~xensrcts/logs/37011/ the
interesting jobs are *-pvgrub, *-pygrub for the bootloader tests and
*-{qcow2,raw,vhd} for the image format stuff, LVM is tested by the
existing -{xl,libvirt} jobs.

Results are a bit of a mixed bag. 64-bit pvgrub worked, but 32-bit
pvgrub failed due to Debian installing a non-PAE kernel (which cannot be
booted), that's probably a bug in the preseeding. The pygrub test
passed.

-qcow2 and -vhd tests passed except the armhf one which couldn't find
the appropriate vmlinuz-xen because mg-debian-installer-update doesn't
create it, and the Wheezy baseline doesn't support running as a Xen
guest on ARM anyway. I could suppress this in make-flight until the
Jessie upgrade.

The -raw tests all failed due to a timeout doing:
dd if=/dev/zero of=/var/lib/xen/images/debian/disk.raw bs=1M count=1

I'm unclear if it timed out after 30 (as the code seems to say) or 60
(as the logs seem to say) but either way something obviously needs
adjusting. I just tried it on my local test box and it took 1m20s and
reported 130 MB/s. I've appended an as yet untested patch which assumed
100 MB/s and calculates a timeout from that.

I could switch to using count=1 seek=1 to create a sparse file,
which should be pretty much instantaneous, but allocating the whole
backing file seemed like a good idea.

Ian.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 18/25] ts-debian-di-install: Refactor root_disk specification

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v4: new patch
---
 ts-debian-di-install | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index 56b6ad8..281e7cb 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -205,12 +205,14 @@ END
OnPowerOff = preserve
 );
 
+my $root_disk = 'phy:$gho-{Lvdev},xvda,w';
+
 prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, END);
 $method_cfg
 extra   = $cmdline
 #
 disk= [
-$extra_disk 'phy:$gho-{Lvdev},xvda,w'
+$extra_disk $root_disk
 ]
 END
 
@@ -234,7 +236,7 @@ END
 $blcfg
 #
 disk= [
-'phy:$gho-{Lvdev},xvda,w'
+$root_disk
 ]
 END
 return;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 06/25] Debian: Refactor installation of overlays, so it can be used for guests too

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
Cc: longtaox.p...@intel.com
---
v3: New patch
---
 Osstest/Debian.pm | 57 ---
 1 file changed, 29 insertions(+), 28 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index db89326..13cd147 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -38,7 +38,9 @@ BEGIN {
   preseed_base
   preseed_create
   preseed_ssh
-  preseed_hook_command preseed_hook_installscript 
preseed_hook_cmds
+  preseed_hook_command preseed_hook_installscript
+  preseed_hook_overlay
+  preseed_hook_cmds
   di_installcmdline_core
   );
 %EXPORT_TAGS = ( );
@@ -700,26 +702,6 @@ sub preseed_create ($$;@) {
 my $d_i= $ho-{Tftp}{Path}.'/'.$ho-{Tftp}{DiBase}.'/'.$r{arch}.'/'.
$c{TftpDiVersion}.'-'.$ho-{Suite};
 
-my $overlays= '';
-my $create_overlay= sub {
-my ($srcdir, $tfilename) = @_;
-my $url= create_webfile($ho, $tfilename$sfx, sub {
-my ($fh) = @_;
-contents_make_cpio($fh, 'ustar', $srcdir);
-});
-$overlays .= END;
-wget -O overlay.tar '$url'
-cd /target
-tar xf \$r/overlay.tar
-cd \$r
-rm overlay.tar
-
-END
-};
-
-$create_overlay-('overlay','overlay.tar');
-$create_overlay-($c{OverlayLocal}, 'overlay-local.tar');
-
 preseed_hook_installscript($ho, $sfx,
   '/lib/partman/init.d', '000override-parted-devices', END);
 #!/bin/sh
@@ -765,19 +747,14 @@ true
 END
 
 preseed_ssh($ho, $sfx);
+preseed_hook_overlay($ho, $sfx, 'overlay', 'overlay.tar');
+preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
 
 preseed_hook_command($ho, 'late_command', $sfx, END);
 #!/bin/sh
 set -ex
 
-r=/target/root
-cd \$r
-
 echo FANCYTTY=0  /target/etc/lsb-base-logging.sh
-
-$overlays
-
-echo latecmd done.
 END
 
 my $dtbs = $d_i/dtbs.tar.gz;
@@ -966,6 +943,30 @@ chmod +x '$installer_pathname'
 END
 }
 
+sub preseed_hook_overlay () {
+my ($ho, $sfx, $srcdir, $tfilename) = @_;
+my $url= create_webfile($ho, $tfilename$sfx, sub {
+my ($fh) = @_;
+contents_make_cpio($fh, 'ustar', $srcdir);
+});
+preseed_hook_command($ho, 'late_command', $sfx, END);
+#!/bin/sh
+set -ex
+
+r=/target/root
+cd \$r
+
+umask 022
+
+wget -O overlay.tar '$url'
+cd /target
+tar xf \$r/overlay.tar
+cd \$r
+rm overlay.tar
+
+END
+}
+
 sub preseed_hook_cmds () {
 my $preseed;
 foreach my $di_key (keys %preseed_cmds) {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories

2015-04-02 Thread Ian Campbell
Causes known_hosts to be consistently created as well as ~osstest/.ssh
to be consistently populated (it previsouly wasn't for HVM guests).

Signed-off-by: Ian Campbell ian.campb...@citrix.com
Cc: longtaox.p...@intel.com
---
v3: New patch
---
 Osstest/Debian.pm | 99 +--
 ts-debian-hvm-install |  5 ++-
 2 files changed, 59 insertions(+), 45 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index ef602b1..db89326 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,6 +37,7 @@ BEGIN {
   %preseed_cmds
   preseed_base
   preseed_create
+  preseed_ssh
   preseed_hook_command preseed_hook_installscript 
preseed_hook_cmds
   di_installcmdline_core
   );
@@ -551,6 +552,60 @@ sub di_installcmdline_core ($$;@) {
 return @cl;
 }
 
+sub preseed_ssh ($$) {
+my ($ho,$sfx) = @_;
+
+my $authkeys_url= create_webfile($ho, authkeys$sfx, authorized_keys());
+
+my $hostkeyfile= $c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub;
+my $hostkey= get_filecontents($hostkeyfile);
+chomp($hostkey); $hostkey.=\n;
+my $knownhosts= '';
+
+my $hostsq= $dbh_tests-prepare(END);
+SELECT val FROM runvars
+ WHERE flight=? AND name LIKE '%host'
+ GROUP BY val
+END
+$hostsq-execute($flight);
+while (my ($node) = $hostsq-fetchrow_array()) {
+my $longname= $node.$c{TestHostDomain};
+my (@hostent)= gethostbyname($longname);
+if (!@hostent) {
+logm(skipping host key for nonexistent host $longname);
+next;
+}
+my $specs= join ',', $longname, $node, map {
+join '.', unpack 'W4', $_;
+} @hostent[4..$#hostent];
+logm(adding host key for $specs);
+$knownhosts.= $specs .$hostkey;
+}
+$hostsq-finish();
+
+$knownhosts.= localhost,127.0.0.1 .$hostkey;
+my $knownhosts_url= create_webfile($ho, known_hosts$sfx, $knownhosts);
+
+preseed_hook_command($ho, 'late_command', $sfx, END);
+#!/bin/sh
+set -ex
+
+r=/target/root
+cd \$r
+
+umask 022
+mkdir .ssh
+wget -O .ssh/authorized_keys '$authkeys_url'
+wget -O .ssh/known_hosts '$knownhosts_url'
+
+u=osstest
+h=/home/\$u
+mkdir /target\$h/.ssh
+cp .ssh/authorized_keys /target\$h/.ssh
+chroot /target chown -R \$u.\$u \$h/.ssh
+END
+}
+
 sub preseed_base ($$$;@) {
 my ($ho,$suite,$extra_packages,%xopts) = @_;
 
@@ -639,43 +694,12 @@ END
 sub preseed_create ($$;@) {
 my ($ho, $sfx, %xopts) = @_;
 
-my $authkeys_url= create_webfile($ho, authkeys$sfx, authorized_keys());
-
-my $hostkeyfile= $c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub;
-my $hostkey= get_filecontents($hostkeyfile);
-chomp($hostkey); $hostkey.=\n;
-my $knownhosts= '';
-
 my $disk= $xopts{DiskDevice} || '/dev/sda';
 my $suite= $xopts{Suite} || $c{DebianSuite};
 
 my $d_i= $ho-{Tftp}{Path}.'/'.$ho-{Tftp}{DiBase}.'/'.$r{arch}.'/'.
$c{TftpDiVersion}.'-'.$ho-{Suite};
 
-my $hostsq= $dbh_tests-prepare(END);
-SELECT val FROM runvars
- WHERE flight=? AND name LIKE '%host'
- GROUP BY val
-END
-$hostsq-execute($flight);
-while (my ($node) = $hostsq-fetchrow_array()) {
-my $longname= $node.$c{TestHostDomain};
-my (@hostent)= gethostbyname($longname);
-if (!@hostent) {
-logm(skipping host key for nonexistent host $longname);
-next;
-}
-my $specs= join ',', $longname, $node, map {
-join '.', unpack 'W4', $_;
-} @hostent[4..$#hostent];
-logm(adding host key for $specs);
-$knownhosts.= $specs .$hostkey;
-}
-$hostsq-finish();
-
-$knownhosts.= localhost,127.0.0.1 .$hostkey;
-my $knownhosts_url= create_webfile($ho, known_hosts$sfx, $knownhosts);
-
 my $overlays= '';
 my $create_overlay= sub {
 my ($srcdir, $tfilename) = @_;
@@ -740,6 +764,8 @@ ls -l /dev/sd*
 true
 END
 
+preseed_ssh($ho, $sfx);
+
 preseed_hook_command($ho, 'late_command', $sfx, END);
 #!/bin/sh
 set -ex
@@ -747,17 +773,6 @@ set -ex
 r=/target/root
 cd \$r
 
-umask 022
-mkdir .ssh
-wget -O .ssh/authorized_keys '$authkeys_url'
-wget -O .ssh/known_hosts '$knownhosts_url'
-
-u=osstest
-h=/home/\$u
-mkdir /target\$h/.ssh
-cp .ssh/authorized_keys /target\$h/.ssh
-chroot /target chown -R \$u.\$u \$h/.ssh
-
 echo FANCYTTY=0  /target/etc/lsb-base-logging.sh
 
 $overlays
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 95fce9a..02c142b 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -44,7 +44,6 @@ our $gho;
 sub preseed () {
 
 my $preseed_file = preseed_base($gho,'wheezy','',());
-my $authkeys = join('\\n', split(/\n/, authorized_keys()));
 
 $preseed_file .= (END);
 d-i netcfg/get_hostname string $gn
@@ -74,10 +73,10 @@ d-i apt-setup/cdrom/set-first 

[Xen-devel] [PATCH OSSTEST v4 11/25] distros: Support pvgrub for Wheezy too.

2015-04-02 Thread Ian Campbell
This requires us to install pv-grub-menu from backports, which we do
using a late_command.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3:
  - Remove spurious () from (END) (and the prexisting  too)
  - Remove $xopts{EnableBackports} and automatically handle the need to add
backports in preseed_base.
  - Install via late_command not apt-setup, since the former has
issues, hence subject drops attempt to...
---
 Osstest/Debian.pm   | 44 +++-
 make-distros-flight |  4 ++--
 2 files changed, 45 insertions(+), 3 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index cbc7172..b37540e 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -689,6 +689,17 @@ END
 d-i clock-setup/ntp-server string $ntpserver
 END
 
+# deb http://ftp.debian.org/debian/ wheezy-backports main
+$preseed .= END if $extra_packages =~ m#\b/$suite-backports\b#;
+d-i apt-setup/local0/repository string \\
+http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+d-i apt-setup/local0/comment string $suite backports
+END
+# Jessie onwards...
+#$preseed .= END if $extra_packages =~ m#\b/$suite-backports\b#;
+#d-i apt-setup/services-select multiselect security, updates, backports
+#END
+
 $preseed .= END;
 
 ### END OF DEBIAN PRESEED BASE
@@ -702,7 +713,38 @@ sub preseed_create_guest ($$;@) {
 
 my $suite= $xopts{Suite} || $c{DebianSuite};
 
-my $extra_packages = pv-grub-menu if $xopts{PvMenuLst};
+my $extra_packages = ;
+if ($xopts{PvMenuLst}) {
+if ($suite =~ m/wheezy/) {
+# pv-grub-menu/wheezy-backports + using apt-setup to add
+# backports results in iproute, ifupdown and
+# isc-dhcp-client getting removed because tasksel's
+# invocation of apt-get install somehow decides the
+# iproute2 from wheezy-backports is a thing it wants to
+# install. So instead lets fake it with a late command...
+#
+# This also has the bonus of working round an issue with
+# 1.2.1~bpo70+1 which created an invalid menu.lst using
+# root(/dev/xvda,0) which pvgrub cannot parse because
+# the Grub device.map isn't present at pkgsel/include time
+# but it is by late_command time. This was fixed by
+# version 1.3 which is in Jessie onwards.
+preseed_hook_command($ho, 'late_command', $sfx, END);
+#!/bin/sh
+set -ex
+
+(
+echo
+echo \\\# $suite backports
+echo deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} 
$suite-backports main
+)  /target/etc/apt/sources.list
+in-target apt-get update
+in-target apt-get install -y -t wheezy-backports pv-grub-menu
+END
+} else {
+$extra_packages = pv-grub-menu;
+}
+}
 
 my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts);
 $preseed_file.= (END);
diff --git a/make-distros-flight b/make-distros-flight
index 80df21b..97df89a 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -65,9 +65,9 @@ test_do_one_netboot () {
 arm*_arm*_*) bootloader=pygrub;; # no pvgrub for arm
 
 # Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
-# workedaround in Jessie+ with pv-grub-menu package.
+# workedaround in Wheezy+ with pv-grub-menu package (backports in Wheezy,
+# in Jessie+ main).
 *_squeeze) bootloader=pygrub;;
-*_wheezy) bootloader=pygrub;;
 
 # pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build.
 i386_amd64_*) bootloader=pygrub;;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file

2015-04-02 Thread Ian Campbell
Call it from ts-debian-hvm-install.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
Cc: longtaox.p...@intel.com
---
v3: New patch
---
 Osstest/Debian.pm | 16 +++-
 ts-debian-hvm-install |  3 +++
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5e6bcba..ef602b1 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,7 +37,7 @@ BEGIN {
   %preseed_cmds
   preseed_base
   preseed_create
-  preseed_hook_command preseed_hook_installscript
+  preseed_hook_command preseed_hook_installscript 
preseed_hook_cmds
   di_installcmdline_core
   );
 %EXPORT_TAGS = ( );
@@ -907,10 +907,7 @@ d-i partman-auto/expert_recipe string  
\\
 
 END
 
-foreach my $di_key (keys %preseed_cmds) {
-$preseed_file .= d-i preseed/$di_key string .
-(join '  ', @{ $preseed_cmds{$di_key} }). \n;
-}
+$preseed_file .= preseed_hook_cmds();
 
 if ($ho-{Flags}{'no-di-kernel'}) {
$preseed_file .= END;
@@ -954,4 +951,13 @@ chmod +x '$installer_pathname'
 END
 }
 
+sub preseed_hook_cmds () {
+my $preseed;
+foreach my $di_key (keys %preseed_cmds) {
+$preseed .= d-i preseed/$di_key string .
+(join '  ', @{ $preseed_cmds{$di_key} }). \n;
+}
+return $preseed;
+}
+
 1;
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index cfd5144..95fce9a 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -77,6 +77,9 @@ d-i preseed/late_command string \\
 in-target mkdir -p /root/.ssh; \\
 in-target sh -c echo -e '$authkeys' /root/.ssh/authorized_keys;
 END
+
+$preseed_file .= preseed_hook_cmds();
+
 return $preseed_file;
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 22/25] make-distros-flight: don't bother building for XSM.

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 make-distros-flight | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/make-distros-flight b/make-distros-flight
index 2724dd2..62c527b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -34,7 +34,10 @@ defsuite=`getconfig DebianSuite`
 defguestsuite=`getconfig GuestDebianSuite`
 
 job_create_build_filter_callback () {
-:
+  case  $*  in
+* enable_xsm=true *) return 1;;
+  esac
+  return 0
 }
 
 if [ x$buildflight = x ]; then
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 08/25] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs()

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3: New patch
---
 make-flight | 4 
 mfi-common  | 4 
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/make-flight b/make-flight
index cc8ecdb..ec0a20c 100755
--- a/make-flight
+++ b/make-flight
@@ -36,10 +36,6 @@ defguestsuite=`getconfig GuestDebianSuite`
 
 if [ x$buildflight = x ]; then
 
-  if [ x$BUILD_LVEXTEND_MAX != x ]; then
- BUILD_RUNVARS+= build_lvextend_max=$BUILD_LVEXTEND_MAX 
-  fi
-
   create_build_jobs
 
 else
diff --git a/mfi-common b/mfi-common
index 16fc8c5..a9e966f 100644
--- a/mfi-common
+++ b/mfi-common
@@ -63,6 +63,10 @@ create_build_jobs () {
   local enable_ovmf
   local build_hostflags
 
+  if [ x$BUILD_LVEXTEND_MAX != x ]; then
+ BUILD_RUNVARS+= build_lvextend_max=$BUILD_LVEXTEND_MAX 
+  fi
+
   for arch in ${BUILD_ARCHES- i386 amd64 armhf }; do
 
 if [ x$arch = xdisable ]; then continue; fi
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 09/25] distros: add support for installing Debian PV guests via d-i, flight and jobs

2015-04-02 Thread Ian Campbell
This patch introduces ts-debian-di-install which can install Debian
from a netboot (PXE) debian installer image. By default it installs
from the d-i image used by osstest (using the special Xen PV guest
enabled flavour where necessary) but it can also install the current
release versions from Squeeze onwards (up to and including Sid) and
the d-i daily builds. This is controlled by runvars {Guest}_diver =
osstest|current and {Guest}_dist = squeeze|...|sid.  The resulting
guests boot the distro kernel using pygrub (pvgrub will follow).

The distros flights differ substantially from the existing flights.
Introduce make-distros-flight using the functionality previously
refactored into mfi-common. The new flight tests all versions of
Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf
from Jessie onwards only) using the usual smoke tests.

Test names are suffixed -pygrub pending the addition of pvgrub
variants in a future commit.

Add the new cases to sg-run-job

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v4: use guest create
v3: $BUILD_LVEXTEND_MAX now handled in mfi-common
Consolidate setting of ruvars
Include $flight and $job in tmpdir name
Use Osstest::Debian::di_installcmdline_core
Document the usage of get_host_property on a guest object
Correct ARM netboot paths
Include bootloader in test name
   Should include -pv too?
console= repetition for Jessie onwards.
Wait for up to an hour for the install. I'd seen timeouts right at
the end of the install with the previous value

fixup: distros: add support for installing Debian PV guests via d-i, flight and 
jobs
---
 Osstest/TestSupport.pm |   3 +
 make-distros-flight|  98 +
 sg-run-job |  11 
 ts-debian-di-install   | 168 +
 4 files changed, 280 insertions(+)
 create mode 100755 make-distros-flight
 create mode 100755 ts-debian-di-install

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 8f67f7e..3cd8d22 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -906,8 +906,11 @@ sub propname_massage ($) {
 return $prop;
 }
 
+# It is fine to call this on a guest object too, in which case it will
+# always return $defval.
 sub get_host_property ($$;$) {
 my ($ho, $prop, $defval) = @_;
+return $defval unless $ho-{Properties};
 my $val = $ho-{Properties}{propname_massage($prop)};
 return defined($val) ? $val : $defval;
 }
diff --git a/make-distros-flight b/make-distros-flight
new file mode 100755
index 000..5067342
--- /dev/null
+++ b/make-distros-flight
@@ -0,0 +1,98 @@
+#!/bin/bash
+
+# This is part of osstest, an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/.
+
+
+set -e
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. cri-common
+. ap-common
+. mfi-common
+
+defsuite=`getconfig DebianSuite`
+defguestsuite=`getconfig GuestDebianSuite`
+
+if [ x$buildflight = x ]; then
+
+  WANT_XEND=false REVISION_LINUX_OLD=disable
+
+  create_build_jobs
+
+else
+
+  bfi=$buildflight.
+
+fi
+
+job_create_test_filter_callback () {
+  if [ $xenarch = i386 ]; then return 1; fi
+  return 0
+}
+
+test_matrix_branch_filter_callback () {
+:
+}
+
+test_do_one_netboot () {
+  job_create_test   \
+   test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub  \
+test-debian-di xl $xenarch $dom0arch\
+  kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
+  debian_arch=$domU \
+  debian_dist=$dist \
+  debian_method=netboot \
+  debian_diver=current  \
+  all_hostflags=$most_hostflags
+}
+
+test_matrix_do_one () {
+  case ${xenarch} in
+  amd64) domUarches=amd64 i386;;
+  armhf) domUarches=armhf;;
+  esac
+
+  for domU in $domUarches ; do
+for dist in squeeze wheezy jessie sid daily ; do
+  case ${domU}_${dist} in
+  armhf_squeeze) continue;; # No armhf in Squeeze
+  armhf_wheezy) continue;; # No armhf guest support in Wheezy
+  *) ;;
+  esac
+
+

[Xen-devel] [PATCH OSSTEST v4 16/25] Debian: Handle lack of bootloader support in d-i on ARM.

2015-04-02 Thread Ian Campbell
Debian doesn't currently know what bootloader to install in a Xen
guest on ARM. We install pv-grub-menu above which actually does what
we need, but the installer doesn't treat that as a bootloader.

Most ARM platforms end up installing a u-boot boot.scr, based on a
platform whitelist. This doesn't seem appropriate for us. Grub is not
available for arm32. For arm64 we will eventually end up with in-guest
UEFI and therefore grub-efi and things will work normally. I'm not
sure what the answer is going to be for arm32.

This patch enables the workaround for Wheezy, Jessie and Sid,
post-Jessie should be enabled as we add them. (Pre-wheezy does not
support running as a Xen guest on ARM so we don't test them at all).

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v4: Handle sid too
v3: New
---
 Osstest/Debian.pm| 14 --
 ts-debian-di-install |  6 --
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 831290c..7f9deaa 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -711,8 +711,8 @@ END
 return $preseed;
 }
 
-sub preseed_create_guest ($$;@) {
-my ($ho, $sfx, %xopts) = @_;
+sub preseed_create_guest ($$$;@) {
+my ($ho, $arch, $sfx, %xopts) = @_;
 
 my $suite= $xopts{Suite} || $c{DebianSuite};
 
@@ -759,6 +759,16 @@ d-i grub-installer/bootdev  string /dev/xvda
 
 END
 
+# Debian doesn't currently know what bootloader to install in a
+# Xen guest on ARM. We install pv-grub-menu above which actually
+# does what we need, but the installer doesn't treat that as a
+# bootloader.
+logm(\$arch is $arch, \$suite is $suite);
+$preseed_file.= (END) if $arch =~ /^arm/  $suite =~ 
/wheezy|jessie|sid/;
+d-i nobootloader/confirmation_common boolean true
+
+END
+
 preseed_ssh($ho, $sfx);
 preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
 
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 62e9bb5..56b6ad8 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -168,7 +168,9 @@ END
 
$suite = sid if $suite eq daily;
 
-   $ps_url = preseed_create_guest($gho, '', Suite=$suite, PvMenuLst=($bl 
eq pvgrub));
+   $ps_url = preseed_create_guest($gho, $arch, '',
+  Suite=$suite,
+  PvMenuLst=($bl eq pvgrub));
 
$extra_disk = ;
 }
@@ -180,7 +182,7 @@ END
 
($method_cfg,$extra_disk) = setup_netinst($tmpdir, $arch, $cd);
 
-   $ps_url = preseed_create_guest($gho, '', CDROM=1);
+   $ps_url = preseed_create_guest($gho, $arch, '', CDROM=1);
 }
 else
 {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case

2015-04-02 Thread Ian Campbell
On Thu, 2015-04-02 at 08:16 +, Pang, LongtaoX wrote:
 
 
  -Original Message-
  From: Ian Campbell [mailto:ian.campb...@citrix.com]
  Sent: Tuesday, March 31, 2015 10:23 PM
  To: Pang, LongtaoX
  Cc: xen-devel@lists.xen.org; ian.jack...@eu.citrix.com; wei.l...@citrix.com;
  Hu, Robert
  Subject: Re: [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case
  
  On Fri, 2015-03-27 at 19:06 -0400, longtao.pang wrote:
   Changes in v7:
   diff --git a/make-flight b/make-flight index 8ac3a87..b8f266f 100755
   --- a/make-flight
   +++ b/make-flight
   @@ -204,6 +204,26 @@ do_hvm_win7_x64_tests () {
all_hostflags=$most_hostflags,hvm  }
  
   +do_hvm_debian_nested_tests () {
   +  if [ $xenarch != amd64 ]; then
   +return
   +  fi
   +  if [ $dom0arch != amd64 ]; then
   +return
   +  fi
  
  You can do these on a line each, or even combine into one test. i.e.
  
  if [ $xenarch != amd64 -o $dom0arch != amd64 ]; then return; fi
  
 I'm sorry I find that the 'if' condition is not appropriate in v7 patch, it 
 should be 
 if [ $xenarch != amd64 -a $dom0arch != amd64 ]; then return; fi

Yes, I think you are right.

   +
   +  job_create_test test-$xenarch$kern-$dom0arch-nested test-nested xl \
   + $xenarch $dom0arch \
   +nested_image=$NESTED_OS_IMAGE \
   +nested2_image=$NESTED_OS_IMAGE \
  
  I think for clarity you should use something like nestedl1 and nestedl2 for 
  the
  runvar names.
  

 'nested' and 'nested2' are guest name of L1 and L2 guest VM. Since
 $specimage is accessed from $r{$gho-{Guest}_image} which
 defined in the function of ' target_put_guest_image'. So, maybe
 'nested' and 'nested2' are available here, I think. 

My point was you should use the clearer names throughout, from point of
definition onwards and in the sg-run-job spec for the recipe too.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v2 06/22] xen/arm: its: Port ITS driver to xen

2015-04-02 Thread Ian Campbell
On Thu, 2015-04-02 at 13:55 +0530, Vijay Kilari wrote:
 On Wed, Apr 1, 2015 at 5:04 PM, Ian Campbell ian.campb...@citrix.com wrote:
  On Thu, 2015-03-19 at 20:07 +0530, vijay.kil...@gmail.com wrote:
  From: Vijaya Kumar K vijaya.ku...@caviumnetworks.com
 
  This patch just makes ITS driver taken from linux
  compiles in xen environment.
 
  What is your intention wrt future updates to this driver?
 
  Are you intending to keep things in sync and import things from the
  Linux side (similar to the smmu drviers) or are you taking the Linux
  code as a starting point and intending that it then be maintained
  independently as a Xen driver from then on?
 
 Yes, I intend to keep things in sync with Linux driver.
 I have kept most the code same as Linux side except removing unused code.

There is a tonne of changes going on if that is your goal, in particular
in this patch but also in some of the following refactoring patches.
When this series is over it seems like the driver would bear very little
resemblance to the Linux one.

If you want to go this route then to aid in future synchronisation from
Linux patches the goal should be to make the changes to the Linux code
as minimal as possible, by defining shim functions and typedefs etc at
the top of the file, e.g. as Julien has tried to do with the smmu
driver.

Unlike the smmu stuff, which has a reasonably small and well-defined
interface to the kernel which can be easily shimmed between Xen and
Linux it's not clear to me that this approach is workable for ITS, the
Xen and Linux interrupt handling systems are rather different and ITS
needs to be more tightly integrated with other bits of Xen, in
particular the GIC drivers.

However if you think maintaining something which can be synchronised
from Linux is viable and desirable then that's ok by me.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] ENC:[Xen-users] Failed to insert 'xen_evtchn': No such device

2015-04-02 Thread Ian Campbell
On Thu, 2015-04-02 at 08:23 +0200, Olaf Hering wrote:
 On Wed, Apr 01, Carlos Gustavo Ramirez Rodriguez wrote:
  Second Problem:
  My boot.log is showing:
  
  
  [ [1;31mFAILED [0m] Failed to start Load Kernel Modules.
  See systemctl status systemd-modules-load.service for details.
  
 
 This is a bug in systemd. It assumes that all modules listed in
 /usr/lib/modules-load.d/ are available in each and every runtime kernel.
 Even grandma knows that this is not true. Just ignore errors from that
 service.

Has someone filed a wishlist bug with systemd?



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running

2015-04-02 Thread Tian, Kevin
 From: Wu, Feng
 Sent: Friday, March 27, 2015 12:58 PM
 
 
 
  -Original Message-
  From: Zhang, Yang Z
  Sent: Friday, March 27, 2015 12:44 PM
  To: Wu, Feng; xen-devel@lists.xen.org
  Cc: jbeul...@suse.com; k...@xen.org; Tian, Kevin
  Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when
 vCPU
  is running
 
  Wu, Feng wrote on 2015-03-27:
  
  
   Zhang, Yang Z wrote on 2015-03-25:
   when vCPU is running
  
   Wu, Feng wrote on 2015-03-25:
   When a vCPU is running in Root mode and a notification event has
   been injected to it. we need to set VCPU_KICK_SOFTIRQ for the
   current cpu, so the pending interrupt in PIRR will be synced to
   vIRR before
   VM-Exit in time.
  
   Shouldn't the pending interrupt be synced unconditionally before next
   vmentry? What happens if we didn't set the softirq?
  
   If we didn't set the softirq in the notification handler, the
   interrupts happened exactly before VM-entry cannot be delivered to
   guest at this time. Please see the following code fragments from
   xen/arch/x86/hvm/vmx/entry.S: (pls pay attention to the comments)
  
   .Lvmx_do_vmentry
  
   ..
 /* If Vt-d engine issues a notification event here,
* it cannot be delivered to guest during this VM-entry
* without raising the softirq in notification handler. */
   cmp  %ecx,(%rdx,%rax,1)
   jnz  .Lvmx_process_softirqs
   ..
  
   je   .Lvmx_launch
   ..
  
  
   .Lvmx_process_softirqs:
   sti
   call do_softirq
   jmp  .Lvmx_do_vmentry
 
  You are right! This helps me to recall why raise the softirq when delivering
 the
  PI.
 
 Yes, __vmx_deliver_posted_interrupt() is the software way to deliver PI, it 
 sets
 the
 softirq for this purpose, however, when VT-d HW delivers PI, we have no
 control to
 the HW itself, hence we need to set this softirq in the Notification Event
 handler.
 

could you include this information in the comment so others can easily
understand this requirement? from code you only mentioned VCPU_KICK
_SOFTIRQ is required, but how it leads to PIRR-VIRR sync is not explained.

Thanks
Kevin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] sudden hang on boot on AMD Notebook

2015-04-02 Thread Olaf Hering
On Wed, Apr 01, Konrad Rzeszutek Wilk wrote:

 You can also use the USB EHCI debug port. It works great on laptops.

Is there an howto for that?

Olaf

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 08/15] Update IRTE according to guest interrupt config changes

2015-04-02 Thread Tian, Kevin
 From: Wu, Feng
 Sent: Thursday, April 02, 2015 2:21 PM
 
 
 
  -Original Message-
  From: Tian, Kevin
  Sent: Thursday, April 02, 2015 1:52 PM
  To: Wu, Feng; xen-devel@lists.xen.org
  Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
  Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config
  changes
 
   From: Wu, Feng
   Sent: Wednesday, March 25, 2015 8:32 PM
  
   When guest changes its interrupt configuration (such as, vector, etc.)
   for direct-assigned devices, we need to update the associated IRTE
   with the new guest vector, so external interrupts from the assigned
   devices can be injected to guests without VM-Exit.
  
   For lowest-priority interrupts, we use vector-hashing mechamisn to find
   the destination vCPU. This follows the hardware behavior, since modern
   Intel CPUs use vector hashing to handle the lowest-priority interrupt.
  
   For multicase/broadcast vCPU, we cannot handle it via interrupt posting,
   still use interrupt remapping.
  
   Signed-off-by: Feng Wu feng...@intel.com
   ---
xen/drivers/passthrough/io.c | 77
   +++-
1 file changed, 76 insertions(+), 1 deletion(-)
  
   diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
   index ae050df..1d9a132 100644
   --- a/xen/drivers/passthrough/io.c
   +++ b/xen/drivers/passthrough/io.c
   @@ -26,6 +26,7 @@
#include asm/hvm/iommu.h
#include asm/hvm/support.h
#include xen/hvm/irq.h
   +#include asm/io_apic.h
  
static DEFINE_PER_CPU(struct list_head, dpci_list);
  
   @@ -199,6 +200,61 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci
 *dpci)
xfree(dpci);
}
  
   +/*
   + * Here we handle the following cases:
   + * - For lowest-priority interrupts, we find the destination vCPU from 
   the
   + *   guest vector using vector-hashing mechamisn and return true. This
   follows
   + *   the hardware behavior, since modern Intel CPUs use vector hashing
 to
   + *   handle the lowest-priority interrupt.
   + * - Otherwise, for single destination interrupt, it is straightforward 
   to
   + *   find the destination vCPU and return true.
   + * - For multicase/broadcast vCPU, we cannot handle it via interrupt
 posting,
   + *   so return false.
   + */
   +static bool_t pi_find_dest_vcpu(struct domain *d, uint8_t dest_id,
   +uint8_t dest_mode, uint8_t
   deliver_mode,
   +uint32_t gvec, struct vcpu
   **dest_vcpu)
   +{
   +struct vcpu *v, **dest_vcpu_array;
   +unsigned int dest_vcpu_num = 0;
   +int ret;
   +
   +if ( deliver_mode == dest_LowestPrio )
   +dest_vcpu_array = xzalloc_array(struct vcpu *, d-max_vcpus);
   +
   +for_each_vcpu ( d, v )
   +{
   +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0,
   +dest_id, dest_mode) )
   +continue;
   +
   +dest_vcpu_num++;
   +
   +if ( deliver_mode == dest_LowestPrio )
   +dest_vcpu_array[dest_vcpu_num] = v;
   +else
   +*dest_vcpu = v;
   +}
   +
   +if ( deliver_mode == dest_LowestPrio )
   +{
   +if (  dest_vcpu_num != 0 )
   +{
   +*dest_vcpu = dest_vcpu_array[gvec % dest_vcpu_num];
   +ret = 1;
   +}
   +else
   +ret = 0;
   +
   +xfree(dest_vcpu_array);
   +return ret;
   +}
   +else if (  dest_vcpu_num == 1 )
   +return 1;
   +else
   +return 0;
   +}
   +
int pt_irq_create_bind(
struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind)
{
   @@ -257,7 +313,7 @@ int pt_irq_create_bind(
{
case PT_IRQ_TYPE_MSI:
{
   -uint8_t dest, dest_mode;
   +uint8_t dest, dest_mode, deliver_mode;
int dest_vcpu_id;
  
if ( !(pirq_dpci-flags  HVM_IRQ_DPCI_MAPPED) )
   @@ -330,11 +386,30 @@ int pt_irq_create_bind(
/* Calculate dest_vcpu_id for MSI-type pirq migration. */
dest = pirq_dpci-gmsi.gflags  VMSI_DEST_ID_MASK;
dest_mode = !!(pirq_dpci-gmsi.gflags  VMSI_DM_MASK);
   +deliver_mode = (pirq_dpci-gmsi.gflags 
   GFLAGS_SHIFT_DELIV_MODE) 
   +VMSI_DELIV_MASK;
dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
pirq_dpci-gmsi.dest_vcpu_id = dest_vcpu_id;
spin_unlock(d-event_lock);
if ( dest_vcpu_id = 0 )
hvm_migrate_pirqs(d-vcpu[dest_vcpu_id]);
   +
   +/* Use interrupt posting if it is supported */
   +if ( iommu_intpost )
   +{
   +struct vcpu *vcpu = NULL;
   +
   +if ( !pi_find_dest_vcpu(d, dest, dest_mode, deliver_mode,
   +pirq_dpci-gmsi.gvec,
 vcpu) )
   +break;
   +
 
  Is it possible this new pi_find_dest_vcpu will return a different 

Re: [Xen-devel] [RFC v1 10/15] vmx: Define two per-cpu variants

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 1:55 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 10/15] vmx: Define two per-cpu variants
 
  From: Wu, Feng
  Sent: Wednesday, March 25, 2015 8:32 PM
 
  This patch defines two per-cpu variants:
 
  blocked_vcpu_on_cpu:
  A list storing the vCPUs which were blocked on this pCPU.
 
  blocked_vcpu_on_cpu_lock:
  The spinlock to protect blocked_vcpu_on_cpu.
 
 since above two are already per-cpu variants, you don't need
 'on_cpu' in the name to duplicate it. How about just call them
 blocked_vcpus and blocked_vcpus_lock? :-)

Sounds great!

Thanks,
Feng

 
 
  Signed-off-by: Feng Wu feng...@intel.com
  ---
   xen/arch/x86/hvm/vmx/vmcs.c   | 3 +++
   xen/arch/x86/hvm/vmx/vmx.c| 7 +++
   xen/include/asm-x86/hvm/vmx/vmx.h | 3 +++
   3 files changed, 13 insertions(+)
 
  diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
  index 942f4b7..1345e69 100644
  --- a/xen/arch/x86/hvm/vmx/vmcs.c
  +++ b/xen/arch/x86/hvm/vmx/vmcs.c
  @@ -585,6 +585,9 @@ int vmx_cpu_up(void)
   if ( cpu_has_vmx_vpid )
   vpid_sync_all();
 
  +INIT_LIST_HEAD(per_cpu(blocked_vcpu_on_cpu, cpu));
  +spin_lock_init(per_cpu(blocked_vcpu_on_cpu_lock, cpu));
  +
   return 0;
   }
 
  diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
  index e1c55ce..ff5544d 100644
  --- a/xen/arch/x86/hvm/vmx/vmx.c
  +++ b/xen/arch/x86/hvm/vmx/vmx.c
  @@ -81,6 +81,13 @@ static int vmx_msr_read_intercept(unsigned int msr,
  uint64_t *msr_content);
   static int vmx_msr_write_intercept(unsigned int msr, uint64_t
 msr_content);
   static void vmx_invlpg_intercept(unsigned long vaddr);
 
  +/*
  + * We maintian a per-CPU linked-list of vCPU, so in PI wakeup handler we
  + * can find which vCPU should be waken up.
  + */
  +DEFINE_PER_CPU(struct list_head, blocked_vcpu_on_cpu);
  +DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
  +
   uint8_t __read_mostly posted_intr_vector;
 
   static int vmx_domain_initialise(struct domain *d)
  diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h
  b/xen/include/asm-x86/hvm/vmx/vmx.h
  index 3cd75eb..e643c3c 100644
  --- a/xen/include/asm-x86/hvm/vmx/vmx.h
  +++ b/xen/include/asm-x86/hvm/vmx/vmx.h
  @@ -30,6 +30,9 @@
   #include asm/hvm/vmx/vmcs.h
   #include asm/apic.h
 
  +DECLARE_PER_CPU(struct list_head, blocked_vcpu_on_cpu);
  +DECLARE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
  +
   extern uint8_t posted_intr_vector;
 
   typedef union {
  --
  2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 08/15] Update IRTE according to guest interrupt config changes

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 1:52 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config
 changes
 
  From: Wu, Feng
  Sent: Wednesday, March 25, 2015 8:32 PM
 
  When guest changes its interrupt configuration (such as, vector, etc.)
  for direct-assigned devices, we need to update the associated IRTE
  with the new guest vector, so external interrupts from the assigned
  devices can be injected to guests without VM-Exit.
 
  For lowest-priority interrupts, we use vector-hashing mechamisn to find
  the destination vCPU. This follows the hardware behavior, since modern
  Intel CPUs use vector hashing to handle the lowest-priority interrupt.
 
  For multicase/broadcast vCPU, we cannot handle it via interrupt posting,
  still use interrupt remapping.
 
  Signed-off-by: Feng Wu feng...@intel.com
  ---
   xen/drivers/passthrough/io.c | 77
  +++-
   1 file changed, 76 insertions(+), 1 deletion(-)
 
  diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
  index ae050df..1d9a132 100644
  --- a/xen/drivers/passthrough/io.c
  +++ b/xen/drivers/passthrough/io.c
  @@ -26,6 +26,7 @@
   #include asm/hvm/iommu.h
   #include asm/hvm/support.h
   #include xen/hvm/irq.h
  +#include asm/io_apic.h
 
   static DEFINE_PER_CPU(struct list_head, dpci_list);
 
  @@ -199,6 +200,61 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci)
   xfree(dpci);
   }
 
  +/*
  + * Here we handle the following cases:
  + * - For lowest-priority interrupts, we find the destination vCPU from the
  + *   guest vector using vector-hashing mechamisn and return true. This
  follows
  + *   the hardware behavior, since modern Intel CPUs use vector hashing to
  + *   handle the lowest-priority interrupt.
  + * - Otherwise, for single destination interrupt, it is straightforward to
  + *   find the destination vCPU and return true.
  + * - For multicase/broadcast vCPU, we cannot handle it via interrupt 
  posting,
  + *   so return false.
  + */
  +static bool_t pi_find_dest_vcpu(struct domain *d, uint8_t dest_id,
  +uint8_t dest_mode, uint8_t
  deliver_mode,
  +uint32_t gvec, struct vcpu
  **dest_vcpu)
  +{
  +struct vcpu *v, **dest_vcpu_array;
  +unsigned int dest_vcpu_num = 0;
  +int ret;
  +
  +if ( deliver_mode == dest_LowestPrio )
  +dest_vcpu_array = xzalloc_array(struct vcpu *, d-max_vcpus);
  +
  +for_each_vcpu ( d, v )
  +{
  +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0,
  +dest_id, dest_mode) )
  +continue;
  +
  +dest_vcpu_num++;
  +
  +if ( deliver_mode == dest_LowestPrio )
  +dest_vcpu_array[dest_vcpu_num] = v;
  +else
  +*dest_vcpu = v;
  +}
  +
  +if ( deliver_mode == dest_LowestPrio )
  +{
  +if (  dest_vcpu_num != 0 )
  +{
  +*dest_vcpu = dest_vcpu_array[gvec % dest_vcpu_num];
  +ret = 1;
  +}
  +else
  +ret = 0;
  +
  +xfree(dest_vcpu_array);
  +return ret;
  +}
  +else if (  dest_vcpu_num == 1 )
  +return 1;
  +else
  +return 0;
  +}
  +
   int pt_irq_create_bind(
   struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind)
   {
  @@ -257,7 +313,7 @@ int pt_irq_create_bind(
   {
   case PT_IRQ_TYPE_MSI:
   {
  -uint8_t dest, dest_mode;
  +uint8_t dest, dest_mode, deliver_mode;
   int dest_vcpu_id;
 
   if ( !(pirq_dpci-flags  HVM_IRQ_DPCI_MAPPED) )
  @@ -330,11 +386,30 @@ int pt_irq_create_bind(
   /* Calculate dest_vcpu_id for MSI-type pirq migration. */
   dest = pirq_dpci-gmsi.gflags  VMSI_DEST_ID_MASK;
   dest_mode = !!(pirq_dpci-gmsi.gflags  VMSI_DM_MASK);
  +deliver_mode = (pirq_dpci-gmsi.gflags 
  GFLAGS_SHIFT_DELIV_MODE) 
  +VMSI_DELIV_MASK;
   dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
   pirq_dpci-gmsi.dest_vcpu_id = dest_vcpu_id;
   spin_unlock(d-event_lock);
   if ( dest_vcpu_id = 0 )
   hvm_migrate_pirqs(d-vcpu[dest_vcpu_id]);
  +
  +/* Use interrupt posting if it is supported */
  +if ( iommu_intpost )
  +{
  +struct vcpu *vcpu = NULL;
  +
  +if ( !pi_find_dest_vcpu(d, dest, dest_mode, deliver_mode,
  +pirq_dpci-gmsi.gvec, vcpu) )
  +break;
  +
 
 Is it possible this new pi_find_dest_vcpu will return a different target from
 earlier hvm_girq_des_2_vcpu_id? if yes it will cause tricky issues since
 earlier pirqs are migrated according to different policy. We need consolidate
 vcpu selection policies together to keep 

Re: [Xen-devel] ENC:[Xen-users] Failed to insert 'xen_evtchn': No such device

2015-04-02 Thread Olaf Hering
On Wed, Apr 01, Carlos Gustavo Ramirez Rodriguez wrote:

 Hello DEVs, can anyone help me?
 
 Ian Campbell pointed me that maybe you could help me, here is the link to the
 last e-mail line that started this problem fixing:
 http://lists.xenproject.org/archives/html/xen-users/2015-04/msg9.html
 
 A brief summary of my current problems after XEN installation:
 
 First Problem: XL Anything, be it INFO, TOP, UPTIME, whatever, returns:
 
 
 xc: error: Could not obtain handle on privileged command interface (2 = No 
 such
 file or directory): Internal error
 libxl: error: libxl.c:109:libxl_ctx_alloc: cannot open libxc handle: No such
 file or directory
 cannot init xl context
 

Which systemd services files from xen are enabled? See the systemd part
of the INSTALL file:
http://xenbits.xen.org/gitweb/?p=xen.git;a=blob_plain;f=INSTALL;hb=staging

systemctl enable xen-qemu-dom0-disk-backend.service
systemctl enable xen-init-dom0.service
systemctl enable xenconsoled.service

 Second Problem:
 My boot.log is showing:
 
 
 [ [1;31mFAILED [0m] Failed to start Load Kernel Modules.
 See systemctl status systemd-modules-load.service for details.
 

This is a bug in systemd. It assumes that all modules listed in
/usr/lib/modules-load.d/ are available in each and every runtime kernel.
Even grandma knows that this is not true. Just ignore errors from that
service.

Olaf

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [qemu-upstream-4.3-testing test] 50282: tolerable FAIL - PUSHED

2015-04-02 Thread osstest service user
flight 50282 qemu-upstream-4.3-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50282/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 debian-hvm-install fail never pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 debian-hvm-install  fail never pass
 test-amd64-i386-libvirt  10 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 10 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-i386-xend-qemut-winxpsp3 17 leak-check/checkfail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xend-winxpsp3 17 leak-check/check fail  never pass

version targeted for testing:
 qemuu7f34050dc014ae8f4078d48aec97ec6553151bf2
baseline version:
 qemuuab689a89ec47b2e1c964c57bea7da68f8ddf89fd


People who touched revisions under test:
  Ian Campbell ian.campb...@citrix.com
  Jan Beulich jbeul...@suse.com


jobs:
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-i386-rhel6hvm-amd pass
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64pass
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail
 test-amd64-amd64-xl-qemut-win7-amd64 fail
 test-amd64-i386-xl-qemut-win7-amd64  fail
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-win7-amd64   fail
 test-amd64-i386-xl-win7-amd64fail
 test-amd64-amd64-xl-credit2  pass
 test-amd64-i386-freebsd10-i386   pass
 test-amd64-i386-rhel6hvm-intel   pass
 test-amd64-i386-qemut-rhel6hvm-intel pass
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-amd64-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-xl-multivcpupass
 test-amd64-amd64-pairpass
 test-amd64-i386-pair pass
 test-amd64-amd64-xl-sedf-pin pass
 test-amd64-amd64-pv  pass
 test-amd64-i386-pv   pass
 test-amd64-amd64-xl-sedf pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 fail
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
 test-amd64-i386-xl-winxpsp3-vcpus1   fail
 test-amd64-i386-xend-qemut-winxpsp3  

Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 2:08 PM
 To: Wu, Feng; Zhang, Yang Z; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org
 Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when vCPU
 is running
 
  From: Wu, Feng
  Sent: Friday, March 27, 2015 12:58 PM
 
 
 
   -Original Message-
   From: Zhang, Yang Z
   Sent: Friday, March 27, 2015 12:44 PM
   To: Wu, Feng; xen-devel@lists.xen.org
   Cc: jbeul...@suse.com; k...@xen.org; Tian, Kevin
   Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when
  vCPU
   is running
  
   Wu, Feng wrote on 2015-03-27:
   
   
Zhang, Yang Z wrote on 2015-03-25:
when vCPU is running
   
Wu, Feng wrote on 2015-03-25:
When a vCPU is running in Root mode and a notification event has
been injected to it. we need to set VCPU_KICK_SOFTIRQ for the
current cpu, so the pending interrupt in PIRR will be synced to
vIRR before
VM-Exit in time.
   
Shouldn't the pending interrupt be synced unconditionally before next
vmentry? What happens if we didn't set the softirq?
   
If we didn't set the softirq in the notification handler, the
interrupts happened exactly before VM-entry cannot be delivered to
guest at this time. Please see the following code fragments from
xen/arch/x86/hvm/vmx/entry.S: (pls pay attention to the comments)
   
.Lvmx_do_vmentry
   
..
/* If Vt-d engine issues a notification event here,
 * it cannot be delivered to guest during this VM-entry
 * without raising the softirq in notification handler. */
cmp  %ecx,(%rdx,%rax,1)
jnz  .Lvmx_process_softirqs
..
   
je   .Lvmx_launch
..
   
   
.Lvmx_process_softirqs:
sti
call do_softirq
jmp  .Lvmx_do_vmentry
  
   You are right! This helps me to recall why raise the softirq when 
   delivering
  the
   PI.
 
  Yes, __vmx_deliver_posted_interrupt() is the software way to deliver PI, it
 sets
  the
  softirq for this purpose, however, when VT-d HW delivers PI, we have no
  control to
  the HW itself, hence we need to set this softirq in the Notification Event
  handler.
 
 
 could you include this information in the comment so others can easily
 understand this requirement? from code you only mentioned VCPU_KICK
 _SOFTIRQ is required, but how it leads to PIRR-VIRR sync is not explained.
 

No problem!

Thanks,
Feng

 Thanks
 Kevin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 08/15] Update IRTE according to guest interrupt config changes

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 2:50 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config
 changes
 
  From: Wu, Feng
  Sent: Thursday, April 02, 2015 2:21 PM
 
 
 
   -Original Message-
   From: Tian, Kevin
   Sent: Thursday, April 02, 2015 1:52 PM
   To: Wu, Feng; xen-devel@lists.xen.org
   Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
   Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt 
   config
   changes
  
From: Wu, Feng
Sent: Wednesday, March 25, 2015 8:32 PM
   
When guest changes its interrupt configuration (such as, vector, etc.)
for direct-assigned devices, we need to update the associated IRTE
with the new guest vector, so external interrupts from the assigned
devices can be injected to guests without VM-Exit.
   
For lowest-priority interrupts, we use vector-hashing mechamisn to find
the destination vCPU. This follows the hardware behavior, since modern
Intel CPUs use vector hashing to handle the lowest-priority interrupt.
   
For multicase/broadcast vCPU, we cannot handle it via interrupt posting,
still use interrupt remapping.
   
Signed-off-by: Feng Wu feng...@intel.com
---
 xen/drivers/passthrough/io.c | 77
+++-
 1 file changed, 76 insertions(+), 1 deletion(-)
   
diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
index ae050df..1d9a132 100644
--- a/xen/drivers/passthrough/io.c
+++ b/xen/drivers/passthrough/io.c
@@ -26,6 +26,7 @@
 #include asm/hvm/iommu.h
 #include asm/hvm/support.h
 #include xen/hvm/irq.h
+#include asm/io_apic.h
   
 static DEFINE_PER_CPU(struct list_head, dpci_list);
   
@@ -199,6 +200,61 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci
  *dpci)
 xfree(dpci);
 }
   
+/*
+ * Here we handle the following cases:
+ * - For lowest-priority interrupts, we find the destination vCPU from 
the
+ *   guest vector using vector-hashing mechamisn and return true. This
follows
+ *   the hardware behavior, since modern Intel CPUs use vector
 hashing
  to
+ *   handle the lowest-priority interrupt.
+ * - Otherwise, for single destination interrupt, it is 
straightforward to
+ *   find the destination vCPU and return true.
+ * - For multicase/broadcast vCPU, we cannot handle it via interrupt
  posting,
+ *   so return false.
+ */
+static bool_t pi_find_dest_vcpu(struct domain *d, uint8_t dest_id,
+uint8_t dest_mode, uint8_t
deliver_mode,
+uint32_t gvec, struct vcpu
**dest_vcpu)
+{
+struct vcpu *v, **dest_vcpu_array;
+unsigned int dest_vcpu_num = 0;
+int ret;
+
+if ( deliver_mode == dest_LowestPrio )
+dest_vcpu_array = xzalloc_array(struct vcpu *, d-max_vcpus);
+
+for_each_vcpu ( d, v )
+{
+if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0,
+dest_id, dest_mode) )
+continue;
+
+dest_vcpu_num++;
+
+if ( deliver_mode == dest_LowestPrio )
+dest_vcpu_array[dest_vcpu_num] = v;
+else
+*dest_vcpu = v;
+}
+
+if ( deliver_mode == dest_LowestPrio )
+{
+if (  dest_vcpu_num != 0 )
+{
+*dest_vcpu = dest_vcpu_array[gvec % dest_vcpu_num];
+ret = 1;
+}
+else
+ret = 0;
+
+xfree(dest_vcpu_array);
+return ret;
+}
+else if (  dest_vcpu_num == 1 )
+return 1;
+else
+return 0;
+}
+
 int pt_irq_create_bind(
 struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind)
 {
@@ -257,7 +313,7 @@ int pt_irq_create_bind(
 {
 case PT_IRQ_TYPE_MSI:
 {
-uint8_t dest, dest_mode;
+uint8_t dest, dest_mode, deliver_mode;
 int dest_vcpu_id;
   
 if ( !(pirq_dpci-flags  HVM_IRQ_DPCI_MAPPED) )
@@ -330,11 +386,30 @@ int pt_irq_create_bind(
 /* Calculate dest_vcpu_id for MSI-type pirq migration. */
 dest = pirq_dpci-gmsi.gflags  VMSI_DEST_ID_MASK;
 dest_mode = !!(pirq_dpci-gmsi.gflags  VMSI_DM_MASK);
+deliver_mode = (pirq_dpci-gmsi.gflags 
GFLAGS_SHIFT_DELIV_MODE) 
+VMSI_DELIV_MASK;
 dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest,
 dest_mode);
 pirq_dpci-gmsi.dest_vcpu_id = dest_vcpu_id;
 spin_unlock(d-event_lock);
 if ( dest_vcpu_id = 0 )
 

[Xen-devel] [PATCH OSSTEST v4 17/25] standalone: propagate result of command from with_logging

2015-04-02 Thread Ian Campbell
Such that the overall script gets the return code of the test/job
which was run.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 standalone | 1 +
 1 file changed, 1 insertion(+)

diff --git a/standalone b/standalone
index caf3fd5..3189e84 100755
--- a/standalone
+++ b/standalone
@@ -156,6 +156,7 @@ with_logging() {
 if [ $rc -ne 0 ] ; then
echo FAILED rc=${rc} 2
 fi
+return $rc
 }
 
 # other potential ops:
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 19/25] make-flight: refactor PV debian tests

2015-04-02 Thread Ian Campbell
No functional change, standalong-generate-dump-flight-runvars confirms
no change to the runvars.

Includes a hook which is not used yet, $recipe_sfx.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v4: new patch
---
 make-flight | 24 ++--
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/make-flight b/make-flight
index ca48b81..909cebe 100755
--- a/make-flight
+++ b/make-flight
@@ -339,6 +339,17 @@ do_pvgrub_tests () {
   all_hostflags=$most_hostflags
 }
 
+do_pv_debian_test_one () {
+  testname=$1; shift
+  recipe_sfx=$1; shift
+  toolstack=$1; shift
+
+  job_create_test test-$xenarch$kern-$dom0arch-$testname\
+ test-debian$recipe_sfx $toolstack  \
+$xenarch $dom0arch  \
+$debian_runvars all_hostflags=$most_hostflags $@
+}
+
 do_pv_debian_tests () {
   xsms=$(xenbranch_xsm_variants)
 
@@ -354,20 +365,13 @@ do_pv_debian_tests () {
   suffix=${platform:+-$platform}
   hostflags=${most_hostflags}${platform:+,platform-$platform}
 
-  job_create_test test-$xenarch$kern-$dom0arch-xl$suffix   \
-  test-debian xl   \
-  $xenarch $dom0arch   \
-  enable_xsm=$xsm  \
-  $debian_runvars all_hostflags=$hostflags
+  do_pv_debian_test_one xl$suffix '' xl enable_xsm=$xsm
+
 done
   done
 
   for xsm in $xsms ; do
-job_create_test test-$xenarch$kern-$dom0arch-libvirt \
-test-debian libvirt  \
-$xenarch $dom0arch   \
-enable_xsm=$xsm  \
-$debian_runvars all_hostflags=$most_hostflags
+do_pv_debian_test_one libvirt '' libvirt enable_xsm=$xsm
   done
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 12/25] distros: support PV guest install from Debian netinst media.

2015-04-02 Thread Ian Campbell
The netinst media are iso images containing a base Debian install and
some (image size dependent) additional tasks.

On x86 the multiarch iso flavour contains a Xen capable kernel for
both i386 and amd64 so use that.

This adds support for two classes of ISO, the CD sized ones which are
built nightly and the DVD sized ones which are built weekly.

The images are downloaded using jigdo which sources the majority of
the data from a local Debian mirror, for this reason I have not
worried about the fact that the i386 and amd64 tests are downloading
the same thing (adding a specific download job would require finding
up to 4GB of scratch space for each flight).

The ISOs booted using pygrub which can extract the kernel and initrd
from a ISO image. The resulting guests are also booted with pygrub
since the pv-grub-menu package is not available on the ISO images and
we have pvgrub coverage from the netboot tests.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v4: Support http_proxy in jigdo download
v3:
  - Indent jigdo-lite script line to improve readability
  - Wrap bootloader_args
  - Fetch URL on target, to get a timeout. Use wget since that is arranged to
be present.
  - include bootloader in test name.
---
 Osstest/Debian.pm|  7 --
 make-distros-flight  | 23 ++
 ts-debian-di-install | 67 
 3 files changed, 95 insertions(+), 2 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index b37540e..831290c 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -616,8 +616,6 @@ sub preseed_base ($$$;@) {
 $xopts{ExtraPreseed} ||= '';
 
 my $preseed = END;
-d-i mirror/suite string $suite
-
 d-i debian-installer/locale string en_GB
 d-i console-keymaps-at/keymap select gb
 d-i keyboard-configuration/xkb-keymap string en_GB
@@ -689,6 +687,11 @@ END
 d-i clock-setup/ntp-server string $ntpserver
 END
 
+# For CDROM the suite is part of the image
+$preseed .= END unless $xopts{CDROM};
+d-i mirror/suite string $suite
+END
+
 # deb http://ftp.debian.org/debian/ wheezy-backports main
 $preseed .= END if $extra_packages =~ m#\b/$suite-backports\b#;
 d-i apt-setup/local0/repository string \\
diff --git a/make-distros-flight b/make-distros-flight
index 97df89a..e3ec8f7 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -85,6 +85,20 @@ test_do_one_netboot () {
   all_hostflags=$most_hostflags
 }
 
+test_do_one_netinst () {
+  # Always pygrub since no pv-grub-menu on CD
+  job_create_test   \
+   test-$xenarch$kern-$dom0arch-$domU-$cd-netinst-pygrub\
+test-debian-di xl $xenarch $dom0arch\
+  kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
+  debian_arch=$domU \
+  debian_cd=$cd \
+  debian_method=netinst \
+  debian_bootloader=pygrub  \
+  all_hostflags=$most_hostflags
+
+}
+
 test_matrix_do_one () {
   case ${xenarch} in
   amd64) domUarches=amd64 i386;;
@@ -103,6 +117,15 @@ test_matrix_do_one () {
 
 done
 
+for cd in current weekly ; do
+  case ${domU}_${dist} in
+  armhf_*) continue;; # No iso targets for armhf
+  *) ;;
+  esac
+
+  test_do_one_netinst
+
+done
   done
 }
 
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 73cb43d..62e9bb5 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -46,6 +46,63 @@ sub prep () {
 target_cmd_root($ho, umount $gho-{Lvdev} ||:);
 }
 
+sub setup_netinst($$$)
+{
+my ($didir, $arch, $cd) = @_;
+my %arch_props = (
+   amd64 = { PathArch = multi-arch, FileArch = amd64-i386, IsoPath 
= /install.amd/xen },
+   i386  = { PathArch = multi-arch, FileArch = amd64-i386, IsoPath 
= /install.386/xen },
+   armhf = { PathArch = armhf,  FileArch = armhf,  IsoPath 
= /install.armhf }
+);
+my $props = $arch_props{$arch} or die Unknown arch $arch;
+
+target_install_packages($ho, qw(jigdo-file));
+
+my $baseurl = $cd eq current ?
+  
http://cdimage.debian.org/debian-cd/current/$props-{PathArch}/jigdo-cd :
+  
http://cdimage.debian.org/cdimage/weekly-builds/$props-{PathArch}/jigdo-cd;
+
+my $filebase;
+
+# Use the MD5SUMs file as an index
+logm(Fetch index from $baseurl/MD5SUMS);
+my $idx = target_cmd_output_root($ho, wget --quiet -O - 
$baseurl/MD5SUMS);
+foreach (split /\n/, $idx) {
+   m/^[0-9a-f]{32}  (debian-.*-$props-{FileArch}-netinst)\.iso$/ or next;
+   $filebase = $1;
+   last;
+}
+
+die unless $filebase;
+
+logm(Downloading $baseurl/$filebase.jigdo);
+my $netinst_jigdo = $baseurl/$filebase.jigdo;
+# Jigdo uses wget internally, and so obeys $http_proxy. This seems
+# simpler than the advice at
+# 

[Xen-devel] [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.

2015-04-02 Thread Ian Campbell
In particular make the path unique by ensuring it includes the host
and guest name in the guest case.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3: New patch
---
 Osstest/TestSupport.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 490e16d..8f67f7e 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1979,6 +1979,8 @@ sub await_webspace_fetch_byleaf ($) {
 sub create_webfile ($$$) {
 my ($ho, $tail, $contents) = @_; # $contents as for file_link_contents
 my $wf_rhs= $ho-{Name}._.$tail;
+# $ho-{Host} is set if $ho is a guest.
+$wf_rhs= $ho-{Host}-{Name}._${wf_rhs} if $ho-{Host};
 my $wf_common= $c{WebspaceCommon}.$wf_rhs;
 my $wf_url= $c{WebspaceUrl}.$wf_common;
 my $wf_file= $c{WebspaceFile}.$wf_common;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 21/25] mfi-common: Allow make-*flight to filter the set of build jobs to include

2015-04-02 Thread Ian Campbell
By using the same job_create_build(_filter_callback) scheme used for
the test jobs.

Will be used in make-distros-flight.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 make-distros-flight |  4 
 make-flight |  4 
 mfi-common  | 21 +++--
 3 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/make-distros-flight b/make-distros-flight
index e3ec8f7..2724dd2 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -33,6 +33,10 @@ flight=`./cs-flight-create $blessing $branch`
 defsuite=`getconfig DebianSuite`
 defguestsuite=`getconfig GuestDebianSuite`
 
+job_create_build_filter_callback () {
+:
+}
+
 if [ x$buildflight = x ]; then
 
   WANT_XEND=false REVISION_LINUX_OLD=disable
diff --git a/make-flight b/make-flight
index c923f21..ef1e422 100755
--- a/make-flight
+++ b/make-flight
@@ -34,6 +34,10 @@ flight=`./cs-flight-create $blessing $branch`
 defsuite=`getconfig DebianSuite`
 defguestsuite=`getconfig GuestDebianSuite`
 
+job_create_build_filter_callback () {
+:
+}
+
 if [ x$buildflight = x ]; then
 
   create_build_jobs
diff --git a/mfi-common b/mfi-common
index a9e966f..a100afb 100644
--- a/mfi-common
+++ b/mfi-common
@@ -54,6 +54,15 @@ xenbranch_xsm_variants () {
 esac
 }
 
+job_create_build () {
+  job_create_build_filter_callback $@ || return 0
+
+  local job=$1; shift
+  local recipe=$1; shift
+
+  ./cs-job-create $flight $job $recipe $@
+}
+
 create_build_jobs () {
 
   local arch
@@ -164,7 +173,7 @@ create_build_jobs () {
   else
 xsm_suffix=
   fi
-  ./cs-job-create $flight build-$arch$xsm_suffix build   \
+  job_create_build build-$arch$xsm_suffix build  \
 arch=$arch enable_xend=$build_defxend enable_ovmf=$enable_ovmf\
 enable_xsm=$enable_xsm   \
 tree_qemu=$TREE_QEMU \
@@ -183,7 +192,7 @@ create_build_jobs () {
 done
 
 if [ $build_extraxend = true ] ; then
-./cs-job-create $flight build-$arch-xend build   \
+job_create_build build-$arch-xend build  \
 arch=$arch enable_xend=true enable_ovmf=$enable_ovmf \
 tree_qemu=$TREE_QEMU \
 tree_qemuu=$TREE_QEMU_UPSTREAM   \
@@ -196,7 +205,7 @@ create_build_jobs () {
 revision_qemuu=$REVISION_QEMU_UPSTREAM
 fi
 
-./cs-job-create $flight build-$arch-pvops build-kern \
+job_create_build build-$arch-pvops build-kern\
 arch=$arch kconfighow=xen-enable-xen-config  \
 $RUNVARS $BUILD_RUNVARS $BUILD_LINUX_RUNVARS $arch_runvars   \
 $suite_runvars   \
@@ -208,7 +217,7 @@ create_build_jobs () {
 
 if [ x$REVISION_LIBVIRT != xdisable ]; then
 
-./cs-job-create $flight build-$arch-libvirt build-libvirt\
+job_create_build build-$arch-libvirt build-libvirt   \
 arch=$arch   \
 tree_xen=$TREE_XEN   \
 $RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \
@@ -223,7 +232,7 @@ create_build_jobs () {
 
 case $arch in
 i386|amd64)
-./cs-job-create $flight build-$arch-rumpuserxen build-rumpuserxen\
+job_create_build build-$arch-rumpuserxen build-rumpuserxen   \
 arch=$arch   \
 tree_xen=$TREE_XEN   \
 $RUNVARS $BUILD_RUNVARS $BUILD_RUMPUSERXEN_RUNVARS 
$arch_runvars \
@@ -252,7 +261,7 @@ create_build_jobs () {
 
 if [ x$REVISION_LINUX_OLD != xdisable ]; then
 
-  ./cs-job-create $flight build-$arch-oldkern build-kern\
+  job_create_build build-$arch-oldkern build-kern   \
 arch=$arch kconfighow=create-config-sh  \
 kimagefile=vmlinux  \
 $RUNVARS $BUILD_RUNVARS $BUILD_LINUX_OLD_RUNVARS\
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 13/25] Test pygrub and pvgrub on the regular flights

2015-04-02 Thread Ian Campbell
Since we now have the ability to test these drop one of each of
pygrub, pvgrub-32 and pvgrub-64 into the standard flights. Omitting
the {Guest}_diver runvar causes ts-debian-di-install to use the d-i
images in the location configured via TftpDiVersion, so they are
Version Controlled along with the d-i version used for the host.

This adds three new jobs:
test-amd64-amd64-amd64-pvgrub:
all_hostflags 
arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch  amd64
buildjob  build-amd64
debian_arch   amd64
debian_bootloader pvgrub
debian_dist   wheezy
debian_method netboot
kernbuildjob  build-amd64-pvops
kernkind  pvops
toolstack xl
xenbuildjob   build-amd64
test-amd64-amd64-i386-pvgrub:
all_hostflags 
arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch  amd64
buildjob  build-amd64
debian_arch   i386
debian_bootloader pvgrub
debian_dist   wheezy
debian_method netboot
kernbuildjob  build-amd64-pvops
kernkind  pvops
toolstack xl
xenbuildjob   build-amd64
test-amd64-amd64-pygrub:
all_hostflags 
arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch  amd64
buildjob  build-amd64
debian_arch   amd64
debian_bootloader pygrub
debian_dist   wheezy
debian_method netboot
kernbuildjob  build-amd64-pvops
kernkind  pvops
toolstack xl
xenbuildjob   build-amd64

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3: added runvar details
---
 make-flight | 39 +++
 1 file changed, 39 insertions(+)

diff --git a/make-flight b/make-flight
index ec0a20c..ca48b81 100755
--- a/make-flight
+++ b/make-flight
@@ -303,6 +303,42 @@ do_passthrough_tests () {
   done
 }
 
+do_pygrub_tests () {
+  if [ $xenarch != amd64 -o $dom0arch != amd64 -o $kern !=  ]; then
+return
+  fi
+
+  job_create_test test-$xenarch$kern-$dom0arch-pygrub   \
+test-debian-di xl $xenarch $dom0arch\
+  debian_arch=amd64 \
+  debian_dist=$guestsuite   \
+  debian_method=netboot \
+  debian_bootloader=pygrub  \
+  all_hostflags=$most_hostflags
+}
+
+do_pvgrub_tests () {
+  if [ $xenarch != amd64 -o $dom0arch != amd64 -o $kern !=  ]; then
+return
+  fi
+
+  job_create_test test-$xenarch$kern-$dom0arch-amd64-pvgrub \
+test-debian-di xl $xenarch $dom0arch\
+  debian_arch=amd64 \
+  debian_dist=$guestsuite   \
+  debian_method=netboot \
+  debian_bootloader=pvgrub  \
+  all_hostflags=$most_hostflags \
+
+  job_create_test test-$xenarch$kern-$dom0arch-i386-pvgrub  \
+test-debian-di xl $xenarch $dom0arch\
+  debian_arch=i386  \
+  debian_dist=$guestsuite   \
+  debian_method=netboot \
+  debian_bootloader=pvgrub  \
+  all_hostflags=$most_hostflags
+}
+
 do_pv_debian_tests () {
   xsms=$(xenbranch_xsm_variants)
 
@@ -427,6 +463,9 @@ test_matrix_do_one () {
 
   fi
   #do_passthrough_tests
+
+  do_pygrub_tests
+  do_pvgrub_tests
 }
 
 test_matrix_iterate
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper

2015-04-02 Thread Ian Campbell
Creates a preseed file suitable for use in a PV guest

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v4: Rebase, pass $ho to preseed_base
v3: Handle $xopts{ExtraPreseed} undefined in preseed_base
---
 Osstest/Debian.pm | 29 +
 1 file changed, 29 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 13cd147..993acc7 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,6 +37,7 @@ BEGIN {
   %preseed_cmds
   preseed_base
   preseed_create
+  preseed_create_guest
   preseed_ssh
   preseed_hook_command preseed_hook_installscript
   preseed_hook_overlay
@@ -611,6 +612,9 @@ END
 sub preseed_base ($$$;@) {
 my ($ho,$suite,$extra_packages,%xopts) = @_;
 
+$extra_packages ||= '';
+$xopts{ExtraPreseed} ||= '';
+
 my $preseed = END;
 d-i mirror/suite string $suite
 
@@ -693,6 +697,31 @@ END
 return $preseed;
 }
 
+sub preseed_create_guest ($$;@) {
+my ($ho, $sfx, %xopts) = @_;
+
+my $suite= $xopts{Suite} || $c{DebianSuite};
+
+my $extra_packages;
+
+my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts);
+$preseed_file.= (END);
+d-i partman-auto/method string regular
+d-i partman-auto/choose_recipe \\
+select All files in one partition (recommended for new users)
+
+d-i grub-installer/bootdev  string /dev/xvda
+
+END
+
+preseed_ssh($ho, $sfx);
+preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
+
+$preseed_file .= preseed_hook_cmds();
+
+return create_webfile($ho, preseed$sfx, $preseed_file);
+}
+
 sub preseed_create ($$;@) {
 my ($ho, $sfx, %xopts) = @_;
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 10/25] distros: support booting Debian PV (d-i installed) guests with pvgrub.

2015-04-02 Thread Ian Campbell
This requires the use of the pv-grub-menu package which is in Jessie
onwards.  (it is in wheezy-backports which is the subject of a
subsequent patch).

The bootloader to use is specified via a runvar {Guest}_bootloader.

Adjust make-distros-flight to use pvgrub for some subset of i386 and
amd64 guests to get coverage.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3: Define and use arch_debian2xen and arch_xen2debian
Avoid pv-grub-x86_64.gz on i386 dom0, we don't built it there.
Fiddle with py vs pv grub stripy a bit.
---
 Osstest.pm   |  7 +++
 Osstest/Debian.pm|  2 +-
 make-distros-flight  | 21 -
 ts-debian-di-install | 11 +--
 4 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/Osstest.pm b/Osstest.pm
index 7989129..c0e3ee7 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -38,6 +38,7 @@ BEGIN {
   db_begin_work
   ensuredir get_filecontents_core_quiet system_checked
   nonempty visible_undef show_abs_time
+  %arch_debian2xen %arch_xen2debian
   );
 %EXPORT_TAGS = ( );
 
@@ -49,6 +50,12 @@ our $mjobdb;
 
 our $dbh_tests;
 
+our %arch_debian2xen = qw(i386 x86_32
+ amd64 x86_64
+ armhf armhf);
+our %arch_xen2debian;
+$arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen;
+
 #-- static default config settings --
 
 our %c = qw(
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 993acc7..cbc7172 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -702,7 +702,7 @@ sub preseed_create_guest ($$;@) {
 
 my $suite= $xopts{Suite} || $c{DebianSuite};
 
-my $extra_packages;
+my $extra_packages = pv-grub-menu if $xopts{PvMenuLst};
 
 my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts);
 $preseed_file.= (END);
diff --git a/make-distros-flight b/make-distros-flight
index 5067342..80df21b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -55,14 +55,33 @@ test_matrix_branch_filter_callback () {
 }
 
 test_do_one_netboot () {
+  stripy bootloader pvgrub pygrub \
+$xenarch amd64 \
+$dom0arch i386 \
+$domU amd64 \
+$dist sid
+
+  case ${dom0arch}_${domU}_${dist} in
+arm*_arm*_*) bootloader=pygrub;; # no pvgrub for arm
+
+# Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
+# workedaround in Jessie+ with pv-grub-menu package.
+*_squeeze) bootloader=pygrub;;
+*_wheezy) bootloader=pygrub;;
+
+# pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build.
+i386_amd64_*) bootloader=pygrub;;
+  esac
+
   job_create_test   \
-   test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub  \
+   test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-$bootloader \
 test-debian-di xl $xenarch $dom0arch\
   kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
   debian_arch=$domU \
   debian_dist=$dist \
   debian_method=netboot \
   debian_diver=current  \
+  debian_bootloader=$bootloader \
   all_hostflags=$most_hostflags
 }
 
diff --git a/ts-debian-di-install b/ts-debian-di-install
index a829185..73cb43d 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -93,6 +93,8 @@ sub ginstall () {
 my $method= $r{$gho-{Guest}_method};
 
 my $tmpdir= /root/$flight-$job-di;
+my $bl= $r{$gho-{Guest}_bootloader};
+
 target_cmd_root($ho, END);
 rm -rf $tmpdir
 mkdir $tmpdir
@@ -109,7 +111,7 @@ END
 
$suite = sid if $suite eq daily;
 
-   $ps_url = preseed_create_guest($gho, '', Suite=$suite);
+   $ps_url = preseed_create_guest($gho, '', Suite=$suite, PvMenuLst=($bl 
eq pvgrub));
 
$extra_disk = ;
 }
@@ -150,7 +152,12 @@ END
 guest_await_shutdown($ho,$gho,3600);
 guest_destroy($gho);
 
-my $blcfg = END;
+my $xenarch = $arch_debian2xen{$arch};
+my $pvgrub = /usr/local/lib/xen/boot/pv-grub-$xenarch.gz;
+my $blcfg = $bl eq pvgrub ? END : END;
+kernel = $pvgrub
+extra = (hd0,0)/boot/grub/menu.lst
+END
 bootloader = pygrub
 END
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 20/25] Add testing of non-LVM/phy disk backends.

2015-04-02 Thread Ian Campbell
xen-create-image makes this tricky to do since it is rather LVM
centric. Now that we have the ability to install from d-i it's
possible to arrange fairly easily that they use non-LVM disk backend
arrangements.

Here we add support to the test script and infra and create a bunch of
new jobs testing the cross product of {xl,libvirt} x {raw,qcow2,vhd}.

The test scripts are modified such that when constructing a domain
with a non-LVM diskfmt runvar:
 - the LVM device is slightly enlarged to account for file format
   headers (1M should be plenty).
 - the LVM device will have an ext3 filesystem created on it instead
   of being used as a phy device for the guest. Reusing the LVM volume
   in this way means we don't need to do more storage management in
   dom0 (i.e. arranging for / to be large enough, or managing a
   special images LV)
 - the relevant type of container is created within the filesystem
   using the appropriate tool.
 - New properties Disk{fmt,spec} are added to all $gho, containing
   the format used for the root disk and the xl diskspec to load it.
 - lvm backed guests use a xend/xm compatible spec, everything
   else uses the improved xl syntax which libvirt also supports.
   We won't test non-LVM on xend.
 - New properties Disk{mnt,img} are added to $gho which are not using
   LVM. These contain the mount point to use (configurable via
   OSSTEST_CONFIG and runvars) and the full path (including mount
   point) to the image itself.
 - When starting or stopping a guest we arrange for the filesystem to
   be (u)mounted.
 - The prepearation when starting a guest copes gracefully with
   the disk already being prepared.
 - Hooks are called from guest_create() and guest_destroy() to
   manipulate the disk as needed.

Using standalong-generate-dump-flight-runvars a representative set of
runvars is:

xen-unstable   test-amd64-amd64-xl-qcow2 
all_hostflags   arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
xen-unstable   test-amd64-amd64-xl-qcow2 arch   
 amd64
xen-unstable   test-amd64-amd64-xl-qcow2 buildjob   
 build-amd64
xen-unstable   test-amd64-amd64-xl-qcow2 
debian_arch amd64
xen-unstable   test-amd64-amd64-xl-qcow2 
debian_bootloader   pygrub
xen-unstable   test-amd64-amd64-xl-qcow2 
debian_diskfmt  qcow2
xen-unstable   test-amd64-amd64-xl-qcow2 
debian_dist wheezy
xen-unstable   test-amd64-amd64-xl-qcow2 
debian_method   netboot
xen-unstable   test-amd64-amd64-xl-qcow2 
kernbuildjobbuild-amd64-pvops
xen-unstable   test-amd64-amd64-xl-qcow2 kernkind   
 pvops
xen-unstable   test-amd64-amd64-xl-qcow2 toolstack  
 xl
xen-unstable   test-amd64-amd64-xl-qcow2 
xenbuildjob build-amd64

Compared to test-amd64-amd64-pygrub (which is the most similar job) and
normalising the test name the difference is:
 xen-unstable   test-amd64-amd64-SUFFIX   
all_hostflags   arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
 xen-unstable   test-amd64-amd64-SUFFIX   arch  
  amd64
 xen-unstable   test-amd64-amd64-SUFFIX   buildjob  
  build-amd64
 xen-unstable   test-amd64-amd64-SUFFIX   
debian_arch amd64
 xen-unstable   test-amd64-amd64-SUFFIX   
debian_bootloader   pygrub
+xen-unstable   test-amd64-amd64-SUFFIX   
debian_diskfmt  qcow2
 xen-unstable   test-amd64-amd64-SUFFIX   
debian_dist wheezy
 xen-unstable   test-amd64-amd64-SUFFIX   
debian_method   netboot
 xen-unstable   test-amd64-amd64-SUFFIX   
kernbuildjobbuild-amd64-pvops
 xen-unstable   test-amd64-amd64-SUFFIX   kernkind  
  pvops
 xen-unstable   test-amd64-amd64-SUFFIX   toolstack 
  xl
 xen-unstable   test-amd64-amd64-SUFFIX   
xenbuildjob build-amd64

These are added to the following flights:
  +libvirt
  +linux-3.0
  +linux-3.10
  +linux-3.14
  +linux-3.4
  +linux-arm-xen
  +linux-linus
  +linux-mingo-tip-master
  +linux-next
  +osstest
  +qemu-mainline
  +qemu-upstream-4.3-testing
  +qemu-upstream-4.4-testing
  +qemu-upstream-unstable
  +seabios
  +xen-4.0-testing
  +xen-4.1-testing
  +xen-4.2-testing
  

[Xen-devel] [PATCH OSSTEST v4 23/25] Debian.pm: Assume 100MB/s dd from /dev/zero when creating a raw disk image

2015-04-02 Thread Ian Campbell
In my local test:
1+0 records in
1+0 records out
1048576 bytes (10 GB) copied, 80.8332 s, 130 MB/s

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 Osstest/TestSupport.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 61fd7bc..391c9b2 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1586,7 +1586,9 @@ sub make_qcow2 ($$$) {
 }
 sub make_raw ($$$) {
 my ($ho, $gho, $disk_mb) = @_;
-target_cmd_root($ho, dd if=/dev/zero of=$gho-{Rootimg} bs=1M 
count=${disk_mb});
+# In local tests this reported 130MB/s, so calculate a timeout assuming 
100MB/s.
+target_cmd_root($ho, dd if=/dev/zero of=$gho-{Rootimg} bs=1M 
count=${disk_mb},
+   ${disk_mb} / 100);
 }
 
 sub prepareguest_part_diskimg ($$$) {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 25/25] ts-debian-di-install: Use ftp.debian.org directly

2015-04-02 Thread Ian Campbell
The local proxy seems to serve stale packages for Jessie etc, I blame
the intercepting cache on the way out of our network, similar to
b5f15136900d mg-debian-installer-update: workaround caching proxies,
except it is between the apt-cache and the world not the osstest vm
and the world.

Since the netboot kernel+initrd are reasonably small, these flights
are infrequent and they are intended to test the current upstream
version I think this is tollerable.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 ts-debian-di-install | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index 839cc27..469c1b6 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -119,7 +119,9 @@ sub setup_netboot($$$)
target_putfile_root($ho, 60, $initrd, $didir/initrd_${suite}_${arch});
 
 } else {
-   my $mirror = http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};;
+   #my $mirror = http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};;
+# XXX local mirror seems to serve up stale files.
+   my $mirror = http://ftp.debian.org/debian;;
 
my $di_url = $suite eq daily ?
http://d-i.debian.org/daily-images/$arch/daily/netboot; :
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3: Make sure wget is installed
---
 Osstest/Debian.pm  | 2 +-
 Osstest/TestSupport.pm | 9 -
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 69530fb..5e6bcba 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -617,7 +617,7 @@ d-i apt-setup/another boolean false
 d-i apt-setup/non-free boolean false
 d-i apt-setup/contrib boolean false
 
-d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, 
chiark-utils-bin, $extra_packages
+d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, 
chiark-utils-bin, wget, $extra_packages
 
 $xopts{ExtraPreseed}
 
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 1053016..c4adc48 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -55,7 +55,7 @@ BEGIN {
   target_putfilecontents_stash
  target_putfilecontents_root_stash
   target_put_guest_image target_editfile
-  target_editfile_cancel
+  target_editfile_cancel target_fetchurl
   target_editfile_root target_file_exists
   target_run_apt
   target_install_packages target_install_packages_norec
@@ -1560,6 +1560,13 @@ END
 return $cfgpath;
 }
 
+sub target_fetchurl($$$;$) {
+my ($ho, $url, $path, $timeo) = @_;
+$timeo ||= 2000;
+target_cmd_root($ho, wget --progress=dot:mega -O $path $url, $timeo);
+}
+
+
 sub target_put_guest_image ($$$) {
 my ($ho, $gho, $default) = @_;
 my $specimage = $r{$gho-{Guest}_image};
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 14/25] distros: add branch infrastructure

2015-04-02 Thread Ian Campbell
Since the distro nightlies are not version controlled we cannot use
the usual mechanisms for detecting regressions. Special case things
appropriately. We use an OLD_REVISION of flight-NNN to signify that
the old revision is another flight and not a tree revision.

A grep over $NEW_REVISION needed adjusting since NEW_REVISION is empty
in this mode, leading to grep filename which hangs waiting for
stdin.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
v3:
  Handle within cr-daily-branch, since ap-fetch-version* don't make sense for
  a branch such as this.
---
 cr-daily-branch | 36 
 cri-common  |  1 +
 2 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/cr-daily-branch b/cr-daily-branch
index c7d1071..55afadf 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -68,23 +68,34 @@ fetch_version () {
printf '%s\n' $fetch_version_result
 }
 
-treeurl=`./ap-print-url $branch`
+case $branch in
+distros)
+   treeurl=none;;
+*)
+   treeurl=`./ap-print-url $branch`;;
+esac
 
 force_baseline=false
 skipidentical=true
 wantpush=$OSSTEST_PUSH
 
-if [ x$OLD_REVISION = x ]; then
-OLD_REVISION=`./ap-fetch-version-old $branch`
-export OLD_REVISION
-fi
-
 check_tested () {
./sg-check-tested --debug --branch=$branch \
  --blessings=${DAILY_BRANCH_TESTED_BLESSING:-$OSSTEST_BLESSING} \
  $@
 }
 
+if [ x$OLD_REVISION = x ]; then
+case $branch in
+   distros)
+   OSSTEST_NO_BASELINE=y
+   OLD_REVISION=flight-`check_tested`
+   ;;
+   *) OLD_REVISION=`./ap-fetch-version-old $branch`;;
+esac
+export OLD_REVISION
+fi
+
 if [ x$OSSTEST_NO_BASELINE != xy ] ; then
testedflight=`check_tested --revision-$tree=$OLD_REVISION`
 
@@ -227,6 +238,11 @@ if [ x$OLD_REVISION = xdetermine-late ]; then
OLD_REVISION=`./ap-fetch-version-baseline-late $branch $NEW_REVISION`
 fi
 
+case $branch in
+distros) makeflight=./make-distros-flight ;;
+*)   makeflight=./make-flight ;;
+esac
+
 if [ x$NEW_REVISION = x$OLD_REVISION ]; then
 wantpush=false
for checkbranch in x $BRANCHES_ALWAYS; do
@@ -241,7 +257,7 @@ if [ x$NEW_REVISION = x$OLD_REVISION ]; then
 fi
 
 $DAILY_BRANCH_PREMAKE_HOOK
-flight=`./make-flight $branch $xenbranch $OSSTEST_BLESSING $@`
+flight=`$makeflight $branch $xenbranch $OSSTEST_BLESSING $@`
 $DAILY_BRANCH_POSTMAKE_HOOK
 
 heading=tmp/$flight.heading-info
@@ -261,6 +277,10 @@ fi
 revlog=tmp/$flight.revision-log
 
 case $NEW_REVISION/$OLD_REVISION in
+/flight-[0-9]*)
+   echo 2 SGR COMPARISON AGAINST ${OLD_REVISION}
+   sgr_args+= --that-flight=${OLD_REVISION#flight-}
+   ;;
 */*[^0-9a-f]* | *[^0-9a-f]*/*)
 echo 2 NO SGR COMPARISON badchar $NEW_REVISION/$OLD_REVISION
 ;;
@@ -321,7 +341,7 @@ start_email $flight $branch $sgr_args $subject_prefix
 push=false
 if grep '^tolerable$' $mrof /dev/null 21; then push=$wantpush; fi
 if test -f $branch.force; then push=$OSSTEST_PUSH; fi
-if grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
+if test -n $NEW_REVISION  grep -xF $NEW_REVISION $branch.force-rev; then 
push=$OSSTEST_PUSH; fi
 if test -f $branch.block; then push=false; fi
 
 if test -e $mrof  test -e $tree_bisect  ! grep '^broken' $mrof; then
diff --git a/cri-common b/cri-common
index ad44546..4bf7548 100644
--- a/cri-common
+++ b/cri-common
@@ -72,6 +72,7 @@ select_xenbranch () {
rumpuserxen)  tree=rumpuserxen; xenbranch=xen-unstable ;;
seabios)tree=seabios;   xenbranch=xen-unstable ;;
ovmf)   tree=ovmf;  xenbranch=xen-unstable ;;
+   distros)tree=none;  xenbranch=xen-unstable ;;
osstest)tree=osstest;   xenbranch=xen-unstable ;;
esac
if [ x$tree = xlinux ]; then
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
--
v3: Use CamelCase for xopts, use the correct condition
---
 Osstest/TestSupport.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index c4adc48..490e16d 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1533,12 +1533,13 @@ sub prepareguest_part_xencfg ($) {
 my $onpoweroff= $xopts-{OnPowerOff} || 'destroy';
 my $oncrash= $xopts-{OnCrash} || 'preserve';
 my $vcpus= guest_var($gho, 'vcpus', $xopts-{DefVcpus} || 2);
+my $viftype= $xopts-{VifType} ? type=$xopts-{VifType}, : ;
 my $xoptcfg= $xopts-{ExtraConfig};
 $xoptcfg='' unless defined $xoptcfg;
 my $xencfg= END;
 name= '$gho-{Name}'
 memory = ${ram_mb}
-vif = [ 'type=ioemu,mac=$gho-{Ether}' ]
+vif = [ '${viftype}mac=$gho-{Ether}' ]
 #
 on_poweroff = '$onpoweroff'
 on_reboot   = '$onreboot'
@@ -1633,6 +1634,7 @@ END
 $cfg .= bios='$bios'\n;
 }
 
+$xopts{VifType} ||= ioemu;
 my $cfgpath= prepareguest_part_xencfg($ho, $gho, $ram_mb, \%xopts, $cfg);
 target_cmd_root($ho, END);
 (echo $passwd; echo $passwd) | vncpasswd $gho-{Guest}.vncpw
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 15/25] distros: Run a flight over the weekend.

2015-04-02 Thread Ian Campbell
Once a week should be sufficient for this test. It involves quite a
large number of jobs so schedule it to start early on Saturday so it
can run over the weekend when, in theory, things should be quieter.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
Acked-by: Ian Jackson ian.jack...@eu.citrix.com
---
 crontab | 1 +
 1 file changed, 1 insertion(+)

diff --git a/crontab b/crontab
index 5dc8675..1917ec4 100644
--- a/crontab
+++ b/crontab
@@ -6,5 +6,6 @@ MAILTO=ian.jack...@citrix.com,ian.campb...@eu.citrix.com
 18 9   * * 1,3,5   cd testing.git  BRANCHES=linux-next   
./cr-for-branches branches -w ./cr-daily-branch --real
 18 4   * * *   cd testing.git  BRANCHES='linux-linus 
linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches 
branches -w ./cr-daily-branch --real
 6-59/15*   * * *   cd testing.git  
EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches 
bisects -w ./cr-try-bisect --real
+46 7   * * 6   cd testing.git  BRANCHES=distros  
./cr-for-branches branches -w ./cr-daily-branch --real
 #8-59/5*   * * *   cd bisects/adhoc.git  
with-lock-ex -q data-tree-lock bash -c ./cr-try-bisect-adhoc; exit $?
 3  4   * * *   savelog -c28 
testing.git/tmp/cr-for-branches.log /dev/null
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST v4 24/25] distros: email only me on play flights

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 daily-cron-email-play--distros | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 daily-cron-email-play--distros

diff --git a/daily-cron-email-play--distros b/daily-cron-email-play--distros
new file mode 100644
index 000..0d82e65
--- /dev/null
+++ b/daily-cron-email-play--distros
@@ -0,0 +1 @@
+To: ian.campb...@citrix.com
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] about cr3 register

2015-04-02 Thread George Dunlap
On Wed, Apr 1, 2015 at 3:37 PM, HANNAS YAYA Issa
issa.hannasy...@enseeiht.fr wrote:
 Hi
 I am working on memory management in xen. I encounter a problem that I hope
 you can help me to solve it.
 I modified the struct mmu_update in xen and linux source code. I add two
 fields cr3 and addr which respectively contains the current cr3 value and
 the current address of the process associated with the pte to be updated.
 the struct mmu_update is sent to the hyervisor via hypercall
 HYPERVISOR_do_mmu_update.

 In the handler of this hypercall (I mean do_mmu_update) when I read the
 value of cr3 via the method read_cr3 it is different from the value I stored
 before in the struct mmu_update. I wonder why?

Without your code there is absolutely no hope of us figuring out
what's going on; and in any case, it's not really our job to debug
your code.  If you have a specific question about the code that's
in-tree, please ask it.

This may also be helpful:

http://wiki.xenproject.org/wiki/Asking_Developer_Questions

(Your last question was closer to being answerable, FWIW.)

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] sudden hang on boot on AMD Notebook

2015-04-02 Thread Olaf Hering
On Wed, Apr 01, Olaf Hering wrote:

 On Wed, Apr 01, Ian Campbell wrote:
 
  Key handlers are only available via serial, I think, and not VGA which
  it sounds like what Olaf is looking at.
 
 This laptop has a serial port, and thats what I used to grab the output.
 No reaction there. Just today I noticed the blinking cursor. Will
 continue to poke this as time permits.

There are other issues now, suspend to ram with non-xen kernel fails as well:
...
[   58.114937] EXT4-fs (sda3): re-mounted. Opts: acl,user_xattr,commit=0
[   60.432380] EXT4-fs (dm-1): re-mounted. Opts: commit=0
[  176.984899] sky2 :02:00.0 eth1: disabling interface
[  179.214856] PM: Syncing filesystems ... done.
[  179.675897] Freezing user space processes ... (elapsed 0.001 seconds) done.
[  179.677286] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) 
done.
[  179.678601] Suspending console(s) (use no_console_suspend to debug)
[  179.679065] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[  179.730066] serial 00:04: disabled
[  179.730070] acpi PNP0501:00: Cannot transition to non-D0 state from D3
[  179.814620] sd 0:0:0:0: [sda] Stopping disk
[  180.957655] PM: suspend of devices complete after 1278.434 msecs
[  180.960953] acpi PNP0C0B:03: Cannot transition to non-D0 state from D3
[  180.960960] dpm_run_callback(): acpi_subsys_suspend_late+0x0/0x22 returns -19
[  180.960962] PM: Device PNP0C0B:03 failed to suspend late: error -19
[  180.964119] PM: early resume of devices complete after 3.149 msecs
[  180.964120] PM: late suspend of devices failed
[  180.964260] ohci-pci :00:12.0: enabling bus mastering
[  180.964281] ahci :00:11.0: restoring config space at offset 0x4 (was 
0x233, writing 0x237)
[  180.964285] ehci-pci :00:12.2: enabling bus mastering
[  180.964312] ohci-pci :00:13.0: enabling bus mastering
[  180.964338] ehci-pci :00:13.2: enabling bus mastering
[  180.964371] ohci-pci :00:14.5: enabling bus mastering
[  180.964383] ohci-pci :00:16.0: enabling bus mastering
[  180.964402] ehci-pci :00:16.2: enabling bus mastering
[  180.966630] sd 0:0:0:0: [sda] Starting disk
[  180.977638] radeon :01:05.0: restoring config space at offset 0x4 (was 
0x13, writing 0x17)
...

These are fan devices. Guess I have to open it some day and see if there are
any cable issues. Have to do it anyway, after HP sent it back from repair they
did not wire up the CDROM device.

Olaf

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 06/15] vt-d: Extend struct iremap_entry to support VT-d Posted-Interrupts

2015-04-02 Thread Tian, Kevin
 From: Jan Beulich [mailto:jbeul...@suse.com]
 Sent: Friday, March 27, 2015 5:58 PM
 
  On 27.03.15 at 02:53, feng...@intel.com wrote:
  From: Andrew Cooper [mailto:andrew.coop...@citrix.com]
  Sent: Friday, March 27, 2015 3:01 AM
  On 25/03/15 12:31, Feng Wu wrote:
   --- a/xen/drivers/passthrough/vtd/iommu.h
   +++ b/xen/drivers/passthrough/vtd/iommu.h
   @@ -303,6 +303,18 @@ struct iremap_entry {
 res_2   : 8,
 dst : 32;
 }lo;
   +struct {
   +u64 p   : 1,
   +fpd : 1,
   +res_1   : 6,
   +avail   : 4,
   +res_2   : 2,
   +urg : 1,
   +im  : 1,
   +vector  : 8,
   +res_3   : 14,
   +pda_l   : 26;
   +}lo_intpost;
   };
   union {
 u64 hi_val;
   @@ -312,6 +324,13 @@ struct iremap_entry {
 svt : 2,
 res_1   : 44;
 }hi;
   +struct {
   +u64 sid : 16,
   +sq  : 2,
   +svt : 2,
   +res_1   : 12,
   +pda_h   : 32;
   +}hi_intpost;
 
  I would prefer if this union was reformatted as I suggested in the
  thread from your design doc, but I won't insist on it as a blocker to 
  entry.
 
  Thanks for the comments. I also considered your sugguestion on the Design
  doc, here is your proposal:
 
  struct iremap_entry {
  union {
  struct { u64 lo, hi; };
  struct { bitfields } norm; (names subject to improvement)
  struct { bitfields } post;
  };
  };
 
  Seems in that way, we need to change some existing code to adapt to
  this new structure. I am okay with both of them, but can we listen
  some voices form some others? Is it okay for you?
 
 I think this would be a good move, but in the end it's the VT-d
 maintainers who got to decide.
 

yes it's a good move. 

Thanks
Kevin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 01/10] VMX: Enable EPT A/D bit support

2015-04-02 Thread Kai Huang



On 03/28/2015 04:38 AM, Andrew Cooper wrote:

On 27/03/15 02:35, Kai Huang wrote:

PML requires A/D bit support so enable it for further use.

Signed-off-by: Kai Huang kai.hu...@linux.intel.com
---
  xen/arch/x86/hvm/vmx/vmcs.c| 1 +
  xen/arch/x86/mm/p2m-ept.c  | 8 +++-
  xen/include/asm-x86/hvm/vmx/vmcs.h | 4 +++-
  xen/include/asm-x86/hvm/vmx/vmx.h  | 5 -
  4 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index d614638..2f645fe 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -103,6 +103,7 @@ static void __init vmx_display_features(void)
  P(cpu_has_vmx_tpr_shadow, APIC TPR shadow);
  P(cpu_has_vmx_ept, Extended Page Tables (EPT));
  P(cpu_has_vmx_vpid, Virtual-Processor Identifiers (VPID));
+P(cpu_has_vmx_ept_ad_bit, EPT A/D bit);
  P(cpu_has_vmx_vnmi, Virtual NMI);
  P(cpu_has_vmx_msr_bitmap, MSR direct-access bitmap);
  P(cpu_has_vmx_unrestricted_guest, Unrestricted Guest);
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index c2d7720..8650092 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -233,6 +233,9 @@ static int ept_split_super_page(struct p2m_domain *p2m, 
ept_entry_t *ept_entry,
  if ( !ept_set_middle_entry(p2m, new_ept) )
  return 0;
  
+/* It's better to copy A bit of Middle entry from original entry */

+new_ept.a = ept_entry-a;

Surely d needs to be propagated as well?  Would it make sense to extend
ept_set_middle_entry() to do all of new_ept setup in one location?


+
  table = map_domain_page(new_ept.mfn);
  trunk = 1UL  ((level - 1) * EPT_TABLE_ORDER);
  
@@ -244,7 +247,7 @@ static int ept_split_super_page(struct p2m_domain *p2m, ept_entry_t *ept_entry,

  epte-sp = (level  1);
  epte-mfn += i * trunk;
  epte-snp = (iommu_enabled  iommu_snoop);
-ASSERT(!epte-rsvd1);
+/* A/D bits are inherited from superpage */
  ASSERT(!epte-avail3);
  
  ept_p2m_type_to_flags(epte, epte-sa_p2mt, epte-access);

@@ -1071,6 +1074,9 @@ int ept_p2m_init(struct p2m_domain *p2m)
  /* set EPT page-walk length, now it's actual walk length - 1, i.e. 3 */
  ept-ept_wl = 3;
  
+/* Enable EPT A/D bit if it's supported by hardware */

+ept-ept_ad = cpu_has_vmx_ept_ad_bit ? 1 : 0;

This will incur overhead on all EPT operations.  It should only be
enabled if pml is going to be in use.  (I think you need reverse patches
1 and 2 in the series, and gate on pml_enable here)

Hi Andrew,

I'd like to also put patch 3 (PML feature detection) before this patch, 
as it's better to use cpu_has_vmx_pml to gate A/D bit enabling here. 
Theoretically simple pml_enable = 1 here doesn't guarantee 
cpu_has_vmx_pml to be true, as PML may be turned off during 
vmx_init_vmcs_config.


And in this case I also want to delete below code, as if PML is not 
enabled, below code will print but actually EPT A/D bits is not enabled 
in hardware.


   P(cpu_has_vmx_ept_ad, EPT A/D bit);

Thanks,
-Kai

+
  if ( !zalloc_cpumask_var(ept-synced_mask) )
  return -ENOMEM;
  
diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h b/xen/include/asm-x86/hvm/vmx/vmcs.h

index 6fce6aa..4528346 100644
--- a/xen/include/asm-x86/hvm/vmx/vmcs.h
+++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
@@ -62,7 +62,8 @@ struct ept_data {
  struct {
  u64 ept_mt :3,
  ept_wl :3,
-rsvd   :6,
+ept_ad :1,
+rsvd   :5,
  asr:52;

While you are making this change, can you add comments similar to
ept_entry_t describing the bits?


  };
  u64 eptp;
@@ -226,6 +227,7 @@ extern u32 vmx_secondary_exec_control;
  #define VMX_EPT_INVEPT_INSTRUCTION  0x0010
  #define VMX_EPT_INVEPT_SINGLE_CONTEXT   0x0200
  #define VMX_EPT_INVEPT_ALL_CONTEXT  0x0400
+#define VMX_EPT_AD_BIT_SUPPORT  0x0020
  
  #define VMX_MISC_VMWRITE_ALL0x2000
  
diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h

index 91c5e18..9afd351 100644
--- a/xen/include/asm-x86/hvm/vmx/vmx.h
+++ b/xen/include/asm-x86/hvm/vmx/vmx.h
@@ -37,7 +37,8 @@ typedef union {
  emt :   3,  /* bits 5:3 - EPT Memory type */
  ipat:   1,  /* bit 6 - Ignore PAT memory type */
  sp  :   1,  /* bit 7 - Is this a superpage? */
-rsvd1   :   2,  /* bits 9:8 - Reserved for future use */
+a   :   1,  /* bit 8 - Access bit */
+d   :   1,  /* bit 9 - Dirty bit */
  recalc  :   1,  /* bit 10 - Software available 1 */
  snp :   1,  /* bit 11 - VT-d snoop control in shared
 EPT/VT-d usage */
@@ -261,6 +262,8 @@ extern uint8_t posted_intr_vector;
  (vmx_ept_vpid_cap  

Re: [Xen-devel] [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d Posted-Interrupts

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 2:01 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d
 Posted-Interrupts
 
  From: Wu, Feng
  Sent: Wednesday, March 25, 2015 8:32 PM
 
  This patch adds a global vector which is used to wake up
  the blocked vCPU when an interrupt is being posted to it.
 
  Signed-off-by: Feng Wu feng...@intel.com
  Suggested-by: Yang Zhang yang.z.zh...@intel.com
  ---
   xen/arch/x86/hvm/vmx/vmx.c| 33
  +
   xen/include/asm-x86/hvm/hvm.h |  1 +
   xen/include/asm-x86/hvm/vmx/vmx.h |  3 +++
   xen/include/xen/sched.h   |  2 ++
   4 files changed, 39 insertions(+)
 
  diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
  index ff5544d..b2b4c26 100644
  --- a/xen/arch/x86/hvm/vmx/vmx.c
  +++ b/xen/arch/x86/hvm/vmx/vmx.c
  @@ -89,6 +89,7 @@ DEFINE_PER_CPU(struct list_head,
  blocked_vcpu_on_cpu);
   DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
 
   uint8_t __read_mostly posted_intr_vector;
  +uint8_t __read_mostly pi_wakeup_vector;
 
   static int vmx_domain_initialise(struct domain *d)
   {
  @@ -131,6 +132,8 @@ static int vmx_vcpu_initialise(struct vcpu *v)
   if ( v-vcpu_id == 0 )
   v-arch.user_regs.eax = 1;
 
  +INIT_LIST_HEAD(v-blocked_vcpu_list);
  +
   return 0;
   }
 
  @@ -1834,11 +1837,19 @@ const struct hvm_function_table * __init
  start_vmx(void)
   }
 
   if ( cpu_has_vmx_posted_intr_processing )
  +{
   alloc_direct_apic_vector(posted_intr_vector,
  event_check_interrupt);
  +
  +if ( iommu_intpost )
  +alloc_direct_apic_vector(pi_wakeup_vector,
  pi_wakeup_interrupt);
  +else
  +vmx_function_table.pi_desc_update = NULL;
  +}
 
 just style issue. Above conditional logic looks not intuitive to me.
 usually we have:
   if ( iommu_intpost )
   vmx_function_table.pi_desc_update = func;
   else
   vmx_function_table.pi_desc_update = NULL;
 
 suppose you will register callback in later patch. then better to
 move the NULL one there too. Putting it here doesn't meet the
 normal if...else implications. :-)

You suggestion is good. Here is my idea about this code fragment:

Here is the place to register notification event handle, so it is better
to register the wakeup event handle for VT-d PI here as well. Just like other
members in vmx_function_table, such as, deliver_posted_intr, sync_pir_to_irr, 
pi_desc_update is initialed to 'vmx_pi_desc_update' in the definition of
vmx_function_table statically. So do you have any ideas to make this
gracefully?

Thanks,
Feng

 
   else
   {
   vmx_function_table.deliver_posted_intr = NULL;
   vmx_function_table.sync_pir_to_irr = NULL;
  +vmx_function_table.pi_desc_update = NULL;
   }
 
   if ( cpu_has_vmx_ept
  @@ -3255,6 +3266,28 @@ void vmx_vmenter_helper(const struct
  cpu_user_regs *regs)
   }
 
   /*
  + * Handle VT-d posted-interrupt when VCPU is blocked.
  + */
  +void pi_wakeup_interrupt(struct cpu_user_regs *regs)
  +{
  +struct vcpu *v;
  +int cpu = smp_processor_id();
  +
  +spin_lock(per_cpu(blocked_vcpu_on_cpu_lock, cpu));
  +list_for_each_entry(v, per_cpu(blocked_vcpu_on_cpu, cpu),
  +blocked_vcpu_list) {
  +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc;
  +
  +if ( pi_test_on(pi_desc) == 1 )
  +tasklet_schedule(v-vcpu_wakeup_tasklet);
 
 why can't we directly call vcpu_unblock here?

Please see the following scenario if we use vcpu_unblock directly here:

pi_wakeup_interrupt() (blocked_vcpu_on_cpu_lock is required) -- vcpu_unblock() 
-- 
vcpu_wake() -- vcpu_runstate_change() -- vmx_ pi_desc_update() (In this 
function we
may need to require blocked_vcpu_on_cpu_lock, this will cause dead lock.)

Thanks,
Feng


 
  +}
  +spin_unlock(per_cpu(blocked_vcpu_on_cpu_lock, cpu));
  +
  +ack_APIC_irq();
  +this_cpu(irq_count)++;
  +}
  +
  +/*
* Local variables:
* mode: C
* c-file-style: BSD
  diff --git a/xen/include/asm-x86/hvm/hvm.h
  b/xen/include/asm-x86/hvm/hvm.h
  index 0dc909b..a11a256 100644
  --- a/xen/include/asm-x86/hvm/hvm.h
  +++ b/xen/include/asm-x86/hvm/hvm.h
  @@ -195,6 +195,7 @@ struct hvm_function_table {
   void (*deliver_posted_intr)(struct vcpu *v, u8 vector);
   void (*sync_pir_to_irr)(struct vcpu *v);
   void (*handle_eoi)(u8 vector);
  +void (*pi_desc_update)(struct vcpu *v, int new_state);
 
   /*Walk nested p2m  */
   int (*nhvm_hap_walk_L1_p2m)(struct vcpu *v, paddr_t L2_gpa,
  diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h
  b/xen/include/asm-x86/hvm/vmx/vmx.h
  index e643c3c..f4296ab 100644
  --- a/xen/include/asm-x86/hvm/vmx/vmx.h
  +++ b/xen/include/asm-x86/hvm/vmx/vmx.h
  @@ -34,6 +34,7 @@ 

[Xen-devel] [PATCH OSSTEST] ts-xen-*: Avoid use of uninitialised $r{enable_xsm}

2015-04-02 Thread Ian Campbell
Use of uninitialized value $r{enable_xsm} in pattern match (m//) at 
./ts-xen-install line 49.

Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 ts-xen-build   | 2 +-
 ts-xen-install | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-xen-build b/ts-xen-build
index e757f68..a530731 100755
--- a/ts-xen-build
+++ b/ts-xen-build
@@ -28,7 +28,7 @@ selectbuildhost(\@ARGV);
 # remaining arguments are passed as targets to make
 builddirsprops();
 
-my $enable_xsm = $r{enable_xsm} =~ m/true/ ? 1 : 0;
+my $enable_xsm = $r{enable_xsm}//'' =~ m/true/ ? 1 : 0;
 
 sub checkout () {
 prepbuilddirs();
diff --git a/ts-xen-install b/ts-xen-install
index b3f4387..6995046 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -46,7 +46,7 @@ if (@ARGV and $ARGV[0] eq '--check') {
 
 our $ho;
 
-my $enable_xsm = $r{enable_xsm} =~ m/true/ ? 1 : 0;
+my $enable_xsm = $r{enable_xsm}//'' =~ m/true/ ? 1 : 0;
 
 my %distpath;
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case

2015-04-02 Thread Pang, LongtaoX


 -Original Message-
 From: Ian Campbell [mailto:ian.campb...@citrix.com]
 Sent: Tuesday, March 31, 2015 10:23 PM
 To: Pang, LongtaoX
 Cc: xen-devel@lists.xen.org; ian.jack...@eu.citrix.com; wei.l...@citrix.com;
 Hu, Robert
 Subject: Re: [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case
 
 On Fri, 2015-03-27 at 19:06 -0400, longtao.pang wrote:
  Changes in v7:
  diff --git a/make-flight b/make-flight index 8ac3a87..b8f266f 100755
  --- a/make-flight
  +++ b/make-flight
  @@ -204,6 +204,26 @@ do_hvm_win7_x64_tests () {
   all_hostflags=$most_hostflags,hvm  }
 
  +do_hvm_debian_nested_tests () {
  +  if [ $xenarch != amd64 ]; then
  +return
  +  fi
  +  if [ $dom0arch != amd64 ]; then
  +return
  +  fi
 
 You can do these on a line each, or even combine into one test. i.e.
 
 if [ $xenarch != amd64 -o $dom0arch != amd64 ]; then return; fi
 
I'm sorry I find that the 'if' condition is not appropriate in v7 patch, it 
should be 
if [ $xenarch != amd64 -a $dom0arch != amd64 ]; then return; fi
  +
  +  job_create_test test-$xenarch$kern-$dom0arch-nested test-nested xl \
  +   $xenarch $dom0arch \
  +nested_image=$NESTED_OS_IMAGE \
  +nested2_image=$NESTED_OS_IMAGE \
 
 I think for clarity you should use something like nestedl1 and nestedl2 for 
 the
 runvar names.
 
'nested' and 'nested2' are guest name of L1 and L2 guest VM. Since $specimage 
is accessed from $r{$gho-{Guest}_image} which defined in the function of ' 
target_put_guest_image'. So, maybe 'nested' and 'nested2' are available here, I 
think. 
  +bios=seabios \
  +kernbuildjob=build-amd64-pvops \
  +kernkind=pvops \
  +nested_vifmodel='e1000' \
  +device_model_version=qemu-xen \
  +all_hostflags=$most_hostflags,hvm }
  +

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 2/2] x86/efi: Fix memcmp check

2015-04-02 Thread Ross Lagerwall
Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com
---
 xen/arch/x86/dmi_scan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
index ec411e6..3749688 100644
--- a/xen/arch/x86/dmi_scan.c
+++ b/xen/arch/x86/dmi_scan.c
@@ -203,7 +203,7 @@ void __init dmi_efi_get_table(const void *smbios, const 
void *smbios3)
const struct smbios_eps *eps = smbios;
const struct smbios3_eps *eps3 = smbios3;
 
-   if (eps3  memcmp(eps3-anchor, _SM3_, 5) 
+   if (eps3  memcmp(eps3-anchor, _SM3_, 5) == 0 
eps3-length = sizeof(*eps3) 
dmi_checksum(eps3, eps3-length)) {
efi_smbios3_address = eps3-address;
-- 
2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 13/15] Update Posted-Interrupts Descriptor during vCPU scheduling

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 2:25 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 13/15] Update Posted-Interrupts Descriptor during vCPU
 scheduling
 
  From: Wu, Feng
  Sent: Wednesday, March 25, 2015 8:32 PM
 
  The basic idea here is:
  1. When vCPU's state is RUNSTATE_running,
  - set 'NV' to 'Notification Vector'.
  - Clear 'SN' to accpet PI.
  - set 'NDST' to the right pCPU.
  2. When vCPU's state is RUNSTATE_blocked,
  - set 'NV' to 'Wake-up Vector', so we can wake up the
related vCPU when posted-interrupt happens for it.
  - Clear 'SN' to accpet PI.
  3. When vCPU's state is RUNSTATE_runnable/RUNSTATE_offline,
  - Set 'SN' to suppress non-urgent interrupts.
(Current, we only support non-urgent interrupts)
  - Set 'NV' back to 'Notification Vector' if needed.
 
  Signed-off-by: Feng Wu feng...@intel.com
  ---
   xen/arch/x86/hvm/vmx/vmx.c | 108
  +
   xen/common/schedule.c  |   3 ++
   2 files changed, 111 insertions(+)
 
  diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
  index b30392c..6323bd6 100644
  --- a/xen/arch/x86/hvm/vmx/vmx.c
  +++ b/xen/arch/x86/hvm/vmx/vmx.c
  @@ -1710,6 +1710,113 @@ static void vmx_handle_eoi(u8 vector)
   __vmwrite(GUEST_INTR_STATUS, status);
   }
 
  +static void vmx_pi_desc_update(struct vcpu *v, int new_state)
  +{
  +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc;
  +struct pi_desc old, new;
  +int old_state = v-runstate.state;
  +unsigned long flags;
  +
  +if ( !iommu_intpost )
  +return;
  +
  +switch ( new_state )
  +{
  +case RUNSTATE_runnable:
  +case RUNSTATE_offline:
  +/*
  + * We don't need to send notification event to a non-running
  + * vcpu, the interrupt information will be delivered to it before
  + * VM-ENTRY when the vcpu is scheduled to run next time.
  + */
  +pi_set_sn(pi_desc);
  +
  +/*
  + * If the state is transferred from RUNSTATE_blocked,
  + * we should set 'NV' feild back to posted_intr_vector,
  + * so the Posted-Interrupts can be delivered to the vCPU
  + * by VT-d HW after it is scheduled to run.
  + */
  +if ( old_state == RUNSTATE_blocked )
  +{
  +do
  +{
  +old.control = new.control = pi_desc-control;
  +new.nv = posted_intr_vector;
  +}
  +while ( cmpxchg(pi_desc-control, old.control, new.control)
  +!= old.control );
  +
  +   /*
  +* Delete the vCPU from the related wakeup queue
  +* if we are resuming from blocked state
  +*/
  +   spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock,
  + v-processor), flags);
  +   list_del(v-blocked_vcpu_list);
  +   spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock,
  +  v-processor), flags);
  +}
  +break;
  +
  +case RUNSTATE_blocked:
  +/*
  + * The vCPU is blocked on the wait queue.
  + * Store the blocked vCPU on the list of the
  + * vcpu-wakeup_cpu, which is the destination
  + * of the wake-up notification event.
  + */
  +spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock,
  +  v-processor), flags);
  +list_add_tail(v-blocked_vcpu_list,
  +  per_cpu(blocked_vcpu_on_cpu, v-processor));
  +spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock,
  +   v-processor), flags);
  +
  +do
  +{
  +old.control = new.control = pi_desc-control;
  +
  +/*
  + * We should not block the vCPU if
  + * an interrupt is posted for it.
  + */
  +
  +if ( pi_test_on(old) == 1 )
  +{
  +tasklet_schedule(v-vcpu_wakeup_tasklet);
  +return;
  +}
 
 so you also need to remove the vcpu from the blocked list, right?

Yes, I need to remove the vcpu here. I thought it could be removed in
another place in this patch, however, I feel it cannot do it after
more thinking about it. I think we can fix this issue in the following two ways:
#1) Just add the remove logic here.
or 
#2) In function vcpu_runstate_change(), call 'hvm_funcs.pi_desc_update()' after
v-runstate.state = new_state; and pass the old_state to 
'hvm_funcs.pi_desc_update()'.
then here is the code path:

tasklet_schedule(v-vcpu_wakeup_tasklet) - vcpu_unblock() - vcpu_wake() -
vcpu_runstate_change(v, RUNSTATE_runnable, NOW()) - vmx_pi_desc_update()

So, the vCPU will be removed in 'case 

Re: [Xen-devel] Migration v2 and related work for 4.6

2015-04-02 Thread Ian Campbell
On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
 I propose that the libxc series be accepted independently of the libxl
 series.

That is most likely a good idea IMHO.

What do you estimate the chances of the libxl bit being done for 4.6 to
be?

Is there an option of actually switching to the new libxc without
switching libxl and fixing libxl later, or would that be as much work as
just fixing libxl?

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v2 13/22] xen/arm: its: Add virtual ITS command support

2015-04-02 Thread Ian Campbell
On Wed, 2015-04-01 at 13:02 +0100, Julien Grall wrote:
 On 01/04/15 12:46, Ian Campbell wrote:
  On Mon, 2015-03-30 at 16:47 +0100, Julien Grall wrote:
  In any case ITS commands are processed in synchronously. So any VCPU that
  send ITS commands is blocked.
  
  What exactly is synchronous here? Is it just the translate vits into
  requests queued with the physical its driver phase or does it also
  include waiting for the physical its' response and translating that back
  into a v response?
 
 From the spec, the processing of command is asynchronous.

I was asking about the implementation of our emulation of it, not about
the hardware itself. I understood that the underlying h/w is
asynchronous.

  The vCPU has
 to poll a register in order to know if the ITS has finished to execute
 the command.
 
 A vCPU may decide to execute other things while the ITS is processing
 commands.
 
 The implementation suggested by Vijay, both the vCPU and CPU is blocked
 while the ITS command are processing.

Can we just enqueue with the hardware and use the guest vcpu polling
loop to trigger us to check for completion? What would happen if a guest
never polled, I suppose we would have to catch it some other way?

 Futhermore, if another vCPU is trying to access the vITS it will be
 blocked too (and therefore the CPU). With this solution we may take down
 Xen.

Yes, we can't have that I'm afraid.

 The translation/queuing can be very long if the queue is big (the
 maximum size of the queue is 2^32). It would at least require some
 preemption in Xen in order to avoid blocking the CPU/vCPU for a long time.

Yes, limiting the number of requests in flight at any one time seems
like a good idea anyway, we can always dribble things in as other ones
complete anyway.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Migration v2 and related work for 4.6

2015-04-02 Thread Andrew Cooper
On 02/04/15 10:03, Ian Campbell wrote:
 On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
 I propose that the libxc series be accepted independently of the libxl
 series.
 That is most likely a good idea IMHO.

 What do you estimate the chances of the libxl bit being done for 4.6 to
 be?

I hope to have everything complete for 4.6, including removal of the
legacy code.

Given the current timescales, I would say most likely.


 Is there an option of actually switching to the new libxc without
 switching libxl and fixing libxl later, or would that be as much work as
 just fixing libxl?

For PV guests, yes.  One can transparently swap the legacy algorithm
from the v2 algorithm and every works.

For HVM guests, no.  The handling of the Qemu save record and toolstack
records needs fixing.

In the upgrade case, libxl also needs to take care of piping the legacy
stream through the conversion script.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] x86/xen: Warn when there's no mfn for pfn

2015-04-02 Thread Ross Lagerwall
I recently had to debug dom0 taking a fatal page fault and it would have
been useful if it warned when creating a non-present pte if there's no
mfn for a pfn.

Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com
---
 arch/x86/xen/mmu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index adca9e2..4cacc11 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -397,6 +397,7 @@ static pteval_t pte_pfn_to_mfn(pteval_t val)
 * pte_mfn_to_pfn is asymmetric.
 */
if (unlikely(mfn == INVALID_P2M_ENTRY)) {
+   printk(KERN_WARNING No mfn for pfn %lx, creating 
non-present pte\n, pfn);
mfn = 0;
flags = 0;
} else
-- 
2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/xen: Warn when there's no mfn for pfn

2015-04-02 Thread Ross Lagerwall

On 04/02/2015 10:00 AM, David Vrabel wrote:

On 02/04/15 08:28, Ross Lagerwall wrote:

I recently had to debug dom0 taking a fatal page fault and it would have
been useful if it warned when creating a non-present pte if there's no
mfn for a pfn.


This will trigger for ever PTE covering memory hotplugged by the balloon
driver, so I don't think we can take this.



That's unfortunate, I thought there may be some issue where it would be 
triggered spuriously.


--
Ross Lagerwall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 2/2] x86/efi: Fix memcmp check

2015-04-02 Thread Andrew Cooper
On 02/04/15 09:24, Ross Lagerwall wrote:
 Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com

Reviewed-by: Andrew Cooper andrew.coop...@citrix.com

 ---
  xen/arch/x86/dmi_scan.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

 diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
 index ec411e6..3749688 100644
 --- a/xen/arch/x86/dmi_scan.c
 +++ b/xen/arch/x86/dmi_scan.c
 @@ -203,7 +203,7 @@ void __init dmi_efi_get_table(const void *smbios, const 
 void *smbios3)
   const struct smbios_eps *eps = smbios;
   const struct smbios3_eps *eps3 = smbios3;
  
 - if (eps3  memcmp(eps3-anchor, _SM3_, 5) 
 + if (eps3  memcmp(eps3-anchor, _SM3_, 5) == 0 
   eps3-length = sizeof(*eps3) 
   dmi_checksum(eps3, eps3-length)) {
   efi_smbios3_address = eps3-address;


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 2/4 v2] INSTALL: mention variables for reproducible builds

2015-04-02 Thread Ian Campbell
On Wed, 2015-04-01 at 13:28 +, Olaf Hering wrote:
 Mention two variables introduced by commit ac977f5 (use more fixed
 strings to build the hypervisor).
 
 Signed-off-by: Olaf Hering o...@aepfle.de
Acked-by: Ian Campbell ian.campb...@citrix.com



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 1/2] x86/efi: Reserve SMBIOS table region when EFI booting

2015-04-02 Thread Andrew Cooper
On 02/04/15 09:24, Ross Lagerwall wrote:
 Some EFI firmware implementations may place the SMBIOS table in RAM
 marked as BootServicesData, which Xen does not consider as reserved.
 When dom0 tries to access the SMBIOS, the region is not contained in the
 initial P2M and it crashes with a page fault. To fix this, reserve the
 SMBIOS region.

 Also, fix the memcmp check for existence of the SMBIOS and the DMI
 checksum calculation.

 Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com

Reviewed-by: Andrew Cooper andrew.coop...@citrix.com

(Personally, I would either have folded the two patches together, or put
the memcmp() one first, fixing both occurrences.  However, as both of
these are candidates for backports, this probably turns more into a
bikeshed activity.)

 ---
  xen/arch/x86/dmi_scan.c | 25 +++--
  1 file changed, 19 insertions(+), 6 deletions(-)

 diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
 index 187c01e..ec411e6 100644
 --- a/xen/arch/x86/dmi_scan.c
 +++ b/xen/arch/x86/dmi_scan.c
 @@ -189,6 +189,8 @@ static inline bool_t __init dmi_checksum(const void 
 __iomem *buf,
  
  static u32 __initdata efi_dmi_address;
  static u32 __initdata efi_dmi_size;
 +static u32 __initdata efi_smbios_address;
 +static u32 __initdata efi_smbios_size;
  static u64 __initdata efi_smbios3_address;
  static u32 __initdata efi_smbios3_size;
  
 @@ -209,13 +211,18 @@ void __init dmi_efi_get_table(const void *smbios, const 
 void *smbios3)
   return;
   }
  
 - if (eps  memcmp(eps-anchor, _SM_, 4) 
 + if (eps  memcmp(eps-anchor, _SM_, 4) == 0 
   eps-length = sizeof(*eps) 
 - dmi_checksum(eps, eps-length) 
 - memcmp(eps-dmi.anchor, _DMI_, 5) == 0 
 - dmi_checksum(eps-dmi, sizeof(eps-dmi))) {
 - efi_dmi_address = eps-dmi.address;
 - efi_dmi_size = eps-dmi.size;
 + dmi_checksum(eps, eps-length)) {
 + efi_smbios_address = (u32)(long)smbios;
 + efi_smbios_size = eps-length;
 +
 + if ( memcmp(eps-dmi.anchor, _DMI_, 5) == 0 
 +  dmi_checksum(eps-dmi,
 +   eps-length - offsetof(struct smbios_eps, 
 dmi)) ) {
 + efi_dmi_address = eps-dmi.address;
 + efi_dmi_size = eps-dmi.size;
 + }
   }
  }
  
 @@ -236,6 +243,12 @@ const char *__init dmi_get_table(paddr_t *base, u32 *len)
   instance |= 2;
   return DMI;
   }
 + if (efi_smbios_size  !(instance  4)) {
 + *base = efi_smbios_address;
 + *len = efi_smbios_size;
 + instance |= 4;
 + return SMBIOS;
 + }
   } else {
   char __iomem *p = maddr_to_virt(0xF), *q;
   union {


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d Posted-Interrupts

2015-04-02 Thread Tian, Kevin
 From: Wu, Feng
 Sent: Wednesday, March 25, 2015 8:32 PM
 
 This patch adds a global vector which is used to wake up
 the blocked vCPU when an interrupt is being posted to it.
 
 Signed-off-by: Feng Wu feng...@intel.com
 Suggested-by: Yang Zhang yang.z.zh...@intel.com
 ---
  xen/arch/x86/hvm/vmx/vmx.c| 33
 +
  xen/include/asm-x86/hvm/hvm.h |  1 +
  xen/include/asm-x86/hvm/vmx/vmx.h |  3 +++
  xen/include/xen/sched.h   |  2 ++
  4 files changed, 39 insertions(+)
 
 diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
 index ff5544d..b2b4c26 100644
 --- a/xen/arch/x86/hvm/vmx/vmx.c
 +++ b/xen/arch/x86/hvm/vmx/vmx.c
 @@ -89,6 +89,7 @@ DEFINE_PER_CPU(struct list_head,
 blocked_vcpu_on_cpu);
  DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
 
  uint8_t __read_mostly posted_intr_vector;
 +uint8_t __read_mostly pi_wakeup_vector;
 
  static int vmx_domain_initialise(struct domain *d)
  {
 @@ -131,6 +132,8 @@ static int vmx_vcpu_initialise(struct vcpu *v)
  if ( v-vcpu_id == 0 )
  v-arch.user_regs.eax = 1;
 
 +INIT_LIST_HEAD(v-blocked_vcpu_list);
 +
  return 0;
  }
 
 @@ -1834,11 +1837,19 @@ const struct hvm_function_table * __init
 start_vmx(void)
  }
 
  if ( cpu_has_vmx_posted_intr_processing )
 +{
  alloc_direct_apic_vector(posted_intr_vector,
 event_check_interrupt);
 +
 +if ( iommu_intpost )
 +alloc_direct_apic_vector(pi_wakeup_vector,
 pi_wakeup_interrupt);
 +else
 +vmx_function_table.pi_desc_update = NULL;
 +}

just style issue. Above conditional logic looks not intuitive to me.
usually we have:
if ( iommu_intpost )
vmx_function_table.pi_desc_update = func;
else
vmx_function_table.pi_desc_update = NULL;

suppose you will register callback in later patch. then better to
move the NULL one there too. Putting it here doesn't meet the
normal if...else implications. :-)

  else
  {
  vmx_function_table.deliver_posted_intr = NULL;
  vmx_function_table.sync_pir_to_irr = NULL;
 +vmx_function_table.pi_desc_update = NULL;
  }
 
  if ( cpu_has_vmx_ept
 @@ -3255,6 +3266,28 @@ void vmx_vmenter_helper(const struct
 cpu_user_regs *regs)
  }
 
  /*
 + * Handle VT-d posted-interrupt when VCPU is blocked.
 + */
 +void pi_wakeup_interrupt(struct cpu_user_regs *regs)
 +{
 +struct vcpu *v;
 +int cpu = smp_processor_id();
 +
 +spin_lock(per_cpu(blocked_vcpu_on_cpu_lock, cpu));
 +list_for_each_entry(v, per_cpu(blocked_vcpu_on_cpu, cpu),
 +blocked_vcpu_list) {
 +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc;
 +
 +if ( pi_test_on(pi_desc) == 1 )
 +tasklet_schedule(v-vcpu_wakeup_tasklet);

why can't we directly call vcpu_unblock here?

 +}
 +spin_unlock(per_cpu(blocked_vcpu_on_cpu_lock, cpu));
 +
 +ack_APIC_irq();
 +this_cpu(irq_count)++;
 +}
 +
 +/*
   * Local variables:
   * mode: C
   * c-file-style: BSD
 diff --git a/xen/include/asm-x86/hvm/hvm.h
 b/xen/include/asm-x86/hvm/hvm.h
 index 0dc909b..a11a256 100644
 --- a/xen/include/asm-x86/hvm/hvm.h
 +++ b/xen/include/asm-x86/hvm/hvm.h
 @@ -195,6 +195,7 @@ struct hvm_function_table {
  void (*deliver_posted_intr)(struct vcpu *v, u8 vector);
  void (*sync_pir_to_irr)(struct vcpu *v);
  void (*handle_eoi)(u8 vector);
 +void (*pi_desc_update)(struct vcpu *v, int new_state);
 
  /*Walk nested p2m  */
  int (*nhvm_hap_walk_L1_p2m)(struct vcpu *v, paddr_t L2_gpa,
 diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h
 b/xen/include/asm-x86/hvm/vmx/vmx.h
 index e643c3c..f4296ab 100644
 --- a/xen/include/asm-x86/hvm/vmx/vmx.h
 +++ b/xen/include/asm-x86/hvm/vmx/vmx.h
 @@ -34,6 +34,7 @@ DECLARE_PER_CPU(struct list_head,
 blocked_vcpu_on_cpu);
  DECLARE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
 
  extern uint8_t posted_intr_vector;
 +extern uint8_t pi_wakeup_vector;
 
  typedef union {
  struct {
 @@ -574,6 +575,8 @@ int alloc_p2m_hap_data(struct p2m_domain *p2m);
  void free_p2m_hap_data(struct p2m_domain *p2m);
  void p2m_init_hap_data(struct p2m_domain *p2m);
 
 +void pi_wakeup_interrupt(struct cpu_user_regs *regs);
 +
  /* EPT violation qualifications definitions */
  #define _EPT_READ_VIOLATION 0
  #define EPT_READ_VIOLATION  (1UL_EPT_READ_VIOLATION)
 diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
 index c874dd4..91f0912 100644
 --- a/xen/include/xen/sched.h
 +++ b/xen/include/xen/sched.h
 @@ -148,6 +148,8 @@ struct vcpu
 
  struct vcpu *next_in_list;
 
 +struct list_head blocked_vcpu_list;
 +
  s_time_t periodic_period;
  s_time_t periodic_last_event;
  struct timer periodic_timer;
 --
 2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 09/15] Add a new per-vCPU tasklet to wakeup the blocked vCPU

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Tian, Kevin
 Sent: Thursday, April 02, 2015 1:53 PM
 To: Wu, Feng; xen-devel@lists.xen.org
 Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z
 Subject: RE: [RFC v1 09/15] Add a new per-vCPU tasklet to wakeup the blocked
 vCPU
 
  From: Wu, Feng
  Sent: Wednesday, March 25, 2015 8:32 PM
 
  This patch adds a new per-vCPU tasklet to wakeup the blocked
  vCPU. It can be used in the case vcpu_unblock cannot be called
  directly.
 
 could you elaborate under which scenario vcpu_unblock can't
 be called directly?

Please see the reply to  [RFC v1 11/15] vmx: Add a global wake-up vector for 
VT-d Posted-Interrupts 
I will elaborate it a bit more in the next post. Thanks!

Thanks,
Feng

 
 
  Signed-off-by: Feng Wu feng...@intel.com
  ---
   xen/common/domain.c | 11 +++
   xen/include/xen/sched.h |  3 +++
   2 files changed, 14 insertions(+)
 
  diff --git a/xen/common/domain.c b/xen/common/domain.c
  index aa78fd7..fe89658 100644
  --- a/xen/common/domain.c
  +++ b/xen/common/domain.c
  @@ -109,6 +109,13 @@ static void vcpu_check_shutdown(struct vcpu *v)
   spin_unlock(d-shutdown_lock);
   }
 
  +static void vcpu_wakeup_tasklet_handler(unsigned long arg)
  +{
  +struct vcpu *v = (void *)arg;
  +
  +vcpu_unblock(v);
  +}
  +
   struct vcpu *alloc_vcpu(
   struct domain *d, unsigned int vcpu_id, unsigned int cpu_id)
   {
  @@ -126,6 +133,9 @@ struct vcpu *alloc_vcpu(
 
   tasklet_init(v-continue_hypercall_tasklet, NULL, 0);
 
  +tasklet_init(v-vcpu_wakeup_tasklet, vcpu_wakeup_tasklet_handler,
  + (unsigned long)v);
  +
   if ( !zalloc_cpumask_var(v-cpu_hard_affinity) ||
!zalloc_cpumask_var(v-cpu_hard_affinity_tmp) ||
!zalloc_cpumask_var(v-cpu_hard_affinity_saved) ||
  @@ -784,6 +794,7 @@ static void complete_domain_destroy(struct
 rcu_head
  *head)
   if ( (v = d-vcpu[i]) == NULL )
   continue;
   tasklet_kill(v-continue_hypercall_tasklet);
  +tasklet_kill(v-vcpu_wakeup_tasklet);
   vcpu_destroy(v);
   sched_destroy_vcpu(v);
   destroy_waitqueue_vcpu(v);
  diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
  index ccd7ed8..c874dd4 100644
  --- a/xen/include/xen/sched.h
  +++ b/xen/include/xen/sched.h
  @@ -239,6 +239,9 @@ struct vcpu
   /* Tasklet for continue_hypercall_on_cpu(). */
   struct tasklet   continue_hypercall_tasklet;
 
  +/* Tasklet for wakeup_blocked_vcpu(). */
  +struct tasklet   vcpu_wakeup_tasklet;
  +
   /* Multicall information. */
   struct mc_state  mc_state;
 
  --
  2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH 1/2] x86/efi: Reserve SMBIOS table region when EFI booting

2015-04-02 Thread Ross Lagerwall
Some EFI firmware implementations may place the SMBIOS table in RAM
marked as BootServicesData, which Xen does not consider as reserved.
When dom0 tries to access the SMBIOS, the region is not contained in the
initial P2M and it crashes with a page fault. To fix this, reserve the
SMBIOS region.

Also, fix the memcmp check for existence of the SMBIOS and the DMI
checksum calculation.

Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com
---
 xen/arch/x86/dmi_scan.c | 25 +++--
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
index 187c01e..ec411e6 100644
--- a/xen/arch/x86/dmi_scan.c
+++ b/xen/arch/x86/dmi_scan.c
@@ -189,6 +189,8 @@ static inline bool_t __init dmi_checksum(const void __iomem 
*buf,
 
 static u32 __initdata efi_dmi_address;
 static u32 __initdata efi_dmi_size;
+static u32 __initdata efi_smbios_address;
+static u32 __initdata efi_smbios_size;
 static u64 __initdata efi_smbios3_address;
 static u32 __initdata efi_smbios3_size;
 
@@ -209,13 +211,18 @@ void __init dmi_efi_get_table(const void *smbios, const 
void *smbios3)
return;
}
 
-   if (eps  memcmp(eps-anchor, _SM_, 4) 
+   if (eps  memcmp(eps-anchor, _SM_, 4) == 0 
eps-length = sizeof(*eps) 
-   dmi_checksum(eps, eps-length) 
-   memcmp(eps-dmi.anchor, _DMI_, 5) == 0 
-   dmi_checksum(eps-dmi, sizeof(eps-dmi))) {
-   efi_dmi_address = eps-dmi.address;
-   efi_dmi_size = eps-dmi.size;
+   dmi_checksum(eps, eps-length)) {
+   efi_smbios_address = (u32)(long)smbios;
+   efi_smbios_size = eps-length;
+
+   if ( memcmp(eps-dmi.anchor, _DMI_, 5) == 0 
+dmi_checksum(eps-dmi,
+ eps-length - offsetof(struct smbios_eps, 
dmi)) ) {
+   efi_dmi_address = eps-dmi.address;
+   efi_dmi_size = eps-dmi.size;
+   }
}
 }
 
@@ -236,6 +243,12 @@ const char *__init dmi_get_table(paddr_t *base, u32 *len)
instance |= 2;
return DMI;
}
+   if (efi_smbios_size  !(instance  4)) {
+   *base = efi_smbios_address;
+   *len = efi_smbios_size;
+   instance |= 4;
+   return SMBIOS;
+   }
} else {
char __iomem *p = maddr_to_virt(0xF), *q;
union {
-- 
2.1.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC PATCH v2 06/22] xen/arm: its: Port ITS driver to xen

2015-04-02 Thread Vijay Kilari
On Wed, Apr 1, 2015 at 5:04 PM, Ian Campbell ian.campb...@citrix.com wrote:
 On Thu, 2015-03-19 at 20:07 +0530, vijay.kil...@gmail.com wrote:
 From: Vijaya Kumar K vijaya.ku...@caviumnetworks.com

 This patch just makes ITS driver taken from linux
 compiles in xen environment.

 What is your intention wrt future updates to this driver?

 Are you intending to keep things in sync and import things from the
 Linux side (similar to the smmu drviers) or are you taking the Linux
 code as a starting point and intending that it then be maintained
 independently as a Xen driver from then on?

Yes, I intend to keep things in sync with Linux driver.
I have kept most the code same as Linux side except removing unused code.

Regards
Vijay

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 4/4] hvmloader: add knob for fixed VGABIOS date string

2015-04-02 Thread Ian Campbell
On Wed, 2015-04-01 at 13:28 +, Olaf Hering wrote:
 To allow reproducible builds of hvmloader introduce a make variable
 VGABIOS_REL_DATE=dd Mon  to provide a fixed date string. Without
 this change the hvmloader binary changes with every rebuild.
 
 Signed-off-by: Olaf Hering o...@aepfle.de
 Cc: Ian Jackson ian.jack...@eu.citrix.com
 Cc: Stefano Stabellini stefano.stabell...@eu.citrix.com

Acked-by: Ian Campbell ian.campb...@citrix.com

It's not clear if vgabios should be maintained by us tools folks or if
it should have been taken under the hvmloader umbrella and maintained by
the hypervisor folks.

I've CC-d Jan, but I'm not going to block on an ack.

 Cc: Wei Liu wei.l...@citrix.com
 ---
  INSTALL | 1 +
  tools/firmware/vgabios/Makefile | 4 ++--
  2 files changed, 3 insertions(+), 2 deletions(-)
 
 diff --git a/INSTALL b/INSTALL
 index bf412ef..a0f2e7b 100644
 --- a/INSTALL
 +++ b/INSTALL
 @@ -217,6 +217,7 @@ can be used to provide fixed timestamps in the expected 
 format.
  XEN_BUILD_DATE=output of date(1)
  XEN_BUILD_TIME=hh:mm:ss
  SMBIOS_REL_DATE=mm/dd/
 +VGABIOS_REL_DATE=dd Mon 
  
  The following variables can be used to tweak some aspects of the
  hypervisor build.
 diff --git a/tools/firmware/vgabios/Makefile b/tools/firmware/vgabios/Makefile
 index 51d9e6e..3284812 100644
 --- a/tools/firmware/vgabios/Makefile
 +++ b/tools/firmware/vgabios/Makefile
 @@ -5,10 +5,10 @@ BCC = bcc
  AS86 = as86
  
  RELEASE = `pwd | sed s-.*/--`
 -RELDATE = `date '+%d %b %Y'`
 +VGABIOS_REL_DATE ?= `date '+%d %b %Y'`
  RELVERS = `pwd | sed s-.*/-- | sed s/vgabios// | sed s/-//`
  
 -VGABIOS_DATE = -DVGABIOS_DATE=\$(RELDATE)\
 +VGABIOS_DATE = -DVGABIOS_DATE=\$(VGABIOS_REL_DATE)\
  
  .PHONY: all
  all: bios cirrus-bios



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 09/47] vidoe: fbdev: atyfb: remove and fix MTRR MMIO hole work around

2015-04-02 Thread Andy Lutomirski
On Thu, Apr 2, 2015 at 12:45 PM, Luis R. Rodriguez mcg...@suse.com wrote:
 On Wed, Apr 01, 2015 at 05:04:08PM -0700, Andy Lutomirski wrote:
 On Wed, Apr 1, 2015 at 4:52 PM, Luis R. Rodriguez mcg...@suse.com wrote:
  On Sat, Mar 28, 2015 at 02:23:34PM +0200, Ville Syrjälä wrote:
  On Sat, Mar 28, 2015 at 01:28:18AM +0100, Luis R. Rodriguez wrote:
   On Fri, Mar 27, 2015 at 03:02:10PM -0700, Andy Lutomirski wrote:
On Fri, Mar 27, 2015 at 2:56 PM, Ville Syrjälä syrj...@sci.fi wrote:
 On Fri, Mar 27, 2015 at 08:57:59PM +0100, Luis R. Rodriguez wrote:
 On Fri, Mar 27, 2015 at 12:43:55PM -0700, Andy Lutomirski wrote:
  On Fri, Mar 27, 2015 at 12:38 PM, Luis R. Rodriguez 
  mcg...@suse.com wrote:
   On Sat, Mar 21, 2015 at 11:15:14AM +0200, Ville Syrjälä wrote:
   On Fri, Mar 20, 2015 at 04:17:59PM -0700, Luis R. Rodriguez 
   wrote:
diff --git a/drivers/video/fbdev/aty/atyfb_base.c 
b/drivers/video/fbdev/aty/atyfb_base.c
index 8025624..8875e56 100644
--- a/drivers/video/fbdev/aty/atyfb_base.c
+++ b/drivers/video/fbdev/aty/atyfb_base.c
@@ -2630,21 +2630,10 @@ static int aty_init(struct fb_info 
*info)
   
 #ifdef CONFIG_MTRR
par-mtrr_aper = -1;
-   par-mtrr_reg = -1;
if (!nomtrr) {
-   /* Cover the whole resource. */
-   par-mtrr_aper = mtrr_add(par-res_start, 
par-res_size,
+   par-mtrr_aper = mtrr_add(info-fix.smem_start,
+ info-fix.smem_len,
  MTRR_TYPE_WRCOMB, 1);
  
   MTRRs need power of two size, so how is this supposed to work?
  
   As per mtrr_add_page() [0] the base and size are just supposed 
   to be in units
   of 4 KiB, although the practice is to use powers of 2 in 
   *some* drivers this
   is not standardized and by no means recorded as a requirement. 
   Obviously
   powers of 2 will work too and you'd end up neatly aligned as 
   well. mtrr_add()
   will use mtrr_check() to verify the the same requirement. 
   Furthermore,
   as per my commit log message:
 
  Whatever the code may or may not do, the x86 architecture uses
  power-of-two MTRR sizes.  So I'm confused.

 There should be no confusion, I simply did not know that *was* the
 requirement for x86, if that is the case we should add a check for 
 that
 and perhaps generalize a helper that does the power of two helper 
 changes,
 the cleanest I found was the vesafb driver solution.

 Thoughts?

 The vesafb solution is bad since you'll only end up covering only
 the first 4MB of the framebuffer instead of the almost 8MB you want.
 Which in practice will mean throwing away half the VRAM since you 
 really
 don't want the massive performance hit from accessing it as UC. And 
 that
 would mean giving up decent display resolutions as well :(

 And the other option of trying to cover the remainder with multiple 
 ever
 smaller MTRRs doesn't work either since you'll run out of MTRRs very
 quickly.

 This is precisely why I used the hole method in atyfb in the first
 place.

 I don't really like the idea of any new mtrr code not supporting 
 that
 use case, especially as these things tend to be present in older 
 machines
 where PAT isn't an option.
   
According to the Intel SDM, volume 3, section 11.5.2.1, table 11-6,
non-PAT CPUs that have a WC MTRR, PCD = 1, and PWT = 1 (aka UC) have
an effective memory type of UC.
 
  This is true but non-PAT systems that use just ioremap() will default to
  _PAGE_CACHE_MODE_UC_MINUS, not _PAGE_CACHE_MODE_UC, and 
  _PAGE_CACHE_MODE_UC_MINUS
  on Linux has PCD = 1, PWT = 0. The list comes from:
 
  uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = {
  [_PAGE_CACHE_MODE_WB  ] = 0 | 0,
  [_PAGE_CACHE_MODE_WC  ] = _PAGE_PWT | 0,
  [_PAGE_CACHE_MODE_UC_MINUS] = 0 | _PAGE_PCD,
  [_PAGE_CACHE_MODE_UC  ] = _PAGE_PWT | _PAGE_PCD,
  [_PAGE_CACHE_MODE_WT  ] = 0 | _PAGE_PCD,
  [_PAGE_CACHE_MODE_WP  ] = 0 | _PAGE_PCD,
  };
 
  This can better be read here:
 
   PAT
   |PCD
   ||PWT
   |||
   000 WB  _PAGE_CACHE_MODE_WB
   001 WC  _PAGE_CACHE_MODE_WC
   010 UC- _PAGE_CACHE_MODE_UC_MINUS
   011 UC  _PAGE_CACHE_MODE_UC
 
  On x86 ioremap() defaults to ioremap_nocache() and right now that uses
  _PAGE_CACHE_MODE_UC_MINUS not _PAGE_CACHE_MODE_UC. We have two cases
  to consider for non-PAT systems then:
 
  a) Right now as ioremap() and ioremap_nocache() default to 
  _PAGE_CACHE_MODE_UC_MINUS
 on x86. In this case using a WC MTRR seems to use PWT=0, PCD=1, and
 table table 

Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Bjorn Helgaas
On Thu, Apr 2, 2015 at 3:20 PM, Luis R. Rodriguez mcg...@suse.com wrote:
 On Thu, Apr 2, 2015 at 1:13 PM, Bjorn Helgaas bhelg...@google.com wrote:

 On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote:

  I'll rephrase this to:
 
  ---
  It is possible to enable CONFIG_MTRR and up with it
  disabled at run time and yet CONFIG_X86_PAT continues
  to kick through with all functionally enabled. This
  can happen for instance on Xen where MTRR is not
  supported but PAT is, this can happen now on Linux as
  of commit 47591df50 by Juergen introduced as of v3.19.

 I still can't parse this.  What does up with it disabled at run time
 mean?

 It  means that technically even if your CPU/BIOS/system did support
 MTRR if you use a kernel with MTRR support enabled you might end up
 with a situation where under one situation MTRR  might be enabled and
 at another run time scenario with the same exact kernel and system you
 will end up with MTRR disabled. Such is the case for example when
 booting with Xen, which disables the CPU bits on the hypervisor code.
 If you boot the same system without Xen you'll get MTRR.

Your text is missing some words.  You seem to be using up as a verb,
but it's not a verb.  Maybe you meant end up?  Even then, it
wouldn't make sense for CONFIG_MTRR to be disabled at run time
because CONFIG_MTRR is a compile-time switch.  The MTRR
*functionality* could certainly be disabled at run-time, but not
CONFIG_MTRR itself.

  And ... continues to kick through?  Probably some idiomatic
 usage I'm just too old to understand :)

 That means for example that in both the above circumstances even if
 MTRR went disabled at run time with Xen, the kernel went through with
 getting PAT enabled.

CONFIG_X86_PAT continues to kick through doesn't seem a very precise
way of describing this.  But maybe it's enough for experts in this
area (which I'm not).

Bjorn

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()

2015-04-02 Thread Bjorn Helgaas
On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez
mcg...@do-not-panic.com wrote:
 From: Luis R. Rodriguez mcg...@suse.com

 Ideally on systems using PAT we can expect a swift
 transition away from MTRR. There can be a few exceptions
 to this, one is where device drivers are known to exist
 on PATs with errata,

This probably makes sense to someone steeped in MTRR and PAT, but not
otherwise.  One exception is where drivers are known to exist on PATs
with errata?  The drivers exist, independent of PAT/MTRR/errata.  Do
you mean there are drivers that can't be converted from MTRR to PAT
because some PATs are broken?

I don't really know anything about MTRR or PAT; I'm just trying to
figure out how to parse this paragraph.

 another situation is observed on
 old device drivers where devices had combined MMIO
 register access with whatever area they typically
 later wanted to end up using MTRR for on the same
 PCI BAR. This situation can still be addressed by
 splitting up ioremap'd PCI BAR into two ioremap'd
 calls, one for MMIO registers, and another for whatever
 is desirable for write-combining -- in order to
 accomplish this though quite a bit of driver
 restructuring is required.

 Device drivers which are known to require large
 amount of re-work in order to split ioremap'd areas
 can use __arch_phys_wc_add() to avoid regressions
 when PAT is enabled.

 For a good example driver where things are neatly
 split up on a PCI BAR refer the infiniband qib
 driver. For a good example of a driver where good
 amount of work is required refer to the infiniband
 ipath driver.

 This is *only* a transitive API -- and as such no new
 drivers are ever expected to use this.

transient?  Do you mean you intend to remove this API in the near future?

Bjorn

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Luis R. Rodriguez
On Thu, Apr 2, 2015 at 1:13 PM, Bjorn Helgaas bhelg...@google.com wrote:

 On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote:

  I'll rephrase this to:
 
  ---
  It is possible to enable CONFIG_MTRR and up with it
  disabled at run time and yet CONFIG_X86_PAT continues
  to kick through with all functionally enabled. This
  can happen for instance on Xen where MTRR is not
  supported but PAT is, this can happen now on Linux as
  of commit 47591df50 by Juergen introduced as of v3.19.

 I still can't parse this.  What does up with it disabled at run time
 mean?

It  means that technically even if your CPU/BIOS/system did support
MTRR if you use a kernel with MTRR support enabled you might end up
with a situation where under one situation MTRR  might be enabled and
at another run time scenario with the same exact kernel and system you
will end up with MTRR disabled. Such is the case for example when
booting with Xen, which disables the CPU bits on the hypervisor code.
If you boot the same system without Xen you'll get MTRR.

  And ... continues to kick through?  Probably some idiomatic
 usage I'm just too old to understand :)

That means for example that in both the above circumstances even if
MTRR went disabled at run time with Xen, the kernel went through with
getting PAT enabled.

 Please use the conventional citation format:

   47591df50512 (xen: Support Xen pv-domains using PAT)

 A one-character typo in a SHA1 makes it completely useless, so it's
 nice to have the summary line both for readability and a bit of
 redundancy.

Sure, fixed.

 Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] implementing a classic Xen front/back pv driver pair to provide a transport for 9P FS Protocol

2015-04-02 Thread Konrad Rzeszutek Wilk
On Thu, Apr 02, 2015 at 01:16:36PM -0600, Linda wrote:
 Thanks Konrad.
 
 I came close to doing your suggestions on git, and got it, sort of, to work.
 I will follow your instructions exactly for updates.

Heh. The git manual is also a very good resource.
 
 There were also, some problems with the way the smtp server needed to be set
 up (compounded by typos of course), which are all fixed.
 
 As far as your answer to my build problems, I'm not sure what this will do:
 
 One way to help with that is if you do:
 
 make 12 2log
 
 And attach the log.
 
 I am semi-new to the linux world - is this to allow someone else to help me?

Correct.
 What does the make command above do?

Build.
 
 Thanks, again.
 
 Linda
 
 On 4/2/2015 12:59 PM, Konrad Rzeszutek Wilk wrote:
 On Wed, Apr 01, 2015 at 07:36:49PM -0600, Linda wrote:
 First, Julien, your suggestion worked like a charm.
 So here's what's happened tonight.  I tried to build the tools directory of
 my git repository.  Although I used the sudo command in my virtual ubuntu, I
 got a permission denied error  126 on xen_foreign.
 One way to help with that is if you do:
 
 make 12 2log
 
 And attach the log.
 Second, I tried to follow the protocol for submitting my patches.  I changed
 libxl_utils.c and libxl_utils.h, in my repository, add and commit them.
 Unfortunately, I didn't include my signature (next page of instructions I
 was following), and couldn't figure how to get back in to add them.
 git commit --amend
 
 Or if you want to do more of them:
 
 git rebase -i origin/staging
 
 (and in the editor change 'pick' to 'r').
 
 Finally, I tried git send-email (took a bit to find I had to install it).
 Now it doesn't like the format of my send-email:
 Again, please copy-n-paste the command line you had.
 
   to the devlopers list above and cc'ing Julien and Wei, followed by:
 1.  following this with the files (even with --no-format-patch), error was
 no subject line
 2.  (different attempt) the repository master  error complained about the
 format patch
 Not sure I understand that. Is that for 'git send-email' or 'git
 format-patch'?
 SO   if anyone is up at an ungodly hour and can explain any of these errors
 to me (I'm in Colorado - so it's 7:30 here), especially with a fix, I'd be
 grateful.  Otherwise, Julien, Wei, I'll start at about 7am my time, maybe a
 little earlier.
 The previous OPW had an article about using git and how to do it with
 kernel patches. It is exactly the same flow - except different email
 address.
 
 Anyhow, what I end up doing is:
 
 1). git format-patch --subject-prefix PATCH RFC origin/staging..
 
 (which generates 0001-, 0002-, etc files - for two patches).
 
 2). git send-email --to xen-de...@xenproject.org --to someotheremail.org
 --compose --subject [PATCH RFC] Patches to fix XYZ. 000*.patch
 
 And in the editor do a little writeup of what the patches have.
 At the end of this, I attach the output from:
 
 git diff --stat origin/staging..
 
 git shortlog origin/staging..
 
 which gives a nice diff output and what the patches are.
 
 And then send it off.
 Thanks.
 
 Linda Jacobson
 
 On 4/1/2015 2:57 PM, Julien Grall wrote:
 
 On 01/04/2015 18:46, Linda wrote:
 I'll try it.  That's the
 
 libvncserver-dev libsdl-dev libjpeg62-dev
 
 Should I keep the libsdl-dev?
 
In the meantime, I'm following the git protocol for patches. I
 successfully cloned xen.git.  The next statement in the directions - I
 can't tell if it's one statement on many lines, or many statements.  It
 starts out git branch -a
 When I type this alone, I get Not a git repository  When I type in the
 many lines as a single command I get the error message:
 origin/master no such file or directory
 You have to type the command git branch -a  in the git repository (i.e
 the directory xen.git).
 
 
 This comes from the line remotes/origin/HEAD-origin
 
 ?
 This is normal. The line starting by '$' is a command. Everything else is
 an example output of the execution of the command.
 
 Obviously, you have to drop the '$' when typing copying the command.
 
 To go further, '$' means a command to execute with your current user and
 '#' a command to execute with root privileges (i.e adding sudo before).
 
 This is usually a standard on Linux/BSD shell documentation.
 
 Although, there is some place within this wiki page where the command
 doesn't have '$'/'#' (see [1]). So you to judge yourself if the line looks
 like a command or not :).
 
 Regards,
 
 [1] 
 http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Git_send-email
 
 
 
 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] implementing a classic Xen front/back pv driver pair to provide a transport for 9P FS Protocol

2015-04-02 Thread Linda Jacobson
Konrad -

Here's the log.  Maybe someone can make sense of this.  Thanks.

Linda

log
Description: Binary data
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Luis R. Rodriguez
On Sat, Mar 28, 2015 at 12:56:30AM +0100, Luis R. Rodriguez wrote:
 On Fri, Mar 27, 2015 at 02:40:17PM -0600, Toshi Kani wrote:
  On Fri, 2015-03-20 at 16:17 -0700, Luis R. Rodriguez wrote:
   :
   @@ -734,6 +742,7 @@ void __init mtrr_bp_init(void)
 }

 if (mtrr_if) {
   + mtrr_enabled = true;
 set_num_var_ranges();
 init_table();
 if (use_intel()) {
  get_mtrr_state();
  
  After setting mtrr_enabled to true, get_mtrr_state() reads
  MSR_MTRRdefType and sets 'mtrr_state.enabled', which also indicates if
  MTRRs are enabled or not on the system.  So, potentially, we could have
  a case that mtrr_enabled is set to true, but mtrr_state.enabled is set
  to disabled when MTRRs are disabled by BIOS.
 
 Thanks for the review, in this case then we should update mtrr_enabled to 
 false.
 
  ps.
  I recently cleaned up this part of the MTRR code in the patch below,
  which is currently available in the -mm  -next trees.
  https://lkml.org/lkml/2015/3/24/1063
 
 Great I will rebase and work with that and try to address this
 consideration you have raised.

OK I'll mesh in this change as well in my next respin:

diff --git a/arch/x86/kernel/cpu/mtrr/generic.c 
b/arch/x86/kernel/cpu/mtrr/generic.c
index a83f27a..ecf7cb9 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -438,7 +438,7 @@ static void __init print_mtrr_state(void)
 }
 
 /* Grab all of the MTRR state for this CPU into *state */
-void __init get_mtrr_state(void)
+bool __init get_mtrr_state(void)
 {
struct mtrr_var_range *vrs;
unsigned long flags;
@@ -482,6 +482,8 @@ void __init get_mtrr_state(void)
 
post_set();
local_irq_restore(flags);
+
+   return !!mtrr_state.enabled;
 }
 
 /* Some BIOS's are messed up and don't set all MTRRs the same! */
diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index ea5f363..f96195e 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -734,22 +742,25 @@ void __init mtrr_bp_init(void)
}
 
if (mtrr_if) {
+   mtrr_enabled = true;
set_num_var_ranges();
init_table();
if (use_intel()) {
-   get_mtrr_state();
+   /* BIOS may override */
+   mtrr_enabled = get_mtrr_state();
 
if (mtrr_cleanup(phys_addr)) {
changed_by_mtrr_cleanup = 1;
@@ -745,11 +755,14 @@ void __init mtrr_bp_init(void)
}
}
}
+
+   if (!mtrr_enabled)
+   pr_info(mtrr: system does not support MTRR\n);
 }
 

diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h
index df5e41f..951884d 100644
--- a/arch/x86/kernel/cpu/mtrr/mtrr.h
+++ b/arch/x86/kernel/cpu/mtrr/mtrr.h
@@ -51,7 +51,7 @@ void set_mtrr_prepare_save(struct set_mtrr_context *ctxt);
 
 void fill_mtrr_var_range(unsigned int index,
u32 base_lo, u32 base_hi, u32 mask_lo, u32 mask_hi);
-void get_mtrr_state(void);
+bool get_mtrr_state(void);
 
 extern void set_mtrr_ops(const struct mtrr_ops *ops);
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Bjorn Helgaas
On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote:

 I'll rephrase this to:

 ---
 It is possible to enable CONFIG_MTRR and up with it
 disabled at run time and yet CONFIG_X86_PAT continues
 to kick through with all functionally enabled. This
 can happen for instance on Xen where MTRR is not
 supported but PAT is, this can happen now on Linux as
 of commit 47591df50 by Juergen introduced as of v3.19.

I still can't parse this.  What does up with it disabled at run time
mean?  And ... continues to kick through?  Probably some idiomatic
usage I'm just too old to understand :)

Please use the conventional citation format:

  47591df50512 (xen: Support Xen pv-domains using PAT)

A one-character typo in a SHA1 makes it completely useless, so it's
nice to have the summary line both for readability and a bit of
redundancy.

Bjorn

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools/libxl new bitmap functions

2015-04-02 Thread Konrad Rzeszutek Wilk
On Thu, Apr 02, 2015 at 11:38:16AM -0600, Linda Jacobson wrote:
 From: Linda lin...@jma3.com
 
 Added bitmap functions for union intersection and difference betweenn two 
 bitmaps
 
 Signed-off-by: Linda lin...@jma3.com
 ---
  tools/libxl/libxl_utils.c | 115 
 ++
  tools/libxl/libxl_utils.h |  10 
  2 files changed, 125 insertions(+)
 
 diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
 index 9053b27..c390ddc 100644
 --- a/tools/libxl/libxl_utils.c
 +++ b/tools/libxl/libxl_utils.c
 @@ -699,6 +699,121 @@ int libxl_bitmap_count_set(const libxl_bitmap *bitmap)
  
  return nr_set_bits;
  }
 +int libxl_bitmap_union(libxl_ctx *ctx, libxl_bitmap *union_bitmap, 

You have an extra space at the end..
 +libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)

And this should really start at the same line length as 'libxl_ctx'

Ditto for the rest of the functions.
 +{
 +int size;
 +int rc;
 +
 +GC_INIT(ctx);
 +
 +// if bitmaps aren't the same size, union should be size of larger bit map

The comments are in:

 /* Comment here. */


 +size = (bitmap1-size  bitmap2-size) ? bitmap1-size : bitmap2-size;

There might be an 'max' macro somwhere in the code that you could use
for this?

 +
 +libxl_bitmap_init(union_bitmap);
 +rc = libxl_bitmap_alloc(ctx, union_bitmap, size);
 +if (rc)
 +{
 +// Following the coding standards here.  
 + //First goto I've written in decades.

Hehe.

No need to add the braces, you can just do:

if (rc)
goto out;

 +goto out;
 +}
 +
 +for (int bit = 0; bit  (size * 8); bit++)

Please move the 'int bit' to the start of the function.

 +{
 +// libxl_bitmap_test returns 0 if past end of bitmap
 +// if the bit is set in either bitmap, set it in their union

I would change that comment to be:

/* NB. libxl_bitmap_test returns 0 if past the end of bitmap. */

 +if (libxl_bitmap_test(bitmap1, bit))
 +{
 +libxl_bitmap_set(union_bitmap, bit);
 +}
 +else if (libxl_bitmap_test(bitmap2, bit))
 +{
 +libxl_bitmap_set(union_bitmap, bit);
 +}

You can ditch the braces.

 +}
 +
 +out:
 +GC_FREE;
 +return rc;
 +}
 +
 +int libxl_bitmap_intersection (libxl_ctx *ctx, libxl_bitmap 
There is space :^ - which should not be there.

 +*intersection_bitmap, libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)
 +{
 +int size;
 +int rc;
 +
 +GC_INIT(ctx);
 +
 +// if bitmaps aren't same size, intersection should be size of 
 +// smaller bit map

I believe the comments I've above apply to the code below as well.

 +size = (bitmap1-size  bitmap2-size) ? bitmap2-size : bitmap1-size;
 +
 +libxl_bitmap_init(intersection_bitmap);
 +rc = libxl_bitmap_alloc(ctx, intersection_bitmap, size);
 +if (rc)
 +{
 +goto out;
 +}
 +
 +for (int bit = 0; bit  (size * 8); bit++)
 +{
 +// libxl_bitmap_test returns 0 if past end of bitmap
 +// if the bit is set in both bitmaps, set it in their intersection
 +if (libxl_bitmap_test (bitmap1, bit) 
 +  libxl_bitmap_test (bitmap2, bit) )

You have an extra space at the end there. Don't think you need that
in libxl code (thought you do for libxc). Yeah, two different
styleguides!

 +{
 +libxl_bitmap_set (intersection_bitmap, bit);
 +}
 +}
 +
 +out:
 +GC_FREE;
 +return rc;
 +}
 +int libxl_bitmap_difference(libxl_ctx *ctx, libxl_bitmap *difference_bitmap, 
 +libxl_bitmap *bitmap1, libxl_bitmap *bitmap2)
 +{
 +int size;
 +int rc;
 +
 +GC_INIT(ctx);
 +
 +// if bitmaps aren't the same size, difference should be size of larger 
 +size = (bitmap1-size  bitmap2-size) ? bitmap1-size : bitmap2-size;
 +
 +libxl_bitmap_init(difference_bitmap);
 +rc = libxl_bitmap_alloc(ctx, difference_bitmap, size);
 +if (rc)
 +{
 +goto out;
 +}
 +
 +for (int bit = 0; bit  (size * 8); bit++)
 +{
 +/* libxl_bitmap_test returns 0 if past end of bitmap
 + if the bit is set in one bitmap and not the other, set it in 
 +their difference
 +NOTE:  if one bit map is larger, this will result in all bits 
 +being set past the size of the smaller bitmap;  if this is not
 +the desired behavior, please let me know

That would make this a bit strange. Perhaps demand that they
must be the same size? If they are not then just return an error?
 +*/
 +
 +if (libxl_bitmap_test (bitmap1, bit) 
You have an extra space at the end there.  ^
 +  (!libxl_bitmap_test (bitmap2, bit)) )
 +{
 +libxl_bitmap_set (difference_bitmap, bit);
 +}
 +}
 +
 +out:
 +GC_FREE;
 +return rc;
 +}
 +
 +
 +
  
  /* NB. caller is responsible for freeing the memory */
  char *libxl_bitmap_to_hex_string(libxl_ctx *ctx, const libxl_bitmap *bitmap)
 

Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()

2015-04-02 Thread Luis R. Rodriguez
On Thu, Apr 02, 2015 at 03:21:22PM -0500, Bjorn Helgaas wrote:
 On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez
 mcg...@do-not-panic.com wrote:
  From: Luis R. Rodriguez mcg...@suse.com
 
  Ideally on systems using PAT we can expect a swift
  transition away from MTRR. There can be a few exceptions
  to this, one is where device drivers are known to exist
  on PATs with errata,
 
 This probably makes sense to someone steeped in MTRR and PAT, but not
 otherwise.  One exception is where drivers are known to exist on PATs
 with errata?  The drivers exist, independent of PAT/MTRR/errata.  Do
 you mean there are drivers that can't be converted from MTRR to PAT
 because some PATs are broken?

Well there is that but it seems we have motivation to
address the PAT broken systems so this would be one of the
lower priority reasons to consider adding this API. The
more important reason is below.

 I don't really know anything about MTRR or PAT; I'm just trying to
 figure out how to parse this paragraph.

Sure.

  another situation is observed on
  old device drivers where devices had combined MMIO
  register access with whatever area they typically
  later wanted to end up using MTRR for on the same
  PCI BAR. This situation can still be addressed by
  splitting up ioremap'd PCI BAR into two ioremap'd
  calls, one for MMIO registers, and another for whatever
  is desirable for write-combining -- in order to
  accomplish this though quite a bit of driver
  restructuring is required.
 
  Device drivers which are known to require large
  amount of re-work in order to split ioremap'd areas
  can use __arch_phys_wc_add() to avoid regressions
  when PAT is enabled.
 
  For a good example driver where things are neatly
  split up on a PCI BAR refer the infiniband qib
  driver. For a good example of a driver where good
  amount of work is required refer to the infiniband
  ipath driver.
 
  This is *only* a transitive API -- and as such no new
  drivers are ever expected to use this.
 
 transient?  Do you mean you intend to remove this API in the near future?

That's correct, the problem is that in order to use PAT cleanly we'd need to
change these drivers to not overlap ioremap'd areas otherwise things can get
quite complex, and changing the way we do the ioremap() calls on a driver might
require a bit of work. The atyfb driver changes I did are an example of the
types of changes that are expected.  In the most complex worst cases there are
MTRR hole tricks used, and as can be observed with the atyfb driver changes
there are a series of things to consider when this is done specially in light
of eventually making strong UC the default instead of UC-.

I might be able to work around not adding this API by reviewing the users I had
in this series again and seeing if something similar to what I will do on atyfb
can be done in the meantime by using ioremap_uc(). Its not clear to me yet.

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 09/47] vidoe: fbdev: atyfb: remove and fix MTRR MMIO hole work around

2015-04-02 Thread Luis R. Rodriguez
On Wed, Apr 01, 2015 at 05:04:08PM -0700, Andy Lutomirski wrote:
 On Wed, Apr 1, 2015 at 4:52 PM, Luis R. Rodriguez mcg...@suse.com wrote:
  On Sat, Mar 28, 2015 at 02:23:34PM +0200, Ville Syrjälä wrote:
  On Sat, Mar 28, 2015 at 01:28:18AM +0100, Luis R. Rodriguez wrote:
   On Fri, Mar 27, 2015 at 03:02:10PM -0700, Andy Lutomirski wrote:
On Fri, Mar 27, 2015 at 2:56 PM, Ville Syrjälä syrj...@sci.fi wrote:
 On Fri, Mar 27, 2015 at 08:57:59PM +0100, Luis R. Rodriguez wrote:
 On Fri, Mar 27, 2015 at 12:43:55PM -0700, Andy Lutomirski wrote:
  On Fri, Mar 27, 2015 at 12:38 PM, Luis R. Rodriguez 
  mcg...@suse.com wrote:
   On Sat, Mar 21, 2015 at 11:15:14AM +0200, Ville Syrjälä wrote:
   On Fri, Mar 20, 2015 at 04:17:59PM -0700, Luis R. Rodriguez 
   wrote:
diff --git a/drivers/video/fbdev/aty/atyfb_base.c 
b/drivers/video/fbdev/aty/atyfb_base.c
index 8025624..8875e56 100644
--- a/drivers/video/fbdev/aty/atyfb_base.c
+++ b/drivers/video/fbdev/aty/atyfb_base.c
@@ -2630,21 +2630,10 @@ static int aty_init(struct fb_info 
*info)
   
 #ifdef CONFIG_MTRR
par-mtrr_aper = -1;
-   par-mtrr_reg = -1;
if (!nomtrr) {
-   /* Cover the whole resource. */
-   par-mtrr_aper = mtrr_add(par-res_start, 
par-res_size,
+   par-mtrr_aper = mtrr_add(info-fix.smem_start,
+ info-fix.smem_len,
  MTRR_TYPE_WRCOMB, 1);
  
   MTRRs need power of two size, so how is this supposed to work?
  
   As per mtrr_add_page() [0] the base and size are just supposed 
   to be in units
   of 4 KiB, although the practice is to use powers of 2 in *some* 
   drivers this
   is not standardized and by no means recorded as a requirement. 
   Obviously
   powers of 2 will work too and you'd end up neatly aligned as 
   well. mtrr_add()
   will use mtrr_check() to verify the the same requirement. 
   Furthermore,
   as per my commit log message:
 
  Whatever the code may or may not do, the x86 architecture uses
  power-of-two MTRR sizes.  So I'm confused.

 There should be no confusion, I simply did not know that *was* the
 requirement for x86, if that is the case we should add a check for 
 that
 and perhaps generalize a helper that does the power of two helper 
 changes,
 the cleanest I found was the vesafb driver solution.

 Thoughts?

 The vesafb solution is bad since you'll only end up covering only
 the first 4MB of the framebuffer instead of the almost 8MB you want.
 Which in practice will mean throwing away half the VRAM since you 
 really
 don't want the massive performance hit from accessing it as UC. And 
 that
 would mean giving up decent display resolutions as well :(

 And the other option of trying to cover the remainder with multiple 
 ever
 smaller MTRRs doesn't work either since you'll run out of MTRRs very
 quickly.

 This is precisely why I used the hole method in atyfb in the first
 place.

 I don't really like the idea of any new mtrr code not supporting that
 use case, especially as these things tend to be present in older 
 machines
 where PAT isn't an option.
   
According to the Intel SDM, volume 3, section 11.5.2.1, table 11-6,
non-PAT CPUs that have a WC MTRR, PCD = 1, and PWT = 1 (aka UC) have
an effective memory type of UC.
 
  This is true but non-PAT systems that use just ioremap() will default to
  _PAGE_CACHE_MODE_UC_MINUS, not _PAGE_CACHE_MODE_UC, and 
  _PAGE_CACHE_MODE_UC_MINUS
  on Linux has PCD = 1, PWT = 0. The list comes from:
 
  uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = {
  [_PAGE_CACHE_MODE_WB  ] = 0 | 0,
  [_PAGE_CACHE_MODE_WC  ] = _PAGE_PWT | 0,
  [_PAGE_CACHE_MODE_UC_MINUS] = 0 | _PAGE_PCD,
  [_PAGE_CACHE_MODE_UC  ] = _PAGE_PWT | _PAGE_PCD,
  [_PAGE_CACHE_MODE_WT  ] = 0 | _PAGE_PCD,
  [_PAGE_CACHE_MODE_WP  ] = 0 | _PAGE_PCD,
  };
 
  This can better be read here:
 
   PAT
   |PCD
   ||PWT
   |||
   000 WB  _PAGE_CACHE_MODE_WB
   001 WC  _PAGE_CACHE_MODE_WC
   010 UC- _PAGE_CACHE_MODE_UC_MINUS
   011 UC  _PAGE_CACHE_MODE_UC
 
  On x86 ioremap() defaults to ioremap_nocache() and right now that uses
  _PAGE_CACHE_MODE_UC_MINUS not _PAGE_CACHE_MODE_UC. We have two cases
  to consider for non-PAT systems then:
 
  a) Right now as ioremap() and ioremap_nocache() default to 
  _PAGE_CACHE_MODE_UC_MINUS
 on x86. In this case using a WC MTRR seems to use PWT=0, PCD=1, and
 table table 11-6 on non-PAT systems seems to place this situation as
 implementation 

Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Luis R. Rodriguez
On Thu, Apr 02, 2015 at 03:28:51PM -0500, Bjorn Helgaas wrote:
 On Thu, Apr 2, 2015 at 3:20 PM, Luis R. Rodriguez mcg...@suse.com wrote:
  On Thu, Apr 2, 2015 at 1:13 PM, Bjorn Helgaas bhelg...@google.com wrote:
 
  On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote:
 
   I'll rephrase this to:
  
   ---
   It is possible to enable CONFIG_MTRR and up with it
   disabled at run time and yet CONFIG_X86_PAT continues
   to kick through with all functionally enabled. This
   can happen for instance on Xen where MTRR is not
   supported but PAT is, this can happen now on Linux as
   of commit 47591df50 by Juergen introduced as of v3.19.
 
  I still can't parse this.  What does up with it disabled at run time
  mean?
 
  It  means that technically even if your CPU/BIOS/system did support
  MTRR if you use a kernel with MTRR support enabled you might end up
  with a situation where under one situation MTRR  might be enabled and
  at another run time scenario with the same exact kernel and system you
  will end up with MTRR disabled. Such is the case for example when
  booting with Xen, which disables the CPU bits on the hypervisor code.
  If you boot the same system without Xen you'll get MTRR.
 
 Your text is missing some words.  You seem to be using up as a verb,
 but it's not a verb.  Maybe you meant end up? 

Indeed.

 Even then, it
 wouldn't make sense for CONFIG_MTRR to be disabled at run time
 because CONFIG_MTRR is a compile-time switch.  The MTRR
 *functionality* could certainly be disabled at run-time, but not
 CONFIG_MTRR itself.

I'll clarify.

   And ... continues to kick through?  Probably some idiomatic
  usage I'm just too old to understand :)
 
  That means for example that in both the above circumstances even if
  MTRR went disabled at run time with Xen, the kernel went through with
  getting PAT enabled.
 
 CONFIG_X86_PAT continues to kick through doesn't seem a very precise
 way of describing this.  But maybe it's enough for experts in this
 area (which I'm not).

I've rephrased this to:

---
It is possible to enable CONFIG_MTRR and CONFIG_X86_PAT 
and end up with a system with MTRR functionality disabled   
PAT functionality enabled. This can happen for instance 
on Xen where MTRR is not supported but PAT is. This can 
happen on Linux as of commit 47591df50 (xen: Support Xen   
pv-domains using PAT) by Juergen, introduced as of v3.19.
---

  Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Bjorn Helgaas
On Thu, Apr 2, 2015 at 4:02 PM, Luis R. Rodriguez mcg...@suse.com wrote:

 ---
 It is possible to enable CONFIG_MTRR and CONFIG_X86_PAT
 and end up with a system with MTRR functionality disabled
 PAT functionality enabled.

This is missing a conjunction or something in MTRR functionality
disabled PAT functionality.

Bjorn

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR

2015-04-02 Thread Toshi Kani
On Thu, 2015-04-02 at 23:49 +0200, Luis R. Rodriguez wrote:
 On Sat, Mar 28, 2015 at 12:56:30AM +0100, Luis R. Rodriguez wrote:
  On Fri, Mar 27, 2015 at 02:40:17PM -0600, Toshi Kani wrote:
   On Fri, 2015-03-20 at 16:17 -0700, Luis R. Rodriguez wrote:
:
@@ -734,6 +742,7 @@ void __init mtrr_bp_init(void)
}
 
if (mtrr_if) {
+   mtrr_enabled = true;
set_num_var_ranges();
init_table();
if (use_intel()) {
   get_mtrr_state();
   
   After setting mtrr_enabled to true, get_mtrr_state() reads
   MSR_MTRRdefType and sets 'mtrr_state.enabled', which also indicates if
   MTRRs are enabled or not on the system.  So, potentially, we could have
   a case that mtrr_enabled is set to true, but mtrr_state.enabled is set
   to disabled when MTRRs are disabled by BIOS.
  
  Thanks for the review, in this case then we should update mtrr_enabled to 
  false.
  
   ps.
   I recently cleaned up this part of the MTRR code in the patch below,
   which is currently available in the -mm  -next trees.
   https://lkml.org/lkml/2015/3/24/1063
  
  Great I will rebase and work with that and try to address this
  consideration you have raised.
 
 OK I'll mesh in this change as well in my next respin:
 
 diff --git a/arch/x86/kernel/cpu/mtrr/generic.c 
 b/arch/x86/kernel/cpu/mtrr/generic.c
 index a83f27a..ecf7cb9 100644
 --- a/arch/x86/kernel/cpu/mtrr/generic.c
 +++ b/arch/x86/kernel/cpu/mtrr/generic.c
 @@ -438,7 +438,7 @@ static void __init print_mtrr_state(void)
  }
  
  /* Grab all of the MTRR state for this CPU into *state */
 -void __init get_mtrr_state(void)
 +bool __init get_mtrr_state(void)
  {
   struct mtrr_var_range *vrs;
   unsigned long flags;
 @@ -482,6 +482,8 @@ void __init get_mtrr_state(void)
  
   post_set();
   local_irq_restore(flags);
 +
 + return !!mtrr_state.enabled;

This should be:
return mtrr_state.enabled  MTRR_STATE_MTRR_ENABLED;

because the MTRR_STATE_MTRR_FIXED_ENABLED flag is ignored when the
MTRR_STATE_MTRR_ENABLED flag is clear.

Thanks,
-Toshi




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-4.2-testing test] 50288: regressions - FAIL

2015-04-02 Thread osstest service user
flight 50288 xen-4.2-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50288/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-amd64 15 guest-start.2 fail REGR. vs. 36512

Tests which did not succeed, but are not blocking:
 test-i386-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 build-amd64-rumpuserxen   5 rumpuserxen-buildfail   never pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 debian-hvm-install fail never pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 debian-hvm-install  fail never pass
 test-amd64-amd64-libvirt 10 migrate-support-checkfail   never pass
 test-i386-i386-libvirt   10 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  10 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop   fail   never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop  fail never pass
 test-i386-i386-xl-qemut-winxpsp3 14 guest-stop fail never pass
 test-i386-i386-xl-qemuu-winxpsp3 14 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop   fail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop   fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop   fail  never pass
 test-i386-i386-xl-winxpsp3   14 guest-stop   fail   never pass
 test-amd64-i386-xend-winxpsp3 17 leak-check/check fail  never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
 build-i386-rumpuserxen5 rumpuserxen-buildfail   never pass
 test-amd64-i386-xend-qemut-winxpsp3 17 leak-check/checkfail never pass

version targeted for testing:
 xen  70bca60d35f02f512548d15b62863d366461be6f
baseline version:
 xen  5bec01c19839e150e489dd04376c65f961830c86


People who touched revisions under test:
  Ian Jackson ian.jack...@eu.citrix.com


jobs:
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  fail
 build-i386-rumpuserxen   fail
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-i386-i386-xlpass
 test-amd64-i386-rhel6hvm-amd pass
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64fail
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-qemuu-freebsd10-amd64pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail
 test-amd64-amd64-rumpuserxen-amd64   blocked 
 test-amd64-amd64-xl-qemut-win7-amd64 fail
 test-amd64-i386-xl-qemut-win7-amd64  fail
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-win7-amd64   fail
 test-amd64-i386-xl-win7-amd64fail
 test-amd64-amd64-xl-credit2  pass
 test-i386-i386-xl-credit2

Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()

2015-04-02 Thread Luis R. Rodriguez
On Thu, Apr 2, 2015 at 3:35 PM, Bjorn Helgaas bhelg...@google.com wrote:
 [-cc Venkatesh, Suresh]

 On Thu, Apr 2, 2015 at 3:55 PM, Luis R. Rodriguez mcg...@suse.com wrote:
 On Thu, Apr 02, 2015 at 03:21:22PM -0500, Bjorn Helgaas wrote:
 On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez
 mcg...@do-not-panic.com wrote:
  From: Luis R. Rodriguez mcg...@suse.com

  This is *only* a transitive API -- and as such no new
  drivers are ever expected to use this.

 transient?  Do you mean you intend to remove this API in the near future?

 That's correct, the problem is that in order to use PAT cleanly we'd need to
 change these drivers ...

 I was just trying to ask whether you intended to write transient
 instead of transitive.  But I'm not doing a very good job :)

Yes, corrected, thanks.

 Luis

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()

2015-04-02 Thread Bjorn Helgaas
[-cc Venkatesh, Suresh]

On Thu, Apr 2, 2015 at 3:55 PM, Luis R. Rodriguez mcg...@suse.com wrote:
 On Thu, Apr 02, 2015 at 03:21:22PM -0500, Bjorn Helgaas wrote:
 On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez
 mcg...@do-not-panic.com wrote:
  From: Luis R. Rodriguez mcg...@suse.com

  This is *only* a transitive API -- and as such no new
  drivers are ever expected to use this.

 transient?  Do you mean you intend to remove this API in the near future?

 That's correct, the problem is that in order to use PAT cleanly we'd need to
 change these drivers ...

I was just trying to ask whether you intended to write transient
instead of transitive.  But I'm not doing a very good job :)

Bjorn

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running

2015-04-02 Thread Wu, Feng


 -Original Message-
 From: Konrad Rzeszutek Wilk [mailto:konrad.w...@oracle.com]
 Sent: Friday, April 03, 2015 3:15 AM
 To: Tian, Kevin
 Cc: Wu, Feng; Zhang, Yang Z; xen-devel@lists.xen.org; k...@xen.org;
 jbeul...@suse.com
 Subject: Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification 
 event
 when vCPU is running
 
 On Thu, Apr 02, 2015 at 06:08:12AM +, Tian, Kevin wrote:
   From: Wu, Feng
   Sent: Friday, March 27, 2015 12:58 PM
  
  
  
-Original Message-
From: Zhang, Yang Z
Sent: Friday, March 27, 2015 12:44 PM
To: Wu, Feng; xen-devel@lists.xen.org
Cc: jbeul...@suse.com; k...@xen.org; Tian, Kevin
Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when
   vCPU
is running
   
Wu, Feng wrote on 2015-03-27:


 Zhang, Yang Z wrote on 2015-03-25:
 when vCPU is running

 Wu, Feng wrote on 2015-03-25:
 When a vCPU is running in Root mode and a notification event has
 been injected to it. we need to set VCPU_KICK_SOFTIRQ for the
 current cpu, so the pending interrupt in PIRR will be synced to
 vIRR before
 
 This would imply that we had VMEXIT-ed due to pending interrupt? And we
 end up calling 'do_IRQ'? If so then the DPCI_SOFTIRQ ends up being set
 and you stll end up calling the softirq code?

No. 

Here is the scenario for the description of this patch:

When vCPU is running in root-mode (such as via hypercall, or any other
reasons which can result in VM-Exit), and before vCPU is back to non-root,
external interrupts happen. Notice that the VM-exit is not caused by this
external interrupt.

Thanks,
Feng

 
 VM-Exit in time.

 Shouldn't the pending interrupt be synced unconditionally before next
 vmentry? What happens if we didn't set the softirq?

 If we didn't set the softirq in the notification handler, the
 interrupts happened exactly before VM-entry cannot be delivered to
 guest at this time. Please see the following code fragments from
 xen/arch/x86/hvm/vmx/entry.S: (pls pay attention to the comments)

 .Lvmx_do_vmentry

 ..
   /* If Vt-d engine issues a notification event here,
  * it cannot be delivered to guest during this VM-entry
  * without raising the softirq in notification handler. */
 cmp  %ecx,(%rdx,%rax,1)
 jnz  .Lvmx_process_softirqs
 ..

 je   .Lvmx_launch
 ..


 .Lvmx_process_softirqs:
 sti
 call do_softirq
 jmp  .Lvmx_do_vmentry
   
You are right! This helps me to recall why raise the softirq when 
delivering
   the
PI.
  
   Yes, __vmx_deliver_posted_interrupt() is the software way to deliver PI, 
   it
 sets
   the
   softirq for this purpose, however, when VT-d HW delivers PI, we have no
   control to
   the HW itself, hence we need to set this softirq in the Notification Event
   handler.
  
 
  could you include this information in the comment so others can easily
  understand this requirement? from code you only mentioned VCPU_KICK
  _SOFTIRQ is required, but how it leads to PIRR-VIRR sync is not explained.
 
  Thanks
  Kevin
 
  ___
  Xen-devel mailing list
  Xen-devel@lists.xen.org
  http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 8/9] qspinlock: Generic paravirt support

2015-04-02 Thread Waiman Long

On 04/02/2015 03:48 PM, Peter Zijlstra wrote:

On Thu, Apr 02, 2015 at 07:20:57PM +0200, Peter Zijlstra wrote:

pv_wait_head():

pv_hash()
/* MB as per cmpxchg */
cmpxchg(l-locked, _Q_LOCKED_VAL, _Q_SLOW_VAL);

VS

__pv_queue_spin_unlock():

if (xchg(l-locked, 0) != _Q_SLOW_VAL)
return;

/* MB as per xchg */
pv_hash_find(lock);




Something like so.. compile tested only.

I took out the LFSR because that was likely over engineering from my
side :-)

--- a/kernel/locking/qspinlock_paravirt.h
+++ b/kernel/locking/qspinlock_paravirt.h
@@ -2,6 +2,8 @@
  #error do not include this file
  #endif

+#includelinux/hash.h
+
  /*
   * Implement paravirt qspinlocks; the general idea is to halt the vcpus 
instead
   * of spinning them.
@@ -107,7 +109,84 @@ static void pv_kick_node(struct mcs_spin
pv_kick(pn-cpu);
  }

-static DEFINE_PER_CPU(struct qspinlock *, __pv_lock_wait);
+/*
+ * Hash table using open addressing with an linear probe sequence.
+ *
+ * Since we should not be holding locks from NMI context (very rare indeed) the
+ * max load factor is 0.75, which is around the point where open adressing
+ * breaks down.
+ *
+ * Instead of probing just the immediate bucket we probe all buckets in the
+ * same cacheline.
+ *
+ * http://en.wikipedia.org/wiki/Hash_table#Open_addressing
+ *
+ */
+
+struct pv_hash_bucket {
+   struct qspinlock *lock;
+   int cpu;
+};
+
+/*
+ * XXX dynamic allocate using nr_cpu_ids instead...
+ */
+#define PV_LOCK_HASH_BITS  (2 + NR_CPUS_BITS)
+
+#if PV_LOCK_HASH_BITS  6
+#undef PV_LOCK_HASH_BITS
+#define PB_LOCK_HASH_BITS  6
+#endif
+
+#define PV_LOCK_HASH_SIZE  (1  PV_LOCK_HASH_BITS)
+
+static struct pv_hash_bucket __pv_lock_hash[PV_LOCK_HASH_SIZE] 
cacheline_aligned;
+
+#define PV_HB_PER_LINE (SMP_CACHE_BYTES / sizeof(struct 
pv_hash_bucket))
+
+static inline u32 hash_align(u32 hash)
+{
+   return hash  ~(PV_HB_PER_LINE - 1);
+}
+
+#define for_each_hash_bucket(hb, off, hash)
\
+   for (hash = hash_align(hash), off = 0, hb =__pv_lock_hash[hash + off];\
+   off  PV_LOCK_HASH_SIZE; \
+   off++, hb =__pv_lock_hash[(hash + off) % PV_LOCK_HASH_SIZE])
+
+static struct pv_hash_bucket *pv_hash_insert(struct qspinlock *lock)
+{
+   u32 offset, hash = hash_ptr(lock, PV_LOCK_HASH_BITS);
+   struct pv_hash_bucket *hb;
+
+   for_each_hash_bucket(hb, offset, hash) {
+   if (!cmpxchg(hb-lock, NULL, lock)) {
+   WRITE_ONCE(hb-cpu, smp_processor_id());
+   return hb;
+   }
+   }
+
+   /*
+* Hard assumes there is an empty bucket somewhere.
+*/
+   BUG();
+}
+
+static struct pv_hash_bucket *pv_hash_find(struct qspinlock *lock)
+{
+   u32 offset, hash = hash_ptr(lock, PV_LOCK_HASH_BITS);
+   struct pv_hash_bucket *hb;
+
+   for_each_hash_bucket(hb, offset, hash) {
+   if (READ_ONCE(hb-lock) == lock)
+   return hb;
+   }
+
+   /*
+* Hard assumes we _WILL_ find a match.
+*/
+   BUG();
+}

  /*
   * Wait for l-locked to become clear; halt the vcpu after a short spin.
@@ -116,7 +195,9 @@ static DEFINE_PER_CPU(struct qspinlock *
  static void pv_wait_head(struct qspinlock *lock)
  {
struct __qspinlock *l = (void *)lock;
+   struct pv_hash_bucket *hb = NULL;
int loop;
+   u8 o;

for (;;) {
for (loop = SPIN_THRESHOLD; loop; loop--) {
@@ -126,29 +207,47 @@ static void pv_wait_head(struct qspinloc
cpu_relax();
}

-   this_cpu_write(__pv_lock_wait, lock);
-   /*
-* __pv_lock_wait must be set before setting _Q_SLOW_VAL
-*
-* [S] __pv_lock_wait = lock[RmW] l = l-locked = 0
-* MB MB
-* [S] l-locked = _Q_SLOW_VAL  [L]   __pv_lock_wait
-*
-* Matches the xchg() in pv_queue_spin_unlock().
-*/
-   if (!cmpxchg(l-locked, _Q_LOCKED_VAL, _Q_SLOW_VAL))
-   goto done;
+   if (!hb) {
+   hb = pv_hash_insert(lock);
+   /*
+* hb  must be set before setting _Q_SLOW_VAL
+*
+* [S]   hb- lock   [RmW] l = l-locked = 0
+*   MB MB
+* [RmW] l-locked ?= _Q_SLOW_VAL [L]   hb
+*
+* Matches the xchg() in pv_queue_spin_unlock().
+*/
+   o = cmpxchg(l-locked, _Q_LOCKED_VAL, _Q_SLOW_VAL);
+   if (!o) {
+   /*
+

Re: [Xen-devel] preparing for 4.5.1

2015-04-02 Thread Ian Campbell
On Thu, 2015-03-26 at 17:07 +, Jan Beulich wrote:
 having been released mid January, it is time to get ready for 4.5.1-rc1.
 Please reply with backport requests which you consider necessary but
 still missing. Please also be patient with them being pulled in - I'll be
 gone the coming two weeks, and I'd hope to get an RC1 put together
 soon thereafter.

On my list for ARM I have this patch, which also touches x86. What do
you think?

commit c2453291b177cc2e89dc104bd4233c3d8bb73922
Author: Vijaya Kumar K vijaya.ku...@caviumnetworks.com
Date:   Tue Mar 24 17:14:47 2015 +0530

xen: Add populate_pt_range interface to reserve non-leaf level table entries

On x86, for the pages mapped with PAGE_HYPERVISOR attribute
non-leaf page tables are allocated with valid pte entries.
and with MAP_SMALL_PAGES attribute only non-leaf page tables are
allocated with invalid (valid bit set to 0) pte entries.
However on arm this is not the case. On arm for the pages
mapped with PAGE_HYPERVISOR and MAP_SMALL_PAGES both
non-leaf and leaf level page table are allocated with valid bit
in pte entries.

This behaviour in arm makes common vmap code fail to
allocate memory beyond 128MB as described below.

In vm_init, map_pages_to_xen() is called for mapping
vm_bitmap. Initially one page of vm_bitmap is allocated
and mapped using PAGE_HYPERVISOR attribute.
For the rest of vm_bitmap pages, MAP_SMALL_PAGES attribute
is used to map.

In ARM for both PAGE_HYPERVISOR and MAP_SMALL_PAGES, valid bit
is set to 1 in pte entry for these mapping.

In vm_alloc(), map_pages_to_xen() is failing for 128MB because
for this next vm_bitmap page the mapping is already set in vm_init()
with valid bit set in pte entry. So map_pages_to_xen() in
ARM returns error.

With this patch, MAP_SMALL_PAGES is dropped and arch specific
populate_pt_range() api is introduced to populate non-leaf
page table entries for the requested pages. Added RESERVE option
to map non-leaf page table entries.

Signed-off-by: Vijaya Kumar Kvijaya.ku...@caviumnetworks.com
Acked-by: Jan Beulich jbeul...@suse.com
Acked-by: Ian Campbell ian.campb...@citrix.com
[ ijc -- rewrote subject line ]

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index 7d4ba0c..d103f8f 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -827,7 +827,8 @@ static int create_xen_table(lpae_t *entry)
 
 enum xenmap_operation {
 INSERT,
-REMOVE
+REMOVE,
+RESERVE
 };
 
 static int create_xen_entries(enum xenmap_operation op,
@@ -859,12 +860,15 @@ static int create_xen_entries(enum xenmap_operation op,
 
 switch ( op ) {
 case INSERT:
+case RESERVE:
 if ( third[third_table_offset(addr)].pt.valid )
 {
 printk(create_xen_entries: trying to replace an existing 
mapping addr=%lx mfn=%lx\n,
addr, mfn);
 return -EINVAL;
 }
+if ( op == RESERVE )
+break;
 pte = mfn_to_xen_entry(mfn, ai);
 pte.pt.table = 1;
 write_pte(third[third_table_offset(addr)], pte);
@@ -898,6 +902,13 @@ int map_pages_to_xen(unsigned long virt,
 {
 return create_xen_entries(INSERT, virt, mfn, nr_mfns, flags);
 }
+
+int populate_pt_range(unsigned long virt, unsigned long mfn,
+  unsigned long nr_mfns)
+{
+return create_xen_entries(RESERVE, virt, mfn, nr_mfns, 0);
+}
+
 void destroy_xen_mappings(unsigned long v, unsigned long e)
 {
 create_xen_entries(REMOVE, v, 0, (e - v)  PAGE_SHIFT, 0);
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 8e29675..786ed47 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5725,6 +5725,12 @@ int map_pages_to_xen(
 return 0;
 }
 
+int populate_pt_range(unsigned long virt, unsigned long mfn,
+  unsigned long nr_mfns)
+{
+return map_pages_to_xen(virt, mfn, nr_mfns, MAP_SMALL_PAGES);
+}
+
 void destroy_xen_mappings(unsigned long s, unsigned long e)
 {
 bool_t locking = system_state  SYS_STATE_boot;
diff --git a/xen/common/vmap.c b/xen/common/vmap.c
index 783cea3..739d468 100644
--- a/xen/common/vmap.c
+++ b/xen/common/vmap.c
@@ -40,7 +40,7 @@ void __init vm_init(void)
 bitmap_fill(vm_bitmap, vm_low);
 
 /* Populate page tables for the bitmap if necessary. */
-map_pages_to_xen(va, 0, vm_low - nr, MAP_SMALL_PAGES);
+populate_pt_range(va, 0, vm_low - nr);
 }
 
 void *vm_alloc(unsigned int nr, unsigned int align)
diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
index 3e7b0ae..8de5e26 100644
--- a/xen/include/asm-arm/page.h
+++ b/xen/include/asm-arm/page.h
@@ -64,7 +64,6 @@
 #define PAGE_HYPERVISOR (WRITEALLOC)
 #define PAGE_HYPERVISOR_NOCACHE (DEV_SHARED)
 #define PAGE_HYPERVISOR_WC  (DEV_WC)
-#define 

Re: [Xen-devel] [PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod 
back to woking):
 Signed-off-by: Ian Campbell ian.campb...@citrix.com
 ---
  production-config-cambridge | 1 +

This is (obviously) fine.  I think that changes to this file from the
Cambridge site operators (ie, each of us two ) should be committed
without review.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST 1/5] cri-args-hostslists: allow instance 
specific settings):
 In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings
 can contain things like export
 OSSTEST_CONFIG=production-config-cambridge to tailor things for a
 particular instance of osstest running in production mode.

Acked-by: Ian Jackson ian.jack...@eu.citrix.com

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [OSSTEST] ms-planner: add a flight summary html report

2015-04-02 Thread Ian Campbell
Resurrecting this old thing...

On Wed, 2014-04-30 at 16:44 +0100, Ian Jackson wrote:
 I think this is a nice thing to have, but:
 
  +while (my ($reso,$evts) = each %{ $plan-{Events} }) {
  +# [osstest real] job 26010.test-amd64-amd64-xl-win7-amd64
  +foreach my $evt ( @{$evts} ) {
  +next unless $evt-{Type} =~ m/^(Start|End)$/;
  +next unless $evt-{Info} =~ m/^\[(\S+) (\S+)\] job ([0-9]+)\
 \.(\S+) (.*)/;
 
 This is rather a layering violation.  Info is for the consumption of
 humans.  Perhaps the plan should have an explicit flight.job field.

I think in principal that's as simple as the two patches below which add
it to the booking in ts-hosts-allocate-Executive and propagate it to the
plan in ms-planner, but I'm not really sure and I don't know how to test
the queuedaemon/planner stuff.

However is even $flight.$job not a layering violation? Or were you
thinking that perhaps generically it would be
thing.sub-thing.sub-sub-thing? Still, the code to generate the
summary would need to know about the concept of flights etc and hence
probably doesn't belong in ms-planner.

 You could compute the [linux-3.4 real] part by looking at the common
 prefix of all the Infos.

Unfortunately the common prefix of the Info is generally at least
[linux-3.4 real] job and depending on the set of jobs currently
running it might even be [linux-3.4 real] job build-, [linux-3.4
real] job test- or even [linux-3.4 real] job test-amd64-amd64.

With all that in mind I've been thinking of coming at this from the
other end and starting with the set of flights which are not finished
and enumerating their jobs as pass/fail/inprogress/planned with the
appropriate times listed.

Only problem is I can't seem to identify in the DB which flights are not
completed, neither the flights table nor the jobs one seems to
consistently reflect that status. Is this possible to find out without
ps?

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking

2015-04-02 Thread Ian Campbell
On Thu, 2015-04-02 at 11:51 +0100, Ian Jackson wrote:
 Ian Campbell writes ([PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod 
 back to woking):
  Signed-off-by: Ian Campbell ian.campb...@citrix.com
  ---
   production-config-cambridge | 1 +
 
 This is (obviously) fine.  I think that changes to this file from the
 Cambridge site operators (ie, each of us two ) should be committed
 without review.

I think you mean to the main osstest#pretest (as opposed to the
Cambridge local one).

Either way that WFM.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch 
a URL on a host):
 Signed-off-by: Ian Campbell ian.campb...@citrix.com

We're not using this to fetch URLs on the public internet, are we ?
If we are then we need to honour $c{HttpProxy}.

Perhaps you could add a comment to the effect that target_fetch_url
should be used for URLs within the test environment, only.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST v4 02/25] TestSupport: allow caller of 
prepareguest_part_xencfg to specify viftype):
 Signed-off-by: Ian Campbell ian.campb...@citrix.com

Acked-by: Ian Jackson ian.jack...@eu.citrix.com

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST v4 03/25] create_webfile: Support use with 
guests as well as hosts.):
 In particular make the path unique by ensuring it includes the host
 and guest name in the guest case.

Acked-by: Ian Jackson ian.jack...@eu.citrix.com

But I have a minor style quibble:

 +# $ho-{Host} is set if $ho is a guest.
 +$wf_rhs= $ho-{Host}-{Name}._${wf_rhs} if $ho-{Host};

It would be usual to write

  +$wf_rhs= $ho-{Host}{Name}._${wf_rhs} if $ho-{Host};

since only the first - is required.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU 
testing flight):
 The failures mainly seem to be some sort of proxy issue returning old
 data (similar to what used to happen to mg-debian-installer-update),

Maybe we can set up an explicit proxy which actually works.  I guess
that might need negotiation with Citrix corporate IT.

Luckily the new colo is not affected by anything like this.

 On a small number I see The installer failed to download a file from
 the mirror. messages, which might be the cache/proxy or might be an
 actual Debian issue.

Hrm.  I find these vague error messages exceptionally frustrating
(debootstrap has this problem too).

 The -raw tests all failed due to a timeout doing:
 dd if=/dev/zero of=/var/lib/xen/images/debian/disk.raw bs=1M 
 count=1
 
 I'm unclear if it timed out after 30 (as the code seems to say) or 60
 (as the logs seem to say) but either way something obviously needs
 adjusting. I just tried it on my local test box and it took 1m20s and
 reported 130 MB/s. I've appended an as yet untested patch which assumed
 100 MB/s and calculates a timeout from that.

Do you intend to fold that patch in ?

 I could switch to using count=1 seek=1 to create a sparse file,
 which should be pretty much instantaneous, but allocating the whole
 backing file seemed like a good idea.

Right.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances

2015-04-02 Thread Ian Campbell
On Thu, 2015-04-02 at 12:13 +0100, Ian Jackson wrote:
 Ian Campbell writes ([PATCH OSSTEST 2/5] Handle osstest's own local push 
 gate in non-master production instances):
  We want to arrange that the master XenProject instance continues to
  test its own pretest branch while any downstream instances will pickup
  changes from the master instance's production (i.e. tested) branch,
  which is published at git://xenbits.xen.org/osstest.git#master.
 ...   ;;
   osstest)
  -   if [ x$OSSTEST_USE_HEAD != xy ] ; then
  +   if [ x$TREEBRANCH_OSSTEST_UPSTREAM != x ] ; then
 
 Shouldn't the OSSTEST_USE_HEAD option override this ?  Ie I think this
 is the wrong way round.

Yes, I think you are right.

I think I just ended up with this because I was testing via the
standalone wrapper which sets USE_HEAD, but I'll test another way.

  +   OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
  +   $TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`
 
 OSSTEST_REVISION_MERGE doesn't seem to be an environment variable.  I
 think it should either be honoured if pre-set, or just be an ordinary
 (lowercase) variable name (in which case it doesn't need the OSSTEST
 prefix).

OK. Any preference as to which?

As it stands the only reason for the variable is to sink the output from
repo_tree_rev_fetch and log it.

 
  +   echo 2 $TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE
  +
  +   rm -rf $repos/osstest-merge 2
  +   git clone -b pretest $HOME/testing.git $repos/osstest-merge 2
  +
  +   git -C $repos/osstest-merge \
  +   fetch $repos/osstest $LOCALREV_OVMF:ap-merge 2
 
 LOCALREV_OVMF (here and earlier).  It's not clear to me that you
 actually need to indirect it; you could just specify a branch name.

OK.

 The rest looks fine.
 
 Having thought about how to override things, particularly:
 
  +   if [ x$TREEBRANCH_OSSTEST_UPSTREAM != x ] ; then
  +   # could push to instance specific location, but
  +   # certainly not to master instance's xenbits repo!
  +   :
  +   else
 
 I think ap-fetch-* and ap-push should have a general purpose hook
 which occurs just after ap-common is loaded, something like this:
 
   diff --git a/ap-fetch-version b/ap-fetch-version
   index 33aaf00..a3028f6 100755
   --- a/ap-fetch-version
   +++ b/ap-fetch-version
   @@ -31,6 +31,8 @@ if info_linux_tree $branch; then
   exit 0
fi
 
   +. ${OSSTEST_HOOK_AP_FETCH-/dev/null}
   +
case $branch in
xen-3.*)
   ./sg-hg-heads sh -ec '
 
 We can postpone actually writing that until someone needs it.

Gladly ;-)

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark

2015-04-02 Thread Ian Campbell
On Thu, 2015-04-02 at 12:16 +0100, Ian Jackson wrote:
 Ian Campbell writes ([PATCH OSSTEST 5/5] cambridge: Stop publishing logs to 
 chiark):
  http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
  points to the live logs directory, so switch PubBaseUrl to that in the
  Cambridge config such that email reports etc contain it. This won't be
  externally accessible but I think that won't matter now that the
  master production instance is elsewhere.
  
  Arrange that cr-publish-flight-logs doesn't publish the corresponding
  thing if either LogsPublish or ResultsPublish is not set, and unset
  them in the Cambridge config.
  
  Likewise arrange that cr-ensure-disk-space doesn't do anything if the
  configuration variable passed as an option is not set, and unset
  Publish (the base for {Logs,Results}Publish) in the Cambridge config.
  
  Signed-off-by: Ian Campbell ian.campb...@citrix.com
  ---
   cr-ensure-disk-space| 2 ++
   cr-publish-flight-logs  | 4 ++--
   production-config-cambridge | 6 +-
   3 files changed, 5 insertions(+), 7 deletions(-)
  
  diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
  index 0ee84c7..84b1890 100755
  --- a/cr-ensure-disk-space
  +++ b/cr-ensure-disk-space
  @@ -40,6 +40,8 @@ die unless @ARGV==1;
   
   our ($cfgbase) = @ARGV;
   
  +exit 0 unless $cfgbase;
   
   csreadconfig();
 
 Don't you mean unless $c{$cfgbase} ?

I think I do, yes.

 And then it has to come after csreadconfig.  (Did you test this ?)

I was running in standalone mode, so I don't think this code will have
actually been run, I was relying on my ability to reason about the code,
which apparently failed :-/.

Shall I run an adhoc flight with this change in it?

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST v4 04/25] Debian: refactor code to add 
preseed commands to the preseed file):
 Call it from ts-debian-hvm-install.

Please expand the commit message.  Something like:

 This means that, in future, ts-debian-hvm-install can use
 preseed_hook_command and preseed_hook_installscript.  No functional
 change for now.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST v4 05/25] Debian: refactor preseeding of 
.ssh directories):
 Causes known_hosts to be consistently created as well as ~osstest/.ssh
 to be consistently populated (it previsouly wasn't for HVM guests).

Most of this is code motion, AFAICT.  The difference is just that
preseed_ssh is now called from ts-debian-hvm-install, too ?

Why should preseed_ssh not be used by preseed_base ?

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC v2 1/3] xen/pvh: enable mmu_update hypercall

2015-04-02 Thread Andrew Cooper
On 02/04/15 11:42, Ian Campbell wrote:
 On Thu, 2015-04-02 at 12:26 +0200, Roger Pau Monne wrote:
 This is needed for performing save/restore of PV guests.
 It's quite a big interface though, isn't it?

 Could we restrict it to a subset of the operations perhaps? Or at least
 justify here how it has been audited and found to be safe to allow an
 HVM guest this access.

It isn't actually very big, but does have quite a lot of PV knowledge
built in.

I would be happer with the safety of this patch if
v-arch.old_guest_table got moved into the pv union, to make the code
much clearer that it is specifically for PV guests.

If I recall, this change only needed for MMU_MACHPHYS_UPDATE against a
foreign domain.  Each of the 3 subops does check for
paging_mode_translate/refcounts() of the target, which does prevent the
hypercall being made against a non-PV domains.  From that point of view,
it should be safe for HVM guests to use, as it is the target domain,
rather than the source domain, which is important. 


However, with migration v2 dropping support for 2-level PV guests (which
died with the 32bit hypervisor build), I believe I have removed all need
for MMU_MACHPHYS_UPDATE hypercalls entirely (unless there are some done
behind the scenes from the kernel).

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC v2 1/3] xen/pvh: enable mmu_update hypercall

2015-04-02 Thread Jürgen Groß

On 04/02/2015 01:50 PM, Andrew Cooper wrote:

On 02/04/15 11:42, Ian Campbell wrote:

On Thu, 2015-04-02 at 12:26 +0200, Roger Pau Monne wrote:

This is needed for performing save/restore of PV guests.

It's quite a big interface though, isn't it?

Could we restrict it to a subset of the operations perhaps? Or at least
justify here how it has been audited and found to be safe to allow an
HVM guest this access.


It isn't actually very big, but does have quite a lot of PV knowledge
built in.

I would be happer with the safety of this patch if
v-arch.old_guest_table got moved into the pv union, to make the code
much clearer that it is specifically for PV guests.

If I recall, this change only needed for MMU_MACHPHYS_UPDATE against a
foreign domain.  Each of the 3 subops does check for
paging_mode_translate/refcounts() of the target, which does prevent the
hypercall being made against a non-PV domains.  From that point of view,
it should be safe for HVM guests to use, as it is the target domain,
rather than the source domain, which is important.


However, with migration v2 dropping support for 2-level PV guests (which
died with the 32bit hypervisor build), I believe I have removed all need
for MMU_MACHPHYS_UPDATE hypercalls entirely (unless there are some done
behind the scenes from the kernel).


There is one usage in the kernel in file arch/x86/xen/setup.c targeting
DOMID_SELF.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking

2015-04-02 Thread Ian Campbell
Signed-off-by: Ian Campbell ian.campb...@citrix.com
---
 production-config-cambridge | 1 +
 1 file changed, 1 insertion(+)

diff --git a/production-config-cambridge b/production-config-cambridge
index b3a7da5..a144cef 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -73,6 +73,7 @@ XenUseUser osstest
 DebianMirrorHost 10.80.16.196
 
 HostProp_NtpServer ntp.uk.xensource.com
+HostProp_DhcpWatchMethod leases dhcp3 woking.cam.xci-test.com:5556
 
 DebianPreseed= 'END'
 END
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark

2015-04-02 Thread Ian Jackson
Ian Campbell writes ([PATCH OSSTEST 5/5] cambridge: Stop publishing logs to 
chiark):
 http://osstest.cam.xci-test.com/~osstest/testlogs already exists and
 points to the live logs directory, so switch PubBaseUrl to that in the
 Cambridge config such that email reports etc contain it. This won't be
 externally accessible but I think that won't matter now that the
 master production instance is elsewhere.
 
 Arrange that cr-publish-flight-logs doesn't publish the corresponding
 thing if either LogsPublish or ResultsPublish is not set, and unset
 them in the Cambridge config.
 
 Likewise arrange that cr-ensure-disk-space doesn't do anything if the
 configuration variable passed as an option is not set, and unset
 Publish (the base for {Logs,Results}Publish) in the Cambridge config.
 
 Signed-off-by: Ian Campbell ian.campb...@citrix.com
 ---
  cr-ensure-disk-space| 2 ++
  cr-publish-flight-logs  | 4 ++--
  production-config-cambridge | 6 +-
  3 files changed, 5 insertions(+), 7 deletions(-)
 
 diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space
 index 0ee84c7..84b1890 100755
 --- a/cr-ensure-disk-space
 +++ b/cr-ensure-disk-space
 @@ -40,6 +40,8 @@ die unless @ARGV==1;
  
  our ($cfgbase) = @ARGV;
  
 +exit 0 unless $cfgbase;
  
  csreadconfig();

Don't you mean unless $c{$cfgbase} ?  And then it has to come after
csreadconfig.  (Did you test this ?)

The rest LGTM.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH RFC v2 1/3] xen/pvh: enable mmu_update hypercall

2015-04-02 Thread Roger Pau Monné
El 02/04/15 a les 12.42, Ian Campbell ha escrit:
 On Thu, 2015-04-02 at 12:26 +0200, Roger Pau Monne wrote:
 This is needed for performing save/restore of PV guests.
 
 It's quite a big interface though, isn't it?

AFAICT it contains MMU_NORMAL_PT_UPDATE, MMU_PT_UPDATE_PRESERVE_AD and
MMU_MACHPHYS_UPDATE.

 Could we restrict it to a subset of the operations perhaps? Or at least
 justify here how it has been audited and found to be safe to allow an
 HVM guest this access.

XSA-109 should have fixed all issues with this operations. IIRC only
MMU_MACHPHYS_UPDATE is needed for save/restore of PV guests, but I will
have to check. If that's the case, I could restrict PVH domains to only
have access to that operation.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


  1   2   3   >