Re: [Xen-devel] [RFC v1 07/15] vt-d: Add API to update IRTE when VT-d PI is used
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 1:34 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 07/15] vt-d: Add API to update IRTE when VT-d PI is used From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM This patch adds an API which is used to update the IRTE for posted-interrupt when guest changes MSI/MSI-X information. Signed-off-by: Feng Wu feng...@intel.com --- xen/drivers/passthrough/vtd/intremap.c | 83 ++ xen/drivers/passthrough/vtd/iommu.h| 3 ++ xen/include/asm-x86/iommu.h| 2 + 3 files changed, 88 insertions(+) diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c index 0333686..f44e74d 100644 --- a/xen/drivers/passthrough/vtd/intremap.c +++ b/xen/drivers/passthrough/vtd/intremap.c @@ -898,3 +898,86 @@ void iommu_disable_x2apic_IR(void) for_each_drhd_unit ( drhd ) disable_qinval(drhd-iommu); } + +/* + * This function is used to update the IRTE for posted-interrupt + * when guest changes MSI/MSI-X information + */ +int pi_update_irte(struct vcpu *v, struct pirq *pirq, uint32_t gvec ) +{ +struct irq_desc *desc; +struct msi_desc *msi_desc; +int remap_index, rc = -1; +struct pci_dev *pci_dev; +struct acpi_drhd_unit *drhd; +struct iommu *iommu; +struct ir_ctrl *ir_ctrl; +struct iremap_entry *iremap_entries = NULL, *p = NULL; +struct iremap_entry new_ire; +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc; +unsigned long flags; + +desc = pirq_spin_lock_irq_desc(pirq, NULL); +if ( !desc ) +return -1; + +msi_desc = desc-msi_desc; +if ( !msi_desc ) +goto unlock_out; + +remap_index = msi_desc-remap_index; +pci_dev = msi_desc-dev; +if ( !pci_dev ) +goto unlock_out; + +drhd = acpi_find_matched_drhd_unit(pci_dev); +if (!drhd) +{ +dprintk(XENLOG_INFO VTDPREFIX, failed to get drhd!\n); +goto unlock_out; +} + +iommu = drhd-iommu; +ir_ctrl = iommu_ir_ctrl(iommu); +if ( !ir_ctrl ) +{ +dprintk(XENLOG_INFO VTDPREFIX, failed to get ir_ctrl!\n); +goto unlock_out; +} + +spin_lock_irqsave(ir_ctrl-iremap_lock, flags); + +GET_IREMAP_ENTRY(ir_ctrl-iremap_maddr, remap_index, iremap_entries, p); + +memcpy(new_ire, p, sizeof(struct iremap_entry)); + +/* Setup/Update interrupt remapping table entry */ +new_ire.lo_intpost.urg = 0; +new_ire.lo_intpost.vector = gvec; +new_ire.lo_intpost.pda_l = (((u64)virt_to_maddr(pi_desc)) +(32 - PDA_LOW_BIT)) ~(-1UL PDA_LOW_BIT); +new_ire.hi_intpost.pda_h = (((u64)virt_to_maddr(pi_desc)) 32) +~(-1UL PDA_HIGH_BIT); + +new_ire.lo_intpost.res_1 = 0; +new_ire.lo_intpost.res_2 = 0; +new_ire.lo_intpost.res_3 = 0; +new_ire.hi_intpost.res_1 = 0; + +new_ire.lo_intpost.im = 1; + Is above code for creating a new entry or updating an existing entry? suppose two purposes would require different steps but here I didn't see a difference... or if you mean every update is equal to what's required for creating a new one, better add a comment for that. It just updates an existing IRTE here to a posted format. Thanks, Feng Thanks Kevin ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 13/15] Update Posted-Interrupts Descriptor during vCPU scheduling
From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM The basic idea here is: 1. When vCPU's state is RUNSTATE_running, - set 'NV' to 'Notification Vector'. - Clear 'SN' to accpet PI. - set 'NDST' to the right pCPU. 2. When vCPU's state is RUNSTATE_blocked, - set 'NV' to 'Wake-up Vector', so we can wake up the related vCPU when posted-interrupt happens for it. - Clear 'SN' to accpet PI. 3. When vCPU's state is RUNSTATE_runnable/RUNSTATE_offline, - Set 'SN' to suppress non-urgent interrupts. (Current, we only support non-urgent interrupts) - Set 'NV' back to 'Notification Vector' if needed. Signed-off-by: Feng Wu feng...@intel.com --- xen/arch/x86/hvm/vmx/vmx.c | 108 + xen/common/schedule.c | 3 ++ 2 files changed, 111 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index b30392c..6323bd6 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -1710,6 +1710,113 @@ static void vmx_handle_eoi(u8 vector) __vmwrite(GUEST_INTR_STATUS, status); } +static void vmx_pi_desc_update(struct vcpu *v, int new_state) +{ +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc; +struct pi_desc old, new; +int old_state = v-runstate.state; +unsigned long flags; + +if ( !iommu_intpost ) +return; + +switch ( new_state ) +{ +case RUNSTATE_runnable: +case RUNSTATE_offline: +/* + * We don't need to send notification event to a non-running + * vcpu, the interrupt information will be delivered to it before + * VM-ENTRY when the vcpu is scheduled to run next time. + */ +pi_set_sn(pi_desc); + +/* + * If the state is transferred from RUNSTATE_blocked, + * we should set 'NV' feild back to posted_intr_vector, + * so the Posted-Interrupts can be delivered to the vCPU + * by VT-d HW after it is scheduled to run. + */ +if ( old_state == RUNSTATE_blocked ) +{ +do +{ +old.control = new.control = pi_desc-control; +new.nv = posted_intr_vector; +} +while ( cmpxchg(pi_desc-control, old.control, new.control) +!= old.control ); + + /* +* Delete the vCPU from the related wakeup queue +* if we are resuming from blocked state +*/ + spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); + list_del(v-blocked_vcpu_list); + spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); +} +break; + +case RUNSTATE_blocked: +/* + * The vCPU is blocked on the wait queue. + * Store the blocked vCPU on the list of the + * vcpu-wakeup_cpu, which is the destination + * of the wake-up notification event. + */ +spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); +list_add_tail(v-blocked_vcpu_list, + per_cpu(blocked_vcpu_on_cpu, v-processor)); +spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); + +do +{ +old.control = new.control = pi_desc-control; + +/* + * We should not block the vCPU if + * an interrupt is posted for it. + */ + +if ( pi_test_on(old) == 1 ) +{ +tasklet_schedule(v-vcpu_wakeup_tasklet); +return; +} so you also need to remove the vcpu from the blocked list, right? and how do you handle ON is set after above check? looks this is better handled behind cmpxchg loop... + +pi_clear_sn(new); +new.nv = pi_wakeup_vector; +} +while ( cmpxchg(pi_desc-control, old.control, new.control) +!= old.control ); +break; + +case RUNSTATE_running: +ASSERT( pi_test_sn(pi_desc) == 1 ); + +do +{ +old.control = new.control = pi_desc-control; +if ( x2apic_enabled ) +new.ndst = cpu_physical_id(v-processor); +else +new.ndst = (cpu_physical_id(v-processor) 8) 0xFF00; + +pi_clear_sn(new); +} +while ( cmpxchg(pi_desc-control, old.control, new.control) +!= old.control ); +break; + +default: +break; +} +} + void vmx_hypervisor_cpuid_leaf(uint32_t sub_idx, uint32_t *eax, uint32_t *ebx,
Re: [Xen-devel] [PATCH] x86/xen: Warn when there's no mfn for pfn
On 02/04/15 08:28, Ross Lagerwall wrote: I recently had to debug dom0 taking a fatal page fault and it would have been useful if it warned when creating a non-present pte if there's no mfn for a pfn. This will trigger for ever PTE covering memory hotplugged by the balloon driver, so I don't think we can take this. David --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -397,6 +397,7 @@ static pteval_t pte_pfn_to_mfn(pteval_t val) * pte_mfn_to_pfn is asymmetric. */ if (unlikely(mfn == INVALID_P2M_ENTRY)) { + printk(KERN_WARNING No mfn for pfn %lx, creating non-present pte\n, pfn); mfn = 0; flags = 0; } else ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight
Hi, It's been a long time since v3 of this series, which was http://article.gmane.org/gmane.comp.emulators.xen.devel/224415. That contains the background which I won't repeat here. Since then I have done the big rebase, addressed the review feed back from last time (I think!) and added a new feature which is a bit unrelated but uses the new ability to do d-i based installs to test qcow, vhd and raw file images in the main flight. There are some patches in here which I think will be useful to the Intel folks doing the nested virt testing, specifically the refactoring of how overlays and ssh host keys are done will be useful for installing a guest to be treated as the L1 host. LongTao, I've CCd you only on the 3 patches which I think will be of interest instead of the full 24 patch series. For the benefit of that work it would be good to get at least the precursor cleanup/refactoring patches into good shape and into the tree sooner rather than later. I've run this as an adhoc job on the Cambridge instance, the results of which are at: http://www.chiark.greenend.org.uk/~xensrcts/logs/36995/ The failures mainly seem to be some sort of proxy issue returning old data (similar to what used to happen to mg-debian-installer-update), so we boot an older d-i kernel+initrd and end up unable to find any kernel modules. I've tried to workaround this with a patch near the end ts-debian-di-install: Use ftp.debian.org directly, the latter I'm not sure how to handle other than by bashing the local proxy over the head. On a small number I see The installer failed to download a file from the mirror. messages, which might be the cache/proxy or might be an actual Debian issue. On the one armhf attempt which didn't suffer from the first problem (daily netboot based) it seems the kernel didn't actual halt when shutdown, which is likely to be a real Debian kernel bug. I've also run an adhoc xen-unstable test of the new jobs there, results at http://www.chiark.greenend.org.uk/~xensrcts/logs/37011/ the interesting jobs are *-pvgrub, *-pygrub for the bootloader tests and *-{qcow2,raw,vhd} for the image format stuff, LVM is tested by the existing -{xl,libvirt} jobs. Results are a bit of a mixed bag. 64-bit pvgrub worked, but 32-bit pvgrub failed due to Debian installing a non-PAE kernel (which cannot be booted), that's probably a bug in the preseeding. The pygrub test passed. -qcow2 and -vhd tests passed except the armhf one which couldn't find the appropriate vmlinuz-xen because mg-debian-installer-update doesn't create it, and the Wheezy baseline doesn't support running as a Xen guest on ARM anyway. I could suppress this in make-flight until the Jessie upgrade. The -raw tests all failed due to a timeout doing: dd if=/dev/zero of=/var/lib/xen/images/debian/disk.raw bs=1M count=1 I'm unclear if it timed out after 30 (as the code seems to say) or 60 (as the logs seem to say) but either way something obviously needs adjusting. I just tried it on my local test box and it took 1m20s and reported 130 MB/s. I've appended an as yet untested patch which assumed 100 MB/s and calculates a timeout from that. I could switch to using count=1 seek=1 to create a sparse file, which should be pretty much instantaneous, but allocating the whole backing file seemed like a good idea. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 18/25] ts-debian-di-install: Refactor root_disk specification
Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v4: new patch --- ts-debian-di-install | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/ts-debian-di-install b/ts-debian-di-install index 56b6ad8..281e7cb 100755 --- a/ts-debian-di-install +++ b/ts-debian-di-install @@ -205,12 +205,14 @@ END OnPowerOff = preserve ); +my $root_disk = 'phy:$gho-{Lvdev},xvda,w'; + prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, END); $method_cfg extra = $cmdline # disk= [ -$extra_disk 'phy:$gho-{Lvdev},xvda,w' +$extra_disk $root_disk ] END @@ -234,7 +236,7 @@ END $blcfg # disk= [ -'phy:$gho-{Lvdev},xvda,w' +$root_disk ] END return; -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 06/25] Debian: Refactor installation of overlays, so it can be used for guests too
Signed-off-by: Ian Campbell ian.campb...@citrix.com Cc: longtaox.p...@intel.com --- v3: New patch --- Osstest/Debian.pm | 57 --- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index db89326..13cd147 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -38,7 +38,9 @@ BEGIN { preseed_base preseed_create preseed_ssh - preseed_hook_command preseed_hook_installscript preseed_hook_cmds + preseed_hook_command preseed_hook_installscript + preseed_hook_overlay + preseed_hook_cmds di_installcmdline_core ); %EXPORT_TAGS = ( ); @@ -700,26 +702,6 @@ sub preseed_create ($$;@) { my $d_i= $ho-{Tftp}{Path}.'/'.$ho-{Tftp}{DiBase}.'/'.$r{arch}.'/'. $c{TftpDiVersion}.'-'.$ho-{Suite}; -my $overlays= ''; -my $create_overlay= sub { -my ($srcdir, $tfilename) = @_; -my $url= create_webfile($ho, $tfilename$sfx, sub { -my ($fh) = @_; -contents_make_cpio($fh, 'ustar', $srcdir); -}); -$overlays .= END; -wget -O overlay.tar '$url' -cd /target -tar xf \$r/overlay.tar -cd \$r -rm overlay.tar - -END -}; - -$create_overlay-('overlay','overlay.tar'); -$create_overlay-($c{OverlayLocal}, 'overlay-local.tar'); - preseed_hook_installscript($ho, $sfx, '/lib/partman/init.d', '000override-parted-devices', END); #!/bin/sh @@ -765,19 +747,14 @@ true END preseed_ssh($ho, $sfx); +preseed_hook_overlay($ho, $sfx, 'overlay', 'overlay.tar'); +preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar'); preseed_hook_command($ho, 'late_command', $sfx, END); #!/bin/sh set -ex -r=/target/root -cd \$r - echo FANCYTTY=0 /target/etc/lsb-base-logging.sh - -$overlays - -echo latecmd done. END my $dtbs = $d_i/dtbs.tar.gz; @@ -966,6 +943,30 @@ chmod +x '$installer_pathname' END } +sub preseed_hook_overlay () { +my ($ho, $sfx, $srcdir, $tfilename) = @_; +my $url= create_webfile($ho, $tfilename$sfx, sub { +my ($fh) = @_; +contents_make_cpio($fh, 'ustar', $srcdir); +}); +preseed_hook_command($ho, 'late_command', $sfx, END); +#!/bin/sh +set -ex + +r=/target/root +cd \$r + +umask 022 + +wget -O overlay.tar '$url' +cd /target +tar xf \$r/overlay.tar +cd \$r +rm overlay.tar + +END +} + sub preseed_hook_cmds () { my $preseed; foreach my $di_key (keys %preseed_cmds) { -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories
Causes known_hosts to be consistently created as well as ~osstest/.ssh to be consistently populated (it previsouly wasn't for HVM guests). Signed-off-by: Ian Campbell ian.campb...@citrix.com Cc: longtaox.p...@intel.com --- v3: New patch --- Osstest/Debian.pm | 99 +-- ts-debian-hvm-install | 5 ++- 2 files changed, 59 insertions(+), 45 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index ef602b1..db89326 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -37,6 +37,7 @@ BEGIN { %preseed_cmds preseed_base preseed_create + preseed_ssh preseed_hook_command preseed_hook_installscript preseed_hook_cmds di_installcmdline_core ); @@ -551,6 +552,60 @@ sub di_installcmdline_core ($$;@) { return @cl; } +sub preseed_ssh ($$) { +my ($ho,$sfx) = @_; + +my $authkeys_url= create_webfile($ho, authkeys$sfx, authorized_keys()); + +my $hostkeyfile= $c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub; +my $hostkey= get_filecontents($hostkeyfile); +chomp($hostkey); $hostkey.=\n; +my $knownhosts= ''; + +my $hostsq= $dbh_tests-prepare(END); +SELECT val FROM runvars + WHERE flight=? AND name LIKE '%host' + GROUP BY val +END +$hostsq-execute($flight); +while (my ($node) = $hostsq-fetchrow_array()) { +my $longname= $node.$c{TestHostDomain}; +my (@hostent)= gethostbyname($longname); +if (!@hostent) { +logm(skipping host key for nonexistent host $longname); +next; +} +my $specs= join ',', $longname, $node, map { +join '.', unpack 'W4', $_; +} @hostent[4..$#hostent]; +logm(adding host key for $specs); +$knownhosts.= $specs .$hostkey; +} +$hostsq-finish(); + +$knownhosts.= localhost,127.0.0.1 .$hostkey; +my $knownhosts_url= create_webfile($ho, known_hosts$sfx, $knownhosts); + +preseed_hook_command($ho, 'late_command', $sfx, END); +#!/bin/sh +set -ex + +r=/target/root +cd \$r + +umask 022 +mkdir .ssh +wget -O .ssh/authorized_keys '$authkeys_url' +wget -O .ssh/known_hosts '$knownhosts_url' + +u=osstest +h=/home/\$u +mkdir /target\$h/.ssh +cp .ssh/authorized_keys /target\$h/.ssh +chroot /target chown -R \$u.\$u \$h/.ssh +END +} + sub preseed_base ($$$;@) { my ($ho,$suite,$extra_packages,%xopts) = @_; @@ -639,43 +694,12 @@ END sub preseed_create ($$;@) { my ($ho, $sfx, %xopts) = @_; -my $authkeys_url= create_webfile($ho, authkeys$sfx, authorized_keys()); - -my $hostkeyfile= $c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub; -my $hostkey= get_filecontents($hostkeyfile); -chomp($hostkey); $hostkey.=\n; -my $knownhosts= ''; - my $disk= $xopts{DiskDevice} || '/dev/sda'; my $suite= $xopts{Suite} || $c{DebianSuite}; my $d_i= $ho-{Tftp}{Path}.'/'.$ho-{Tftp}{DiBase}.'/'.$r{arch}.'/'. $c{TftpDiVersion}.'-'.$ho-{Suite}; -my $hostsq= $dbh_tests-prepare(END); -SELECT val FROM runvars - WHERE flight=? AND name LIKE '%host' - GROUP BY val -END -$hostsq-execute($flight); -while (my ($node) = $hostsq-fetchrow_array()) { -my $longname= $node.$c{TestHostDomain}; -my (@hostent)= gethostbyname($longname); -if (!@hostent) { -logm(skipping host key for nonexistent host $longname); -next; -} -my $specs= join ',', $longname, $node, map { -join '.', unpack 'W4', $_; -} @hostent[4..$#hostent]; -logm(adding host key for $specs); -$knownhosts.= $specs .$hostkey; -} -$hostsq-finish(); - -$knownhosts.= localhost,127.0.0.1 .$hostkey; -my $knownhosts_url= create_webfile($ho, known_hosts$sfx, $knownhosts); - my $overlays= ''; my $create_overlay= sub { my ($srcdir, $tfilename) = @_; @@ -740,6 +764,8 @@ ls -l /dev/sd* true END +preseed_ssh($ho, $sfx); + preseed_hook_command($ho, 'late_command', $sfx, END); #!/bin/sh set -ex @@ -747,17 +773,6 @@ set -ex r=/target/root cd \$r -umask 022 -mkdir .ssh -wget -O .ssh/authorized_keys '$authkeys_url' -wget -O .ssh/known_hosts '$knownhosts_url' - -u=osstest -h=/home/\$u -mkdir /target\$h/.ssh -cp .ssh/authorized_keys /target\$h/.ssh -chroot /target chown -R \$u.\$u \$h/.ssh - echo FANCYTTY=0 /target/etc/lsb-base-logging.sh $overlays diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install index 95fce9a..02c142b 100755 --- a/ts-debian-hvm-install +++ b/ts-debian-hvm-install @@ -44,7 +44,6 @@ our $gho; sub preseed () { my $preseed_file = preseed_base($gho,'wheezy','',()); -my $authkeys = join('\\n', split(/\n/, authorized_keys())); $preseed_file .= (END); d-i netcfg/get_hostname string $gn @@ -74,10 +73,10 @@ d-i apt-setup/cdrom/set-first
[Xen-devel] [PATCH OSSTEST v4 11/25] distros: Support pvgrub for Wheezy too.
This requires us to install pv-grub-menu from backports, which we do using a late_command. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: - Remove spurious () from (END) (and the prexisting too) - Remove $xopts{EnableBackports} and automatically handle the need to add backports in preseed_base. - Install via late_command not apt-setup, since the former has issues, hence subject drops attempt to... --- Osstest/Debian.pm | 44 +++- make-distros-flight | 4 ++-- 2 files changed, 45 insertions(+), 3 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index cbc7172..b37540e 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -689,6 +689,17 @@ END d-i clock-setup/ntp-server string $ntpserver END +# deb http://ftp.debian.org/debian/ wheezy-backports main +$preseed .= END if $extra_packages =~ m#\b/$suite-backports\b#; +d-i apt-setup/local0/repository string \\ +http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main +d-i apt-setup/local0/comment string $suite backports +END +# Jessie onwards... +#$preseed .= END if $extra_packages =~ m#\b/$suite-backports\b#; +#d-i apt-setup/services-select multiselect security, updates, backports +#END + $preseed .= END; ### END OF DEBIAN PRESEED BASE @@ -702,7 +713,38 @@ sub preseed_create_guest ($$;@) { my $suite= $xopts{Suite} || $c{DebianSuite}; -my $extra_packages = pv-grub-menu if $xopts{PvMenuLst}; +my $extra_packages = ; +if ($xopts{PvMenuLst}) { +if ($suite =~ m/wheezy/) { +# pv-grub-menu/wheezy-backports + using apt-setup to add +# backports results in iproute, ifupdown and +# isc-dhcp-client getting removed because tasksel's +# invocation of apt-get install somehow decides the +# iproute2 from wheezy-backports is a thing it wants to +# install. So instead lets fake it with a late command... +# +# This also has the bonus of working round an issue with +# 1.2.1~bpo70+1 which created an invalid menu.lst using +# root(/dev/xvda,0) which pvgrub cannot parse because +# the Grub device.map isn't present at pkgsel/include time +# but it is by late_command time. This was fixed by +# version 1.3 which is in Jessie onwards. +preseed_hook_command($ho, 'late_command', $sfx, END); +#!/bin/sh +set -ex + +( +echo +echo \\\# $suite backports +echo deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main +) /target/etc/apt/sources.list +in-target apt-get update +in-target apt-get install -y -t wheezy-backports pv-grub-menu +END +} else { +$extra_packages = pv-grub-menu; +} +} my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts); $preseed_file.= (END); diff --git a/make-distros-flight b/make-distros-flight index 80df21b..97df89a 100755 --- a/make-distros-flight +++ b/make-distros-flight @@ -65,9 +65,9 @@ test_do_one_netboot () { arm*_arm*_*) bootloader=pygrub;; # no pvgrub for arm # Needs a menu.lst, not present in Squeeze+ due to switch to grub2, -# workedaround in Jessie+ with pv-grub-menu package. +# workedaround in Wheezy+ with pv-grub-menu package (backports in Wheezy, +# in Jessie+ main). *_squeeze) bootloader=pygrub;; -*_wheezy) bootloader=pygrub;; # pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build. i386_amd64_*) bootloader=pygrub;; -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file
Call it from ts-debian-hvm-install. Signed-off-by: Ian Campbell ian.campb...@citrix.com Cc: longtaox.p...@intel.com --- v3: New patch --- Osstest/Debian.pm | 16 +++- ts-debian-hvm-install | 3 +++ 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index 5e6bcba..ef602b1 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -37,7 +37,7 @@ BEGIN { %preseed_cmds preseed_base preseed_create - preseed_hook_command preseed_hook_installscript + preseed_hook_command preseed_hook_installscript preseed_hook_cmds di_installcmdline_core ); %EXPORT_TAGS = ( ); @@ -907,10 +907,7 @@ d-i partman-auto/expert_recipe string \\ END -foreach my $di_key (keys %preseed_cmds) { -$preseed_file .= d-i preseed/$di_key string . -(join ' ', @{ $preseed_cmds{$di_key} }). \n; -} +$preseed_file .= preseed_hook_cmds(); if ($ho-{Flags}{'no-di-kernel'}) { $preseed_file .= END; @@ -954,4 +951,13 @@ chmod +x '$installer_pathname' END } +sub preseed_hook_cmds () { +my $preseed; +foreach my $di_key (keys %preseed_cmds) { +$preseed .= d-i preseed/$di_key string . +(join ' ', @{ $preseed_cmds{$di_key} }). \n; +} +return $preseed; +} + 1; diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install index cfd5144..95fce9a 100755 --- a/ts-debian-hvm-install +++ b/ts-debian-hvm-install @@ -77,6 +77,9 @@ d-i preseed/late_command string \\ in-target mkdir -p /root/.ssh; \\ in-target sh -c echo -e '$authkeys' /root/.ssh/authorized_keys; END + +$preseed_file .= preseed_hook_cmds(); + return $preseed_file; } -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 22/25] make-distros-flight: don't bother building for XSM.
Signed-off-by: Ian Campbell ian.campb...@citrix.com --- make-distros-flight | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/make-distros-flight b/make-distros-flight index 2724dd2..62c527b 100755 --- a/make-distros-flight +++ b/make-distros-flight @@ -34,7 +34,10 @@ defsuite=`getconfig DebianSuite` defguestsuite=`getconfig GuestDebianSuite` job_create_build_filter_callback () { -: + case $* in +* enable_xsm=true *) return 1;; + esac + return 0 } if [ x$buildflight = x ]; then -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 08/25] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs()
Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: New patch --- make-flight | 4 mfi-common | 4 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/make-flight b/make-flight index cc8ecdb..ec0a20c 100755 --- a/make-flight +++ b/make-flight @@ -36,10 +36,6 @@ defguestsuite=`getconfig GuestDebianSuite` if [ x$buildflight = x ]; then - if [ x$BUILD_LVEXTEND_MAX != x ]; then - BUILD_RUNVARS+= build_lvextend_max=$BUILD_LVEXTEND_MAX - fi - create_build_jobs else diff --git a/mfi-common b/mfi-common index 16fc8c5..a9e966f 100644 --- a/mfi-common +++ b/mfi-common @@ -63,6 +63,10 @@ create_build_jobs () { local enable_ovmf local build_hostflags + if [ x$BUILD_LVEXTEND_MAX != x ]; then + BUILD_RUNVARS+= build_lvextend_max=$BUILD_LVEXTEND_MAX + fi + for arch in ${BUILD_ARCHES- i386 amd64 armhf }; do if [ x$arch = xdisable ]; then continue; fi -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 09/25] distros: add support for installing Debian PV guests via d-i, flight and jobs
This patch introduces ts-debian-di-install which can install Debian from a netboot (PXE) debian installer image. By default it installs from the d-i image used by osstest (using the special Xen PV guest enabled flavour where necessary) but it can also install the current release versions from Squeeze onwards (up to and including Sid) and the d-i daily builds. This is controlled by runvars {Guest}_diver = osstest|current and {Guest}_dist = squeeze|...|sid. The resulting guests boot the distro kernel using pygrub (pvgrub will follow). The distros flights differ substantially from the existing flights. Introduce make-distros-flight using the functionality previously refactored into mfi-common. The new flight tests all versions of Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf from Jessie onwards only) using the usual smoke tests. Test names are suffixed -pygrub pending the addition of pvgrub variants in a future commit. Add the new cases to sg-run-job Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v4: use guest create v3: $BUILD_LVEXTEND_MAX now handled in mfi-common Consolidate setting of ruvars Include $flight and $job in tmpdir name Use Osstest::Debian::di_installcmdline_core Document the usage of get_host_property on a guest object Correct ARM netboot paths Include bootloader in test name Should include -pv too? console= repetition for Jessie onwards. Wait for up to an hour for the install. I'd seen timeouts right at the end of the install with the previous value fixup: distros: add support for installing Debian PV guests via d-i, flight and jobs --- Osstest/TestSupport.pm | 3 + make-distros-flight| 98 + sg-run-job | 11 ts-debian-di-install | 168 + 4 files changed, 280 insertions(+) create mode 100755 make-distros-flight create mode 100755 ts-debian-di-install diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index 8f67f7e..3cd8d22 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -906,8 +906,11 @@ sub propname_massage ($) { return $prop; } +# It is fine to call this on a guest object too, in which case it will +# always return $defval. sub get_host_property ($$;$) { my ($ho, $prop, $defval) = @_; +return $defval unless $ho-{Properties}; my $val = $ho-{Properties}{propname_massage($prop)}; return defined($val) ? $val : $defval; } diff --git a/make-distros-flight b/make-distros-flight new file mode 100755 index 000..5067342 --- /dev/null +++ b/make-distros-flight @@ -0,0 +1,98 @@ +#!/bin/bash + +# This is part of osstest, an automated testing framework for Xen. +# Copyright (C) 2009-2013 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see http://www.gnu.org/licenses/. + + +set -e + +branch=$1 +xenbranch=$2 +blessing=$3 +buildflight=$4 + +flight=`./cs-flight-create $blessing $branch` + +. cri-common +. ap-common +. mfi-common + +defsuite=`getconfig DebianSuite` +defguestsuite=`getconfig GuestDebianSuite` + +if [ x$buildflight = x ]; then + + WANT_XEND=false REVISION_LINUX_OLD=disable + + create_build_jobs + +else + + bfi=$buildflight. + +fi + +job_create_test_filter_callback () { + if [ $xenarch = i386 ]; then return 1; fi + return 0 +} + +test_matrix_branch_filter_callback () { +: +} + +test_do_one_netboot () { + job_create_test \ + test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub \ +test-debian-di xl $xenarch $dom0arch\ + kernbuildjob=${bfi}build-$dom0arch-$kernbuild \ + debian_arch=$domU \ + debian_dist=$dist \ + debian_method=netboot \ + debian_diver=current \ + all_hostflags=$most_hostflags +} + +test_matrix_do_one () { + case ${xenarch} in + amd64) domUarches=amd64 i386;; + armhf) domUarches=armhf;; + esac + + for domU in $domUarches ; do +for dist in squeeze wheezy jessie sid daily ; do + case ${domU}_${dist} in + armhf_squeeze) continue;; # No armhf in Squeeze + armhf_wheezy) continue;; # No armhf guest support in Wheezy + *) ;; + esac + +
[Xen-devel] [PATCH OSSTEST v4 16/25] Debian: Handle lack of bootloader support in d-i on ARM.
Debian doesn't currently know what bootloader to install in a Xen guest on ARM. We install pv-grub-menu above which actually does what we need, but the installer doesn't treat that as a bootloader. Most ARM platforms end up installing a u-boot boot.scr, based on a platform whitelist. This doesn't seem appropriate for us. Grub is not available for arm32. For arm64 we will eventually end up with in-guest UEFI and therefore grub-efi and things will work normally. I'm not sure what the answer is going to be for arm32. This patch enables the workaround for Wheezy, Jessie and Sid, post-Jessie should be enabled as we add them. (Pre-wheezy does not support running as a Xen guest on ARM so we don't test them at all). Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v4: Handle sid too v3: New --- Osstest/Debian.pm| 14 -- ts-debian-di-install | 6 -- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index 831290c..7f9deaa 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -711,8 +711,8 @@ END return $preseed; } -sub preseed_create_guest ($$;@) { -my ($ho, $sfx, %xopts) = @_; +sub preseed_create_guest ($$$;@) { +my ($ho, $arch, $sfx, %xopts) = @_; my $suite= $xopts{Suite} || $c{DebianSuite}; @@ -759,6 +759,16 @@ d-i grub-installer/bootdev string /dev/xvda END +# Debian doesn't currently know what bootloader to install in a +# Xen guest on ARM. We install pv-grub-menu above which actually +# does what we need, but the installer doesn't treat that as a +# bootloader. +logm(\$arch is $arch, \$suite is $suite); +$preseed_file.= (END) if $arch =~ /^arm/ $suite =~ /wheezy|jessie|sid/; +d-i nobootloader/confirmation_common boolean true + +END + preseed_ssh($ho, $sfx); preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar'); diff --git a/ts-debian-di-install b/ts-debian-di-install index 62e9bb5..56b6ad8 100755 --- a/ts-debian-di-install +++ b/ts-debian-di-install @@ -168,7 +168,9 @@ END $suite = sid if $suite eq daily; - $ps_url = preseed_create_guest($gho, '', Suite=$suite, PvMenuLst=($bl eq pvgrub)); + $ps_url = preseed_create_guest($gho, $arch, '', + Suite=$suite, + PvMenuLst=($bl eq pvgrub)); $extra_disk = ; } @@ -180,7 +182,7 @@ END ($method_cfg,$extra_disk) = setup_netinst($tmpdir, $arch, $cd); - $ps_url = preseed_create_guest($gho, '', CDROM=1); + $ps_url = preseed_create_guest($gho, $arch, '', CDROM=1); } else { -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case
On Thu, 2015-04-02 at 08:16 +, Pang, LongtaoX wrote: -Original Message- From: Ian Campbell [mailto:ian.campb...@citrix.com] Sent: Tuesday, March 31, 2015 10:23 PM To: Pang, LongtaoX Cc: xen-devel@lists.xen.org; ian.jack...@eu.citrix.com; wei.l...@citrix.com; Hu, Robert Subject: Re: [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case On Fri, 2015-03-27 at 19:06 -0400, longtao.pang wrote: Changes in v7: diff --git a/make-flight b/make-flight index 8ac3a87..b8f266f 100755 --- a/make-flight +++ b/make-flight @@ -204,6 +204,26 @@ do_hvm_win7_x64_tests () { all_hostflags=$most_hostflags,hvm } +do_hvm_debian_nested_tests () { + if [ $xenarch != amd64 ]; then +return + fi + if [ $dom0arch != amd64 ]; then +return + fi You can do these on a line each, or even combine into one test. i.e. if [ $xenarch != amd64 -o $dom0arch != amd64 ]; then return; fi I'm sorry I find that the 'if' condition is not appropriate in v7 patch, it should be if [ $xenarch != amd64 -a $dom0arch != amd64 ]; then return; fi Yes, I think you are right. + + job_create_test test-$xenarch$kern-$dom0arch-nested test-nested xl \ + $xenarch $dom0arch \ +nested_image=$NESTED_OS_IMAGE \ +nested2_image=$NESTED_OS_IMAGE \ I think for clarity you should use something like nestedl1 and nestedl2 for the runvar names. 'nested' and 'nested2' are guest name of L1 and L2 guest VM. Since $specimage is accessed from $r{$gho-{Guest}_image} which defined in the function of ' target_put_guest_image'. So, maybe 'nested' and 'nested2' are available here, I think. My point was you should use the clearer names throughout, from point of definition onwards and in the sg-run-job spec for the recipe too. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC PATCH v2 06/22] xen/arm: its: Port ITS driver to xen
On Thu, 2015-04-02 at 13:55 +0530, Vijay Kilari wrote: On Wed, Apr 1, 2015 at 5:04 PM, Ian Campbell ian.campb...@citrix.com wrote: On Thu, 2015-03-19 at 20:07 +0530, vijay.kil...@gmail.com wrote: From: Vijaya Kumar K vijaya.ku...@caviumnetworks.com This patch just makes ITS driver taken from linux compiles in xen environment. What is your intention wrt future updates to this driver? Are you intending to keep things in sync and import things from the Linux side (similar to the smmu drviers) or are you taking the Linux code as a starting point and intending that it then be maintained independently as a Xen driver from then on? Yes, I intend to keep things in sync with Linux driver. I have kept most the code same as Linux side except removing unused code. There is a tonne of changes going on if that is your goal, in particular in this patch but also in some of the following refactoring patches. When this series is over it seems like the driver would bear very little resemblance to the Linux one. If you want to go this route then to aid in future synchronisation from Linux patches the goal should be to make the changes to the Linux code as minimal as possible, by defining shim functions and typedefs etc at the top of the file, e.g. as Julien has tried to do with the smmu driver. Unlike the smmu stuff, which has a reasonably small and well-defined interface to the kernel which can be easily shimmed between Xen and Linux it's not clear to me that this approach is workable for ITS, the Xen and Linux interrupt handling systems are rather different and ITS needs to be more tightly integrated with other bits of Xen, in particular the GIC drivers. However if you think maintaining something which can be synchronised from Linux is viable and desirable then that's ok by me. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] ENC:[Xen-users] Failed to insert 'xen_evtchn': No such device
On Thu, 2015-04-02 at 08:23 +0200, Olaf Hering wrote: On Wed, Apr 01, Carlos Gustavo Ramirez Rodriguez wrote: Second Problem: My boot.log is showing: [ [1;31mFAILED [0m] Failed to start Load Kernel Modules. See systemctl status systemd-modules-load.service for details. This is a bug in systemd. It assumes that all modules listed in /usr/lib/modules-load.d/ are available in each and every runtime kernel. Even grandma knows that this is not true. Just ignore errors from that service. Has someone filed a wishlist bug with systemd? ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running
From: Wu, Feng Sent: Friday, March 27, 2015 12:58 PM -Original Message- From: Zhang, Yang Z Sent: Friday, March 27, 2015 12:44 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Tian, Kevin Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running Wu, Feng wrote on 2015-03-27: Zhang, Yang Z wrote on 2015-03-25: when vCPU is running Wu, Feng wrote on 2015-03-25: When a vCPU is running in Root mode and a notification event has been injected to it. we need to set VCPU_KICK_SOFTIRQ for the current cpu, so the pending interrupt in PIRR will be synced to vIRR before VM-Exit in time. Shouldn't the pending interrupt be synced unconditionally before next vmentry? What happens if we didn't set the softirq? If we didn't set the softirq in the notification handler, the interrupts happened exactly before VM-entry cannot be delivered to guest at this time. Please see the following code fragments from xen/arch/x86/hvm/vmx/entry.S: (pls pay attention to the comments) .Lvmx_do_vmentry .. /* If Vt-d engine issues a notification event here, * it cannot be delivered to guest during this VM-entry * without raising the softirq in notification handler. */ cmp %ecx,(%rdx,%rax,1) jnz .Lvmx_process_softirqs .. je .Lvmx_launch .. .Lvmx_process_softirqs: sti call do_softirq jmp .Lvmx_do_vmentry You are right! This helps me to recall why raise the softirq when delivering the PI. Yes, __vmx_deliver_posted_interrupt() is the software way to deliver PI, it sets the softirq for this purpose, however, when VT-d HW delivers PI, we have no control to the HW itself, hence we need to set this softirq in the Notification Event handler. could you include this information in the comment so others can easily understand this requirement? from code you only mentioned VCPU_KICK _SOFTIRQ is required, but how it leads to PIRR-VIRR sync is not explained. Thanks Kevin ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] sudden hang on boot on AMD Notebook
On Wed, Apr 01, Konrad Rzeszutek Wilk wrote: You can also use the USB EHCI debug port. It works great on laptops. Is there an howto for that? Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 08/15] Update IRTE according to guest interrupt config changes
From: Wu, Feng Sent: Thursday, April 02, 2015 2:21 PM -Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 1:52 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config changes From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM When guest changes its interrupt configuration (such as, vector, etc.) for direct-assigned devices, we need to update the associated IRTE with the new guest vector, so external interrupts from the assigned devices can be injected to guests without VM-Exit. For lowest-priority interrupts, we use vector-hashing mechamisn to find the destination vCPU. This follows the hardware behavior, since modern Intel CPUs use vector hashing to handle the lowest-priority interrupt. For multicase/broadcast vCPU, we cannot handle it via interrupt posting, still use interrupt remapping. Signed-off-by: Feng Wu feng...@intel.com --- xen/drivers/passthrough/io.c | 77 +++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index ae050df..1d9a132 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -26,6 +26,7 @@ #include asm/hvm/iommu.h #include asm/hvm/support.h #include xen/hvm/irq.h +#include asm/io_apic.h static DEFINE_PER_CPU(struct list_head, dpci_list); @@ -199,6 +200,61 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci) xfree(dpci); } +/* + * Here we handle the following cases: + * - For lowest-priority interrupts, we find the destination vCPU from the + * guest vector using vector-hashing mechamisn and return true. This follows + * the hardware behavior, since modern Intel CPUs use vector hashing to + * handle the lowest-priority interrupt. + * - Otherwise, for single destination interrupt, it is straightforward to + * find the destination vCPU and return true. + * - For multicase/broadcast vCPU, we cannot handle it via interrupt posting, + * so return false. + */ +static bool_t pi_find_dest_vcpu(struct domain *d, uint8_t dest_id, +uint8_t dest_mode, uint8_t deliver_mode, +uint32_t gvec, struct vcpu **dest_vcpu) +{ +struct vcpu *v, **dest_vcpu_array; +unsigned int dest_vcpu_num = 0; +int ret; + +if ( deliver_mode == dest_LowestPrio ) +dest_vcpu_array = xzalloc_array(struct vcpu *, d-max_vcpus); + +for_each_vcpu ( d, v ) +{ +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0, +dest_id, dest_mode) ) +continue; + +dest_vcpu_num++; + +if ( deliver_mode == dest_LowestPrio ) +dest_vcpu_array[dest_vcpu_num] = v; +else +*dest_vcpu = v; +} + +if ( deliver_mode == dest_LowestPrio ) +{ +if ( dest_vcpu_num != 0 ) +{ +*dest_vcpu = dest_vcpu_array[gvec % dest_vcpu_num]; +ret = 1; +} +else +ret = 0; + +xfree(dest_vcpu_array); +return ret; +} +else if ( dest_vcpu_num == 1 ) +return 1; +else +return 0; +} + int pt_irq_create_bind( struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind) { @@ -257,7 +313,7 @@ int pt_irq_create_bind( { case PT_IRQ_TYPE_MSI: { -uint8_t dest, dest_mode; +uint8_t dest, dest_mode, deliver_mode; int dest_vcpu_id; if ( !(pirq_dpci-flags HVM_IRQ_DPCI_MAPPED) ) @@ -330,11 +386,30 @@ int pt_irq_create_bind( /* Calculate dest_vcpu_id for MSI-type pirq migration. */ dest = pirq_dpci-gmsi.gflags VMSI_DEST_ID_MASK; dest_mode = !!(pirq_dpci-gmsi.gflags VMSI_DM_MASK); +deliver_mode = (pirq_dpci-gmsi.gflags GFLAGS_SHIFT_DELIV_MODE) +VMSI_DELIV_MASK; dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); pirq_dpci-gmsi.dest_vcpu_id = dest_vcpu_id; spin_unlock(d-event_lock); if ( dest_vcpu_id = 0 ) hvm_migrate_pirqs(d-vcpu[dest_vcpu_id]); + +/* Use interrupt posting if it is supported */ +if ( iommu_intpost ) +{ +struct vcpu *vcpu = NULL; + +if ( !pi_find_dest_vcpu(d, dest, dest_mode, deliver_mode, +pirq_dpci-gmsi.gvec, vcpu) ) +break; + Is it possible this new pi_find_dest_vcpu will return a different
Re: [Xen-devel] [RFC v1 10/15] vmx: Define two per-cpu variants
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 1:55 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 10/15] vmx: Define two per-cpu variants From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM This patch defines two per-cpu variants: blocked_vcpu_on_cpu: A list storing the vCPUs which were blocked on this pCPU. blocked_vcpu_on_cpu_lock: The spinlock to protect blocked_vcpu_on_cpu. since above two are already per-cpu variants, you don't need 'on_cpu' in the name to duplicate it. How about just call them blocked_vcpus and blocked_vcpus_lock? :-) Sounds great! Thanks, Feng Signed-off-by: Feng Wu feng...@intel.com --- xen/arch/x86/hvm/vmx/vmcs.c | 3 +++ xen/arch/x86/hvm/vmx/vmx.c| 7 +++ xen/include/asm-x86/hvm/vmx/vmx.h | 3 +++ 3 files changed, 13 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 942f4b7..1345e69 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -585,6 +585,9 @@ int vmx_cpu_up(void) if ( cpu_has_vmx_vpid ) vpid_sync_all(); +INIT_LIST_HEAD(per_cpu(blocked_vcpu_on_cpu, cpu)); +spin_lock_init(per_cpu(blocked_vcpu_on_cpu_lock, cpu)); + return 0; } diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index e1c55ce..ff5544d 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -81,6 +81,13 @@ static int vmx_msr_read_intercept(unsigned int msr, uint64_t *msr_content); static int vmx_msr_write_intercept(unsigned int msr, uint64_t msr_content); static void vmx_invlpg_intercept(unsigned long vaddr); +/* + * We maintian a per-CPU linked-list of vCPU, so in PI wakeup handler we + * can find which vCPU should be waken up. + */ +DEFINE_PER_CPU(struct list_head, blocked_vcpu_on_cpu); +DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock); + uint8_t __read_mostly posted_intr_vector; static int vmx_domain_initialise(struct domain *d) diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index 3cd75eb..e643c3c 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -30,6 +30,9 @@ #include asm/hvm/vmx/vmcs.h #include asm/apic.h +DECLARE_PER_CPU(struct list_head, blocked_vcpu_on_cpu); +DECLARE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock); + extern uint8_t posted_intr_vector; typedef union { -- 2.1.0 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 08/15] Update IRTE according to guest interrupt config changes
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 1:52 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config changes From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM When guest changes its interrupt configuration (such as, vector, etc.) for direct-assigned devices, we need to update the associated IRTE with the new guest vector, so external interrupts from the assigned devices can be injected to guests without VM-Exit. For lowest-priority interrupts, we use vector-hashing mechamisn to find the destination vCPU. This follows the hardware behavior, since modern Intel CPUs use vector hashing to handle the lowest-priority interrupt. For multicase/broadcast vCPU, we cannot handle it via interrupt posting, still use interrupt remapping. Signed-off-by: Feng Wu feng...@intel.com --- xen/drivers/passthrough/io.c | 77 +++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index ae050df..1d9a132 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -26,6 +26,7 @@ #include asm/hvm/iommu.h #include asm/hvm/support.h #include xen/hvm/irq.h +#include asm/io_apic.h static DEFINE_PER_CPU(struct list_head, dpci_list); @@ -199,6 +200,61 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci) xfree(dpci); } +/* + * Here we handle the following cases: + * - For lowest-priority interrupts, we find the destination vCPU from the + * guest vector using vector-hashing mechamisn and return true. This follows + * the hardware behavior, since modern Intel CPUs use vector hashing to + * handle the lowest-priority interrupt. + * - Otherwise, for single destination interrupt, it is straightforward to + * find the destination vCPU and return true. + * - For multicase/broadcast vCPU, we cannot handle it via interrupt posting, + * so return false. + */ +static bool_t pi_find_dest_vcpu(struct domain *d, uint8_t dest_id, +uint8_t dest_mode, uint8_t deliver_mode, +uint32_t gvec, struct vcpu **dest_vcpu) +{ +struct vcpu *v, **dest_vcpu_array; +unsigned int dest_vcpu_num = 0; +int ret; + +if ( deliver_mode == dest_LowestPrio ) +dest_vcpu_array = xzalloc_array(struct vcpu *, d-max_vcpus); + +for_each_vcpu ( d, v ) +{ +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0, +dest_id, dest_mode) ) +continue; + +dest_vcpu_num++; + +if ( deliver_mode == dest_LowestPrio ) +dest_vcpu_array[dest_vcpu_num] = v; +else +*dest_vcpu = v; +} + +if ( deliver_mode == dest_LowestPrio ) +{ +if ( dest_vcpu_num != 0 ) +{ +*dest_vcpu = dest_vcpu_array[gvec % dest_vcpu_num]; +ret = 1; +} +else +ret = 0; + +xfree(dest_vcpu_array); +return ret; +} +else if ( dest_vcpu_num == 1 ) +return 1; +else +return 0; +} + int pt_irq_create_bind( struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind) { @@ -257,7 +313,7 @@ int pt_irq_create_bind( { case PT_IRQ_TYPE_MSI: { -uint8_t dest, dest_mode; +uint8_t dest, dest_mode, deliver_mode; int dest_vcpu_id; if ( !(pirq_dpci-flags HVM_IRQ_DPCI_MAPPED) ) @@ -330,11 +386,30 @@ int pt_irq_create_bind( /* Calculate dest_vcpu_id for MSI-type pirq migration. */ dest = pirq_dpci-gmsi.gflags VMSI_DEST_ID_MASK; dest_mode = !!(pirq_dpci-gmsi.gflags VMSI_DM_MASK); +deliver_mode = (pirq_dpci-gmsi.gflags GFLAGS_SHIFT_DELIV_MODE) +VMSI_DELIV_MASK; dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); pirq_dpci-gmsi.dest_vcpu_id = dest_vcpu_id; spin_unlock(d-event_lock); if ( dest_vcpu_id = 0 ) hvm_migrate_pirqs(d-vcpu[dest_vcpu_id]); + +/* Use interrupt posting if it is supported */ +if ( iommu_intpost ) +{ +struct vcpu *vcpu = NULL; + +if ( !pi_find_dest_vcpu(d, dest, dest_mode, deliver_mode, +pirq_dpci-gmsi.gvec, vcpu) ) +break; + Is it possible this new pi_find_dest_vcpu will return a different target from earlier hvm_girq_des_2_vcpu_id? if yes it will cause tricky issues since earlier pirqs are migrated according to different policy. We need consolidate vcpu selection policies together to keep
Re: [Xen-devel] ENC:[Xen-users] Failed to insert 'xen_evtchn': No such device
On Wed, Apr 01, Carlos Gustavo Ramirez Rodriguez wrote: Hello DEVs, can anyone help me? Ian Campbell pointed me that maybe you could help me, here is the link to the last e-mail line that started this problem fixing: http://lists.xenproject.org/archives/html/xen-users/2015-04/msg9.html A brief summary of my current problems after XEN installation: First Problem: XL Anything, be it INFO, TOP, UPTIME, whatever, returns: xc: error: Could not obtain handle on privileged command interface (2 = No such file or directory): Internal error libxl: error: libxl.c:109:libxl_ctx_alloc: cannot open libxc handle: No such file or directory cannot init xl context Which systemd services files from xen are enabled? See the systemd part of the INSTALL file: http://xenbits.xen.org/gitweb/?p=xen.git;a=blob_plain;f=INSTALL;hb=staging systemctl enable xen-qemu-dom0-disk-backend.service systemctl enable xen-init-dom0.service systemctl enable xenconsoled.service Second Problem: My boot.log is showing: [ [1;31mFAILED [0m] Failed to start Load Kernel Modules. See systemctl status systemd-modules-load.service for details. This is a bug in systemd. It assumes that all modules listed in /usr/lib/modules-load.d/ are available in each and every runtime kernel. Even grandma knows that this is not true. Just ignore errors from that service. Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [qemu-upstream-4.3-testing test] 50282: tolerable FAIL - PUSHED
flight 50282 qemu-upstream-4.3-testing real [real] http://logs.test-lab.xenproject.org/osstest/logs/50282/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-qemuu-ovmf-amd64 7 debian-hvm-install fail never pass test-amd64-i386-xl-qemuu-ovmf-amd64 7 debian-hvm-install fail never pass test-amd64-i386-libvirt 10 migrate-support-checkfail never pass test-amd64-amd64-libvirt 10 migrate-support-checkfail never pass test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop fail never pass test-amd64-amd64-xl-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop fail never pass test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass test-amd64-amd64-xl-winxpsp3 14 guest-stop fail never pass test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xend-qemut-winxpsp3 17 leak-check/checkfail never pass test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop fail never pass test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop fail never pass test-amd64-i386-xend-winxpsp3 17 leak-check/check fail never pass version targeted for testing: qemuu7f34050dc014ae8f4078d48aec97ec6553151bf2 baseline version: qemuuab689a89ec47b2e1c964c57bea7da68f8ddf89fd People who touched revisions under test: Ian Campbell ian.campb...@citrix.com Jan Beulich jbeul...@suse.com jobs: build-amd64 pass build-i386 pass build-amd64-libvirt pass build-i386-libvirt pass build-amd64-pvopspass build-i386-pvops pass test-amd64-amd64-xl pass test-amd64-i386-xl pass test-amd64-i386-rhel6hvm-amd pass test-amd64-i386-qemut-rhel6hvm-amd pass test-amd64-i386-qemuu-rhel6hvm-amd pass test-amd64-amd64-xl-qemut-debianhvm-amd64pass test-amd64-i386-xl-qemut-debianhvm-amd64 pass test-amd64-amd64-xl-qemuu-debianhvm-amd64pass test-amd64-i386-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-freebsd10-amd64 pass test-amd64-amd64-xl-qemuu-ovmf-amd64 fail test-amd64-i386-xl-qemuu-ovmf-amd64 fail test-amd64-amd64-xl-qemut-win7-amd64 fail test-amd64-i386-xl-qemut-win7-amd64 fail test-amd64-amd64-xl-qemuu-win7-amd64 fail test-amd64-i386-xl-qemuu-win7-amd64 fail test-amd64-amd64-xl-win7-amd64 fail test-amd64-i386-xl-win7-amd64fail test-amd64-amd64-xl-credit2 pass test-amd64-i386-freebsd10-i386 pass test-amd64-i386-rhel6hvm-intel pass test-amd64-i386-qemut-rhel6hvm-intel pass test-amd64-i386-qemuu-rhel6hvm-intel pass test-amd64-amd64-libvirt pass test-amd64-i386-libvirt pass test-amd64-amd64-xl-multivcpupass test-amd64-amd64-pairpass test-amd64-i386-pair pass test-amd64-amd64-xl-sedf-pin pass test-amd64-amd64-pv pass test-amd64-i386-pv pass test-amd64-amd64-xl-sedf pass test-amd64-i386-xl-qemut-winxpsp3-vcpus1 fail test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail test-amd64-i386-xl-winxpsp3-vcpus1 fail test-amd64-i386-xend-qemut-winxpsp3
Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 2:08 PM To: Wu, Feng; Zhang, Yang Z; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running From: Wu, Feng Sent: Friday, March 27, 2015 12:58 PM -Original Message- From: Zhang, Yang Z Sent: Friday, March 27, 2015 12:44 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Tian, Kevin Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running Wu, Feng wrote on 2015-03-27: Zhang, Yang Z wrote on 2015-03-25: when vCPU is running Wu, Feng wrote on 2015-03-25: When a vCPU is running in Root mode and a notification event has been injected to it. we need to set VCPU_KICK_SOFTIRQ for the current cpu, so the pending interrupt in PIRR will be synced to vIRR before VM-Exit in time. Shouldn't the pending interrupt be synced unconditionally before next vmentry? What happens if we didn't set the softirq? If we didn't set the softirq in the notification handler, the interrupts happened exactly before VM-entry cannot be delivered to guest at this time. Please see the following code fragments from xen/arch/x86/hvm/vmx/entry.S: (pls pay attention to the comments) .Lvmx_do_vmentry .. /* If Vt-d engine issues a notification event here, * it cannot be delivered to guest during this VM-entry * without raising the softirq in notification handler. */ cmp %ecx,(%rdx,%rax,1) jnz .Lvmx_process_softirqs .. je .Lvmx_launch .. .Lvmx_process_softirqs: sti call do_softirq jmp .Lvmx_do_vmentry You are right! This helps me to recall why raise the softirq when delivering the PI. Yes, __vmx_deliver_posted_interrupt() is the software way to deliver PI, it sets the softirq for this purpose, however, when VT-d HW delivers PI, we have no control to the HW itself, hence we need to set this softirq in the Notification Event handler. could you include this information in the comment so others can easily understand this requirement? from code you only mentioned VCPU_KICK _SOFTIRQ is required, but how it leads to PIRR-VIRR sync is not explained. No problem! Thanks, Feng Thanks Kevin ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 08/15] Update IRTE according to guest interrupt config changes
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 2:50 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config changes From: Wu, Feng Sent: Thursday, April 02, 2015 2:21 PM -Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 1:52 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 08/15] Update IRTE according to guest interrupt config changes From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM When guest changes its interrupt configuration (such as, vector, etc.) for direct-assigned devices, we need to update the associated IRTE with the new guest vector, so external interrupts from the assigned devices can be injected to guests without VM-Exit. For lowest-priority interrupts, we use vector-hashing mechamisn to find the destination vCPU. This follows the hardware behavior, since modern Intel CPUs use vector hashing to handle the lowest-priority interrupt. For multicase/broadcast vCPU, we cannot handle it via interrupt posting, still use interrupt remapping. Signed-off-by: Feng Wu feng...@intel.com --- xen/drivers/passthrough/io.c | 77 +++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index ae050df..1d9a132 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -26,6 +26,7 @@ #include asm/hvm/iommu.h #include asm/hvm/support.h #include xen/hvm/irq.h +#include asm/io_apic.h static DEFINE_PER_CPU(struct list_head, dpci_list); @@ -199,6 +200,61 @@ void free_hvm_irq_dpci(struct hvm_irq_dpci *dpci) xfree(dpci); } +/* + * Here we handle the following cases: + * - For lowest-priority interrupts, we find the destination vCPU from the + * guest vector using vector-hashing mechamisn and return true. This follows + * the hardware behavior, since modern Intel CPUs use vector hashing to + * handle the lowest-priority interrupt. + * - Otherwise, for single destination interrupt, it is straightforward to + * find the destination vCPU and return true. + * - For multicase/broadcast vCPU, we cannot handle it via interrupt posting, + * so return false. + */ +static bool_t pi_find_dest_vcpu(struct domain *d, uint8_t dest_id, +uint8_t dest_mode, uint8_t deliver_mode, +uint32_t gvec, struct vcpu **dest_vcpu) +{ +struct vcpu *v, **dest_vcpu_array; +unsigned int dest_vcpu_num = 0; +int ret; + +if ( deliver_mode == dest_LowestPrio ) +dest_vcpu_array = xzalloc_array(struct vcpu *, d-max_vcpus); + +for_each_vcpu ( d, v ) +{ +if ( !vlapic_match_dest(vcpu_vlapic(v), NULL, 0, +dest_id, dest_mode) ) +continue; + +dest_vcpu_num++; + +if ( deliver_mode == dest_LowestPrio ) +dest_vcpu_array[dest_vcpu_num] = v; +else +*dest_vcpu = v; +} + +if ( deliver_mode == dest_LowestPrio ) +{ +if ( dest_vcpu_num != 0 ) +{ +*dest_vcpu = dest_vcpu_array[gvec % dest_vcpu_num]; +ret = 1; +} +else +ret = 0; + +xfree(dest_vcpu_array); +return ret; +} +else if ( dest_vcpu_num == 1 ) +return 1; +else +return 0; +} + int pt_irq_create_bind( struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind) { @@ -257,7 +313,7 @@ int pt_irq_create_bind( { case PT_IRQ_TYPE_MSI: { -uint8_t dest, dest_mode; +uint8_t dest, dest_mode, deliver_mode; int dest_vcpu_id; if ( !(pirq_dpci-flags HVM_IRQ_DPCI_MAPPED) ) @@ -330,11 +386,30 @@ int pt_irq_create_bind( /* Calculate dest_vcpu_id for MSI-type pirq migration. */ dest = pirq_dpci-gmsi.gflags VMSI_DEST_ID_MASK; dest_mode = !!(pirq_dpci-gmsi.gflags VMSI_DM_MASK); +deliver_mode = (pirq_dpci-gmsi.gflags GFLAGS_SHIFT_DELIV_MODE) +VMSI_DELIV_MASK; dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode); pirq_dpci-gmsi.dest_vcpu_id = dest_vcpu_id; spin_unlock(d-event_lock); if ( dest_vcpu_id = 0 )
[Xen-devel] [PATCH OSSTEST v4 17/25] standalone: propagate result of command from with_logging
Such that the overall script gets the return code of the test/job which was run. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- standalone | 1 + 1 file changed, 1 insertion(+) diff --git a/standalone b/standalone index caf3fd5..3189e84 100755 --- a/standalone +++ b/standalone @@ -156,6 +156,7 @@ with_logging() { if [ $rc -ne 0 ] ; then echo FAILED rc=${rc} 2 fi +return $rc } # other potential ops: -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 19/25] make-flight: refactor PV debian tests
No functional change, standalong-generate-dump-flight-runvars confirms no change to the runvars. Includes a hook which is not used yet, $recipe_sfx. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v4: new patch --- make-flight | 24 ++-- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/make-flight b/make-flight index ca48b81..909cebe 100755 --- a/make-flight +++ b/make-flight @@ -339,6 +339,17 @@ do_pvgrub_tests () { all_hostflags=$most_hostflags } +do_pv_debian_test_one () { + testname=$1; shift + recipe_sfx=$1; shift + toolstack=$1; shift + + job_create_test test-$xenarch$kern-$dom0arch-$testname\ + test-debian$recipe_sfx $toolstack \ +$xenarch $dom0arch \ +$debian_runvars all_hostflags=$most_hostflags $@ +} + do_pv_debian_tests () { xsms=$(xenbranch_xsm_variants) @@ -354,20 +365,13 @@ do_pv_debian_tests () { suffix=${platform:+-$platform} hostflags=${most_hostflags}${platform:+,platform-$platform} - job_create_test test-$xenarch$kern-$dom0arch-xl$suffix \ - test-debian xl \ - $xenarch $dom0arch \ - enable_xsm=$xsm \ - $debian_runvars all_hostflags=$hostflags + do_pv_debian_test_one xl$suffix '' xl enable_xsm=$xsm + done done for xsm in $xsms ; do -job_create_test test-$xenarch$kern-$dom0arch-libvirt \ -test-debian libvirt \ -$xenarch $dom0arch \ -enable_xsm=$xsm \ -$debian_runvars all_hostflags=$most_hostflags +do_pv_debian_test_one libvirt '' libvirt enable_xsm=$xsm done } -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 12/25] distros: support PV guest install from Debian netinst media.
The netinst media are iso images containing a base Debian install and some (image size dependent) additional tasks. On x86 the multiarch iso flavour contains a Xen capable kernel for both i386 and amd64 so use that. This adds support for two classes of ISO, the CD sized ones which are built nightly and the DVD sized ones which are built weekly. The images are downloaded using jigdo which sources the majority of the data from a local Debian mirror, for this reason I have not worried about the fact that the i386 and amd64 tests are downloading the same thing (adding a specific download job would require finding up to 4GB of scratch space for each flight). The ISOs booted using pygrub which can extract the kernel and initrd from a ISO image. The resulting guests are also booted with pygrub since the pv-grub-menu package is not available on the ISO images and we have pvgrub coverage from the netboot tests. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v4: Support http_proxy in jigdo download v3: - Indent jigdo-lite script line to improve readability - Wrap bootloader_args - Fetch URL on target, to get a timeout. Use wget since that is arranged to be present. - include bootloader in test name. --- Osstest/Debian.pm| 7 -- make-distros-flight | 23 ++ ts-debian-di-install | 67 3 files changed, 95 insertions(+), 2 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index b37540e..831290c 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -616,8 +616,6 @@ sub preseed_base ($$$;@) { $xopts{ExtraPreseed} ||= ''; my $preseed = END; -d-i mirror/suite string $suite - d-i debian-installer/locale string en_GB d-i console-keymaps-at/keymap select gb d-i keyboard-configuration/xkb-keymap string en_GB @@ -689,6 +687,11 @@ END d-i clock-setup/ntp-server string $ntpserver END +# For CDROM the suite is part of the image +$preseed .= END unless $xopts{CDROM}; +d-i mirror/suite string $suite +END + # deb http://ftp.debian.org/debian/ wheezy-backports main $preseed .= END if $extra_packages =~ m#\b/$suite-backports\b#; d-i apt-setup/local0/repository string \\ diff --git a/make-distros-flight b/make-distros-flight index 97df89a..e3ec8f7 100755 --- a/make-distros-flight +++ b/make-distros-flight @@ -85,6 +85,20 @@ test_do_one_netboot () { all_hostflags=$most_hostflags } +test_do_one_netinst () { + # Always pygrub since no pv-grub-menu on CD + job_create_test \ + test-$xenarch$kern-$dom0arch-$domU-$cd-netinst-pygrub\ +test-debian-di xl $xenarch $dom0arch\ + kernbuildjob=${bfi}build-$dom0arch-$kernbuild \ + debian_arch=$domU \ + debian_cd=$cd \ + debian_method=netinst \ + debian_bootloader=pygrub \ + all_hostflags=$most_hostflags + +} + test_matrix_do_one () { case ${xenarch} in amd64) domUarches=amd64 i386;; @@ -103,6 +117,15 @@ test_matrix_do_one () { done +for cd in current weekly ; do + case ${domU}_${dist} in + armhf_*) continue;; # No iso targets for armhf + *) ;; + esac + + test_do_one_netinst + +done done } diff --git a/ts-debian-di-install b/ts-debian-di-install index 73cb43d..62e9bb5 100755 --- a/ts-debian-di-install +++ b/ts-debian-di-install @@ -46,6 +46,63 @@ sub prep () { target_cmd_root($ho, umount $gho-{Lvdev} ||:); } +sub setup_netinst($$$) +{ +my ($didir, $arch, $cd) = @_; +my %arch_props = ( + amd64 = { PathArch = multi-arch, FileArch = amd64-i386, IsoPath = /install.amd/xen }, + i386 = { PathArch = multi-arch, FileArch = amd64-i386, IsoPath = /install.386/xen }, + armhf = { PathArch = armhf, FileArch = armhf, IsoPath = /install.armhf } +); +my $props = $arch_props{$arch} or die Unknown arch $arch; + +target_install_packages($ho, qw(jigdo-file)); + +my $baseurl = $cd eq current ? + http://cdimage.debian.org/debian-cd/current/$props-{PathArch}/jigdo-cd : + http://cdimage.debian.org/cdimage/weekly-builds/$props-{PathArch}/jigdo-cd; + +my $filebase; + +# Use the MD5SUMs file as an index +logm(Fetch index from $baseurl/MD5SUMS); +my $idx = target_cmd_output_root($ho, wget --quiet -O - $baseurl/MD5SUMS); +foreach (split /\n/, $idx) { + m/^[0-9a-f]{32} (debian-.*-$props-{FileArch}-netinst)\.iso$/ or next; + $filebase = $1; + last; +} + +die unless $filebase; + +logm(Downloading $baseurl/$filebase.jigdo); +my $netinst_jigdo = $baseurl/$filebase.jigdo; +# Jigdo uses wget internally, and so obeys $http_proxy. This seems +# simpler than the advice at +#
[Xen-devel] [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.
In particular make the path unique by ensuring it includes the host and guest name in the guest case. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: New patch --- Osstest/TestSupport.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index 490e16d..8f67f7e 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -1979,6 +1979,8 @@ sub await_webspace_fetch_byleaf ($) { sub create_webfile ($$$) { my ($ho, $tail, $contents) = @_; # $contents as for file_link_contents my $wf_rhs= $ho-{Name}._.$tail; +# $ho-{Host} is set if $ho is a guest. +$wf_rhs= $ho-{Host}-{Name}._${wf_rhs} if $ho-{Host}; my $wf_common= $c{WebspaceCommon}.$wf_rhs; my $wf_url= $c{WebspaceUrl}.$wf_common; my $wf_file= $c{WebspaceFile}.$wf_common; -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 21/25] mfi-common: Allow make-*flight to filter the set of build jobs to include
By using the same job_create_build(_filter_callback) scheme used for the test jobs. Will be used in make-distros-flight. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- make-distros-flight | 4 make-flight | 4 mfi-common | 21 +++-- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/make-distros-flight b/make-distros-flight index e3ec8f7..2724dd2 100755 --- a/make-distros-flight +++ b/make-distros-flight @@ -33,6 +33,10 @@ flight=`./cs-flight-create $blessing $branch` defsuite=`getconfig DebianSuite` defguestsuite=`getconfig GuestDebianSuite` +job_create_build_filter_callback () { +: +} + if [ x$buildflight = x ]; then WANT_XEND=false REVISION_LINUX_OLD=disable diff --git a/make-flight b/make-flight index c923f21..ef1e422 100755 --- a/make-flight +++ b/make-flight @@ -34,6 +34,10 @@ flight=`./cs-flight-create $blessing $branch` defsuite=`getconfig DebianSuite` defguestsuite=`getconfig GuestDebianSuite` +job_create_build_filter_callback () { +: +} + if [ x$buildflight = x ]; then create_build_jobs diff --git a/mfi-common b/mfi-common index a9e966f..a100afb 100644 --- a/mfi-common +++ b/mfi-common @@ -54,6 +54,15 @@ xenbranch_xsm_variants () { esac } +job_create_build () { + job_create_build_filter_callback $@ || return 0 + + local job=$1; shift + local recipe=$1; shift + + ./cs-job-create $flight $job $recipe $@ +} + create_build_jobs () { local arch @@ -164,7 +173,7 @@ create_build_jobs () { else xsm_suffix= fi - ./cs-job-create $flight build-$arch$xsm_suffix build \ + job_create_build build-$arch$xsm_suffix build \ arch=$arch enable_xend=$build_defxend enable_ovmf=$enable_ovmf\ enable_xsm=$enable_xsm \ tree_qemu=$TREE_QEMU \ @@ -183,7 +192,7 @@ create_build_jobs () { done if [ $build_extraxend = true ] ; then -./cs-job-create $flight build-$arch-xend build \ +job_create_build build-$arch-xend build \ arch=$arch enable_xend=true enable_ovmf=$enable_ovmf \ tree_qemu=$TREE_QEMU \ tree_qemuu=$TREE_QEMU_UPSTREAM \ @@ -196,7 +205,7 @@ create_build_jobs () { revision_qemuu=$REVISION_QEMU_UPSTREAM fi -./cs-job-create $flight build-$arch-pvops build-kern \ +job_create_build build-$arch-pvops build-kern\ arch=$arch kconfighow=xen-enable-xen-config \ $RUNVARS $BUILD_RUNVARS $BUILD_LINUX_RUNVARS $arch_runvars \ $suite_runvars \ @@ -208,7 +217,7 @@ create_build_jobs () { if [ x$REVISION_LIBVIRT != xdisable ]; then -./cs-job-create $flight build-$arch-libvirt build-libvirt\ +job_create_build build-$arch-libvirt build-libvirt \ arch=$arch \ tree_xen=$TREE_XEN \ $RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \ @@ -223,7 +232,7 @@ create_build_jobs () { case $arch in i386|amd64) -./cs-job-create $flight build-$arch-rumpuserxen build-rumpuserxen\ +job_create_build build-$arch-rumpuserxen build-rumpuserxen \ arch=$arch \ tree_xen=$TREE_XEN \ $RUNVARS $BUILD_RUNVARS $BUILD_RUMPUSERXEN_RUNVARS $arch_runvars \ @@ -252,7 +261,7 @@ create_build_jobs () { if [ x$REVISION_LINUX_OLD != xdisable ]; then - ./cs-job-create $flight build-$arch-oldkern build-kern\ + job_create_build build-$arch-oldkern build-kern \ arch=$arch kconfighow=create-config-sh \ kimagefile=vmlinux \ $RUNVARS $BUILD_RUNVARS $BUILD_LINUX_OLD_RUNVARS\ -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 13/25] Test pygrub and pvgrub on the regular flights
Since we now have the ability to test these drop one of each of pygrub, pvgrub-32 and pvgrub-64 into the standard flights. Omitting the {Guest}_diver runvar causes ts-debian-di-install to use the d-i images in the location configured via TftpDiVersion, so they are Version Controlled along with the d-i version used for the host. This adds three new jobs: test-amd64-amd64-amd64-pvgrub: all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test arch amd64 buildjob build-amd64 debian_arch amd64 debian_bootloader pvgrub debian_dist wheezy debian_method netboot kernbuildjob build-amd64-pvops kernkind pvops toolstack xl xenbuildjob build-amd64 test-amd64-amd64-i386-pvgrub: all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test arch amd64 buildjob build-amd64 debian_arch i386 debian_bootloader pvgrub debian_dist wheezy debian_method netboot kernbuildjob build-amd64-pvops kernkind pvops toolstack xl xenbuildjob build-amd64 test-amd64-amd64-pygrub: all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test arch amd64 buildjob build-amd64 debian_arch amd64 debian_bootloader pygrub debian_dist wheezy debian_method netboot kernbuildjob build-amd64-pvops kernkind pvops toolstack xl xenbuildjob build-amd64 Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: added runvar details --- make-flight | 39 +++ 1 file changed, 39 insertions(+) diff --git a/make-flight b/make-flight index ec0a20c..ca48b81 100755 --- a/make-flight +++ b/make-flight @@ -303,6 +303,42 @@ do_passthrough_tests () { done } +do_pygrub_tests () { + if [ $xenarch != amd64 -o $dom0arch != amd64 -o $kern != ]; then +return + fi + + job_create_test test-$xenarch$kern-$dom0arch-pygrub \ +test-debian-di xl $xenarch $dom0arch\ + debian_arch=amd64 \ + debian_dist=$guestsuite \ + debian_method=netboot \ + debian_bootloader=pygrub \ + all_hostflags=$most_hostflags +} + +do_pvgrub_tests () { + if [ $xenarch != amd64 -o $dom0arch != amd64 -o $kern != ]; then +return + fi + + job_create_test test-$xenarch$kern-$dom0arch-amd64-pvgrub \ +test-debian-di xl $xenarch $dom0arch\ + debian_arch=amd64 \ + debian_dist=$guestsuite \ + debian_method=netboot \ + debian_bootloader=pvgrub \ + all_hostflags=$most_hostflags \ + + job_create_test test-$xenarch$kern-$dom0arch-i386-pvgrub \ +test-debian-di xl $xenarch $dom0arch\ + debian_arch=i386 \ + debian_dist=$guestsuite \ + debian_method=netboot \ + debian_bootloader=pvgrub \ + all_hostflags=$most_hostflags +} + do_pv_debian_tests () { xsms=$(xenbranch_xsm_variants) @@ -427,6 +463,9 @@ test_matrix_do_one () { fi #do_passthrough_tests + + do_pygrub_tests + do_pvgrub_tests } test_matrix_iterate -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper
Creates a preseed file suitable for use in a PV guest Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v4: Rebase, pass $ho to preseed_base v3: Handle $xopts{ExtraPreseed} undefined in preseed_base --- Osstest/Debian.pm | 29 + 1 file changed, 29 insertions(+) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index 13cd147..993acc7 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -37,6 +37,7 @@ BEGIN { %preseed_cmds preseed_base preseed_create + preseed_create_guest preseed_ssh preseed_hook_command preseed_hook_installscript preseed_hook_overlay @@ -611,6 +612,9 @@ END sub preseed_base ($$$;@) { my ($ho,$suite,$extra_packages,%xopts) = @_; +$extra_packages ||= ''; +$xopts{ExtraPreseed} ||= ''; + my $preseed = END; d-i mirror/suite string $suite @@ -693,6 +697,31 @@ END return $preseed; } +sub preseed_create_guest ($$;@) { +my ($ho, $sfx, %xopts) = @_; + +my $suite= $xopts{Suite} || $c{DebianSuite}; + +my $extra_packages; + +my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts); +$preseed_file.= (END); +d-i partman-auto/method string regular +d-i partman-auto/choose_recipe \\ +select All files in one partition (recommended for new users) + +d-i grub-installer/bootdev string /dev/xvda + +END + +preseed_ssh($ho, $sfx); +preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar'); + +$preseed_file .= preseed_hook_cmds(); + +return create_webfile($ho, preseed$sfx, $preseed_file); +} + sub preseed_create ($$;@) { my ($ho, $sfx, %xopts) = @_; -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 10/25] distros: support booting Debian PV (d-i installed) guests with pvgrub.
This requires the use of the pv-grub-menu package which is in Jessie onwards. (it is in wheezy-backports which is the subject of a subsequent patch). The bootloader to use is specified via a runvar {Guest}_bootloader. Adjust make-distros-flight to use pvgrub for some subset of i386 and amd64 guests to get coverage. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: Define and use arch_debian2xen and arch_xen2debian Avoid pv-grub-x86_64.gz on i386 dom0, we don't built it there. Fiddle with py vs pv grub stripy a bit. --- Osstest.pm | 7 +++ Osstest/Debian.pm| 2 +- make-distros-flight | 21 - ts-debian-di-install | 11 +-- 4 files changed, 37 insertions(+), 4 deletions(-) diff --git a/Osstest.pm b/Osstest.pm index 7989129..c0e3ee7 100644 --- a/Osstest.pm +++ b/Osstest.pm @@ -38,6 +38,7 @@ BEGIN { db_begin_work ensuredir get_filecontents_core_quiet system_checked nonempty visible_undef show_abs_time + %arch_debian2xen %arch_xen2debian ); %EXPORT_TAGS = ( ); @@ -49,6 +50,12 @@ our $mjobdb; our $dbh_tests; +our %arch_debian2xen = qw(i386 x86_32 + amd64 x86_64 + armhf armhf); +our %arch_xen2debian; +$arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen; + #-- static default config settings -- our %c = qw( diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index 993acc7..cbc7172 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -702,7 +702,7 @@ sub preseed_create_guest ($$;@) { my $suite= $xopts{Suite} || $c{DebianSuite}; -my $extra_packages; +my $extra_packages = pv-grub-menu if $xopts{PvMenuLst}; my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts); $preseed_file.= (END); diff --git a/make-distros-flight b/make-distros-flight index 5067342..80df21b 100755 --- a/make-distros-flight +++ b/make-distros-flight @@ -55,14 +55,33 @@ test_matrix_branch_filter_callback () { } test_do_one_netboot () { + stripy bootloader pvgrub pygrub \ +$xenarch amd64 \ +$dom0arch i386 \ +$domU amd64 \ +$dist sid + + case ${dom0arch}_${domU}_${dist} in +arm*_arm*_*) bootloader=pygrub;; # no pvgrub for arm + +# Needs a menu.lst, not present in Squeeze+ due to switch to grub2, +# workedaround in Jessie+ with pv-grub-menu package. +*_squeeze) bootloader=pygrub;; +*_wheezy) bootloader=pygrub;; + +# pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build. +i386_amd64_*) bootloader=pygrub;; + esac + job_create_test \ - test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub \ + test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-$bootloader \ test-debian-di xl $xenarch $dom0arch\ kernbuildjob=${bfi}build-$dom0arch-$kernbuild \ debian_arch=$domU \ debian_dist=$dist \ debian_method=netboot \ debian_diver=current \ + debian_bootloader=$bootloader \ all_hostflags=$most_hostflags } diff --git a/ts-debian-di-install b/ts-debian-di-install index a829185..73cb43d 100755 --- a/ts-debian-di-install +++ b/ts-debian-di-install @@ -93,6 +93,8 @@ sub ginstall () { my $method= $r{$gho-{Guest}_method}; my $tmpdir= /root/$flight-$job-di; +my $bl= $r{$gho-{Guest}_bootloader}; + target_cmd_root($ho, END); rm -rf $tmpdir mkdir $tmpdir @@ -109,7 +111,7 @@ END $suite = sid if $suite eq daily; - $ps_url = preseed_create_guest($gho, '', Suite=$suite); + $ps_url = preseed_create_guest($gho, '', Suite=$suite, PvMenuLst=($bl eq pvgrub)); $extra_disk = ; } @@ -150,7 +152,12 @@ END guest_await_shutdown($ho,$gho,3600); guest_destroy($gho); -my $blcfg = END; +my $xenarch = $arch_debian2xen{$arch}; +my $pvgrub = /usr/local/lib/xen/boot/pv-grub-$xenarch.gz; +my $blcfg = $bl eq pvgrub ? END : END; +kernel = $pvgrub +extra = (hd0,0)/boot/grub/menu.lst +END bootloader = pygrub END -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 20/25] Add testing of non-LVM/phy disk backends.
xen-create-image makes this tricky to do since it is rather LVM centric. Now that we have the ability to install from d-i it's possible to arrange fairly easily that they use non-LVM disk backend arrangements. Here we add support to the test script and infra and create a bunch of new jobs testing the cross product of {xl,libvirt} x {raw,qcow2,vhd}. The test scripts are modified such that when constructing a domain with a non-LVM diskfmt runvar: - the LVM device is slightly enlarged to account for file format headers (1M should be plenty). - the LVM device will have an ext3 filesystem created on it instead of being used as a phy device for the guest. Reusing the LVM volume in this way means we don't need to do more storage management in dom0 (i.e. arranging for / to be large enough, or managing a special images LV) - the relevant type of container is created within the filesystem using the appropriate tool. - New properties Disk{fmt,spec} are added to all $gho, containing the format used for the root disk and the xl diskspec to load it. - lvm backed guests use a xend/xm compatible spec, everything else uses the improved xl syntax which libvirt also supports. We won't test non-LVM on xend. - New properties Disk{mnt,img} are added to $gho which are not using LVM. These contain the mount point to use (configurable via OSSTEST_CONFIG and runvars) and the full path (including mount point) to the image itself. - When starting or stopping a guest we arrange for the filesystem to be (u)mounted. - The prepearation when starting a guest copes gracefully with the disk already being prepared. - Hooks are called from guest_create() and guest_destroy() to manipulate the disk as needed. Using standalong-generate-dump-flight-runvars a representative set of runvars is: xen-unstable test-amd64-amd64-xl-qcow2 all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test xen-unstable test-amd64-amd64-xl-qcow2 arch amd64 xen-unstable test-amd64-amd64-xl-qcow2 buildjob build-amd64 xen-unstable test-amd64-amd64-xl-qcow2 debian_arch amd64 xen-unstable test-amd64-amd64-xl-qcow2 debian_bootloader pygrub xen-unstable test-amd64-amd64-xl-qcow2 debian_diskfmt qcow2 xen-unstable test-amd64-amd64-xl-qcow2 debian_dist wheezy xen-unstable test-amd64-amd64-xl-qcow2 debian_method netboot xen-unstable test-amd64-amd64-xl-qcow2 kernbuildjobbuild-amd64-pvops xen-unstable test-amd64-amd64-xl-qcow2 kernkind pvops xen-unstable test-amd64-amd64-xl-qcow2 toolstack xl xen-unstable test-amd64-amd64-xl-qcow2 xenbuildjob build-amd64 Compared to test-amd64-amd64-pygrub (which is the most similar job) and normalising the test name the difference is: xen-unstable test-amd64-amd64-SUFFIX all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test xen-unstable test-amd64-amd64-SUFFIX arch amd64 xen-unstable test-amd64-amd64-SUFFIX buildjob build-amd64 xen-unstable test-amd64-amd64-SUFFIX debian_arch amd64 xen-unstable test-amd64-amd64-SUFFIX debian_bootloader pygrub +xen-unstable test-amd64-amd64-SUFFIX debian_diskfmt qcow2 xen-unstable test-amd64-amd64-SUFFIX debian_dist wheezy xen-unstable test-amd64-amd64-SUFFIX debian_method netboot xen-unstable test-amd64-amd64-SUFFIX kernbuildjobbuild-amd64-pvops xen-unstable test-amd64-amd64-SUFFIX kernkind pvops xen-unstable test-amd64-amd64-SUFFIX toolstack xl xen-unstable test-amd64-amd64-SUFFIX xenbuildjob build-amd64 These are added to the following flights: +libvirt +linux-3.0 +linux-3.10 +linux-3.14 +linux-3.4 +linux-arm-xen +linux-linus +linux-mingo-tip-master +linux-next +osstest +qemu-mainline +qemu-upstream-4.3-testing +qemu-upstream-4.4-testing +qemu-upstream-unstable +seabios +xen-4.0-testing +xen-4.1-testing +xen-4.2-testing
[Xen-devel] [PATCH OSSTEST v4 23/25] Debian.pm: Assume 100MB/s dd from /dev/zero when creating a raw disk image
In my local test: 1+0 records in 1+0 records out 1048576 bytes (10 GB) copied, 80.8332 s, 130 MB/s Signed-off-by: Ian Campbell ian.campb...@citrix.com --- Osstest/TestSupport.pm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index 61fd7bc..391c9b2 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -1586,7 +1586,9 @@ sub make_qcow2 ($$$) { } sub make_raw ($$$) { my ($ho, $gho, $disk_mb) = @_; -target_cmd_root($ho, dd if=/dev/zero of=$gho-{Rootimg} bs=1M count=${disk_mb}); +# In local tests this reported 130MB/s, so calculate a timeout assuming 100MB/s. +target_cmd_root($ho, dd if=/dev/zero of=$gho-{Rootimg} bs=1M count=${disk_mb}, + ${disk_mb} / 100); } sub prepareguest_part_diskimg ($$$) { -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 25/25] ts-debian-di-install: Use ftp.debian.org directly
The local proxy seems to serve stale packages for Jessie etc, I blame the intercepting cache on the way out of our network, similar to b5f15136900d mg-debian-installer-update: workaround caching proxies, except it is between the apt-cache and the world not the osstest vm and the world. Since the netboot kernel+initrd are reasonably small, these flights are infrequent and they are intended to test the current upstream version I think this is tollerable. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- ts-debian-di-install | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ts-debian-di-install b/ts-debian-di-install index 839cc27..469c1b6 100755 --- a/ts-debian-di-install +++ b/ts-debian-di-install @@ -119,7 +119,9 @@ sub setup_netboot($$$) target_putfile_root($ho, 60, $initrd, $didir/initrd_${suite}_${arch}); } else { - my $mirror = http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};; + #my $mirror = http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};; +# XXX local mirror seems to serve up stale files. + my $mirror = http://ftp.debian.org/debian;; my $di_url = $suite eq daily ? http://d-i.debian.org/daily-images/$arch/daily/netboot; : -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host
Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: Make sure wget is installed --- Osstest/Debian.pm | 2 +- Osstest/TestSupport.pm | 9 - 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index 69530fb..5e6bcba 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -617,7 +617,7 @@ d-i apt-setup/another boolean false d-i apt-setup/non-free boolean false d-i apt-setup/contrib boolean false -d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, $extra_packages +d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, wget, $extra_packages $xopts{ExtraPreseed} diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index 1053016..c4adc48 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -55,7 +55,7 @@ BEGIN { target_putfilecontents_stash target_putfilecontents_root_stash target_put_guest_image target_editfile - target_editfile_cancel + target_editfile_cancel target_fetchurl target_editfile_root target_file_exists target_run_apt target_install_packages target_install_packages_norec @@ -1560,6 +1560,13 @@ END return $cfgpath; } +sub target_fetchurl($$$;$) { +my ($ho, $url, $path, $timeo) = @_; +$timeo ||= 2000; +target_cmd_root($ho, wget --progress=dot:mega -O $path $url, $timeo); +} + + sub target_put_guest_image ($$$) { my ($ho, $gho, $default) = @_; my $specimage = $r{$gho-{Guest}_image}; -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 14/25] distros: add branch infrastructure
Since the distro nightlies are not version controlled we cannot use the usual mechanisms for detecting regressions. Special case things appropriately. We use an OLD_REVISION of flight-NNN to signify that the old revision is another flight and not a tree revision. A grep over $NEW_REVISION needed adjusting since NEW_REVISION is empty in this mode, leading to grep filename which hangs waiting for stdin. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- v3: Handle within cr-daily-branch, since ap-fetch-version* don't make sense for a branch such as this. --- cr-daily-branch | 36 cri-common | 1 + 2 files changed, 29 insertions(+), 8 deletions(-) diff --git a/cr-daily-branch b/cr-daily-branch index c7d1071..55afadf 100755 --- a/cr-daily-branch +++ b/cr-daily-branch @@ -68,23 +68,34 @@ fetch_version () { printf '%s\n' $fetch_version_result } -treeurl=`./ap-print-url $branch` +case $branch in +distros) + treeurl=none;; +*) + treeurl=`./ap-print-url $branch`;; +esac force_baseline=false skipidentical=true wantpush=$OSSTEST_PUSH -if [ x$OLD_REVISION = x ]; then -OLD_REVISION=`./ap-fetch-version-old $branch` -export OLD_REVISION -fi - check_tested () { ./sg-check-tested --debug --branch=$branch \ --blessings=${DAILY_BRANCH_TESTED_BLESSING:-$OSSTEST_BLESSING} \ $@ } +if [ x$OLD_REVISION = x ]; then +case $branch in + distros) + OSSTEST_NO_BASELINE=y + OLD_REVISION=flight-`check_tested` + ;; + *) OLD_REVISION=`./ap-fetch-version-old $branch`;; +esac +export OLD_REVISION +fi + if [ x$OSSTEST_NO_BASELINE != xy ] ; then testedflight=`check_tested --revision-$tree=$OLD_REVISION` @@ -227,6 +238,11 @@ if [ x$OLD_REVISION = xdetermine-late ]; then OLD_REVISION=`./ap-fetch-version-baseline-late $branch $NEW_REVISION` fi +case $branch in +distros) makeflight=./make-distros-flight ;; +*) makeflight=./make-flight ;; +esac + if [ x$NEW_REVISION = x$OLD_REVISION ]; then wantpush=false for checkbranch in x $BRANCHES_ALWAYS; do @@ -241,7 +257,7 @@ if [ x$NEW_REVISION = x$OLD_REVISION ]; then fi $DAILY_BRANCH_PREMAKE_HOOK -flight=`./make-flight $branch $xenbranch $OSSTEST_BLESSING $@` +flight=`$makeflight $branch $xenbranch $OSSTEST_BLESSING $@` $DAILY_BRANCH_POSTMAKE_HOOK heading=tmp/$flight.heading-info @@ -261,6 +277,10 @@ fi revlog=tmp/$flight.revision-log case $NEW_REVISION/$OLD_REVISION in +/flight-[0-9]*) + echo 2 SGR COMPARISON AGAINST ${OLD_REVISION} + sgr_args+= --that-flight=${OLD_REVISION#flight-} + ;; */*[^0-9a-f]* | *[^0-9a-f]*/*) echo 2 NO SGR COMPARISON badchar $NEW_REVISION/$OLD_REVISION ;; @@ -321,7 +341,7 @@ start_email $flight $branch $sgr_args $subject_prefix push=false if grep '^tolerable$' $mrof /dev/null 21; then push=$wantpush; fi if test -f $branch.force; then push=$OSSTEST_PUSH; fi -if grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi +if test -n $NEW_REVISION grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi if test -f $branch.block; then push=false; fi if test -e $mrof test -e $tree_bisect ! grep '^broken' $mrof; then diff --git a/cri-common b/cri-common index ad44546..4bf7548 100644 --- a/cri-common +++ b/cri-common @@ -72,6 +72,7 @@ select_xenbranch () { rumpuserxen) tree=rumpuserxen; xenbranch=xen-unstable ;; seabios)tree=seabios; xenbranch=xen-unstable ;; ovmf) tree=ovmf; xenbranch=xen-unstable ;; + distros)tree=none; xenbranch=xen-unstable ;; osstest)tree=osstest; xenbranch=xen-unstable ;; esac if [ x$tree = xlinux ]; then -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype
Signed-off-by: Ian Campbell ian.campb...@citrix.com -- v3: Use CamelCase for xopts, use the correct condition --- Osstest/TestSupport.pm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index c4adc48..490e16d 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -1533,12 +1533,13 @@ sub prepareguest_part_xencfg ($) { my $onpoweroff= $xopts-{OnPowerOff} || 'destroy'; my $oncrash= $xopts-{OnCrash} || 'preserve'; my $vcpus= guest_var($gho, 'vcpus', $xopts-{DefVcpus} || 2); +my $viftype= $xopts-{VifType} ? type=$xopts-{VifType}, : ; my $xoptcfg= $xopts-{ExtraConfig}; $xoptcfg='' unless defined $xoptcfg; my $xencfg= END; name= '$gho-{Name}' memory = ${ram_mb} -vif = [ 'type=ioemu,mac=$gho-{Ether}' ] +vif = [ '${viftype}mac=$gho-{Ether}' ] # on_poweroff = '$onpoweroff' on_reboot = '$onreboot' @@ -1633,6 +1634,7 @@ END $cfg .= bios='$bios'\n; } +$xopts{VifType} ||= ioemu; my $cfgpath= prepareguest_part_xencfg($ho, $gho, $ram_mb, \%xopts, $cfg); target_cmd_root($ho, END); (echo $passwd; echo $passwd) | vncpasswd $gho-{Guest}.vncpw -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 15/25] distros: Run a flight over the weekend.
Once a week should be sufficient for this test. It involves quite a large number of jobs so schedule it to start early on Saturday so it can run over the weekend when, in theory, things should be quieter. Signed-off-by: Ian Campbell ian.campb...@citrix.com Acked-by: Ian Jackson ian.jack...@eu.citrix.com --- crontab | 1 + 1 file changed, 1 insertion(+) diff --git a/crontab b/crontab index 5dc8675..1917ec4 100644 --- a/crontab +++ b/crontab @@ -6,5 +6,6 @@ MAILTO=ian.jack...@citrix.com,ian.campb...@eu.citrix.com 18 9 * * 1,3,5 cd testing.git BRANCHES=linux-next ./cr-for-branches branches -w ./cr-daily-branch --real 18 4 * * * cd testing.git BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w ./cr-daily-branch --real 6-59/15* * * * cd testing.git EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches bisects -w ./cr-try-bisect --real +46 7 * * 6 cd testing.git BRANCHES=distros ./cr-for-branches branches -w ./cr-daily-branch --real #8-59/5* * * * cd bisects/adhoc.git with-lock-ex -q data-tree-lock bash -c ./cr-try-bisect-adhoc; exit $? 3 4 * * * savelog -c28 testing.git/tmp/cr-for-branches.log /dev/null -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST v4 24/25] distros: email only me on play flights
Signed-off-by: Ian Campbell ian.campb...@citrix.com --- daily-cron-email-play--distros | 1 + 1 file changed, 1 insertion(+) create mode 100644 daily-cron-email-play--distros diff --git a/daily-cron-email-play--distros b/daily-cron-email-play--distros new file mode 100644 index 000..0d82e65 --- /dev/null +++ b/daily-cron-email-play--distros @@ -0,0 +1 @@ +To: ian.campb...@citrix.com -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] about cr3 register
On Wed, Apr 1, 2015 at 3:37 PM, HANNAS YAYA Issa issa.hannasy...@enseeiht.fr wrote: Hi I am working on memory management in xen. I encounter a problem that I hope you can help me to solve it. I modified the struct mmu_update in xen and linux source code. I add two fields cr3 and addr which respectively contains the current cr3 value and the current address of the process associated with the pte to be updated. the struct mmu_update is sent to the hyervisor via hypercall HYPERVISOR_do_mmu_update. In the handler of this hypercall (I mean do_mmu_update) when I read the value of cr3 via the method read_cr3 it is different from the value I stored before in the struct mmu_update. I wonder why? Without your code there is absolutely no hope of us figuring out what's going on; and in any case, it's not really our job to debug your code. If you have a specific question about the code that's in-tree, please ask it. This may also be helpful: http://wiki.xenproject.org/wiki/Asking_Developer_Questions (Your last question was closer to being answerable, FWIW.) -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] sudden hang on boot on AMD Notebook
On Wed, Apr 01, Olaf Hering wrote: On Wed, Apr 01, Ian Campbell wrote: Key handlers are only available via serial, I think, and not VGA which it sounds like what Olaf is looking at. This laptop has a serial port, and thats what I used to grab the output. No reaction there. Just today I noticed the blinking cursor. Will continue to poke this as time permits. There are other issues now, suspend to ram with non-xen kernel fails as well: ... [ 58.114937] EXT4-fs (sda3): re-mounted. Opts: acl,user_xattr,commit=0 [ 60.432380] EXT4-fs (dm-1): re-mounted. Opts: commit=0 [ 176.984899] sky2 :02:00.0 eth1: disabling interface [ 179.214856] PM: Syncing filesystems ... done. [ 179.675897] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 179.677286] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 179.678601] Suspending console(s) (use no_console_suspend to debug) [ 179.679065] sd 0:0:0:0: [sda] Synchronizing SCSI cache [ 179.730066] serial 00:04: disabled [ 179.730070] acpi PNP0501:00: Cannot transition to non-D0 state from D3 [ 179.814620] sd 0:0:0:0: [sda] Stopping disk [ 180.957655] PM: suspend of devices complete after 1278.434 msecs [ 180.960953] acpi PNP0C0B:03: Cannot transition to non-D0 state from D3 [ 180.960960] dpm_run_callback(): acpi_subsys_suspend_late+0x0/0x22 returns -19 [ 180.960962] PM: Device PNP0C0B:03 failed to suspend late: error -19 [ 180.964119] PM: early resume of devices complete after 3.149 msecs [ 180.964120] PM: late suspend of devices failed [ 180.964260] ohci-pci :00:12.0: enabling bus mastering [ 180.964281] ahci :00:11.0: restoring config space at offset 0x4 (was 0x233, writing 0x237) [ 180.964285] ehci-pci :00:12.2: enabling bus mastering [ 180.964312] ohci-pci :00:13.0: enabling bus mastering [ 180.964338] ehci-pci :00:13.2: enabling bus mastering [ 180.964371] ohci-pci :00:14.5: enabling bus mastering [ 180.964383] ohci-pci :00:16.0: enabling bus mastering [ 180.964402] ehci-pci :00:16.2: enabling bus mastering [ 180.966630] sd 0:0:0:0: [sda] Starting disk [ 180.977638] radeon :01:05.0: restoring config space at offset 0x4 (was 0x13, writing 0x17) ... These are fan devices. Guess I have to open it some day and see if there are any cable issues. Have to do it anyway, after HP sent it back from repair they did not wire up the CDROM device. Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 06/15] vt-d: Extend struct iremap_entry to support VT-d Posted-Interrupts
From: Jan Beulich [mailto:jbeul...@suse.com] Sent: Friday, March 27, 2015 5:58 PM On 27.03.15 at 02:53, feng...@intel.com wrote: From: Andrew Cooper [mailto:andrew.coop...@citrix.com] Sent: Friday, March 27, 2015 3:01 AM On 25/03/15 12:31, Feng Wu wrote: --- a/xen/drivers/passthrough/vtd/iommu.h +++ b/xen/drivers/passthrough/vtd/iommu.h @@ -303,6 +303,18 @@ struct iremap_entry { res_2 : 8, dst : 32; }lo; +struct { +u64 p : 1, +fpd : 1, +res_1 : 6, +avail : 4, +res_2 : 2, +urg : 1, +im : 1, +vector : 8, +res_3 : 14, +pda_l : 26; +}lo_intpost; }; union { u64 hi_val; @@ -312,6 +324,13 @@ struct iremap_entry { svt : 2, res_1 : 44; }hi; +struct { +u64 sid : 16, +sq : 2, +svt : 2, +res_1 : 12, +pda_h : 32; +}hi_intpost; I would prefer if this union was reformatted as I suggested in the thread from your design doc, but I won't insist on it as a blocker to entry. Thanks for the comments. I also considered your sugguestion on the Design doc, here is your proposal: struct iremap_entry { union { struct { u64 lo, hi; }; struct { bitfields } norm; (names subject to improvement) struct { bitfields } post; }; }; Seems in that way, we need to change some existing code to adapt to this new structure. I am okay with both of them, but can we listen some voices form some others? Is it okay for you? I think this would be a good move, but in the end it's the VT-d maintainers who got to decide. yes it's a good move. Thanks Kevin ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 01/10] VMX: Enable EPT A/D bit support
On 03/28/2015 04:38 AM, Andrew Cooper wrote: On 27/03/15 02:35, Kai Huang wrote: PML requires A/D bit support so enable it for further use. Signed-off-by: Kai Huang kai.hu...@linux.intel.com --- xen/arch/x86/hvm/vmx/vmcs.c| 1 + xen/arch/x86/mm/p2m-ept.c | 8 +++- xen/include/asm-x86/hvm/vmx/vmcs.h | 4 +++- xen/include/asm-x86/hvm/vmx/vmx.h | 5 - 4 files changed, 15 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index d614638..2f645fe 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -103,6 +103,7 @@ static void __init vmx_display_features(void) P(cpu_has_vmx_tpr_shadow, APIC TPR shadow); P(cpu_has_vmx_ept, Extended Page Tables (EPT)); P(cpu_has_vmx_vpid, Virtual-Processor Identifiers (VPID)); +P(cpu_has_vmx_ept_ad_bit, EPT A/D bit); P(cpu_has_vmx_vnmi, Virtual NMI); P(cpu_has_vmx_msr_bitmap, MSR direct-access bitmap); P(cpu_has_vmx_unrestricted_guest, Unrestricted Guest); diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c index c2d7720..8650092 100644 --- a/xen/arch/x86/mm/p2m-ept.c +++ b/xen/arch/x86/mm/p2m-ept.c @@ -233,6 +233,9 @@ static int ept_split_super_page(struct p2m_domain *p2m, ept_entry_t *ept_entry, if ( !ept_set_middle_entry(p2m, new_ept) ) return 0; +/* It's better to copy A bit of Middle entry from original entry */ +new_ept.a = ept_entry-a; Surely d needs to be propagated as well? Would it make sense to extend ept_set_middle_entry() to do all of new_ept setup in one location? + table = map_domain_page(new_ept.mfn); trunk = 1UL ((level - 1) * EPT_TABLE_ORDER); @@ -244,7 +247,7 @@ static int ept_split_super_page(struct p2m_domain *p2m, ept_entry_t *ept_entry, epte-sp = (level 1); epte-mfn += i * trunk; epte-snp = (iommu_enabled iommu_snoop); -ASSERT(!epte-rsvd1); +/* A/D bits are inherited from superpage */ ASSERT(!epte-avail3); ept_p2m_type_to_flags(epte, epte-sa_p2mt, epte-access); @@ -1071,6 +1074,9 @@ int ept_p2m_init(struct p2m_domain *p2m) /* set EPT page-walk length, now it's actual walk length - 1, i.e. 3 */ ept-ept_wl = 3; +/* Enable EPT A/D bit if it's supported by hardware */ +ept-ept_ad = cpu_has_vmx_ept_ad_bit ? 1 : 0; This will incur overhead on all EPT operations. It should only be enabled if pml is going to be in use. (I think you need reverse patches 1 and 2 in the series, and gate on pml_enable here) Hi Andrew, I'd like to also put patch 3 (PML feature detection) before this patch, as it's better to use cpu_has_vmx_pml to gate A/D bit enabling here. Theoretically simple pml_enable = 1 here doesn't guarantee cpu_has_vmx_pml to be true, as PML may be turned off during vmx_init_vmcs_config. And in this case I also want to delete below code, as if PML is not enabled, below code will print but actually EPT A/D bits is not enabled in hardware. P(cpu_has_vmx_ept_ad, EPT A/D bit); Thanks, -Kai + if ( !zalloc_cpumask_var(ept-synced_mask) ) return -ENOMEM; diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h b/xen/include/asm-x86/hvm/vmx/vmcs.h index 6fce6aa..4528346 100644 --- a/xen/include/asm-x86/hvm/vmx/vmcs.h +++ b/xen/include/asm-x86/hvm/vmx/vmcs.h @@ -62,7 +62,8 @@ struct ept_data { struct { u64 ept_mt :3, ept_wl :3, -rsvd :6, +ept_ad :1, +rsvd :5, asr:52; While you are making this change, can you add comments similar to ept_entry_t describing the bits? }; u64 eptp; @@ -226,6 +227,7 @@ extern u32 vmx_secondary_exec_control; #define VMX_EPT_INVEPT_INSTRUCTION 0x0010 #define VMX_EPT_INVEPT_SINGLE_CONTEXT 0x0200 #define VMX_EPT_INVEPT_ALL_CONTEXT 0x0400 +#define VMX_EPT_AD_BIT_SUPPORT 0x0020 #define VMX_MISC_VMWRITE_ALL0x2000 diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index 91c5e18..9afd351 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -37,7 +37,8 @@ typedef union { emt : 3, /* bits 5:3 - EPT Memory type */ ipat: 1, /* bit 6 - Ignore PAT memory type */ sp : 1, /* bit 7 - Is this a superpage? */ -rsvd1 : 2, /* bits 9:8 - Reserved for future use */ +a : 1, /* bit 8 - Access bit */ +d : 1, /* bit 9 - Dirty bit */ recalc : 1, /* bit 10 - Software available 1 */ snp : 1, /* bit 11 - VT-d snoop control in shared EPT/VT-d usage */ @@ -261,6 +262,8 @@ extern uint8_t posted_intr_vector; (vmx_ept_vpid_cap
Re: [Xen-devel] [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d Posted-Interrupts
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 2:01 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d Posted-Interrupts From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM This patch adds a global vector which is used to wake up the blocked vCPU when an interrupt is being posted to it. Signed-off-by: Feng Wu feng...@intel.com Suggested-by: Yang Zhang yang.z.zh...@intel.com --- xen/arch/x86/hvm/vmx/vmx.c| 33 + xen/include/asm-x86/hvm/hvm.h | 1 + xen/include/asm-x86/hvm/vmx/vmx.h | 3 +++ xen/include/xen/sched.h | 2 ++ 4 files changed, 39 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index ff5544d..b2b4c26 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -89,6 +89,7 @@ DEFINE_PER_CPU(struct list_head, blocked_vcpu_on_cpu); DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock); uint8_t __read_mostly posted_intr_vector; +uint8_t __read_mostly pi_wakeup_vector; static int vmx_domain_initialise(struct domain *d) { @@ -131,6 +132,8 @@ static int vmx_vcpu_initialise(struct vcpu *v) if ( v-vcpu_id == 0 ) v-arch.user_regs.eax = 1; +INIT_LIST_HEAD(v-blocked_vcpu_list); + return 0; } @@ -1834,11 +1837,19 @@ const struct hvm_function_table * __init start_vmx(void) } if ( cpu_has_vmx_posted_intr_processing ) +{ alloc_direct_apic_vector(posted_intr_vector, event_check_interrupt); + +if ( iommu_intpost ) +alloc_direct_apic_vector(pi_wakeup_vector, pi_wakeup_interrupt); +else +vmx_function_table.pi_desc_update = NULL; +} just style issue. Above conditional logic looks not intuitive to me. usually we have: if ( iommu_intpost ) vmx_function_table.pi_desc_update = func; else vmx_function_table.pi_desc_update = NULL; suppose you will register callback in later patch. then better to move the NULL one there too. Putting it here doesn't meet the normal if...else implications. :-) You suggestion is good. Here is my idea about this code fragment: Here is the place to register notification event handle, so it is better to register the wakeup event handle for VT-d PI here as well. Just like other members in vmx_function_table, such as, deliver_posted_intr, sync_pir_to_irr, pi_desc_update is initialed to 'vmx_pi_desc_update' in the definition of vmx_function_table statically. So do you have any ideas to make this gracefully? Thanks, Feng else { vmx_function_table.deliver_posted_intr = NULL; vmx_function_table.sync_pir_to_irr = NULL; +vmx_function_table.pi_desc_update = NULL; } if ( cpu_has_vmx_ept @@ -3255,6 +3266,28 @@ void vmx_vmenter_helper(const struct cpu_user_regs *regs) } /* + * Handle VT-d posted-interrupt when VCPU is blocked. + */ +void pi_wakeup_interrupt(struct cpu_user_regs *regs) +{ +struct vcpu *v; +int cpu = smp_processor_id(); + +spin_lock(per_cpu(blocked_vcpu_on_cpu_lock, cpu)); +list_for_each_entry(v, per_cpu(blocked_vcpu_on_cpu, cpu), +blocked_vcpu_list) { +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc; + +if ( pi_test_on(pi_desc) == 1 ) +tasklet_schedule(v-vcpu_wakeup_tasklet); why can't we directly call vcpu_unblock here? Please see the following scenario if we use vcpu_unblock directly here: pi_wakeup_interrupt() (blocked_vcpu_on_cpu_lock is required) -- vcpu_unblock() -- vcpu_wake() -- vcpu_runstate_change() -- vmx_ pi_desc_update() (In this function we may need to require blocked_vcpu_on_cpu_lock, this will cause dead lock.) Thanks, Feng +} +spin_unlock(per_cpu(blocked_vcpu_on_cpu_lock, cpu)); + +ack_APIC_irq(); +this_cpu(irq_count)++; +} + +/* * Local variables: * mode: C * c-file-style: BSD diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h index 0dc909b..a11a256 100644 --- a/xen/include/asm-x86/hvm/hvm.h +++ b/xen/include/asm-x86/hvm/hvm.h @@ -195,6 +195,7 @@ struct hvm_function_table { void (*deliver_posted_intr)(struct vcpu *v, u8 vector); void (*sync_pir_to_irr)(struct vcpu *v); void (*handle_eoi)(u8 vector); +void (*pi_desc_update)(struct vcpu *v, int new_state); /*Walk nested p2m */ int (*nhvm_hap_walk_L1_p2m)(struct vcpu *v, paddr_t L2_gpa, diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index e643c3c..f4296ab 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -34,6 +34,7 @@
[Xen-devel] [PATCH OSSTEST] ts-xen-*: Avoid use of uninitialised $r{enable_xsm}
Use of uninitialized value $r{enable_xsm} in pattern match (m//) at ./ts-xen-install line 49. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- ts-xen-build | 2 +- ts-xen-install | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ts-xen-build b/ts-xen-build index e757f68..a530731 100755 --- a/ts-xen-build +++ b/ts-xen-build @@ -28,7 +28,7 @@ selectbuildhost(\@ARGV); # remaining arguments are passed as targets to make builddirsprops(); -my $enable_xsm = $r{enable_xsm} =~ m/true/ ? 1 : 0; +my $enable_xsm = $r{enable_xsm}//'' =~ m/true/ ? 1 : 0; sub checkout () { prepbuilddirs(); diff --git a/ts-xen-install b/ts-xen-install index b3f4387..6995046 100755 --- a/ts-xen-install +++ b/ts-xen-install @@ -46,7 +46,7 @@ if (@ARGV and $ARGV[0] eq '--check') { our $ho; -my $enable_xsm = $r{enable_xsm} =~ m/true/ ? 1 : 0; +my $enable_xsm = $r{enable_xsm}//'' =~ m/true/ ? 1 : 0; my %distpath; -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case
-Original Message- From: Ian Campbell [mailto:ian.campb...@citrix.com] Sent: Tuesday, March 31, 2015 10:23 PM To: Pang, LongtaoX Cc: xen-devel@lists.xen.org; ian.jack...@eu.citrix.com; wei.l...@citrix.com; Hu, Robert Subject: Re: [OSSTEST Nested PATCH v7 5/6] Add test job for nest test case On Fri, 2015-03-27 at 19:06 -0400, longtao.pang wrote: Changes in v7: diff --git a/make-flight b/make-flight index 8ac3a87..b8f266f 100755 --- a/make-flight +++ b/make-flight @@ -204,6 +204,26 @@ do_hvm_win7_x64_tests () { all_hostflags=$most_hostflags,hvm } +do_hvm_debian_nested_tests () { + if [ $xenarch != amd64 ]; then +return + fi + if [ $dom0arch != amd64 ]; then +return + fi You can do these on a line each, or even combine into one test. i.e. if [ $xenarch != amd64 -o $dom0arch != amd64 ]; then return; fi I'm sorry I find that the 'if' condition is not appropriate in v7 patch, it should be if [ $xenarch != amd64 -a $dom0arch != amd64 ]; then return; fi + + job_create_test test-$xenarch$kern-$dom0arch-nested test-nested xl \ + $xenarch $dom0arch \ +nested_image=$NESTED_OS_IMAGE \ +nested2_image=$NESTED_OS_IMAGE \ I think for clarity you should use something like nestedl1 and nestedl2 for the runvar names. 'nested' and 'nested2' are guest name of L1 and L2 guest VM. Since $specimage is accessed from $r{$gho-{Guest}_image} which defined in the function of ' target_put_guest_image'. So, maybe 'nested' and 'nested2' are available here, I think. +bios=seabios \ +kernbuildjob=build-amd64-pvops \ +kernkind=pvops \ +nested_vifmodel='e1000' \ +device_model_version=qemu-xen \ +all_hostflags=$most_hostflags,hvm } + ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH 2/2] x86/efi: Fix memcmp check
Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com --- xen/arch/x86/dmi_scan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c index ec411e6..3749688 100644 --- a/xen/arch/x86/dmi_scan.c +++ b/xen/arch/x86/dmi_scan.c @@ -203,7 +203,7 @@ void __init dmi_efi_get_table(const void *smbios, const void *smbios3) const struct smbios_eps *eps = smbios; const struct smbios3_eps *eps3 = smbios3; - if (eps3 memcmp(eps3-anchor, _SM3_, 5) + if (eps3 memcmp(eps3-anchor, _SM3_, 5) == 0 eps3-length = sizeof(*eps3) dmi_checksum(eps3, eps3-length)) { efi_smbios3_address = eps3-address; -- 2.1.0 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 13/15] Update Posted-Interrupts Descriptor during vCPU scheduling
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 2:25 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 13/15] Update Posted-Interrupts Descriptor during vCPU scheduling From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM The basic idea here is: 1. When vCPU's state is RUNSTATE_running, - set 'NV' to 'Notification Vector'. - Clear 'SN' to accpet PI. - set 'NDST' to the right pCPU. 2. When vCPU's state is RUNSTATE_blocked, - set 'NV' to 'Wake-up Vector', so we can wake up the related vCPU when posted-interrupt happens for it. - Clear 'SN' to accpet PI. 3. When vCPU's state is RUNSTATE_runnable/RUNSTATE_offline, - Set 'SN' to suppress non-urgent interrupts. (Current, we only support non-urgent interrupts) - Set 'NV' back to 'Notification Vector' if needed. Signed-off-by: Feng Wu feng...@intel.com --- xen/arch/x86/hvm/vmx/vmx.c | 108 + xen/common/schedule.c | 3 ++ 2 files changed, 111 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index b30392c..6323bd6 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -1710,6 +1710,113 @@ static void vmx_handle_eoi(u8 vector) __vmwrite(GUEST_INTR_STATUS, status); } +static void vmx_pi_desc_update(struct vcpu *v, int new_state) +{ +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc; +struct pi_desc old, new; +int old_state = v-runstate.state; +unsigned long flags; + +if ( !iommu_intpost ) +return; + +switch ( new_state ) +{ +case RUNSTATE_runnable: +case RUNSTATE_offline: +/* + * We don't need to send notification event to a non-running + * vcpu, the interrupt information will be delivered to it before + * VM-ENTRY when the vcpu is scheduled to run next time. + */ +pi_set_sn(pi_desc); + +/* + * If the state is transferred from RUNSTATE_blocked, + * we should set 'NV' feild back to posted_intr_vector, + * so the Posted-Interrupts can be delivered to the vCPU + * by VT-d HW after it is scheduled to run. + */ +if ( old_state == RUNSTATE_blocked ) +{ +do +{ +old.control = new.control = pi_desc-control; +new.nv = posted_intr_vector; +} +while ( cmpxchg(pi_desc-control, old.control, new.control) +!= old.control ); + + /* +* Delete the vCPU from the related wakeup queue +* if we are resuming from blocked state +*/ + spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); + list_del(v-blocked_vcpu_list); + spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); +} +break; + +case RUNSTATE_blocked: +/* + * The vCPU is blocked on the wait queue. + * Store the blocked vCPU on the list of the + * vcpu-wakeup_cpu, which is the destination + * of the wake-up notification event. + */ +spin_lock_irqsave(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); +list_add_tail(v-blocked_vcpu_list, + per_cpu(blocked_vcpu_on_cpu, v-processor)); +spin_unlock_irqrestore(per_cpu(blocked_vcpu_on_cpu_lock, + v-processor), flags); + +do +{ +old.control = new.control = pi_desc-control; + +/* + * We should not block the vCPU if + * an interrupt is posted for it. + */ + +if ( pi_test_on(old) == 1 ) +{ +tasklet_schedule(v-vcpu_wakeup_tasklet); +return; +} so you also need to remove the vcpu from the blocked list, right? Yes, I need to remove the vcpu here. I thought it could be removed in another place in this patch, however, I feel it cannot do it after more thinking about it. I think we can fix this issue in the following two ways: #1) Just add the remove logic here. or #2) In function vcpu_runstate_change(), call 'hvm_funcs.pi_desc_update()' after v-runstate.state = new_state; and pass the old_state to 'hvm_funcs.pi_desc_update()'. then here is the code path: tasklet_schedule(v-vcpu_wakeup_tasklet) - vcpu_unblock() - vcpu_wake() - vcpu_runstate_change(v, RUNSTATE_runnable, NOW()) - vmx_pi_desc_update() So, the vCPU will be removed in 'case
Re: [Xen-devel] Migration v2 and related work for 4.6
On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote: I propose that the libxc series be accepted independently of the libxl series. That is most likely a good idea IMHO. What do you estimate the chances of the libxl bit being done for 4.6 to be? Is there an option of actually switching to the new libxc without switching libxl and fixing libxl later, or would that be as much work as just fixing libxl? Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC PATCH v2 13/22] xen/arm: its: Add virtual ITS command support
On Wed, 2015-04-01 at 13:02 +0100, Julien Grall wrote: On 01/04/15 12:46, Ian Campbell wrote: On Mon, 2015-03-30 at 16:47 +0100, Julien Grall wrote: In any case ITS commands are processed in synchronously. So any VCPU that send ITS commands is blocked. What exactly is synchronous here? Is it just the translate vits into requests queued with the physical its driver phase or does it also include waiting for the physical its' response and translating that back into a v response? From the spec, the processing of command is asynchronous. I was asking about the implementation of our emulation of it, not about the hardware itself. I understood that the underlying h/w is asynchronous. The vCPU has to poll a register in order to know if the ITS has finished to execute the command. A vCPU may decide to execute other things while the ITS is processing commands. The implementation suggested by Vijay, both the vCPU and CPU is blocked while the ITS command are processing. Can we just enqueue with the hardware and use the guest vcpu polling loop to trigger us to check for completion? What would happen if a guest never polled, I suppose we would have to catch it some other way? Futhermore, if another vCPU is trying to access the vITS it will be blocked too (and therefore the CPU). With this solution we may take down Xen. Yes, we can't have that I'm afraid. The translation/queuing can be very long if the queue is big (the maximum size of the queue is 2^32). It would at least require some preemption in Xen in order to avoid blocking the CPU/vCPU for a long time. Yes, limiting the number of requests in flight at any one time seems like a good idea anyway, we can always dribble things in as other ones complete anyway. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Migration v2 and related work for 4.6
On 02/04/15 10:03, Ian Campbell wrote: On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote: I propose that the libxc series be accepted independently of the libxl series. That is most likely a good idea IMHO. What do you estimate the chances of the libxl bit being done for 4.6 to be? I hope to have everything complete for 4.6, including removal of the legacy code. Given the current timescales, I would say most likely. Is there an option of actually switching to the new libxc without switching libxl and fixing libxl later, or would that be as much work as just fixing libxl? For PV guests, yes. One can transparently swap the legacy algorithm from the v2 algorithm and every works. For HVM guests, no. The handling of the Qemu save record and toolstack records needs fixing. In the upgrade case, libxl also needs to take care of piping the legacy stream through the conversion script. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH] x86/xen: Warn when there's no mfn for pfn
I recently had to debug dom0 taking a fatal page fault and it would have been useful if it warned when creating a non-present pte if there's no mfn for a pfn. Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com --- arch/x86/xen/mmu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index adca9e2..4cacc11 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -397,6 +397,7 @@ static pteval_t pte_pfn_to_mfn(pteval_t val) * pte_mfn_to_pfn is asymmetric. */ if (unlikely(mfn == INVALID_P2M_ENTRY)) { + printk(KERN_WARNING No mfn for pfn %lx, creating non-present pte\n, pfn); mfn = 0; flags = 0; } else -- 2.1.0 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH] x86/xen: Warn when there's no mfn for pfn
On 04/02/2015 10:00 AM, David Vrabel wrote: On 02/04/15 08:28, Ross Lagerwall wrote: I recently had to debug dom0 taking a fatal page fault and it would have been useful if it warned when creating a non-present pte if there's no mfn for a pfn. This will trigger for ever PTE covering memory hotplugged by the balloon driver, so I don't think we can take this. That's unfortunate, I thought there may be some issue where it would be triggered spuriously. -- Ross Lagerwall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 2/2] x86/efi: Fix memcmp check
On 02/04/15 09:24, Ross Lagerwall wrote: Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com Reviewed-by: Andrew Cooper andrew.coop...@citrix.com --- xen/arch/x86/dmi_scan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c index ec411e6..3749688 100644 --- a/xen/arch/x86/dmi_scan.c +++ b/xen/arch/x86/dmi_scan.c @@ -203,7 +203,7 @@ void __init dmi_efi_get_table(const void *smbios, const void *smbios3) const struct smbios_eps *eps = smbios; const struct smbios3_eps *eps3 = smbios3; - if (eps3 memcmp(eps3-anchor, _SM3_, 5) + if (eps3 memcmp(eps3-anchor, _SM3_, 5) == 0 eps3-length = sizeof(*eps3) dmi_checksum(eps3, eps3-length)) { efi_smbios3_address = eps3-address; ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 2/4 v2] INSTALL: mention variables for reproducible builds
On Wed, 2015-04-01 at 13:28 +, Olaf Hering wrote: Mention two variables introduced by commit ac977f5 (use more fixed strings to build the hypervisor). Signed-off-by: Olaf Hering o...@aepfle.de Acked-by: Ian Campbell ian.campb...@citrix.com ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 1/2] x86/efi: Reserve SMBIOS table region when EFI booting
On 02/04/15 09:24, Ross Lagerwall wrote: Some EFI firmware implementations may place the SMBIOS table in RAM marked as BootServicesData, which Xen does not consider as reserved. When dom0 tries to access the SMBIOS, the region is not contained in the initial P2M and it crashes with a page fault. To fix this, reserve the SMBIOS region. Also, fix the memcmp check for existence of the SMBIOS and the DMI checksum calculation. Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com Reviewed-by: Andrew Cooper andrew.coop...@citrix.com (Personally, I would either have folded the two patches together, or put the memcmp() one first, fixing both occurrences. However, as both of these are candidates for backports, this probably turns more into a bikeshed activity.) --- xen/arch/x86/dmi_scan.c | 25 +++-- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c index 187c01e..ec411e6 100644 --- a/xen/arch/x86/dmi_scan.c +++ b/xen/arch/x86/dmi_scan.c @@ -189,6 +189,8 @@ static inline bool_t __init dmi_checksum(const void __iomem *buf, static u32 __initdata efi_dmi_address; static u32 __initdata efi_dmi_size; +static u32 __initdata efi_smbios_address; +static u32 __initdata efi_smbios_size; static u64 __initdata efi_smbios3_address; static u32 __initdata efi_smbios3_size; @@ -209,13 +211,18 @@ void __init dmi_efi_get_table(const void *smbios, const void *smbios3) return; } - if (eps memcmp(eps-anchor, _SM_, 4) + if (eps memcmp(eps-anchor, _SM_, 4) == 0 eps-length = sizeof(*eps) - dmi_checksum(eps, eps-length) - memcmp(eps-dmi.anchor, _DMI_, 5) == 0 - dmi_checksum(eps-dmi, sizeof(eps-dmi))) { - efi_dmi_address = eps-dmi.address; - efi_dmi_size = eps-dmi.size; + dmi_checksum(eps, eps-length)) { + efi_smbios_address = (u32)(long)smbios; + efi_smbios_size = eps-length; + + if ( memcmp(eps-dmi.anchor, _DMI_, 5) == 0 + dmi_checksum(eps-dmi, + eps-length - offsetof(struct smbios_eps, dmi)) ) { + efi_dmi_address = eps-dmi.address; + efi_dmi_size = eps-dmi.size; + } } } @@ -236,6 +243,12 @@ const char *__init dmi_get_table(paddr_t *base, u32 *len) instance |= 2; return DMI; } + if (efi_smbios_size !(instance 4)) { + *base = efi_smbios_address; + *len = efi_smbios_size; + instance |= 4; + return SMBIOS; + } } else { char __iomem *p = maddr_to_virt(0xF), *q; union { ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d Posted-Interrupts
From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM This patch adds a global vector which is used to wake up the blocked vCPU when an interrupt is being posted to it. Signed-off-by: Feng Wu feng...@intel.com Suggested-by: Yang Zhang yang.z.zh...@intel.com --- xen/arch/x86/hvm/vmx/vmx.c| 33 + xen/include/asm-x86/hvm/hvm.h | 1 + xen/include/asm-x86/hvm/vmx/vmx.h | 3 +++ xen/include/xen/sched.h | 2 ++ 4 files changed, 39 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index ff5544d..b2b4c26 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -89,6 +89,7 @@ DEFINE_PER_CPU(struct list_head, blocked_vcpu_on_cpu); DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock); uint8_t __read_mostly posted_intr_vector; +uint8_t __read_mostly pi_wakeup_vector; static int vmx_domain_initialise(struct domain *d) { @@ -131,6 +132,8 @@ static int vmx_vcpu_initialise(struct vcpu *v) if ( v-vcpu_id == 0 ) v-arch.user_regs.eax = 1; +INIT_LIST_HEAD(v-blocked_vcpu_list); + return 0; } @@ -1834,11 +1837,19 @@ const struct hvm_function_table * __init start_vmx(void) } if ( cpu_has_vmx_posted_intr_processing ) +{ alloc_direct_apic_vector(posted_intr_vector, event_check_interrupt); + +if ( iommu_intpost ) +alloc_direct_apic_vector(pi_wakeup_vector, pi_wakeup_interrupt); +else +vmx_function_table.pi_desc_update = NULL; +} just style issue. Above conditional logic looks not intuitive to me. usually we have: if ( iommu_intpost ) vmx_function_table.pi_desc_update = func; else vmx_function_table.pi_desc_update = NULL; suppose you will register callback in later patch. then better to move the NULL one there too. Putting it here doesn't meet the normal if...else implications. :-) else { vmx_function_table.deliver_posted_intr = NULL; vmx_function_table.sync_pir_to_irr = NULL; +vmx_function_table.pi_desc_update = NULL; } if ( cpu_has_vmx_ept @@ -3255,6 +3266,28 @@ void vmx_vmenter_helper(const struct cpu_user_regs *regs) } /* + * Handle VT-d posted-interrupt when VCPU is blocked. + */ +void pi_wakeup_interrupt(struct cpu_user_regs *regs) +{ +struct vcpu *v; +int cpu = smp_processor_id(); + +spin_lock(per_cpu(blocked_vcpu_on_cpu_lock, cpu)); +list_for_each_entry(v, per_cpu(blocked_vcpu_on_cpu, cpu), +blocked_vcpu_list) { +struct pi_desc *pi_desc = v-arch.hvm_vmx.pi_desc; + +if ( pi_test_on(pi_desc) == 1 ) +tasklet_schedule(v-vcpu_wakeup_tasklet); why can't we directly call vcpu_unblock here? +} +spin_unlock(per_cpu(blocked_vcpu_on_cpu_lock, cpu)); + +ack_APIC_irq(); +this_cpu(irq_count)++; +} + +/* * Local variables: * mode: C * c-file-style: BSD diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h index 0dc909b..a11a256 100644 --- a/xen/include/asm-x86/hvm/hvm.h +++ b/xen/include/asm-x86/hvm/hvm.h @@ -195,6 +195,7 @@ struct hvm_function_table { void (*deliver_posted_intr)(struct vcpu *v, u8 vector); void (*sync_pir_to_irr)(struct vcpu *v); void (*handle_eoi)(u8 vector); +void (*pi_desc_update)(struct vcpu *v, int new_state); /*Walk nested p2m */ int (*nhvm_hap_walk_L1_p2m)(struct vcpu *v, paddr_t L2_gpa, diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index e643c3c..f4296ab 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -34,6 +34,7 @@ DECLARE_PER_CPU(struct list_head, blocked_vcpu_on_cpu); DECLARE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock); extern uint8_t posted_intr_vector; +extern uint8_t pi_wakeup_vector; typedef union { struct { @@ -574,6 +575,8 @@ int alloc_p2m_hap_data(struct p2m_domain *p2m); void free_p2m_hap_data(struct p2m_domain *p2m); void p2m_init_hap_data(struct p2m_domain *p2m); +void pi_wakeup_interrupt(struct cpu_user_regs *regs); + /* EPT violation qualifications definitions */ #define _EPT_READ_VIOLATION 0 #define EPT_READ_VIOLATION (1UL_EPT_READ_VIOLATION) diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index c874dd4..91f0912 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -148,6 +148,8 @@ struct vcpu struct vcpu *next_in_list; +struct list_head blocked_vcpu_list; + s_time_t periodic_period; s_time_t periodic_last_event; struct timer periodic_timer; -- 2.1.0 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 09/15] Add a new per-vCPU tasklet to wakeup the blocked vCPU
-Original Message- From: Tian, Kevin Sent: Thursday, April 02, 2015 1:53 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Zhang, Yang Z Subject: RE: [RFC v1 09/15] Add a new per-vCPU tasklet to wakeup the blocked vCPU From: Wu, Feng Sent: Wednesday, March 25, 2015 8:32 PM This patch adds a new per-vCPU tasklet to wakeup the blocked vCPU. It can be used in the case vcpu_unblock cannot be called directly. could you elaborate under which scenario vcpu_unblock can't be called directly? Please see the reply to [RFC v1 11/15] vmx: Add a global wake-up vector for VT-d Posted-Interrupts I will elaborate it a bit more in the next post. Thanks! Thanks, Feng Signed-off-by: Feng Wu feng...@intel.com --- xen/common/domain.c | 11 +++ xen/include/xen/sched.h | 3 +++ 2 files changed, 14 insertions(+) diff --git a/xen/common/domain.c b/xen/common/domain.c index aa78fd7..fe89658 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -109,6 +109,13 @@ static void vcpu_check_shutdown(struct vcpu *v) spin_unlock(d-shutdown_lock); } +static void vcpu_wakeup_tasklet_handler(unsigned long arg) +{ +struct vcpu *v = (void *)arg; + +vcpu_unblock(v); +} + struct vcpu *alloc_vcpu( struct domain *d, unsigned int vcpu_id, unsigned int cpu_id) { @@ -126,6 +133,9 @@ struct vcpu *alloc_vcpu( tasklet_init(v-continue_hypercall_tasklet, NULL, 0); +tasklet_init(v-vcpu_wakeup_tasklet, vcpu_wakeup_tasklet_handler, + (unsigned long)v); + if ( !zalloc_cpumask_var(v-cpu_hard_affinity) || !zalloc_cpumask_var(v-cpu_hard_affinity_tmp) || !zalloc_cpumask_var(v-cpu_hard_affinity_saved) || @@ -784,6 +794,7 @@ static void complete_domain_destroy(struct rcu_head *head) if ( (v = d-vcpu[i]) == NULL ) continue; tasklet_kill(v-continue_hypercall_tasklet); +tasklet_kill(v-vcpu_wakeup_tasklet); vcpu_destroy(v); sched_destroy_vcpu(v); destroy_waitqueue_vcpu(v); diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index ccd7ed8..c874dd4 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -239,6 +239,9 @@ struct vcpu /* Tasklet for continue_hypercall_on_cpu(). */ struct tasklet continue_hypercall_tasklet; +/* Tasklet for wakeup_blocked_vcpu(). */ +struct tasklet vcpu_wakeup_tasklet; + /* Multicall information. */ struct mc_state mc_state; -- 2.1.0 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH 1/2] x86/efi: Reserve SMBIOS table region when EFI booting
Some EFI firmware implementations may place the SMBIOS table in RAM marked as BootServicesData, which Xen does not consider as reserved. When dom0 tries to access the SMBIOS, the region is not contained in the initial P2M and it crashes with a page fault. To fix this, reserve the SMBIOS region. Also, fix the memcmp check for existence of the SMBIOS and the DMI checksum calculation. Signed-off-by: Ross Lagerwall ross.lagerw...@citrix.com --- xen/arch/x86/dmi_scan.c | 25 +++-- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c index 187c01e..ec411e6 100644 --- a/xen/arch/x86/dmi_scan.c +++ b/xen/arch/x86/dmi_scan.c @@ -189,6 +189,8 @@ static inline bool_t __init dmi_checksum(const void __iomem *buf, static u32 __initdata efi_dmi_address; static u32 __initdata efi_dmi_size; +static u32 __initdata efi_smbios_address; +static u32 __initdata efi_smbios_size; static u64 __initdata efi_smbios3_address; static u32 __initdata efi_smbios3_size; @@ -209,13 +211,18 @@ void __init dmi_efi_get_table(const void *smbios, const void *smbios3) return; } - if (eps memcmp(eps-anchor, _SM_, 4) + if (eps memcmp(eps-anchor, _SM_, 4) == 0 eps-length = sizeof(*eps) - dmi_checksum(eps, eps-length) - memcmp(eps-dmi.anchor, _DMI_, 5) == 0 - dmi_checksum(eps-dmi, sizeof(eps-dmi))) { - efi_dmi_address = eps-dmi.address; - efi_dmi_size = eps-dmi.size; + dmi_checksum(eps, eps-length)) { + efi_smbios_address = (u32)(long)smbios; + efi_smbios_size = eps-length; + + if ( memcmp(eps-dmi.anchor, _DMI_, 5) == 0 +dmi_checksum(eps-dmi, + eps-length - offsetof(struct smbios_eps, dmi)) ) { + efi_dmi_address = eps-dmi.address; + efi_dmi_size = eps-dmi.size; + } } } @@ -236,6 +243,12 @@ const char *__init dmi_get_table(paddr_t *base, u32 *len) instance |= 2; return DMI; } + if (efi_smbios_size !(instance 4)) { + *base = efi_smbios_address; + *len = efi_smbios_size; + instance |= 4; + return SMBIOS; + } } else { char __iomem *p = maddr_to_virt(0xF), *q; union { -- 2.1.0 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC PATCH v2 06/22] xen/arm: its: Port ITS driver to xen
On Wed, Apr 1, 2015 at 5:04 PM, Ian Campbell ian.campb...@citrix.com wrote: On Thu, 2015-03-19 at 20:07 +0530, vijay.kil...@gmail.com wrote: From: Vijaya Kumar K vijaya.ku...@caviumnetworks.com This patch just makes ITS driver taken from linux compiles in xen environment. What is your intention wrt future updates to this driver? Are you intending to keep things in sync and import things from the Linux side (similar to the smmu drviers) or are you taking the Linux code as a starting point and intending that it then be maintained independently as a Xen driver from then on? Yes, I intend to keep things in sync with Linux driver. I have kept most the code same as Linux side except removing unused code. Regards Vijay ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 4/4] hvmloader: add knob for fixed VGABIOS date string
On Wed, 2015-04-01 at 13:28 +, Olaf Hering wrote: To allow reproducible builds of hvmloader introduce a make variable VGABIOS_REL_DATE=dd Mon to provide a fixed date string. Without this change the hvmloader binary changes with every rebuild. Signed-off-by: Olaf Hering o...@aepfle.de Cc: Ian Jackson ian.jack...@eu.citrix.com Cc: Stefano Stabellini stefano.stabell...@eu.citrix.com Acked-by: Ian Campbell ian.campb...@citrix.com It's not clear if vgabios should be maintained by us tools folks or if it should have been taken under the hvmloader umbrella and maintained by the hypervisor folks. I've CC-d Jan, but I'm not going to block on an ack. Cc: Wei Liu wei.l...@citrix.com --- INSTALL | 1 + tools/firmware/vgabios/Makefile | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/INSTALL b/INSTALL index bf412ef..a0f2e7b 100644 --- a/INSTALL +++ b/INSTALL @@ -217,6 +217,7 @@ can be used to provide fixed timestamps in the expected format. XEN_BUILD_DATE=output of date(1) XEN_BUILD_TIME=hh:mm:ss SMBIOS_REL_DATE=mm/dd/ +VGABIOS_REL_DATE=dd Mon The following variables can be used to tweak some aspects of the hypervisor build. diff --git a/tools/firmware/vgabios/Makefile b/tools/firmware/vgabios/Makefile index 51d9e6e..3284812 100644 --- a/tools/firmware/vgabios/Makefile +++ b/tools/firmware/vgabios/Makefile @@ -5,10 +5,10 @@ BCC = bcc AS86 = as86 RELEASE = `pwd | sed s-.*/--` -RELDATE = `date '+%d %b %Y'` +VGABIOS_REL_DATE ?= `date '+%d %b %Y'` RELVERS = `pwd | sed s-.*/-- | sed s/vgabios// | sed s/-//` -VGABIOS_DATE = -DVGABIOS_DATE=\$(RELDATE)\ +VGABIOS_DATE = -DVGABIOS_DATE=\$(VGABIOS_REL_DATE)\ .PHONY: all all: bios cirrus-bios ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 09/47] vidoe: fbdev: atyfb: remove and fix MTRR MMIO hole work around
On Thu, Apr 2, 2015 at 12:45 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Wed, Apr 01, 2015 at 05:04:08PM -0700, Andy Lutomirski wrote: On Wed, Apr 1, 2015 at 4:52 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Sat, Mar 28, 2015 at 02:23:34PM +0200, Ville Syrjälä wrote: On Sat, Mar 28, 2015 at 01:28:18AM +0100, Luis R. Rodriguez wrote: On Fri, Mar 27, 2015 at 03:02:10PM -0700, Andy Lutomirski wrote: On Fri, Mar 27, 2015 at 2:56 PM, Ville Syrjälä syrj...@sci.fi wrote: On Fri, Mar 27, 2015 at 08:57:59PM +0100, Luis R. Rodriguez wrote: On Fri, Mar 27, 2015 at 12:43:55PM -0700, Andy Lutomirski wrote: On Fri, Mar 27, 2015 at 12:38 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Sat, Mar 21, 2015 at 11:15:14AM +0200, Ville Syrjälä wrote: On Fri, Mar 20, 2015 at 04:17:59PM -0700, Luis R. Rodriguez wrote: diff --git a/drivers/video/fbdev/aty/atyfb_base.c b/drivers/video/fbdev/aty/atyfb_base.c index 8025624..8875e56 100644 --- a/drivers/video/fbdev/aty/atyfb_base.c +++ b/drivers/video/fbdev/aty/atyfb_base.c @@ -2630,21 +2630,10 @@ static int aty_init(struct fb_info *info) #ifdef CONFIG_MTRR par-mtrr_aper = -1; - par-mtrr_reg = -1; if (!nomtrr) { - /* Cover the whole resource. */ - par-mtrr_aper = mtrr_add(par-res_start, par-res_size, + par-mtrr_aper = mtrr_add(info-fix.smem_start, + info-fix.smem_len, MTRR_TYPE_WRCOMB, 1); MTRRs need power of two size, so how is this supposed to work? As per mtrr_add_page() [0] the base and size are just supposed to be in units of 4 KiB, although the practice is to use powers of 2 in *some* drivers this is not standardized and by no means recorded as a requirement. Obviously powers of 2 will work too and you'd end up neatly aligned as well. mtrr_add() will use mtrr_check() to verify the the same requirement. Furthermore, as per my commit log message: Whatever the code may or may not do, the x86 architecture uses power-of-two MTRR sizes. So I'm confused. There should be no confusion, I simply did not know that *was* the requirement for x86, if that is the case we should add a check for that and perhaps generalize a helper that does the power of two helper changes, the cleanest I found was the vesafb driver solution. Thoughts? The vesafb solution is bad since you'll only end up covering only the first 4MB of the framebuffer instead of the almost 8MB you want. Which in practice will mean throwing away half the VRAM since you really don't want the massive performance hit from accessing it as UC. And that would mean giving up decent display resolutions as well :( And the other option of trying to cover the remainder with multiple ever smaller MTRRs doesn't work either since you'll run out of MTRRs very quickly. This is precisely why I used the hole method in atyfb in the first place. I don't really like the idea of any new mtrr code not supporting that use case, especially as these things tend to be present in older machines where PAT isn't an option. According to the Intel SDM, volume 3, section 11.5.2.1, table 11-6, non-PAT CPUs that have a WC MTRR, PCD = 1, and PWT = 1 (aka UC) have an effective memory type of UC. This is true but non-PAT systems that use just ioremap() will default to _PAGE_CACHE_MODE_UC_MINUS, not _PAGE_CACHE_MODE_UC, and _PAGE_CACHE_MODE_UC_MINUS on Linux has PCD = 1, PWT = 0. The list comes from: uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { [_PAGE_CACHE_MODE_WB ] = 0 | 0, [_PAGE_CACHE_MODE_WC ] = _PAGE_PWT | 0, [_PAGE_CACHE_MODE_UC_MINUS] = 0 | _PAGE_PCD, [_PAGE_CACHE_MODE_UC ] = _PAGE_PWT | _PAGE_PCD, [_PAGE_CACHE_MODE_WT ] = 0 | _PAGE_PCD, [_PAGE_CACHE_MODE_WP ] = 0 | _PAGE_PCD, }; This can better be read here: PAT |PCD ||PWT ||| 000 WB _PAGE_CACHE_MODE_WB 001 WC _PAGE_CACHE_MODE_WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS 011 UC _PAGE_CACHE_MODE_UC On x86 ioremap() defaults to ioremap_nocache() and right now that uses _PAGE_CACHE_MODE_UC_MINUS not _PAGE_CACHE_MODE_UC. We have two cases to consider for non-PAT systems then: a) Right now as ioremap() and ioremap_nocache() default to _PAGE_CACHE_MODE_UC_MINUS on x86. In this case using a WC MTRR seems to use PWT=0, PCD=1, and table table
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Thu, Apr 2, 2015 at 3:20 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Thu, Apr 2, 2015 at 1:13 PM, Bjorn Helgaas bhelg...@google.com wrote: On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote: I'll rephrase this to: --- It is possible to enable CONFIG_MTRR and up with it disabled at run time and yet CONFIG_X86_PAT continues to kick through with all functionally enabled. This can happen for instance on Xen where MTRR is not supported but PAT is, this can happen now on Linux as of commit 47591df50 by Juergen introduced as of v3.19. I still can't parse this. What does up with it disabled at run time mean? It means that technically even if your CPU/BIOS/system did support MTRR if you use a kernel with MTRR support enabled you might end up with a situation where under one situation MTRR might be enabled and at another run time scenario with the same exact kernel and system you will end up with MTRR disabled. Such is the case for example when booting with Xen, which disables the CPU bits on the hypervisor code. If you boot the same system without Xen you'll get MTRR. Your text is missing some words. You seem to be using up as a verb, but it's not a verb. Maybe you meant end up? Even then, it wouldn't make sense for CONFIG_MTRR to be disabled at run time because CONFIG_MTRR is a compile-time switch. The MTRR *functionality* could certainly be disabled at run-time, but not CONFIG_MTRR itself. And ... continues to kick through? Probably some idiomatic usage I'm just too old to understand :) That means for example that in both the above circumstances even if MTRR went disabled at run time with Xen, the kernel went through with getting PAT enabled. CONFIG_X86_PAT continues to kick through doesn't seem a very precise way of describing this. But maybe it's enough for experts in this area (which I'm not). Bjorn ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()
On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez mcg...@do-not-panic.com wrote: From: Luis R. Rodriguez mcg...@suse.com Ideally on systems using PAT we can expect a swift transition away from MTRR. There can be a few exceptions to this, one is where device drivers are known to exist on PATs with errata, This probably makes sense to someone steeped in MTRR and PAT, but not otherwise. One exception is where drivers are known to exist on PATs with errata? The drivers exist, independent of PAT/MTRR/errata. Do you mean there are drivers that can't be converted from MTRR to PAT because some PATs are broken? I don't really know anything about MTRR or PAT; I'm just trying to figure out how to parse this paragraph. another situation is observed on old device drivers where devices had combined MMIO register access with whatever area they typically later wanted to end up using MTRR for on the same PCI BAR. This situation can still be addressed by splitting up ioremap'd PCI BAR into two ioremap'd calls, one for MMIO registers, and another for whatever is desirable for write-combining -- in order to accomplish this though quite a bit of driver restructuring is required. Device drivers which are known to require large amount of re-work in order to split ioremap'd areas can use __arch_phys_wc_add() to avoid regressions when PAT is enabled. For a good example driver where things are neatly split up on a PCI BAR refer the infiniband qib driver. For a good example of a driver where good amount of work is required refer to the infiniband ipath driver. This is *only* a transitive API -- and as such no new drivers are ever expected to use this. transient? Do you mean you intend to remove this API in the near future? Bjorn ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Thu, Apr 2, 2015 at 1:13 PM, Bjorn Helgaas bhelg...@google.com wrote: On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote: I'll rephrase this to: --- It is possible to enable CONFIG_MTRR and up with it disabled at run time and yet CONFIG_X86_PAT continues to kick through with all functionally enabled. This can happen for instance on Xen where MTRR is not supported but PAT is, this can happen now on Linux as of commit 47591df50 by Juergen introduced as of v3.19. I still can't parse this. What does up with it disabled at run time mean? It means that technically even if your CPU/BIOS/system did support MTRR if you use a kernel with MTRR support enabled you might end up with a situation where under one situation MTRR might be enabled and at another run time scenario with the same exact kernel and system you will end up with MTRR disabled. Such is the case for example when booting with Xen, which disables the CPU bits on the hypervisor code. If you boot the same system without Xen you'll get MTRR. And ... continues to kick through? Probably some idiomatic usage I'm just too old to understand :) That means for example that in both the above circumstances even if MTRR went disabled at run time with Xen, the kernel went through with getting PAT enabled. Please use the conventional citation format: 47591df50512 (xen: Support Xen pv-domains using PAT) A one-character typo in a SHA1 makes it completely useless, so it's nice to have the summary line both for readability and a bit of redundancy. Sure, fixed. Luis ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] implementing a classic Xen front/back pv driver pair to provide a transport for 9P FS Protocol
On Thu, Apr 02, 2015 at 01:16:36PM -0600, Linda wrote: Thanks Konrad. I came close to doing your suggestions on git, and got it, sort of, to work. I will follow your instructions exactly for updates. Heh. The git manual is also a very good resource. There were also, some problems with the way the smtp server needed to be set up (compounded by typos of course), which are all fixed. As far as your answer to my build problems, I'm not sure what this will do: One way to help with that is if you do: make 12 2log And attach the log. I am semi-new to the linux world - is this to allow someone else to help me? Correct. What does the make command above do? Build. Thanks, again. Linda On 4/2/2015 12:59 PM, Konrad Rzeszutek Wilk wrote: On Wed, Apr 01, 2015 at 07:36:49PM -0600, Linda wrote: First, Julien, your suggestion worked like a charm. So here's what's happened tonight. I tried to build the tools directory of my git repository. Although I used the sudo command in my virtual ubuntu, I got a permission denied error 126 on xen_foreign. One way to help with that is if you do: make 12 2log And attach the log. Second, I tried to follow the protocol for submitting my patches. I changed libxl_utils.c and libxl_utils.h, in my repository, add and commit them. Unfortunately, I didn't include my signature (next page of instructions I was following), and couldn't figure how to get back in to add them. git commit --amend Or if you want to do more of them: git rebase -i origin/staging (and in the editor change 'pick' to 'r'). Finally, I tried git send-email (took a bit to find I had to install it). Now it doesn't like the format of my send-email: Again, please copy-n-paste the command line you had. to the devlopers list above and cc'ing Julien and Wei, followed by: 1. following this with the files (even with --no-format-patch), error was no subject line 2. (different attempt) the repository master error complained about the format patch Not sure I understand that. Is that for 'git send-email' or 'git format-patch'? SO if anyone is up at an ungodly hour and can explain any of these errors to me (I'm in Colorado - so it's 7:30 here), especially with a fix, I'd be grateful. Otherwise, Julien, Wei, I'll start at about 7am my time, maybe a little earlier. The previous OPW had an article about using git and how to do it with kernel patches. It is exactly the same flow - except different email address. Anyhow, what I end up doing is: 1). git format-patch --subject-prefix PATCH RFC origin/staging.. (which generates 0001-, 0002-, etc files - for two patches). 2). git send-email --to xen-de...@xenproject.org --to someotheremail.org --compose --subject [PATCH RFC] Patches to fix XYZ. 000*.patch And in the editor do a little writeup of what the patches have. At the end of this, I attach the output from: git diff --stat origin/staging.. git shortlog origin/staging.. which gives a nice diff output and what the patches are. And then send it off. Thanks. Linda Jacobson On 4/1/2015 2:57 PM, Julien Grall wrote: On 01/04/2015 18:46, Linda wrote: I'll try it. That's the libvncserver-dev libsdl-dev libjpeg62-dev Should I keep the libsdl-dev? In the meantime, I'm following the git protocol for patches. I successfully cloned xen.git. The next statement in the directions - I can't tell if it's one statement on many lines, or many statements. It starts out git branch -a When I type this alone, I get Not a git repository When I type in the many lines as a single command I get the error message: origin/master no such file or directory You have to type the command git branch -a in the git repository (i.e the directory xen.git). This comes from the line remotes/origin/HEAD-origin ? This is normal. The line starting by '$' is a command. Everything else is an example output of the execution of the command. Obviously, you have to drop the '$' when typing copying the command. To go further, '$' means a command to execute with your current user and '#' a command to execute with root privileges (i.e adding sudo before). This is usually a standard on Linux/BSD shell documentation. Although, there is some place within this wiki page where the command doesn't have '$'/'#' (see [1]). So you to judge yourself if the line looks like a command or not :). Regards, [1] http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Git_send-email ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] implementing a classic Xen front/back pv driver pair to provide a transport for 9P FS Protocol
Konrad - Here's the log. Maybe someone can make sense of this. Thanks. Linda log Description: Binary data ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Sat, Mar 28, 2015 at 12:56:30AM +0100, Luis R. Rodriguez wrote: On Fri, Mar 27, 2015 at 02:40:17PM -0600, Toshi Kani wrote: On Fri, 2015-03-20 at 16:17 -0700, Luis R. Rodriguez wrote: : @@ -734,6 +742,7 @@ void __init mtrr_bp_init(void) } if (mtrr_if) { + mtrr_enabled = true; set_num_var_ranges(); init_table(); if (use_intel()) { get_mtrr_state(); After setting mtrr_enabled to true, get_mtrr_state() reads MSR_MTRRdefType and sets 'mtrr_state.enabled', which also indicates if MTRRs are enabled or not on the system. So, potentially, we could have a case that mtrr_enabled is set to true, but mtrr_state.enabled is set to disabled when MTRRs are disabled by BIOS. Thanks for the review, in this case then we should update mtrr_enabled to false. ps. I recently cleaned up this part of the MTRR code in the patch below, which is currently available in the -mm -next trees. https://lkml.org/lkml/2015/3/24/1063 Great I will rebase and work with that and try to address this consideration you have raised. OK I'll mesh in this change as well in my next respin: diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index a83f27a..ecf7cb9 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -438,7 +438,7 @@ static void __init print_mtrr_state(void) } /* Grab all of the MTRR state for this CPU into *state */ -void __init get_mtrr_state(void) +bool __init get_mtrr_state(void) { struct mtrr_var_range *vrs; unsigned long flags; @@ -482,6 +482,8 @@ void __init get_mtrr_state(void) post_set(); local_irq_restore(flags); + + return !!mtrr_state.enabled; } /* Some BIOS's are messed up and don't set all MTRRs the same! */ diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c index ea5f363..f96195e 100644 --- a/arch/x86/kernel/cpu/mtrr/main.c +++ b/arch/x86/kernel/cpu/mtrr/main.c @@ -734,22 +742,25 @@ void __init mtrr_bp_init(void) } if (mtrr_if) { + mtrr_enabled = true; set_num_var_ranges(); init_table(); if (use_intel()) { - get_mtrr_state(); + /* BIOS may override */ + mtrr_enabled = get_mtrr_state(); if (mtrr_cleanup(phys_addr)) { changed_by_mtrr_cleanup = 1; @@ -745,11 +755,14 @@ void __init mtrr_bp_init(void) } } } + + if (!mtrr_enabled) + pr_info(mtrr: system does not support MTRR\n); } diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h index df5e41f..951884d 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.h +++ b/arch/x86/kernel/cpu/mtrr/mtrr.h @@ -51,7 +51,7 @@ void set_mtrr_prepare_save(struct set_mtrr_context *ctxt); void fill_mtrr_var_range(unsigned int index, u32 base_lo, u32 base_hi, u32 mask_lo, u32 mask_hi); -void get_mtrr_state(void); +bool get_mtrr_state(void); extern void set_mtrr_ops(const struct mtrr_ops *ops); ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote: I'll rephrase this to: --- It is possible to enable CONFIG_MTRR and up with it disabled at run time and yet CONFIG_X86_PAT continues to kick through with all functionally enabled. This can happen for instance on Xen where MTRR is not supported but PAT is, this can happen now on Linux as of commit 47591df50 by Juergen introduced as of v3.19. I still can't parse this. What does up with it disabled at run time mean? And ... continues to kick through? Probably some idiomatic usage I'm just too old to understand :) Please use the conventional citation format: 47591df50512 (xen: Support Xen pv-domains using PAT) A one-character typo in a SHA1 makes it completely useless, so it's nice to have the summary line both for readability and a bit of redundancy. Bjorn ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH] tools/libxl new bitmap functions
On Thu, Apr 02, 2015 at 11:38:16AM -0600, Linda Jacobson wrote: From: Linda lin...@jma3.com Added bitmap functions for union intersection and difference betweenn two bitmaps Signed-off-by: Linda lin...@jma3.com --- tools/libxl/libxl_utils.c | 115 ++ tools/libxl/libxl_utils.h | 10 2 files changed, 125 insertions(+) diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c index 9053b27..c390ddc 100644 --- a/tools/libxl/libxl_utils.c +++ b/tools/libxl/libxl_utils.c @@ -699,6 +699,121 @@ int libxl_bitmap_count_set(const libxl_bitmap *bitmap) return nr_set_bits; } +int libxl_bitmap_union(libxl_ctx *ctx, libxl_bitmap *union_bitmap, You have an extra space at the end.. +libxl_bitmap *bitmap1, libxl_bitmap *bitmap2) And this should really start at the same line length as 'libxl_ctx' Ditto for the rest of the functions. +{ +int size; +int rc; + +GC_INIT(ctx); + +// if bitmaps aren't the same size, union should be size of larger bit map The comments are in: /* Comment here. */ +size = (bitmap1-size bitmap2-size) ? bitmap1-size : bitmap2-size; There might be an 'max' macro somwhere in the code that you could use for this? + +libxl_bitmap_init(union_bitmap); +rc = libxl_bitmap_alloc(ctx, union_bitmap, size); +if (rc) +{ +// Following the coding standards here. + //First goto I've written in decades. Hehe. No need to add the braces, you can just do: if (rc) goto out; +goto out; +} + +for (int bit = 0; bit (size * 8); bit++) Please move the 'int bit' to the start of the function. +{ +// libxl_bitmap_test returns 0 if past end of bitmap +// if the bit is set in either bitmap, set it in their union I would change that comment to be: /* NB. libxl_bitmap_test returns 0 if past the end of bitmap. */ +if (libxl_bitmap_test(bitmap1, bit)) +{ +libxl_bitmap_set(union_bitmap, bit); +} +else if (libxl_bitmap_test(bitmap2, bit)) +{ +libxl_bitmap_set(union_bitmap, bit); +} You can ditch the braces. +} + +out: +GC_FREE; +return rc; +} + +int libxl_bitmap_intersection (libxl_ctx *ctx, libxl_bitmap There is space :^ - which should not be there. +*intersection_bitmap, libxl_bitmap *bitmap1, libxl_bitmap *bitmap2) +{ +int size; +int rc; + +GC_INIT(ctx); + +// if bitmaps aren't same size, intersection should be size of +// smaller bit map I believe the comments I've above apply to the code below as well. +size = (bitmap1-size bitmap2-size) ? bitmap2-size : bitmap1-size; + +libxl_bitmap_init(intersection_bitmap); +rc = libxl_bitmap_alloc(ctx, intersection_bitmap, size); +if (rc) +{ +goto out; +} + +for (int bit = 0; bit (size * 8); bit++) +{ +// libxl_bitmap_test returns 0 if past end of bitmap +// if the bit is set in both bitmaps, set it in their intersection +if (libxl_bitmap_test (bitmap1, bit) + libxl_bitmap_test (bitmap2, bit) ) You have an extra space at the end there. Don't think you need that in libxl code (thought you do for libxc). Yeah, two different styleguides! +{ +libxl_bitmap_set (intersection_bitmap, bit); +} +} + +out: +GC_FREE; +return rc; +} +int libxl_bitmap_difference(libxl_ctx *ctx, libxl_bitmap *difference_bitmap, +libxl_bitmap *bitmap1, libxl_bitmap *bitmap2) +{ +int size; +int rc; + +GC_INIT(ctx); + +// if bitmaps aren't the same size, difference should be size of larger +size = (bitmap1-size bitmap2-size) ? bitmap1-size : bitmap2-size; + +libxl_bitmap_init(difference_bitmap); +rc = libxl_bitmap_alloc(ctx, difference_bitmap, size); +if (rc) +{ +goto out; +} + +for (int bit = 0; bit (size * 8); bit++) +{ +/* libxl_bitmap_test returns 0 if past end of bitmap + if the bit is set in one bitmap and not the other, set it in +their difference +NOTE: if one bit map is larger, this will result in all bits +being set past the size of the smaller bitmap; if this is not +the desired behavior, please let me know That would make this a bit strange. Perhaps demand that they must be the same size? If they are not then just return an error? +*/ + +if (libxl_bitmap_test (bitmap1, bit) You have an extra space at the end there. ^ + (!libxl_bitmap_test (bitmap2, bit)) ) +{ +libxl_bitmap_set (difference_bitmap, bit); +} +} + +out: +GC_FREE; +return rc; +} + + + /* NB. caller is responsible for freeing the memory */ char *libxl_bitmap_to_hex_string(libxl_ctx *ctx, const libxl_bitmap *bitmap)
Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()
On Thu, Apr 02, 2015 at 03:21:22PM -0500, Bjorn Helgaas wrote: On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez mcg...@do-not-panic.com wrote: From: Luis R. Rodriguez mcg...@suse.com Ideally on systems using PAT we can expect a swift transition away from MTRR. There can be a few exceptions to this, one is where device drivers are known to exist on PATs with errata, This probably makes sense to someone steeped in MTRR and PAT, but not otherwise. One exception is where drivers are known to exist on PATs with errata? The drivers exist, independent of PAT/MTRR/errata. Do you mean there are drivers that can't be converted from MTRR to PAT because some PATs are broken? Well there is that but it seems we have motivation to address the PAT broken systems so this would be one of the lower priority reasons to consider adding this API. The more important reason is below. I don't really know anything about MTRR or PAT; I'm just trying to figure out how to parse this paragraph. Sure. another situation is observed on old device drivers where devices had combined MMIO register access with whatever area they typically later wanted to end up using MTRR for on the same PCI BAR. This situation can still be addressed by splitting up ioremap'd PCI BAR into two ioremap'd calls, one for MMIO registers, and another for whatever is desirable for write-combining -- in order to accomplish this though quite a bit of driver restructuring is required. Device drivers which are known to require large amount of re-work in order to split ioremap'd areas can use __arch_phys_wc_add() to avoid regressions when PAT is enabled. For a good example driver where things are neatly split up on a PCI BAR refer the infiniband qib driver. For a good example of a driver where good amount of work is required refer to the infiniband ipath driver. This is *only* a transitive API -- and as such no new drivers are ever expected to use this. transient? Do you mean you intend to remove this API in the near future? That's correct, the problem is that in order to use PAT cleanly we'd need to change these drivers to not overlap ioremap'd areas otherwise things can get quite complex, and changing the way we do the ioremap() calls on a driver might require a bit of work. The atyfb driver changes I did are an example of the types of changes that are expected. In the most complex worst cases there are MTRR hole tricks used, and as can be observed with the atyfb driver changes there are a series of things to consider when this is done specially in light of eventually making strong UC the default instead of UC-. I might be able to work around not adding this API by reviewing the users I had in this series again and seeing if something similar to what I will do on atyfb can be done in the meantime by using ioremap_uc(). Its not clear to me yet. Luis ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 09/47] vidoe: fbdev: atyfb: remove and fix MTRR MMIO hole work around
On Wed, Apr 01, 2015 at 05:04:08PM -0700, Andy Lutomirski wrote: On Wed, Apr 1, 2015 at 4:52 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Sat, Mar 28, 2015 at 02:23:34PM +0200, Ville Syrjälä wrote: On Sat, Mar 28, 2015 at 01:28:18AM +0100, Luis R. Rodriguez wrote: On Fri, Mar 27, 2015 at 03:02:10PM -0700, Andy Lutomirski wrote: On Fri, Mar 27, 2015 at 2:56 PM, Ville Syrjälä syrj...@sci.fi wrote: On Fri, Mar 27, 2015 at 08:57:59PM +0100, Luis R. Rodriguez wrote: On Fri, Mar 27, 2015 at 12:43:55PM -0700, Andy Lutomirski wrote: On Fri, Mar 27, 2015 at 12:38 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Sat, Mar 21, 2015 at 11:15:14AM +0200, Ville Syrjälä wrote: On Fri, Mar 20, 2015 at 04:17:59PM -0700, Luis R. Rodriguez wrote: diff --git a/drivers/video/fbdev/aty/atyfb_base.c b/drivers/video/fbdev/aty/atyfb_base.c index 8025624..8875e56 100644 --- a/drivers/video/fbdev/aty/atyfb_base.c +++ b/drivers/video/fbdev/aty/atyfb_base.c @@ -2630,21 +2630,10 @@ static int aty_init(struct fb_info *info) #ifdef CONFIG_MTRR par-mtrr_aper = -1; - par-mtrr_reg = -1; if (!nomtrr) { - /* Cover the whole resource. */ - par-mtrr_aper = mtrr_add(par-res_start, par-res_size, + par-mtrr_aper = mtrr_add(info-fix.smem_start, + info-fix.smem_len, MTRR_TYPE_WRCOMB, 1); MTRRs need power of two size, so how is this supposed to work? As per mtrr_add_page() [0] the base and size are just supposed to be in units of 4 KiB, although the practice is to use powers of 2 in *some* drivers this is not standardized and by no means recorded as a requirement. Obviously powers of 2 will work too and you'd end up neatly aligned as well. mtrr_add() will use mtrr_check() to verify the the same requirement. Furthermore, as per my commit log message: Whatever the code may or may not do, the x86 architecture uses power-of-two MTRR sizes. So I'm confused. There should be no confusion, I simply did not know that *was* the requirement for x86, if that is the case we should add a check for that and perhaps generalize a helper that does the power of two helper changes, the cleanest I found was the vesafb driver solution. Thoughts? The vesafb solution is bad since you'll only end up covering only the first 4MB of the framebuffer instead of the almost 8MB you want. Which in practice will mean throwing away half the VRAM since you really don't want the massive performance hit from accessing it as UC. And that would mean giving up decent display resolutions as well :( And the other option of trying to cover the remainder with multiple ever smaller MTRRs doesn't work either since you'll run out of MTRRs very quickly. This is precisely why I used the hole method in atyfb in the first place. I don't really like the idea of any new mtrr code not supporting that use case, especially as these things tend to be present in older machines where PAT isn't an option. According to the Intel SDM, volume 3, section 11.5.2.1, table 11-6, non-PAT CPUs that have a WC MTRR, PCD = 1, and PWT = 1 (aka UC) have an effective memory type of UC. This is true but non-PAT systems that use just ioremap() will default to _PAGE_CACHE_MODE_UC_MINUS, not _PAGE_CACHE_MODE_UC, and _PAGE_CACHE_MODE_UC_MINUS on Linux has PCD = 1, PWT = 0. The list comes from: uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { [_PAGE_CACHE_MODE_WB ] = 0 | 0, [_PAGE_CACHE_MODE_WC ] = _PAGE_PWT | 0, [_PAGE_CACHE_MODE_UC_MINUS] = 0 | _PAGE_PCD, [_PAGE_CACHE_MODE_UC ] = _PAGE_PWT | _PAGE_PCD, [_PAGE_CACHE_MODE_WT ] = 0 | _PAGE_PCD, [_PAGE_CACHE_MODE_WP ] = 0 | _PAGE_PCD, }; This can better be read here: PAT |PCD ||PWT ||| 000 WB _PAGE_CACHE_MODE_WB 001 WC _PAGE_CACHE_MODE_WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS 011 UC _PAGE_CACHE_MODE_UC On x86 ioremap() defaults to ioremap_nocache() and right now that uses _PAGE_CACHE_MODE_UC_MINUS not _PAGE_CACHE_MODE_UC. We have two cases to consider for non-PAT systems then: a) Right now as ioremap() and ioremap_nocache() default to _PAGE_CACHE_MODE_UC_MINUS on x86. In this case using a WC MTRR seems to use PWT=0, PCD=1, and table table 11-6 on non-PAT systems seems to place this situation as implementation
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Thu, Apr 02, 2015 at 03:28:51PM -0500, Bjorn Helgaas wrote: On Thu, Apr 2, 2015 at 3:20 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Thu, Apr 2, 2015 at 1:13 PM, Bjorn Helgaas bhelg...@google.com wrote: On Thu, Mar 26, 2015 at 6:35 PM, Luis R. Rodriguez mcg...@suse.com wrote: I'll rephrase this to: --- It is possible to enable CONFIG_MTRR and up with it disabled at run time and yet CONFIG_X86_PAT continues to kick through with all functionally enabled. This can happen for instance on Xen where MTRR is not supported but PAT is, this can happen now on Linux as of commit 47591df50 by Juergen introduced as of v3.19. I still can't parse this. What does up with it disabled at run time mean? It means that technically even if your CPU/BIOS/system did support MTRR if you use a kernel with MTRR support enabled you might end up with a situation where under one situation MTRR might be enabled and at another run time scenario with the same exact kernel and system you will end up with MTRR disabled. Such is the case for example when booting with Xen, which disables the CPU bits on the hypervisor code. If you boot the same system without Xen you'll get MTRR. Your text is missing some words. You seem to be using up as a verb, but it's not a verb. Maybe you meant end up? Indeed. Even then, it wouldn't make sense for CONFIG_MTRR to be disabled at run time because CONFIG_MTRR is a compile-time switch. The MTRR *functionality* could certainly be disabled at run-time, but not CONFIG_MTRR itself. I'll clarify. And ... continues to kick through? Probably some idiomatic usage I'm just too old to understand :) That means for example that in both the above circumstances even if MTRR went disabled at run time with Xen, the kernel went through with getting PAT enabled. CONFIG_X86_PAT continues to kick through doesn't seem a very precise way of describing this. But maybe it's enough for experts in this area (which I'm not). I've rephrased this to: --- It is possible to enable CONFIG_MTRR and CONFIG_X86_PAT and end up with a system with MTRR functionality disabled PAT functionality enabled. This can happen for instance on Xen where MTRR is not supported but PAT is. This can happen on Linux as of commit 47591df50 (xen: Support Xen pv-domains using PAT) by Juergen, introduced as of v3.19. --- Luis ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Thu, Apr 2, 2015 at 4:02 PM, Luis R. Rodriguez mcg...@suse.com wrote: --- It is possible to enable CONFIG_MTRR and CONFIG_X86_PAT and end up with a system with MTRR functionality disabled PAT functionality enabled. This is missing a conjunction or something in MTRR functionality disabled PAT functionality. Bjorn ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 02/47] x86: mtrr: generalize run time disabling of MTRR
On Thu, 2015-04-02 at 23:49 +0200, Luis R. Rodriguez wrote: On Sat, Mar 28, 2015 at 12:56:30AM +0100, Luis R. Rodriguez wrote: On Fri, Mar 27, 2015 at 02:40:17PM -0600, Toshi Kani wrote: On Fri, 2015-03-20 at 16:17 -0700, Luis R. Rodriguez wrote: : @@ -734,6 +742,7 @@ void __init mtrr_bp_init(void) } if (mtrr_if) { + mtrr_enabled = true; set_num_var_ranges(); init_table(); if (use_intel()) { get_mtrr_state(); After setting mtrr_enabled to true, get_mtrr_state() reads MSR_MTRRdefType and sets 'mtrr_state.enabled', which also indicates if MTRRs are enabled or not on the system. So, potentially, we could have a case that mtrr_enabled is set to true, but mtrr_state.enabled is set to disabled when MTRRs are disabled by BIOS. Thanks for the review, in this case then we should update mtrr_enabled to false. ps. I recently cleaned up this part of the MTRR code in the patch below, which is currently available in the -mm -next trees. https://lkml.org/lkml/2015/3/24/1063 Great I will rebase and work with that and try to address this consideration you have raised. OK I'll mesh in this change as well in my next respin: diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index a83f27a..ecf7cb9 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -438,7 +438,7 @@ static void __init print_mtrr_state(void) } /* Grab all of the MTRR state for this CPU into *state */ -void __init get_mtrr_state(void) +bool __init get_mtrr_state(void) { struct mtrr_var_range *vrs; unsigned long flags; @@ -482,6 +482,8 @@ void __init get_mtrr_state(void) post_set(); local_irq_restore(flags); + + return !!mtrr_state.enabled; This should be: return mtrr_state.enabled MTRR_STATE_MTRR_ENABLED; because the MTRR_STATE_MTRR_FIXED_ENABLED flag is ignored when the MTRR_STATE_MTRR_ENABLED flag is clear. Thanks, -Toshi ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [xen-4.2-testing test] 50288: regressions - FAIL
flight 50288 xen-4.2-testing real [real] http://logs.test-lab.xenproject.org/osstest/logs/50288/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemut-debianhvm-amd64 15 guest-start.2 fail REGR. vs. 36512 Tests which did not succeed, but are not blocking: test-i386-i386-rumpuserxen-i386 1 build-check(1) blocked n/a test-amd64-i386-rumpuserxen-i386 1 build-check(1) blocked n/a test-amd64-amd64-rumpuserxen-amd64 1 build-check(1) blocked n/a build-amd64-rumpuserxen 5 rumpuserxen-buildfail never pass test-amd64-amd64-xl-qemuu-ovmf-amd64 7 debian-hvm-install fail never pass test-amd64-i386-xl-qemuu-ovmf-amd64 7 debian-hvm-install fail never pass test-amd64-amd64-libvirt 10 migrate-support-checkfail never pass test-i386-i386-libvirt 10 migrate-support-checkfail never pass test-amd64-i386-libvirt 10 migrate-support-checkfail never pass test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass test-amd64-amd64-xl-winxpsp3 14 guest-stop fail never pass test-amd64-amd64-xl-win7-amd64 14 guest-stop fail never pass test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop fail never pass test-i386-i386-xl-qemut-winxpsp3 14 guest-stop fail never pass test-i386-i386-xl-qemuu-winxpsp3 14 guest-stop fail never pass test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop fail never pass test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop fail never pass test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop fail never pass test-amd64-i386-xl-win7-amd64 14 guest-stop fail never pass test-i386-i386-xl-winxpsp3 14 guest-stop fail never pass test-amd64-i386-xend-winxpsp3 17 leak-check/check fail never pass test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass build-i386-rumpuserxen5 rumpuserxen-buildfail never pass test-amd64-i386-xend-qemut-winxpsp3 17 leak-check/checkfail never pass version targeted for testing: xen 70bca60d35f02f512548d15b62863d366461be6f baseline version: xen 5bec01c19839e150e489dd04376c65f961830c86 People who touched revisions under test: Ian Jackson ian.jack...@eu.citrix.com jobs: build-amd64 pass build-i386 pass build-amd64-libvirt pass build-i386-libvirt pass build-amd64-pvopspass build-i386-pvops pass build-amd64-rumpuserxen fail build-i386-rumpuserxen fail test-amd64-amd64-xl pass test-amd64-i386-xl pass test-i386-i386-xlpass test-amd64-i386-rhel6hvm-amd pass test-amd64-i386-qemut-rhel6hvm-amd pass test-amd64-i386-qemuu-rhel6hvm-amd pass test-amd64-amd64-xl-qemut-debianhvm-amd64fail test-amd64-i386-xl-qemut-debianhvm-amd64 pass test-amd64-amd64-xl-qemuu-debianhvm-amd64pass test-amd64-i386-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-qemuu-freebsd10-amd64pass test-amd64-amd64-xl-qemuu-ovmf-amd64 fail test-amd64-i386-xl-qemuu-ovmf-amd64 fail test-amd64-amd64-rumpuserxen-amd64 blocked test-amd64-amd64-xl-qemut-win7-amd64 fail test-amd64-i386-xl-qemut-win7-amd64 fail test-amd64-amd64-xl-qemuu-win7-amd64 fail test-amd64-i386-xl-qemuu-win7-amd64 fail test-amd64-amd64-xl-win7-amd64 fail test-amd64-i386-xl-win7-amd64fail test-amd64-amd64-xl-credit2 pass test-i386-i386-xl-credit2
Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()
On Thu, Apr 2, 2015 at 3:35 PM, Bjorn Helgaas bhelg...@google.com wrote: [-cc Venkatesh, Suresh] On Thu, Apr 2, 2015 at 3:55 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Thu, Apr 02, 2015 at 03:21:22PM -0500, Bjorn Helgaas wrote: On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez mcg...@do-not-panic.com wrote: From: Luis R. Rodriguez mcg...@suse.com This is *only* a transitive API -- and as such no new drivers are ever expected to use this. transient? Do you mean you intend to remove this API in the near future? That's correct, the problem is that in order to use PAT cleanly we'd need to change these drivers ... I was just trying to ask whether you intended to write transient instead of transitive. But I'm not doing a very good job :) Yes, corrected, thanks. Luis ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 06/47] mtrr: add __arch_phys_wc_add()
[-cc Venkatesh, Suresh] On Thu, Apr 2, 2015 at 3:55 PM, Luis R. Rodriguez mcg...@suse.com wrote: On Thu, Apr 02, 2015 at 03:21:22PM -0500, Bjorn Helgaas wrote: On Fri, Mar 20, 2015 at 6:17 PM, Luis R. Rodriguez mcg...@do-not-panic.com wrote: From: Luis R. Rodriguez mcg...@suse.com This is *only* a transitive API -- and as such no new drivers are ever expected to use this. transient? Do you mean you intend to remove this API in the near future? That's correct, the problem is that in order to use PAT cleanly we'd need to change these drivers ... I was just trying to ask whether you intended to write transient instead of transitive. But I'm not doing a very good job :) Bjorn ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running
-Original Message- From: Konrad Rzeszutek Wilk [mailto:konrad.w...@oracle.com] Sent: Friday, April 03, 2015 3:15 AM To: Tian, Kevin Cc: Wu, Feng; Zhang, Yang Z; xen-devel@lists.xen.org; k...@xen.org; jbeul...@suse.com Subject: Re: [Xen-devel] [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running On Thu, Apr 02, 2015 at 06:08:12AM +, Tian, Kevin wrote: From: Wu, Feng Sent: Friday, March 27, 2015 12:58 PM -Original Message- From: Zhang, Yang Z Sent: Friday, March 27, 2015 12:44 PM To: Wu, Feng; xen-devel@lists.xen.org Cc: jbeul...@suse.com; k...@xen.org; Tian, Kevin Subject: RE: [RFC v1 12/15] vmx: Properly handle notification event when vCPU is running Wu, Feng wrote on 2015-03-27: Zhang, Yang Z wrote on 2015-03-25: when vCPU is running Wu, Feng wrote on 2015-03-25: When a vCPU is running in Root mode and a notification event has been injected to it. we need to set VCPU_KICK_SOFTIRQ for the current cpu, so the pending interrupt in PIRR will be synced to vIRR before This would imply that we had VMEXIT-ed due to pending interrupt? And we end up calling 'do_IRQ'? If so then the DPCI_SOFTIRQ ends up being set and you stll end up calling the softirq code? No. Here is the scenario for the description of this patch: When vCPU is running in root-mode (such as via hypercall, or any other reasons which can result in VM-Exit), and before vCPU is back to non-root, external interrupts happen. Notice that the VM-exit is not caused by this external interrupt. Thanks, Feng VM-Exit in time. Shouldn't the pending interrupt be synced unconditionally before next vmentry? What happens if we didn't set the softirq? If we didn't set the softirq in the notification handler, the interrupts happened exactly before VM-entry cannot be delivered to guest at this time. Please see the following code fragments from xen/arch/x86/hvm/vmx/entry.S: (pls pay attention to the comments) .Lvmx_do_vmentry .. /* If Vt-d engine issues a notification event here, * it cannot be delivered to guest during this VM-entry * without raising the softirq in notification handler. */ cmp %ecx,(%rdx,%rax,1) jnz .Lvmx_process_softirqs .. je .Lvmx_launch .. .Lvmx_process_softirqs: sti call do_softirq jmp .Lvmx_do_vmentry You are right! This helps me to recall why raise the softirq when delivering the PI. Yes, __vmx_deliver_posted_interrupt() is the software way to deliver PI, it sets the softirq for this purpose, however, when VT-d HW delivers PI, we have no control to the HW itself, hence we need to set this softirq in the Notification Event handler. could you include this information in the comment so others can easily understand this requirement? from code you only mentioned VCPU_KICK _SOFTIRQ is required, but how it leads to PIRR-VIRR sync is not explained. Thanks Kevin ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH 8/9] qspinlock: Generic paravirt support
On 04/02/2015 03:48 PM, Peter Zijlstra wrote: On Thu, Apr 02, 2015 at 07:20:57PM +0200, Peter Zijlstra wrote: pv_wait_head(): pv_hash() /* MB as per cmpxchg */ cmpxchg(l-locked, _Q_LOCKED_VAL, _Q_SLOW_VAL); VS __pv_queue_spin_unlock(): if (xchg(l-locked, 0) != _Q_SLOW_VAL) return; /* MB as per xchg */ pv_hash_find(lock); Something like so.. compile tested only. I took out the LFSR because that was likely over engineering from my side :-) --- a/kernel/locking/qspinlock_paravirt.h +++ b/kernel/locking/qspinlock_paravirt.h @@ -2,6 +2,8 @@ #error do not include this file #endif +#includelinux/hash.h + /* * Implement paravirt qspinlocks; the general idea is to halt the vcpus instead * of spinning them. @@ -107,7 +109,84 @@ static void pv_kick_node(struct mcs_spin pv_kick(pn-cpu); } -static DEFINE_PER_CPU(struct qspinlock *, __pv_lock_wait); +/* + * Hash table using open addressing with an linear probe sequence. + * + * Since we should not be holding locks from NMI context (very rare indeed) the + * max load factor is 0.75, which is around the point where open adressing + * breaks down. + * + * Instead of probing just the immediate bucket we probe all buckets in the + * same cacheline. + * + * http://en.wikipedia.org/wiki/Hash_table#Open_addressing + * + */ + +struct pv_hash_bucket { + struct qspinlock *lock; + int cpu; +}; + +/* + * XXX dynamic allocate using nr_cpu_ids instead... + */ +#define PV_LOCK_HASH_BITS (2 + NR_CPUS_BITS) + +#if PV_LOCK_HASH_BITS 6 +#undef PV_LOCK_HASH_BITS +#define PB_LOCK_HASH_BITS 6 +#endif + +#define PV_LOCK_HASH_SIZE (1 PV_LOCK_HASH_BITS) + +static struct pv_hash_bucket __pv_lock_hash[PV_LOCK_HASH_SIZE] cacheline_aligned; + +#define PV_HB_PER_LINE (SMP_CACHE_BYTES / sizeof(struct pv_hash_bucket)) + +static inline u32 hash_align(u32 hash) +{ + return hash ~(PV_HB_PER_LINE - 1); +} + +#define for_each_hash_bucket(hb, off, hash) \ + for (hash = hash_align(hash), off = 0, hb =__pv_lock_hash[hash + off];\ + off PV_LOCK_HASH_SIZE; \ + off++, hb =__pv_lock_hash[(hash + off) % PV_LOCK_HASH_SIZE]) + +static struct pv_hash_bucket *pv_hash_insert(struct qspinlock *lock) +{ + u32 offset, hash = hash_ptr(lock, PV_LOCK_HASH_BITS); + struct pv_hash_bucket *hb; + + for_each_hash_bucket(hb, offset, hash) { + if (!cmpxchg(hb-lock, NULL, lock)) { + WRITE_ONCE(hb-cpu, smp_processor_id()); + return hb; + } + } + + /* +* Hard assumes there is an empty bucket somewhere. +*/ + BUG(); +} + +static struct pv_hash_bucket *pv_hash_find(struct qspinlock *lock) +{ + u32 offset, hash = hash_ptr(lock, PV_LOCK_HASH_BITS); + struct pv_hash_bucket *hb; + + for_each_hash_bucket(hb, offset, hash) { + if (READ_ONCE(hb-lock) == lock) + return hb; + } + + /* +* Hard assumes we _WILL_ find a match. +*/ + BUG(); +} /* * Wait for l-locked to become clear; halt the vcpu after a short spin. @@ -116,7 +195,9 @@ static DEFINE_PER_CPU(struct qspinlock * static void pv_wait_head(struct qspinlock *lock) { struct __qspinlock *l = (void *)lock; + struct pv_hash_bucket *hb = NULL; int loop; + u8 o; for (;;) { for (loop = SPIN_THRESHOLD; loop; loop--) { @@ -126,29 +207,47 @@ static void pv_wait_head(struct qspinloc cpu_relax(); } - this_cpu_write(__pv_lock_wait, lock); - /* -* __pv_lock_wait must be set before setting _Q_SLOW_VAL -* -* [S] __pv_lock_wait = lock[RmW] l = l-locked = 0 -* MB MB -* [S] l-locked = _Q_SLOW_VAL [L] __pv_lock_wait -* -* Matches the xchg() in pv_queue_spin_unlock(). -*/ - if (!cmpxchg(l-locked, _Q_LOCKED_VAL, _Q_SLOW_VAL)) - goto done; + if (!hb) { + hb = pv_hash_insert(lock); + /* +* hb must be set before setting _Q_SLOW_VAL +* +* [S] hb- lock [RmW] l = l-locked = 0 +* MB MB +* [RmW] l-locked ?= _Q_SLOW_VAL [L] hb +* +* Matches the xchg() in pv_queue_spin_unlock(). +*/ + o = cmpxchg(l-locked, _Q_LOCKED_VAL, _Q_SLOW_VAL); + if (!o) { + /* +
Re: [Xen-devel] preparing for 4.5.1
On Thu, 2015-03-26 at 17:07 +, Jan Beulich wrote: having been released mid January, it is time to get ready for 4.5.1-rc1. Please reply with backport requests which you consider necessary but still missing. Please also be patient with them being pulled in - I'll be gone the coming two weeks, and I'd hope to get an RC1 put together soon thereafter. On my list for ARM I have this patch, which also touches x86. What do you think? commit c2453291b177cc2e89dc104bd4233c3d8bb73922 Author: Vijaya Kumar K vijaya.ku...@caviumnetworks.com Date: Tue Mar 24 17:14:47 2015 +0530 xen: Add populate_pt_range interface to reserve non-leaf level table entries On x86, for the pages mapped with PAGE_HYPERVISOR attribute non-leaf page tables are allocated with valid pte entries. and with MAP_SMALL_PAGES attribute only non-leaf page tables are allocated with invalid (valid bit set to 0) pte entries. However on arm this is not the case. On arm for the pages mapped with PAGE_HYPERVISOR and MAP_SMALL_PAGES both non-leaf and leaf level page table are allocated with valid bit in pte entries. This behaviour in arm makes common vmap code fail to allocate memory beyond 128MB as described below. In vm_init, map_pages_to_xen() is called for mapping vm_bitmap. Initially one page of vm_bitmap is allocated and mapped using PAGE_HYPERVISOR attribute. For the rest of vm_bitmap pages, MAP_SMALL_PAGES attribute is used to map. In ARM for both PAGE_HYPERVISOR and MAP_SMALL_PAGES, valid bit is set to 1 in pte entry for these mapping. In vm_alloc(), map_pages_to_xen() is failing for 128MB because for this next vm_bitmap page the mapping is already set in vm_init() with valid bit set in pte entry. So map_pages_to_xen() in ARM returns error. With this patch, MAP_SMALL_PAGES is dropped and arch specific populate_pt_range() api is introduced to populate non-leaf page table entries for the requested pages. Added RESERVE option to map non-leaf page table entries. Signed-off-by: Vijaya Kumar Kvijaya.ku...@caviumnetworks.com Acked-by: Jan Beulich jbeul...@suse.com Acked-by: Ian Campbell ian.campb...@citrix.com [ ijc -- rewrote subject line ] diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 7d4ba0c..d103f8f 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -827,7 +827,8 @@ static int create_xen_table(lpae_t *entry) enum xenmap_operation { INSERT, -REMOVE +REMOVE, +RESERVE }; static int create_xen_entries(enum xenmap_operation op, @@ -859,12 +860,15 @@ static int create_xen_entries(enum xenmap_operation op, switch ( op ) { case INSERT: +case RESERVE: if ( third[third_table_offset(addr)].pt.valid ) { printk(create_xen_entries: trying to replace an existing mapping addr=%lx mfn=%lx\n, addr, mfn); return -EINVAL; } +if ( op == RESERVE ) +break; pte = mfn_to_xen_entry(mfn, ai); pte.pt.table = 1; write_pte(third[third_table_offset(addr)], pte); @@ -898,6 +902,13 @@ int map_pages_to_xen(unsigned long virt, { return create_xen_entries(INSERT, virt, mfn, nr_mfns, flags); } + +int populate_pt_range(unsigned long virt, unsigned long mfn, + unsigned long nr_mfns) +{ +return create_xen_entries(RESERVE, virt, mfn, nr_mfns, 0); +} + void destroy_xen_mappings(unsigned long v, unsigned long e) { create_xen_entries(REMOVE, v, 0, (e - v) PAGE_SHIFT, 0); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 8e29675..786ed47 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5725,6 +5725,12 @@ int map_pages_to_xen( return 0; } +int populate_pt_range(unsigned long virt, unsigned long mfn, + unsigned long nr_mfns) +{ +return map_pages_to_xen(virt, mfn, nr_mfns, MAP_SMALL_PAGES); +} + void destroy_xen_mappings(unsigned long s, unsigned long e) { bool_t locking = system_state SYS_STATE_boot; diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 783cea3..739d468 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -40,7 +40,7 @@ void __init vm_init(void) bitmap_fill(vm_bitmap, vm_low); /* Populate page tables for the bitmap if necessary. */ -map_pages_to_xen(va, 0, vm_low - nr, MAP_SMALL_PAGES); +populate_pt_range(va, 0, vm_low - nr); } void *vm_alloc(unsigned int nr, unsigned int align) diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 3e7b0ae..8de5e26 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -64,7 +64,6 @@ #define PAGE_HYPERVISOR (WRITEALLOC) #define PAGE_HYPERVISOR_NOCACHE (DEV_SHARED) #define PAGE_HYPERVISOR_WC (DEV_WC) -#define
Re: [Xen-devel] [PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking
Ian Campbell writes ([PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking): Signed-off-by: Ian Campbell ian.campb...@citrix.com --- production-config-cambridge | 1 + This is (obviously) fine. I think that changes to this file from the Cambridge site operators (ie, each of us two ) should be committed without review. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings
Ian Campbell writes ([PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings): In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like export OSSTEST_CONFIG=production-config-cambridge to tailor things for a particular instance of osstest running in production mode. Acked-by: Ian Jackson ian.jack...@eu.citrix.com ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [OSSTEST] ms-planner: add a flight summary html report
Resurrecting this old thing... On Wed, 2014-04-30 at 16:44 +0100, Ian Jackson wrote: I think this is a nice thing to have, but: +while (my ($reso,$evts) = each %{ $plan-{Events} }) { +# [osstest real] job 26010.test-amd64-amd64-xl-win7-amd64 +foreach my $evt ( @{$evts} ) { +next unless $evt-{Type} =~ m/^(Start|End)$/; +next unless $evt-{Info} =~ m/^\[(\S+) (\S+)\] job ([0-9]+)\ \.(\S+) (.*)/; This is rather a layering violation. Info is for the consumption of humans. Perhaps the plan should have an explicit flight.job field. I think in principal that's as simple as the two patches below which add it to the booking in ts-hosts-allocate-Executive and propagate it to the plan in ms-planner, but I'm not really sure and I don't know how to test the queuedaemon/planner stuff. However is even $flight.$job not a layering violation? Or were you thinking that perhaps generically it would be thing.sub-thing.sub-sub-thing? Still, the code to generate the summary would need to know about the concept of flights etc and hence probably doesn't belong in ms-planner. You could compute the [linux-3.4 real] part by looking at the common prefix of all the Infos. Unfortunately the common prefix of the Info is generally at least [linux-3.4 real] job and depending on the set of jobs currently running it might even be [linux-3.4 real] job build-, [linux-3.4 real] job test- or even [linux-3.4 real] job test-amd64-amd64. With all that in mind I've been thinking of coming at this from the other end and starting with the set of flights which are not finished and enumerating their jobs as pass/fail/inprogress/planned with the appropriate times listed. Only problem is I can't seem to identify in the DB which flights are not completed, neither the flights table nor the jobs one seems to consistently reflect that status. Is this possible to find out without ps? Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking
On Thu, 2015-04-02 at 11:51 +0100, Ian Jackson wrote: Ian Campbell writes ([PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking): Signed-off-by: Ian Campbell ian.campb...@citrix.com --- production-config-cambridge | 1 + This is (obviously) fine. I think that changes to this file from the Cambridge site operators (ie, each of us two ) should be committed without review. I think you mean to the main osstest#pretest (as opposed to the Cambridge local one). Either way that WFM. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host
Ian Campbell writes ([PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host): Signed-off-by: Ian Campbell ian.campb...@citrix.com We're not using this to fetch URLs on the public internet, are we ? If we are then we need to honour $c{HttpProxy}. Perhaps you could add a comment to the effect that target_fetch_url should be used for URLs within the test environment, only. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype
Ian Campbell writes ([PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype): Signed-off-by: Ian Campbell ian.campb...@citrix.com Acked-by: Ian Jackson ian.jack...@eu.citrix.com ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.
Ian Campbell writes ([PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.): In particular make the path unique by ensuring it includes the host and guest name in the guest case. Acked-by: Ian Jackson ian.jack...@eu.citrix.com But I have a minor style quibble: +# $ho-{Host} is set if $ho is a guest. +$wf_rhs= $ho-{Host}-{Name}._${wf_rhs} if $ho-{Host}; It would be usual to write +$wf_rhs= $ho-{Host}{Name}._${wf_rhs} if $ho-{Host}; since only the first - is required. Thanks, Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight
Ian Campbell writes ([Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight): The failures mainly seem to be some sort of proxy issue returning old data (similar to what used to happen to mg-debian-installer-update), Maybe we can set up an explicit proxy which actually works. I guess that might need negotiation with Citrix corporate IT. Luckily the new colo is not affected by anything like this. On a small number I see The installer failed to download a file from the mirror. messages, which might be the cache/proxy or might be an actual Debian issue. Hrm. I find these vague error messages exceptionally frustrating (debootstrap has this problem too). The -raw tests all failed due to a timeout doing: dd if=/dev/zero of=/var/lib/xen/images/debian/disk.raw bs=1M count=1 I'm unclear if it timed out after 30 (as the code seems to say) or 60 (as the logs seem to say) but either way something obviously needs adjusting. I just tried it on my local test box and it took 1m20s and reported 130 MB/s. I've appended an as yet untested patch which assumed 100 MB/s and calculates a timeout from that. Do you intend to fold that patch in ? I could switch to using count=1 seek=1 to create a sparse file, which should be pretty much instantaneous, but allocating the whole backing file seemed like a good idea. Right. Thanks, Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances
On Thu, 2015-04-02 at 12:13 +0100, Ian Jackson wrote: Ian Campbell writes ([PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances): We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master. ... ;; osstest) - if [ x$OSSTEST_USE_HEAD != xy ] ; then + if [ x$TREEBRANCH_OSSTEST_UPSTREAM != x ] ; then Shouldn't the OSSTEST_USE_HEAD option override this ? Ie I think this is the wrong way round. Yes, I think you are right. I think I just ended up with this because I was testing via the standalone wrapper which sets USE_HEAD, but I'll test another way. + OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \ + $TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF` OSSTEST_REVISION_MERGE doesn't seem to be an environment variable. I think it should either be honoured if pre-set, or just be an ordinary (lowercase) variable name (in which case it doesn't need the OSSTEST prefix). OK. Any preference as to which? As it stands the only reason for the variable is to sink the output from repo_tree_rev_fetch and log it. + echo 2 $TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE + + rm -rf $repos/osstest-merge 2 + git clone -b pretest $HOME/testing.git $repos/osstest-merge 2 + + git -C $repos/osstest-merge \ + fetch $repos/osstest $LOCALREV_OVMF:ap-merge 2 LOCALREV_OVMF (here and earlier). It's not clear to me that you actually need to indirect it; you could just specify a branch name. OK. The rest looks fine. Having thought about how to override things, particularly: + if [ x$TREEBRANCH_OSSTEST_UPSTREAM != x ] ; then + # could push to instance specific location, but + # certainly not to master instance's xenbits repo! + : + else I think ap-fetch-* and ap-push should have a general purpose hook which occurs just after ap-common is loaded, something like this: diff --git a/ap-fetch-version b/ap-fetch-version index 33aaf00..a3028f6 100755 --- a/ap-fetch-version +++ b/ap-fetch-version @@ -31,6 +31,8 @@ if info_linux_tree $branch; then exit 0 fi +. ${OSSTEST_HOOK_AP_FETCH-/dev/null} + case $branch in xen-3.*) ./sg-hg-heads sh -ec ' We can postpone actually writing that until someone needs it. Gladly ;-) Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
On Thu, 2015-04-02 at 12:16 +0100, Ian Jackson wrote: Ian Campbell writes ([PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark): http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere. Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config. Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- cr-ensure-disk-space| 2 ++ cr-publish-flight-logs | 4 ++-- production-config-cambridge | 6 +- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space index 0ee84c7..84b1890 100755 --- a/cr-ensure-disk-space +++ b/cr-ensure-disk-space @@ -40,6 +40,8 @@ die unless @ARGV==1; our ($cfgbase) = @ARGV; +exit 0 unless $cfgbase; csreadconfig(); Don't you mean unless $c{$cfgbase} ? I think I do, yes. And then it has to come after csreadconfig. (Did you test this ?) I was running in standalone mode, so I don't think this code will have actually been run, I was relying on my ability to reason about the code, which apparently failed :-/. Shall I run an adhoc flight with this change in it? Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file
Ian Campbell writes ([PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file): Call it from ts-debian-hvm-install. Please expand the commit message. Something like: This means that, in future, ts-debian-hvm-install can use preseed_hook_command and preseed_hook_installscript. No functional change for now. Thanks, Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories
Ian Campbell writes ([PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories): Causes known_hosts to be consistently created as well as ~osstest/.ssh to be consistently populated (it previsouly wasn't for HVM guests). Most of this is code motion, AFAICT. The difference is just that preseed_ssh is now called from ts-debian-hvm-install, too ? Why should preseed_ssh not be used by preseed_base ? Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC v2 1/3] xen/pvh: enable mmu_update hypercall
On 02/04/15 11:42, Ian Campbell wrote: On Thu, 2015-04-02 at 12:26 +0200, Roger Pau Monne wrote: This is needed for performing save/restore of PV guests. It's quite a big interface though, isn't it? Could we restrict it to a subset of the operations perhaps? Or at least justify here how it has been audited and found to be safe to allow an HVM guest this access. It isn't actually very big, but does have quite a lot of PV knowledge built in. I would be happer with the safety of this patch if v-arch.old_guest_table got moved into the pv union, to make the code much clearer that it is specifically for PV guests. If I recall, this change only needed for MMU_MACHPHYS_UPDATE against a foreign domain. Each of the 3 subops does check for paging_mode_translate/refcounts() of the target, which does prevent the hypercall being made against a non-PV domains. From that point of view, it should be safe for HVM guests to use, as it is the target domain, rather than the source domain, which is important. However, with migration v2 dropping support for 2-level PV guests (which died with the 32bit hypervisor build), I believe I have removed all need for MMU_MACHPHYS_UPDATE hypercalls entirely (unless there are some done behind the scenes from the kernel). ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC v2 1/3] xen/pvh: enable mmu_update hypercall
On 04/02/2015 01:50 PM, Andrew Cooper wrote: On 02/04/15 11:42, Ian Campbell wrote: On Thu, 2015-04-02 at 12:26 +0200, Roger Pau Monne wrote: This is needed for performing save/restore of PV guests. It's quite a big interface though, isn't it? Could we restrict it to a subset of the operations perhaps? Or at least justify here how it has been audited and found to be safe to allow an HVM guest this access. It isn't actually very big, but does have quite a lot of PV knowledge built in. I would be happer with the safety of this patch if v-arch.old_guest_table got moved into the pv union, to make the code much clearer that it is specifically for PV guests. If I recall, this change only needed for MMU_MACHPHYS_UPDATE against a foreign domain. Each of the 3 subops does check for paging_mode_translate/refcounts() of the target, which does prevent the hypercall being made against a non-PV domains. From that point of view, it should be safe for HVM guests to use, as it is the target domain, rather than the source domain, which is important. However, with migration v2 dropping support for 2-level PV guests (which died with the 32bit hypervisor build), I believe I have removed all need for MMU_MACHPHYS_UPDATE hypercalls entirely (unless there are some done behind the scenes from the kernel). There is one usage in the kernel in file arch/x86/xen/setup.c targeting DOMID_SELF. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH OSSTEST] cambridge: Set HostProp_DhcpWatchMethod back to woking
Signed-off-by: Ian Campbell ian.campb...@citrix.com --- production-config-cambridge | 1 + 1 file changed, 1 insertion(+) diff --git a/production-config-cambridge b/production-config-cambridge index b3a7da5..a144cef 100644 --- a/production-config-cambridge +++ b/production-config-cambridge @@ -73,6 +73,7 @@ XenUseUser osstest DebianMirrorHost 10.80.16.196 HostProp_NtpServer ntp.uk.xensource.com +HostProp_DhcpWatchMethod leases dhcp3 woking.cam.xci-test.com:5556 DebianPreseed= 'END' END -- 2.1.4 ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
Ian Campbell writes ([PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark): http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere. Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config. Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config. Signed-off-by: Ian Campbell ian.campb...@citrix.com --- cr-ensure-disk-space| 2 ++ cr-publish-flight-logs | 4 ++-- production-config-cambridge | 6 +- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space index 0ee84c7..84b1890 100755 --- a/cr-ensure-disk-space +++ b/cr-ensure-disk-space @@ -40,6 +40,8 @@ die unless @ARGV==1; our ($cfgbase) = @ARGV; +exit 0 unless $cfgbase; csreadconfig(); Don't you mean unless $c{$cfgbase} ? And then it has to come after csreadconfig. (Did you test this ?) The rest LGTM. Thanks, Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC v2 1/3] xen/pvh: enable mmu_update hypercall
El 02/04/15 a les 12.42, Ian Campbell ha escrit: On Thu, 2015-04-02 at 12:26 +0200, Roger Pau Monne wrote: This is needed for performing save/restore of PV guests. It's quite a big interface though, isn't it? AFAICT it contains MMU_NORMAL_PT_UPDATE, MMU_PT_UPDATE_PRESERVE_AD and MMU_MACHPHYS_UPDATE. Could we restrict it to a subset of the operations perhaps? Or at least justify here how it has been audited and found to be safe to allow an HVM guest this access. XSA-109 should have fixed all issues with this operations. IIRC only MMU_MACHPHYS_UPDATE is needed for save/restore of PV guests, but I will have to check. If that's the case, I could restrict PVH domains to only have access to that operation. Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel