[COMMIT master] ksm support

2009-10-05 Thread Avi Kivity
From: Izik Eidus iei...@redhat.com

Call madvise(MADV_MERGEABLE) on the memory allocations to allow the
kernel to merge them.

Signed-off-by: Izik Eidus iei...@redhat.com
Signed-off-by: Avi Kivity a...@redhat.com

diff --git a/exec.c b/exec.c
index 5c9edf7..406d2cb 100644
--- a/exec.c
+++ b/exec.c
@@ -2538,6 +2538,9 @@ ram_addr_t qemu_ram_alloc(ram_addr_t size)
 new_block-host = file_ram_alloc(size, mem_path);
 if (!new_block-host) {
 new_block-host = qemu_vmalloc(size);
+#ifdef MADV_MERGEABLE
+madvise(new_block-host, size, MADV_MERGEABLE);
+#endif
 }
 new_block-offset = last_ram_offset;
 new_block-length = size;
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] Merge branch 'upstream-merge'

2009-10-05 Thread Avi Kivity
From: Marcelo Tosatti mtosa...@redhat.com

* upstream-merge: (55 commits)
  target-ppc: log instructions start in TCG code
  target-mips: log instructions start in TCG code
  Win32: avoid a warning
  BSD user: suppress a warning
  BSD user: implement GUEST_BASE
  tcg/i386: generates dec/inc instead of sub/add when possible
  tcg/i386: optimize and $0xff(ff), reg
  Compile some user files only once for all targets
  tcg/x86_64: generated dec/inc instead of sub/add when possible
  Don't compile roms if not building system targets
  Fix user emulator breakage
  tcg/ppc: always use tcg_out_call
  vl: Add failure check for SetEvent
  vl: Do not use perror after failed Win32 API calls
  posix-aio-compat: avoid signal race when spawning a thread
  target-i386: kill a tmp register
  target-i386: use subfi instead of sub with a non-freed constant
  eepro100: Fix format strings in debug messages
  eepro100: Replace sprintf by snprintf
  eepro100: Remove unused code
  ...

Signed-off-by: Marcelo Tosatti mtosa...@redhat.com
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: fix LAPIC timer period overflow

2009-10-05 Thread Avi Kivity
From: Aurelien Jarno aurel...@aurel32.net

Don't overflow when computing the 64-bit period from 32-bit registers.

Fixes sourceforge bug #2826486.

Signed-off-by: Aurelien Jarno aurel...@aurel32.net
Cc: sta...@kernel.org
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 9c8f901..3ca7767 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -658,7 +658,7 @@ static void start_apic_timer(struct kvm_lapic *apic)
 {
ktime_t now = apic-lapic_timer.timer.base-get_time();
 
-   apic-lapic_timer.period = apic_get_reg(apic, APIC_TMICT) *
+   apic-lapic_timer.period = (u64)apic_get_reg(apic, APIC_TMICT) *
APIC_BUS_CYCLE_NS * apic-divide_count;
atomic_set(apic-lapic_timer.pending, 0);
 
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: add support for change_pte mmu notifiers

2009-10-05 Thread Avi Kivity
From: Izik Eidus iei...@redhat.com

this is needed for kvm if it want ksm to directly map pages into its
shadow page tables.

[marcelo: cast pfn assignment to u64]

Signed-off-by: Izik Eidus iei...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 299cc1b..295c7c4 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -794,6 +794,7 @@ asmlinkage void kvm_handle_fault_on_reboot(void);
 #define KVM_ARCH_WANT_MMU_NOTIFIER
 int kvm_unmap_hva(struct kvm *kvm, unsigned long hva);
 int kvm_age_hva(struct kvm *kvm, unsigned long hva);
+void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte);
 int cpuid_maxphyaddr(struct kvm_vcpu *vcpu);
 int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu);
 int kvm_arch_interrupt_allowed(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index a7151b8..20a2cd1 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -748,7 +748,7 @@ static int rmap_write_protect(struct kvm *kvm, u64 gfn)
return write_protected;
 }
 
-static int kvm_unmap_rmapp(struct kvm *kvm, unsigned long *rmapp)
+static int kvm_unmap_rmapp(struct kvm *kvm, unsigned long *rmapp, u64 data)
 {
u64 *spte;
int need_tlb_flush = 0;
@@ -763,8 +763,45 @@ static int kvm_unmap_rmapp(struct kvm *kvm, unsigned long 
*rmapp)
return need_tlb_flush;
 }
 
-static int kvm_handle_hva(struct kvm *kvm, unsigned long hva,
- int (*handler)(struct kvm *kvm, unsigned long *rmapp))
+static int kvm_set_pte_rmapp(struct kvm *kvm, unsigned long *rmapp, u64 data)
+{
+   int need_flush = 0;
+   u64 *spte, new_spte;
+   pte_t *ptep = (pte_t *)data;
+   pfn_t new_pfn;
+
+   WARN_ON(pte_huge(*ptep));
+   new_pfn = pte_pfn(*ptep);
+   spte = rmap_next(kvm, rmapp, NULL);
+   while (spte) {
+   BUG_ON(!is_shadow_present_pte(*spte));
+   rmap_printk(kvm_set_pte_rmapp: spte %p %llx\n, spte, *spte);
+   need_flush = 1;
+   if (pte_write(*ptep)) {
+   rmap_remove(kvm, spte);
+   __set_spte(spte, shadow_trap_nonpresent_pte);
+   spte = rmap_next(kvm, rmapp, NULL);
+   } else {
+   new_spte = *spte ~ (PT64_BASE_ADDR_MASK);
+   new_spte |= (u64)new_pfn  PAGE_SHIFT;
+
+   new_spte = ~PT_WRITABLE_MASK;
+   new_spte = ~SPTE_HOST_WRITEABLE;
+   if (is_writeble_pte(*spte))
+   kvm_set_pfn_dirty(spte_to_pfn(*spte));
+   __set_spte(spte, new_spte);
+   spte = rmap_next(kvm, rmapp, spte);
+   }
+   }
+   if (need_flush)
+   kvm_flush_remote_tlbs(kvm);
+
+   return 0;
+}
+
+static int kvm_handle_hva(struct kvm *kvm, unsigned long hva, u64 data,
+ int (*handler)(struct kvm *kvm, unsigned long *rmapp,
+u64 data))
 {
int i, j;
int retval = 0;
@@ -786,13 +823,15 @@ static int kvm_handle_hva(struct kvm *kvm, unsigned long 
hva,
if (hva = start  hva  end) {
gfn_t gfn_offset = (hva - start)  PAGE_SHIFT;
 
-   retval |= handler(kvm, memslot-rmap[gfn_offset]);
+   retval |= handler(kvm, memslot-rmap[gfn_offset],
+ data);
 
for (j = 0; j  KVM_NR_PAGE_SIZES - 1; ++j) {
int idx = gfn_offset;
idx /= KVM_PAGES_PER_HPAGE(PT_DIRECTORY_LEVEL + 
j);
retval |= handler(kvm,
-   memslot-lpage_info[j][idx].rmap_pde);
+   memslot-lpage_info[j][idx].rmap_pde,
+   data);
}
}
}
@@ -802,10 +841,15 @@ static int kvm_handle_hva(struct kvm *kvm, unsigned long 
hva,
 
 int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
 {
-   return kvm_handle_hva(kvm, hva, kvm_unmap_rmapp);
+   return kvm_handle_hva(kvm, hva, 0, kvm_unmap_rmapp);
+}
+
+void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte)
+{
+   kvm_handle_hva(kvm, hva, (u64)pte, kvm_set_pte_rmapp);
 }
 
-static int kvm_age_rmapp(struct kvm *kvm, unsigned long *rmapp)
+static int kvm_age_rmapp(struct kvm *kvm, unsigned long *rmapp, u64 data)
 {
u64 *spte;
int young = 0;
@@ -841,13 +885,13 @@ static void rmap_recycle(struct kvm_vcpu *vcpu, u64 
*spte, gfn_t gfn)
gfn = unalias_gfn(vcpu-kvm, gfn);
rmapp = gfn_to_rmap(vcpu-kvm, gfn, sp-role.level);
 
-   kvm_unmap_rmapp(vcpu-kvm, rmapp);
+   kvm_unmap_rmapp(vcpu-kvm, rmapp, 0);

[COMMIT master] KVM: MMU: add SPTE_HOST_WRITEABLE flag to the shadow ptes

2009-10-05 Thread Avi Kivity
From: Izik Eidus iei...@redhat.com

this flag notify that the host physical page we are pointing to from
the spte is write protected, and therefore we cant change its access
to be write unless we run get_user_pages(write = 1).

(this is needed for change_pte support in kvm)

Signed-off-by: Izik Eidus iei...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 62d2f86..a7151b8 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -156,6 +156,8 @@ module_param(oos_shadow, bool, 0644);
 #define CREATE_TRACE_POINTS
 #include mmutrace.h
 
+#define SPTE_HOST_WRITEABLE (1ULL  PT_FIRST_AVAIL_BITS_SHIFT)
+
 #define SHADOW_PT_INDEX(addr, level) PT64_INDEX(addr, level)
 
 struct kvm_rmap_desc {
@@ -1754,7 +1756,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
unsigned pte_access, int user_fault,
int write_fault, int dirty, int level,
gfn_t gfn, pfn_t pfn, bool speculative,
-   bool can_unsync)
+   bool can_unsync, bool reset_host_protection)
 {
u64 spte;
int ret = 0;
@@ -1781,6 +1783,9 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
spte |= kvm_x86_ops-get_mt_mask(vcpu, gfn,
kvm_is_mmio_pfn(pfn));
 
+   if (reset_host_protection)
+   spte |= SPTE_HOST_WRITEABLE;
+
spte |= (u64)pfn  PAGE_SHIFT;
 
if ((pte_access  ACC_WRITE_MASK)
@@ -1826,7 +1831,8 @@ static void mmu_set_spte(struct kvm_vcpu *vcpu, u64 
*sptep,
 unsigned pt_access, unsigned pte_access,
 int user_fault, int write_fault, int dirty,
 int *ptwrite, int level, gfn_t gfn,
-pfn_t pfn, bool speculative)
+pfn_t pfn, bool speculative,
+bool reset_host_protection)
 {
int was_rmapped = 0;
int was_writeble = is_writeble_pte(*sptep);
@@ -1858,7 +1864,8 @@ static void mmu_set_spte(struct kvm_vcpu *vcpu, u64 
*sptep,
}
 
if (set_spte(vcpu, sptep, pte_access, user_fault, write_fault,
- dirty, level, gfn, pfn, speculative, true)) {
+ dirty, level, gfn, pfn, speculative, true,
+ reset_host_protection)) {
if (write_fault)
*ptwrite = 1;
kvm_x86_ops-tlb_flush(vcpu);
@@ -1906,7 +1913,7 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t v, 
int write,
if (iterator.level == level) {
mmu_set_spte(vcpu, iterator.sptep, ACC_ALL, ACC_ALL,
 0, write, 1, pt_write,
-level, gfn, pfn, false);
+level, gfn, pfn, false, true);
++vcpu-stat.pf_fixed;
break;
}
diff --git a/arch/x86/kvm/paging_tmpl.h b/arch/x86/kvm/paging_tmpl.h
index d2fec9c..72558f8 100644
--- a/arch/x86/kvm/paging_tmpl.h
+++ b/arch/x86/kvm/paging_tmpl.h
@@ -273,9 +273,13 @@ static void FNAME(update_pte)(struct kvm_vcpu *vcpu, 
struct kvm_mmu_page *page,
if (mmu_notifier_retry(vcpu, vcpu-arch.update_pte.mmu_seq))
return;
kvm_get_pfn(pfn);
+   /*
+* we call mmu_set_spte() with reset_host_protection = true beacuse that
+* vcpu-arch.update_pte.pfn was fetched from get_user_pages(write = 1).
+*/
mmu_set_spte(vcpu, spte, page-role.access, pte_access, 0, 0,
 gpte  PT_DIRTY_MASK, NULL, PT_PAGE_TABLE_LEVEL,
-gpte_to_gfn(gpte), pfn, true);
+gpte_to_gfn(gpte), pfn, true, true);
 }
 
 /*
@@ -308,7 +312,7 @@ static u64 *FNAME(fetch)(struct kvm_vcpu *vcpu, gva_t addr,
 user_fault, write_fault,
 gw-ptes[gw-level-1]  PT_DIRTY_MASK,
 ptwrite, level,
-gw-gfn, pfn, false);
+gw-gfn, pfn, false, true);
break;
}
 
@@ -558,6 +562,7 @@ static void FNAME(prefetch_page)(struct kvm_vcpu *vcpu,
 static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)
 {
int i, offset, nr_present;
+   bool reset_host_protection;
 
offset = nr_present = 0;
 
@@ -595,9 +600,16 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct 
kvm_mmu_page *sp)
 
nr_present++;
pte_access = sp-role.access  FNAME(gpte_access)(vcpu, gpte);
+   if (!(sp-spt[i]  SPTE_HOST_WRITEABLE)) {
+   pte_access = ~ACC_WRITE_MASK;
+   reset_host_protection = 0;
+   } else {
+   reset_host_protection = 1;
+   }
  

[COMMIT master] KVM: Separate timer intialization into an indepedent function

2009-10-05 Thread Avi Kivity
From: Zachary Amsden zams...@redhat.com

Signed-off-by: Zachary Amsden zams...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index fedac9d..15d2ace 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3116,9 +3116,22 @@ static struct notifier_block 
kvmclock_cpufreq_notifier_block = {
 .notifier_call  = kvmclock_cpufreq_notifier
 };
 
+static void kvm_timer_init(void)
+{
+   int cpu;
+
+   for_each_possible_cpu(cpu)
+   per_cpu(cpu_tsc_khz, cpu) = tsc_khz;
+   if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) {
+   tsc_khz_ref = tsc_khz;
+   cpufreq_register_notifier(kvmclock_cpufreq_notifier_block,
+ CPUFREQ_TRANSITION_NOTIFIER);
+   }
+}
+
 int kvm_arch_init(void *opaque)
 {
-   int r, cpu;
+   int r;
struct kvm_x86_ops *ops = (struct kvm_x86_ops *)opaque;
 
if (kvm_x86_ops) {
@@ -3150,13 +3163,7 @@ int kvm_arch_init(void *opaque)
kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
PT_DIRTY_MASK, PT64_NX_MASK, 0);
 
-   for_each_possible_cpu(cpu)
-   per_cpu(cpu_tsc_khz, cpu) = tsc_khz;
-   if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) {
-   tsc_khz_ref = tsc_khz;
-   cpufreq_register_notifier(kvmclock_cpufreq_notifier_block,
- CPUFREQ_TRANSITION_NOTIFIER);
-   }
+   kvm_timer_init();
 
return 0;
 
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: Kill the confusing tsc_ref_khz and ref_freq variables

2009-10-05 Thread Avi Kivity
From: Zachary Amsden zams...@redhat.com

They are globals, not clearly protected by any ordering or locking, and
vulnerable to various startup races.

Instead, for variable TSC machines, register the cpufreq notifier and get
the TSC frequency directly from the cpufreq machinery.  Not only is it
always right, it is also perfectly accurate, as no error prone measurement
is required.

On such machines, when a new CPU online is brought online, it isn't clear what
frequency it will start with, and it may not correspond to the reference, thus
in hardware_enable we clear the cpu_tsc_khz variable to zero and make sure
it is set before running on a VCPU.

Signed-off-by: Zachary Amsden zams...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 15d2ace..de4ce8f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1326,6 +1326,8 @@ out:
 void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 {
kvm_x86_ops-vcpu_load(vcpu, cpu);
+   if (unlikely(per_cpu(cpu_tsc_khz, cpu) == 0))
+   per_cpu(cpu_tsc_khz, cpu) = cpufreq_quick_get(cpu);
kvm_request_guest_time_update(vcpu);
 }
 
@@ -3061,9 +3063,6 @@ static void bounce_off(void *info)
/* nothing */
 }
 
-static unsigned int  ref_freq;
-static unsigned long tsc_khz_ref;
-
 static int kvmclock_cpufreq_notifier(struct notifier_block *nb, unsigned long 
val,
 void *data)
 {
@@ -3072,14 +3071,11 @@ static int kvmclock_cpufreq_notifier(struct 
notifier_block *nb, unsigned long va
struct kvm_vcpu *vcpu;
int i, send_ipi = 0;
 
-   if (!ref_freq)
-   ref_freq = freq-old;
-
if (val == CPUFREQ_PRECHANGE  freq-old  freq-new)
return 0;
if (val == CPUFREQ_POSTCHANGE  freq-old  freq-new)
return 0;
-   per_cpu(cpu_tsc_khz, freq-cpu) = cpufreq_scale(tsc_khz_ref, ref_freq, 
freq-new);
+   per_cpu(cpu_tsc_khz, freq-cpu) = freq-new;
 
spin_lock(kvm_lock);
list_for_each_entry(kvm, vm_list, vm_list) {
@@ -3120,12 +3116,14 @@ static void kvm_timer_init(void)
 {
int cpu;
 
-   for_each_possible_cpu(cpu)
-   per_cpu(cpu_tsc_khz, cpu) = tsc_khz;
if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) {
-   tsc_khz_ref = tsc_khz;
cpufreq_register_notifier(kvmclock_cpufreq_notifier_block,
  CPUFREQ_TRANSITION_NOTIFIER);
+   for_each_online_cpu(cpu)
+   per_cpu(cpu_tsc_khz, cpu) = cpufreq_get(cpu);
+   } else {
+   for_each_possible_cpu(cpu)
+   per_cpu(cpu_tsc_khz, cpu) = tsc_khz;
}
 }
 
@@ -4698,6 +4696,14 @@ int kvm_arch_vcpu_reset(struct kvm_vcpu *vcpu)
 
 int kvm_arch_hardware_enable(void *garbage)
 {
+   /*
+* Since this may be called from a hotplug notifcation,
+* we can't get the CPU frequency directly.
+*/
+   if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) {
+   int cpu = raw_smp_processor_id();
+   per_cpu(cpu_tsc_khz, cpu) = 0;
+   }
return kvm_x86_ops-hardware_enable(garbage);
 }
 
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: Fix hotplug of CPUs

2009-10-05 Thread Avi Kivity
From: Zachary Amsden zams...@redhat.com

Both VMX and SVM require per-cpu memory allocation, which is done at module
init time, for only online cpus.

Backend was not allocating enough structure for all possible CPUs, so
new CPUs coming online could not be hardware enabled.

Signed-off-by: Zachary Amsden zams...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index d1036ce..02a4269 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -482,7 +482,7 @@ static __init int svm_hardware_setup(void)
kvm_enable_efer_bits(EFER_SVME);
}
 
-   for_each_online_cpu(cpu) {
+   for_each_possible_cpu(cpu) {
r = svm_cpu_init(cpu);
if (r)
goto err;
@@ -516,7 +516,7 @@ static __exit void svm_hardware_unsetup(void)
 {
int cpu;
 
-   for_each_online_cpu(cpu)
+   for_each_possible_cpu(cpu)
svm_cpu_uninit(cpu);
 
__free_pages(pfn_to_page(iopm_base  PAGE_SHIFT), IOPM_ALLOC_ORDER);
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 3fe0d42..e86f1a6 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -1350,15 +1350,17 @@ static void free_kvm_area(void)
 {
int cpu;
 
-   for_each_online_cpu(cpu)
+   for_each_possible_cpu(cpu) {
free_vmcs(per_cpu(vmxarea, cpu));
+   per_cpu(vmxarea, cpu) = NULL;
+   }
 }
 
 static __init int alloc_kvm_area(void)
 {
int cpu;
 
-   for_each_online_cpu(cpu) {
+   for_each_possible_cpu(cpu) {
struct vmcs *vmcs;
 
vmcs = alloc_vmcs_cpu(cpu);
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6

2009-10-05 Thread Avi Kivity
From: Avi Kivity a...@redhat.com

Signed-off-by: Avi Kivity a...@redhat.com
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: VMX: flush TLB with INVEPT on cpu migration

2009-10-05 Thread Avi Kivity
From: Marcelo Tosatti mtosa...@redhat.com

It is possible that stale EPTP-tagged mappings are used, if a
vcpu migrates to a different pcpu.

Set KVM_REQ_TLB_FLUSH in vmx_vcpu_load, when switching pcpus, which
will invalidate both VPID and EPT mappings on the next vm-entry.

Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index e86f1a6..97f4265 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -708,7 +708,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
if (vcpu-cpu != cpu) {
vcpu_clear(vmx);
kvm_migrate_timers(vcpu);
-   vpid_sync_vcpu_all(vmx);
+   set_bit(KVM_REQ_TLB_FLUSH, vcpu-requests);
local_irq_disable();
list_add(vmx-local_vcpus_link,
 per_cpu(vcpus_on_cpu, cpu));
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: Fix printk name error in svm.c

2009-10-05 Thread Avi Kivity
From: Zachary Amsden zams...@redhat.com

Signed-off-by: Zachary Amsden zams...@redhat.com
Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 9a4daca..d1036ce 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -330,13 +330,14 @@ static int svm_hardware_enable(void *garbage)
return -EBUSY;
 
if (!has_svm()) {
-   printk(KERN_ERR svm_cpu_init: err EOPNOTSUPP on %d\n, me);
+   printk(KERN_ERR svm_hardware_enable: err EOPNOTSUPP on %d\n,
+  me);
return -EINVAL;
}
svm_data = per_cpu(svm_data, me);
 
if (!svm_data) {
-   printk(KERN_ERR svm_cpu_init: svm_data is NULL on %d\n,
+   printk(KERN_ERR svm_hardware_enable: svm_data is NULL on %d\n,
   me);
return -EINVAL;
}
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: Prevent overflow in KVM_GET_SUPPORTED_CPUID

2009-10-05 Thread Avi Kivity
From: Avi Kivity a...@redhat.com

The number of entries is multiplied by the entry size, which can
overflow on 32-bit hosts.  Bound the entry count instead.

Reported-by: David Wagner d...@cs.berkeley.edu
Signed-off-by: Avi Kivity a...@redhat.com

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index ffccb5c..b2f4f13 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1593,6 +1593,8 @@ static int kvm_dev_ioctl_get_supported_cpuid(struct 
kvm_cpuid2 *cpuid,
 
if (cpuid-nent  1)
goto out;
+   if (cpuid-nent  KVM_MAX_CPUID_ENTRIES)
+   cpuid-nent = KVM_MAX_CPUID_ENTRIES;
r = -ENOMEM;
cpuid_entries = vmalloc(sizeof(struct kvm_cpuid_entry2) * cpuid-nent);
if (!cpuid_entries)
--
To unsubscribe from this list: send the line unsubscribe kvm-commits in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[COMMIT master] KVM: x86: Rework guest single-step flag injection and filtering

2009-10-05 Thread Avi Kivity
From: Jan Kiszka jan.kis...@siemens.com

Push TF and RF injection and filtering on guest single-stepping into the
vender get/set_rflags callbacks. This makes the whole mechanism more
robust wrt user space IOCTL order and instruction emulations.

Signed-off-by: Jan Kiszka jan.kis...@siemens.com
Signed-off-by: Avi Kivity a...@redhat.com

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index e7f8708..179a919 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -614,6 +614,9 @@ void kvm_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, 
int *l);
 int kvm_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
 int kvm_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data);
 
+unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu);
+void kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags);
+
 void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_inject_page_fault(struct kvm_vcpu *vcpu, unsigned long cr2,
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b2e196e..11a6f2f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -235,6 +235,25 @@ bool kvm_require_cpl(struct kvm_vcpu *vcpu, int 
required_cpl)
 }
 EXPORT_SYMBOL_GPL(kvm_require_cpl);
 
+unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu)
+{
+   unsigned long rflags;
+
+   rflags = kvm_x86_ops-get_rflags(vcpu);
+   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
+   rflags = ~(unsigned long)(X86_EFLAGS_TF | X86_EFLAGS_RF);
+   return rflags;
+}
+EXPORT_SYMBOL_GPL(kvm_get_rflags);
+
+void kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
+{
+   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
+   rflags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
+   kvm_x86_ops-set_rflags(vcpu, rflags);
+}
+EXPORT_SYMBOL_GPL(kvm_set_rflags);
+
 /*
  * Load the pae pdptrs.  Return true is they are all valid.
  */
@@ -2777,7 +2796,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
kvm_x86_ops-get_cs_db_l_bits(vcpu, cs_db, cs_l);
 
vcpu-arch.emulate_ctxt.vcpu = vcpu;
-   vcpu-arch.emulate_ctxt.eflags = kvm_x86_ops-get_rflags(vcpu);
+   vcpu-arch.emulate_ctxt.eflags = kvm_get_rflags(vcpu);
vcpu-arch.emulate_ctxt.mode =
(vcpu-arch.emulate_ctxt.eflags  X86_EFLAGS_VM)
? X86EMUL_MODE_REAL : cs_l
@@ -2855,7 +2874,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
return EMULATE_DO_MMIO;
}
 
-   kvm_x86_ops-set_rflags(vcpu, vcpu-arch.emulate_ctxt.eflags);
+   kvm_set_rflags(vcpu, vcpu-arch.emulate_ctxt.eflags);
 
if (vcpu-mmio_is_write) {
vcpu-mmio_needed = 0;
@@ -3291,7 +3310,7 @@ void realmode_lmsw(struct kvm_vcpu *vcpu, unsigned long 
msw,
   unsigned long *rflags)
 {
kvm_lmsw(vcpu, msw);
-   *rflags = kvm_x86_ops-get_rflags(vcpu);
+   *rflags = kvm_get_rflags(vcpu);
 }
 
 unsigned long realmode_get_cr(struct kvm_vcpu *vcpu, int cr)
@@ -3329,7 +3348,7 @@ void realmode_set_cr(struct kvm_vcpu *vcpu, int cr, 
unsigned long val,
switch (cr) {
case 0:
kvm_set_cr0(vcpu, mk_cr_64(vcpu-arch.cr0, val));
-   *rflags = kvm_x86_ops-get_rflags(vcpu);
+   *rflags = kvm_get_rflags(vcpu);
break;
case 2:
vcpu-arch.cr2 = val;
@@ -3460,7 +3479,7 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu)
 {
struct kvm_run *kvm_run = vcpu-run;
 
-   kvm_run-if_flag = (kvm_x86_ops-get_rflags(vcpu)  X86_EFLAGS_IF) != 0;
+   kvm_run-if_flag = (kvm_get_rflags(vcpu)  X86_EFLAGS_IF) != 0;
kvm_run-cr8 = kvm_get_cr8(vcpu);
kvm_run-apic_base = kvm_get_apic_base(vcpu);
if (irqchip_in_kernel(vcpu-kvm))
@@ -3840,13 +3859,7 @@ int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, 
struct kvm_regs *regs)
 #endif
 
regs-rip = kvm_rip_read(vcpu);
-   regs-rflags = kvm_x86_ops-get_rflags(vcpu);
-
-   /*
-* Don't leak debug flags in case they were set for guest debugging
-*/
-   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
-   regs-rflags = ~(X86_EFLAGS_TF | X86_EFLAGS_RF);
+   regs-rflags = kvm_get_rflags(vcpu);
 
vcpu_put(vcpu);
 
@@ -3874,12 +3887,10 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, 
struct kvm_regs *regs)
kvm_register_write(vcpu, VCPU_REGS_R13, regs-r13);
kvm_register_write(vcpu, VCPU_REGS_R14, regs-r14);
kvm_register_write(vcpu, VCPU_REGS_R15, regs-r15);
-
 #endif
 
kvm_rip_write(vcpu, regs-rip);
-   kvm_x86_ops-set_rflags(vcpu, regs-rflags);
-
+   kvm_set_rflags(vcpu, regs-rflags);
 
vcpu-arch.exception.pending = false;
 
@@ -4098,7 +4109,7 @@ static int is_vm86_segment(struct kvm_vcpu *vcpu, int seg)
 {
 

[COMMIT master] KVM: x86: Refactor guest debug IOCTL handling

2009-10-05 Thread Avi Kivity
From: Jan Kiszka jan.kis...@web.de

Much of so far vendor-specific code for setting up guest debug can
actually be handled by the generic code. This also fixes a minor deficit
in the SVM part /wrt processing KVM_GUESTDBG_ENABLE.

Signed-off-by: Jan Kiszka jan.kis...@siemens.com
Signed-off-by: Avi Kivity a...@redhat.com

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 295c7c4..e7f8708 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -475,8 +475,8 @@ struct kvm_x86_ops {
void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
void (*vcpu_put)(struct kvm_vcpu *vcpu);
 
-   int (*set_guest_debug)(struct kvm_vcpu *vcpu,
-  struct kvm_guest_debug *dbg);
+   void (*set_guest_debug)(struct kvm_vcpu *vcpu,
+   struct kvm_guest_debug *dbg);
int (*get_msr)(struct kvm_vcpu *vcpu, u32 msr_index, u64 *pdata);
int (*set_msr)(struct kvm_vcpu *vcpu, u32 msr_index, u64 data);
u64 (*get_segment_base)(struct kvm_vcpu *vcpu, int seg);
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 02a4269..279a2ae 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -1065,26 +1065,16 @@ static void update_db_intercept(struct kvm_vcpu *vcpu)
vcpu-guest_debug = 0;
 }
 
-static int svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg)
+static void svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg)
 {
-   int old_debug = vcpu-guest_debug;
struct vcpu_svm *svm = to_svm(vcpu);
 
-   vcpu-guest_debug = dbg-control;
-
-   update_db_intercept(vcpu);
-
if (vcpu-guest_debug  KVM_GUESTDBG_USE_HW_BP)
svm-vmcb-save.dr7 = dbg-arch.debugreg[7];
else
svm-vmcb-save.dr7 = vcpu-arch.dr7;
 
-   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
-   svm-vmcb-save.rflags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
-   else if (old_debug  KVM_GUESTDBG_SINGLESTEP)
-   svm-vmcb-save.rflags = ~(X86_EFLAGS_TF | X86_EFLAGS_RF);
-
-   return 0;
+   update_db_intercept(vcpu);
 }
 
 static void load_host_msrs(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 97f4265..70020e5 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -1096,30 +1096,14 @@ static void vmx_cache_reg(struct kvm_vcpu *vcpu, enum 
kvm_reg reg)
}
 }
 
-static int set_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg)
+static void set_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg)
 {
-   int old_debug = vcpu-guest_debug;
-   unsigned long flags;
-
-   vcpu-guest_debug = dbg-control;
-   if (!(vcpu-guest_debug  KVM_GUESTDBG_ENABLE))
-   vcpu-guest_debug = 0;
-
if (vcpu-guest_debug  KVM_GUESTDBG_USE_HW_BP)
vmcs_writel(GUEST_DR7, dbg-arch.debugreg[7]);
else
vmcs_writel(GUEST_DR7, vcpu-arch.dr7);
 
-   flags = vmcs_readl(GUEST_RFLAGS);
-   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
-   flags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
-   else if (old_debug  KVM_GUESTDBG_SINGLESTEP)
-   flags = ~(X86_EFLAGS_TF | X86_EFLAGS_RF);
-   vmcs_writel(GUEST_RFLAGS, flags);
-
update_exception_bitmap(vcpu);
-
-   return 0;
 }
 
 static __init int cpu_has_kvm_support(void)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b2f4f13..2bc9693 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4472,12 +4472,19 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
struct kvm_guest_debug *dbg)
 {
-   int i, r;
+   unsigned long rflags;
+   int old_debug;
+   int i;
 
vcpu_load(vcpu);
 
-   if ((dbg-control  (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_HW_BP)) ==
-   (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_HW_BP)) {
+   old_debug = vcpu-guest_debug;
+
+   vcpu-guest_debug = dbg-control;
+   if (!(vcpu-guest_debug  KVM_GUESTDBG_ENABLE))
+   vcpu-guest_debug = 0;
+
+   if (vcpu-guest_debug  KVM_GUESTDBG_USE_HW_BP) {
for (i = 0; i  KVM_NR_DB_REGS; ++i)
vcpu-arch.eff_db[i] = dbg-arch.debugreg[i];
vcpu-arch.switch_db_regs =
@@ -4488,16 +4495,23 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu 
*vcpu,
vcpu-arch.switch_db_regs = (vcpu-arch.dr7  DR7_BP_EN_MASK);
}
 
-   r = kvm_x86_ops-set_guest_debug(vcpu, dbg);
+   rflags = kvm_x86_ops-get_rflags(vcpu);
+   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
+   rflags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
+   else if (old_debug  KVM_GUESTDBG_SINGLESTEP)
+   rflags = ~(X86_EFLAGS_TF | X86_EFLAGS_RF);
+   kvm_x86_ops-set_rflags(vcpu, rflags);
 
-   if 

Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Avi Kivity

On 10/04/2009 09:07 PM, Jan Kiszka wrote:

btw, instead of adding a new ioctl, perhaps it makes sense to define a
new KVM_VCPU_STATE structure that holds all current and future state
(with generous reserved space), instead of separating state over a dozen
ioctls.

 

OK, makes sense. With our without lapic state?


I'm in two minds.  I'm leaning towards including lapic but would welcome 
arguments either way.


Note we have to be careful with timers such as the tsc and lapic timer.  
Maybe have a bitmask at the front specifying which elements are active.



How much future space?
   


avx will change the sse registers from 16x16 to 16x32, with a hint of 
more to come.  Nested vmx needs the vmptr and some more bits.  MSRs are 
potentially endless.  Lots of space.


--
Do not meddle in the internals of kernels, for they are subtle and quick to 
panic.

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Jan Kiszka
Avi Kivity wrote:
 On 10/04/2009 09:07 PM, Jan Kiszka wrote:
 btw, instead of adding a new ioctl, perhaps it makes sense to define a
 new KVM_VCPU_STATE structure that holds all current and future state
 (with generous reserved space), instead of separating state over a dozen
 ioctls.

  
 OK, makes sense. With our without lapic state?
 
 I'm in two minds.  I'm leaning towards including lapic but would welcome
 arguments either way.

The lapic is optional and, thus, typically handled in different code
modules by user space. QEMU even creates a separate device that holds
the state. I'm not sure user space will benefit from a unified query/set
interface with regard to this.

 
 Note we have to be careful with timers such as the tsc and lapic timer. 
 Maybe have a bitmask at the front specifying which elements are active.

...and the lapic timers are another argument.

Regarding TSC, which means MSRs: I tend to include only states into the
this meta state which have fixed sizes. Otherwise things will get very
hairy. And the GET/SET_MSRS interface is already fairly flexible, the
question would be again: What can we gain by unifying?

 
 How much future space?

 
 avx will change the sse registers from 16x16 to 16x32, with a hint of
 more to come.  Nested vmx needs the vmptr and some more bits.  MSRs are
 potentially endless.  Lots of space.
 

Hmm, a some kB then (even without MSRs)...

Jan



signature.asc
Description: OpenPGP digital signature


Re: avoid soft lockups

2009-10-05 Thread James Brackinshaw
On Fri, Oct 2, 2009 at 7:47 PM, Marcelo Tosatti mtosa...@redhat.com wrote:
 On Fri, Oct 02, 2009 at 01:54:22PM +0200, James Brackinshaw wrote:
 Hello,

 If I suspend and resume a guest under kvm, or I migrate it from one
 host to another, I sometimes get a soft lockup.

 Is there a timer mode to prevent or reduce the likelihood of these?

 Not yet. For now you can either disable softlockup in the guest, or live
 with the spurious warnings.

Oh good, so they really are nothing to worry about - thanks!
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Avi Kivity

On 10/05/2009 09:43 AM, Jan Kiszka wrote:

Avi Kivity wrote:
   

On 10/04/2009 09:07 PM, Jan Kiszka wrote:
 

btw, instead of adding a new ioctl, perhaps it makes sense to define a
new KVM_VCPU_STATE structure that holds all current and future state
(with generous reserved space), instead of separating state over a dozen
ioctls.


 

OK, makes sense. With our without lapic state?
   

I'm in two minds.  I'm leaning towards including lapic but would welcome
arguments either way.
 

The lapic is optional and, thus, typically handled in different code
modules by user space. QEMU even creates a separate device that holds
the state.


avx registers, nested vmx are optional as well.


I'm not sure user space will benefit from a unified query/set
interface with regard to this.
   


The main benefit is to avoid creating an ioctl each time we find a 
missing bit.



Note we have to be careful with timers such as the tsc and lapic timer.
Maybe have a bitmask at the front specifying which elements are active.
 

...and the lapic timers are another argument.

Regarding TSC, which means MSRs: I tend to include only states into the
this meta state which have fixed sizes. Otherwise things will get very
hairy. And the GET/SET_MSRS interface is already fairly flexible, the
question would be again: What can we gain by unifying?
   


For MSRs, not much.

Note we can make it work, by storing an offset/length at a fixed 
location and letting userspace point it into the reserved area.



How much future space?

   

avx will change the sse registers from 16x16 to 16x32, with a hint of
more to come.  Nested vmx needs the vmptr and some more bits.  MSRs are
potentially endless.  Lots of space.

 

Hmm, a some kB then (even without MSRs)...
   


Yup, it's amazing how much state modern processors hold.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: kvm guest: hrtimer: interrupt too slow

2009-10-05 Thread Michael Tokarev

Marcelo Tosatti wrote:
[]

You should see min_delta_ns increase to a much smaller value, hopefully in
the 2000-1 range.


That's what I see:

Oct  4 16:13:09 paltus kernel: [boot]
Oct  5 12:35:51 paltus kernel: hrtimer: interrupt too slow, forcing clock min 
delta to 1500 ns
Oct  5 12:47:29 paltus kernel: hrtimer: interrupt too slow, forcing clock min 
delta to 2250 ns

I tried to stress-load the machine yesterday (Oct-4), including massive
network usage (I think it's the network that triggers the issue most
reliable), but wasn't able to trigger it.  Today, at Monday, at normal
system load, it triggered by its own, increasing min_delta to 50%
as in the patch.  The system runs smoothly so far, but it's to be
expected (till it increases min delta to some 10-digit number :).

But I'm unsure still if that's the right approach...  The root
cause seems to be the problem, and this patch just tries to
work around it in a better-than-before way.  But it might be
just me... ;)

I'll continue watching it, since the machine(s) in question are
important for us.

Thanks!

/mjt
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] qemu-kvm: kvm headers update

2009-10-05 Thread Michael S. Tsirkin
Update headers from linux 2.6.32-rc1, this mainly
adds irqfd which will make it easier to add vhost,
down the line. Also reduce code duplication by including
kvm_types.h instead of copying it.

Signed-off-by: Michael S. Tsirkin m...@redhat.com
---
 kvm/include/linux/kvm.h|  131 +++--
 kvm/include/linux/kvm_para.h   |   56 +---
 kvm/include/x86/asm/kvm.h  |   46 +
 kvm/include/x86/asm/kvm_para.h |  141 +---
 4 files changed, 61 insertions(+), 313 deletions(-)

diff --git a/kvm/include/linux/kvm.h b/kvm/include/linux/kvm.h
index 801865d..2c9a209 100644
--- a/kvm/include/linux/kvm.h
+++ b/kvm/include/linux/kvm.h
@@ -1,43 +1,4 @@
-#ifndef KVM_UNIFDEF_H
-#define KVM_UNIFDEF_H
-
-#ifdef __i386__
-#ifndef CONFIG_X86_32
-#define CONFIG_X86_32 1
-#endif
-#endif
-
-#ifdef __x86_64__
-#ifndef CONFIG_X86_64
-#define CONFIG_X86_64 1
-#endif
-#endif
-
-#if defined(__i386__) || defined (__x86_64__)
-#ifndef CONFIG_X86
-#define CONFIG_X86 1
-#endif
-#endif
-
-#ifdef __ia64__
-#ifndef CONFIG_IA64
-#define CONFIG_IA64 1
-#endif
-#endif
-
-#ifdef __PPC__
-#ifndef CONFIG_PPC
-#define CONFIG_PPC 1
-#endif
-#endif
-
-#ifdef __s390__
-#ifndef CONFIG_S390
-#define CONFIG_S390 1
-#endif
-#endif
-
-#endif
+#include linux/kvm_types.h
 #ifndef __LINUX_KVM_H
 #define __LINUX_KVM_H
 
@@ -47,14 +8,14 @@
  * Note: you must update KVM_API_VERSION if you change this interface.
  */
 
-#include asm/types.h
+#include linux/types.h
 
 #include linux/ioctl.h
 #include asm/kvm.h
 
 #define KVM_API_VERSION 12
 
-/* for KVM_TRACE_ENABLE */
+/* for KVM_TRACE_ENABLE, deprecated */
 struct kvm_user_trace_setup {
__u32 buf_size; /* sub_buffer size of each per-cpu */
__u32 buf_nr; /* the number of sub_buffers of each per-cpu */
@@ -281,7 +242,7 @@ struct kvm_dirty_log {
__u32 slot;
__u32 padding1;
union {
-   void   *dirty_bitmap; /* one bit per page */
+   void *dirty_bitmap; /* one bit per page */
__u64 padding2;
};
 };
@@ -347,6 +308,28 @@ struct kvm_guest_debug {
struct kvm_guest_debug_arch arch;
 };
 
+enum {
+   kvm_ioeventfd_flag_nr_datamatch,
+   kvm_ioeventfd_flag_nr_pio,
+   kvm_ioeventfd_flag_nr_deassign,
+   kvm_ioeventfd_flag_nr_max,
+};
+
+#define KVM_IOEVENTFD_FLAG_DATAMATCH (1  kvm_ioeventfd_flag_nr_datamatch)
+#define KVM_IOEVENTFD_FLAG_PIO   (1  kvm_ioeventfd_flag_nr_pio)
+#define KVM_IOEVENTFD_FLAG_DEASSIGN  (1  kvm_ioeventfd_flag_nr_deassign)
+
+#define KVM_IOEVENTFD_VALID_FLAG_MASK  ((1  kvm_ioeventfd_flag_nr_max) - 1)
+
+struct kvm_ioeventfd {
+   __u64 datamatch;
+   __u64 addr;/* legal pio/mmio address */
+   __u32 len; /* 1, 2, 4, or 8 bytes*/
+   __s32 fd;
+   __u32 flags;
+   __u8  pad[36];
+};
+
 #define KVM_TRC_SHIFT   16
 /*
  * kvm trace categories
@@ -365,35 +348,6 @@ struct kvm_guest_debug {
 #define KVM_TRC_CYCLE_SIZE  8
 #define KVM_TRC_EXTRA_MAX   7
 
-/* This structure represents a single trace buffer record. */
-struct kvm_trace_rec {
-   /* variable rec_val
-* is split into:
-* bits 0 - 27  - event id
-* bits 28 -30  - number of extra data args of size u32
-* bits 31  - binary indicator for if tsc is in record
-*/
-   __u32 rec_val;
-   __u32 pid;
-   __u32 vcpu_id;
-   union {
-   struct {
-   __u64 timestamp;
-   __u32 extra_u32[KVM_TRC_EXTRA_MAX];
-   } __attribute__((packed)) timestamp;
-   struct {
-   __u32 extra_u32[KVM_TRC_EXTRA_MAX];
-   } notimestamp;
-   } u;
-};
-
-#define TRACE_REC_EVENT_ID(val) \
-   (0x0fff  (val))
-#define TRACE_REC_NUM_DATA_ARGS(val) \
-   (0x7000  ((val)  28))
-#define TRACE_REC_TCS(val) \
-   (0x8000  ((val)  31))
-
 #define KVMIO 0xAE
 
 /*
@@ -473,8 +427,15 @@ struct kvm_trace_rec {
 #ifdef __KVM_HAVE_MCE
 #define KVM_CAP_MCE 31
 #endif
+#define KVM_CAP_IRQFD 32
+#ifdef __KVM_HAVE_PIT
 #define KVM_CAP_PIT2 33
+#endif
+#define KVM_CAP_SET_BOOT_CPU_ID 34
+#ifdef __KVM_HAVE_PIT_STATE2
 #define KVM_CAP_PIT_STATE2 35
+#endif
+#define KVM_CAP_IOEVENTFD 36
 #define KVM_CAP_SET_IDENTITY_MAP_ADDR 37
 
 #ifdef KVM_CAP_IRQ_ROUTING
@@ -528,16 +489,19 @@ struct kvm_x86_mce {
 };
 #endif
 
+#define KVM_IRQFD_FLAG_DEASSIGN (1  0)
+
+struct kvm_irqfd {
+   __u32 fd;
+   __u32 gsi;
+   __u32 flags;
+   __u8  pad[20];
+};
+
 /*
  * ioctls for VM fds
  */
 #define KVM_SET_MEMORY_REGION _IOW(KVMIO, 0x40, struct kvm_memory_region)
-#define KVM_SET_NR_MMU_PAGES  _IO(KVMIO, 0x44)
-#define KVM_GET_NR_MMU_PAGES  _IO(KVMIO, 0x45)
-#define KVM_SET_USER_MEMORY_REGION _IOW(KVMIO, 0x46,\
-   struct kvm_userspace_memory_region)
-#define 

[GIT PULL] KVM updates for 2.6.32-rc3

2009-10-05 Thread Avi Kivity

Linus, please find in

  git://git.kernel.org/pub/scm/virt/kvm/kvm.git kvm-updates/2.6.32

some KVM fixes, most notably an EPT TLB fix on vcpu migration.  Also
included is glue code for ksm.  While KVM will work without this code,
any shared pages will be removed from the shadow page tables and unshared
on the next guest access, resulting in much thrashing.  With the new code,
the shadow ptes are simply downgraded to read-only access.

Aurelien Jarno (1):
  KVM: fix LAPIC timer period overflow

Avi Kivity (1):
  KVM: Prevent overflow in KVM_GET_SUPPORTED_CPUID

Christian Borntraeger (1):
  KVM: s390: fix memsize = 4G

Izik Eidus (3):
  KVM: MMU: dont hold pagecount reference for mapped sptes pages
  KVM: MMU: add SPTE_HOST_WRITEABLE flag to the shadow ptes
  KVM: add support for change_pte mmu notifiers

Joerg Roedel (2):
  KVM: SVM: Fix tsc offset adjustment when running nested
  KVM: SVM: Handle tsc in svm_get_msr/svm_set_msr correctly

Marcelo Tosatti (1):
  KVM: VMX: flush TLB with INVEPT on cpu migration

 arch/s390/kvm/kvm-s390.h|2 +-
 arch/x86/include/asm/kvm_host.h |1 +
 arch/x86/kvm/lapic.c|2 +-
 arch/x86/kvm/mmu.c  |   84 
++

 arch/x86/kvm/paging_tmpl.h  |   18 +++-
 arch/x86/kvm/svm.c  |   25 +---
 arch/x86/kvm/vmx.c  |2 +-
 arch/x86/kvm/x86.c  |2 +
 virt/kvm/kvm_main.c |   14 ++
 9 files changed, 120 insertions(+), 30 deletions(-)

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 2/2] KVM: x86: Preserve guest single-stepping on register

2009-10-05 Thread Avi Kivity

On 10/04/2009 09:02 PM, Jan Kiszka wrote:


H, good point. Mind reverting 2/2 and applying this one instead?

Jan

-

KVM: x86: Rework guest single-step flag injection and filtering

Push TF and RF injection and filtering on guest single-stepping into the
vender get/set_rflags callbacks. This makes the whole mechanism more
robust /wrt user space IOTCTL order and instruction emulations.

Signed-off-by: Jan Kiszkajan.kis...@siemens.com
---

  arch/x86/kvm/svm.c |8 +++-
  arch/x86/kvm/vmx.c |4 
  arch/x86/kvm/x86.c |   24 +---
  3 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 279a2ae..407e1a7 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -797,11 +797,17 @@ static void svm_vcpu_put(struct kvm_vcpu *vcpu)

  static unsigned long svm_get_rflags(struct kvm_vcpu *vcpu)
  {
-   return to_svm(vcpu)-vmcb-save.rflags;
+   unsigned long rflags = to_svm(vcpu)-vmcb-save.rflags;
+
+   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
+   rflags= ~(unsigned long)(X86_EFLAGS_TF | X86_EFLAGS_RF);
+   return rflags;
  }

  static void svm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
  {
+   if (vcpu-guest_debug  KVM_GUESTDBG_SINGLESTEP)
+   rflags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
to_svm(vcpu)-vmcb-save.rflags = rflags;
  }
   


This code is duplicated in vmx.  How about kvm_[gs]et_rflags to contain it?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: kvm guest: hrtimer: interrupt too slow

2009-10-05 Thread Avi Kivity

On 09/29/2009 03:58 PM, Michael Tokarev wrote:

Avi Kivity wrote:

On 09/29/2009 03:12 PM, Michael Tokarev wrote:

[]

The thing is that after some uptime, kvm
guest prints something like this:

hrtimer: interrupt too slow, forcing clock min delta to 461487495 ns

[]

What happens if you use hpet or pmtimer as guest clocksource?


For all the guests I have handy, only 2 clocksources
are available: kvm-clock and acpi_pm.  The host itself
has hpet turned off because it itself had issues with
hpet, and after many tries we finally turned it off
(there were several long and painy threads on lkml
about this).

Dunno why pmtimer isn't available.


acpi_pm is pmtimer.



I tried switching to acpi_pm on a running guest (which
is still runnining in very slow mode), but that did not
make any difference - i.e., it did not become fast
again.


Please post host /proc/cpuinfo.


Here's the cpuinfo from host (for last core):

flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext 
fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good 
nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm 
extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs


Has both constant_tsc and nonstop_tsc, wonder why it's making so much 
trouble.



--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Biweekly KVM Test report, kernel 94252... qemu 5cc3c...

2009-10-05 Thread Avi Kivity

On 09/29/2009 05:34 AM, Xu, Jiajun wrote:

Hi All,

This Weekly KVM Testing Report against lastest kvm.git
94252a58662dc4ca6191eac479efb40e0716865c and qemu-kvm.git
5cc3cfb6c2254483ae324da407a13307fe7355f3.

Qemu-kvm tree build issue is fixed by qemu commit 
781774b38c90797add71d029b7fbee43200c66d4.
There is no other new bug found in this two weeks. There are 7 old bugs open in 
bug tracking.


Seven Old Issues:

1. Guest hang with exhausted IRQ sources error if 8 VFs assigned
https://sourceforge.net/tracker/?func=detailaid=2847560group_id=180599atid=893831
   


Does the attached patch fix this issue?

--
error compiling committee.c: too many arguments to function

diff --git a/virt/kvm/assigned-dev.c b/virt/kvm/assigned-dev.c
index fd9c097..91b3d99 100644
--- a/virt/kvm/assigned-dev.c
+++ b/virt/kvm/assigned-dev.c
@@ -142,7 +142,8 @@ static void deassign_guest_irq(struct kvm *kvm,
 	kvm_unregister_irq_ack_notifier(kvm, assigned_dev-ack_notifier);
 	assigned_dev-ack_notifier.gsi = -1;
 
-	if (assigned_dev-irq_source_id != -1)
+	if (assigned_dev-irq_source_id != -1
+	 assigned_dev-irq_source_id != KVM_USERSPACE_IRQ_SOURCE_ID)
 		kvm_free_irq_source_id(kvm, assigned_dev-irq_source_id);
 	assigned_dev-irq_source_id = -1;
 	assigned_dev-irq_requested_type = ~(KVM_DEV_IRQ_GUEST_MASK);
@@ -396,20 +397,17 @@ static int assign_guest_irq(struct kvm *kvm,
 			struct kvm_assigned_irq *irq,
 			unsigned long guest_irq_type)
 {
-	int id;
+	int id = KVM_USERSPACE_IRQ_SOURCE_ID;
 	int r = -EEXIST;
 
 	if (dev-irq_requested_type  KVM_DEV_IRQ_GUEST_MASK)
 		return r;
 
-	id = kvm_request_irq_source_id(kvm);
-	if (id  0)
-		return id;
-
-	dev-irq_source_id = id;
-
 	switch (guest_irq_type) {
 	case KVM_DEV_IRQ_GUEST_INTX:
+		id = kvm_request_irq_source_id(kvm);
+		if (id  0)
+			return id;
 		r = assigned_device_enable_guest_intx(kvm, dev, irq);
 		break;
 #ifdef __KVM_HAVE_MSI
@@ -426,11 +424,15 @@ static int assign_guest_irq(struct kvm *kvm,
 		r = -EINVAL;
 	}
 
+	dev-irq_source_id = id;
+
 	if (!r) {
 		dev-irq_requested_type |= guest_irq_type;
 		kvm_register_irq_ack_notifier(kvm, dev-ack_notifier);
-	} else
-		kvm_free_irq_source_id(kvm, dev-irq_source_id);
+	} else {
+		if (dev-irq_source_id != KVM_USERSPACE_IRQ_SOURCE_ID)
+			kvm_free_irq_source_id(kvm, dev-irq_source_id);
+	}
 
 	return r;
 }


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Jan Kiszka
Avi Kivity wrote:
 On 10/05/2009 09:43 AM, Jan Kiszka wrote:
 Avi Kivity wrote:
   
 On 10/04/2009 09:07 PM, Jan Kiszka wrote:
 
 btw, instead of adding a new ioctl, perhaps it makes sense to define a
 new KVM_VCPU_STATE structure that holds all current and future state
 (with generous reserved space), instead of separating state over a
 dozen
 ioctls.


  
 OK, makes sense. With our without lapic state?

 I'm in two minds.  I'm leaning towards including lapic but would welcome
 arguments either way.
  
 The lapic is optional and, thus, typically handled in different code
 modules by user space. QEMU even creates a separate device that holds
 the state.
 
 avx registers, nested vmx are optional as well.
 
 I'm not sure user space will benefit from a unified query/set
 interface with regard to this.

 
 The main benefit is to avoid creating an ioctl each time we find a
 missing bit.
 
 Note we have to be careful with timers such as the tsc and lapic timer.
 Maybe have a bitmask at the front specifying which elements are active.
  
 ...and the lapic timers are another argument.

 Regarding TSC, which means MSRs: I tend to include only states into the
 this meta state which have fixed sizes. Otherwise things will get very
 hairy. And the GET/SET_MSRS interface is already fairly flexible, the
 question would be again: What can we gain by unifying?

 
 For MSRs, not much.
 
 Note we can make it work, by storing an offset/length at a fixed
 location and letting userspace point it into the reserved area.

Hmm, pointers... That makes me think of a meta IOCTL like this:

struct kvm_vcpu_state {
int substates;
void __user *substate[0];
};

#define KVM_VCPU_STATE_REGS  0 /* i.e. substate[0] points to kvm_regs */
#define KVM_VCPU_STATE_SREGS 1
#define KVM_VCPU_STATE_LAPIC 2
...

We could easily extend the call with more substates just by defining new
pointer slots. Moreover, user space could define which substates should
be get/set by simply passing NULL or a valid pointer for substate[n] (or
by limiting the substates field).

The only ugliness I see is the missing type safety as we would have to
deal with void pointers to the substate structures here.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Avi Kivity

On 10/05/2009 01:18 PM, Jan Kiszka wrote:

Avi Kivity wrote:
   

On 10/05/2009 09:43 AM, Jan Kiszka wrote:
 

Avi Kivity wrote:

   

On 10/04/2009 09:07 PM, Jan Kiszka wrote:

 

btw, instead of adding a new ioctl, perhaps it makes sense to define a
new KVM_VCPU_STATE structure that holds all current and future state
(with generous reserved space), instead of separating state over a
dozen
ioctls.



 

OK, makes sense. With our without lapic state?

   

I'm in two minds.  I'm leaning towards including lapic but would welcome
arguments either way.

 

The lapic is optional and, thus, typically handled in different code
modules by user space. QEMU even creates a separate device that holds
the state.
   

avx registers, nested vmx are optional as well.

 

I'm not sure user space will benefit from a unified query/set
interface with regard to this.

   

The main benefit is to avoid creating an ioctl each time we find a
missing bit.

 

Note we have to be careful with timers such as the tsc and lapic timer.
Maybe have a bitmask at the front specifying which elements are active.

 

...and the lapic timers are another argument.

Regarding TSC, which means MSRs: I tend to include only states into the
this meta state which have fixed sizes. Otherwise things will get very
hairy. And the GET/SET_MSRS interface is already fairly flexible, the
question would be again: What can we gain by unifying?

   

For MSRs, not much.

Note we can make it work, by storing an offset/length at a fixed
location and letting userspace point it into the reserved area.
 

Hmm, pointers... That makes me think of a meta IOCTL like this:

struct kvm_vcpu_state {
int substates;
void __user *substate[0];
};

   


True pointers are no go since we have to deal with compat code (31/32 
bit userspace calling into a 64 bit kernel).  Offsets into the state 
structure are easier.



#define KVM_VCPU_STATE_REGS  0 /* i.e. substate[0] points to kvm_regs */
#define KVM_VCPU_STATE_SREGS 1
#define KVM_VCPU_STATE_LAPIC 2
...

We could easily extend the call with more substates just by defining new
pointer slots. Moreover, user space could define which substates should
be get/set by simply passing NULL or a valid pointer for substate[n] (or
by limiting the substates field).

The only ugliness I see is the missing type safety as we would have to
deal with void pointers to the substate structures here.
   


For fixed sized state a feature bitmap is sufficient I think.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: migrate_set_downtime bug

2009-10-05 Thread Avi Kivity

On 09/30/2009 08:41 PM, Dietmar Maurer wrote:


I just think of common scenarios like 'maintanace mode', where all VM should 
migrate to another host. A endless migrate task can make that fail.

For me, it is totally unclear what value I should set for 'max_downtime' to 
avoid that behavior?

   


We used to have a heuristic that said 'if an iteration transfers more 
pages than the previous iteration, we've stopped converging'.  Why 
wouldn't that work?


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Jan Kiszka
Avi Kivity wrote:
 On 10/05/2009 01:18 PM, Jan Kiszka wrote:
 Avi Kivity wrote:

 On 10/05/2009 09:43 AM, Jan Kiszka wrote:
  
 Avi Kivity wrote:


 On 10/04/2009 09:07 PM, Jan Kiszka wrote:

  
 btw, instead of adding a new ioctl, perhaps it makes sense to define a
 new KVM_VCPU_STATE structure that holds all current and future state
 (with generous reserved space), instead of separating state over a
 dozen
 ioctls.



  
 OK, makes sense. With our without lapic state?


 I'm in two minds.  I'm leaning towards including lapic but would welcome
 arguments either way.

  
 The lapic is optional and, thus, typically handled in different code
 modules by user space. QEMU even creates a separate device that holds
 the state.

 avx registers, nested vmx are optional as well.

  
 I'm not sure user space will benefit from a unified query/set
 interface with regard to this.


 The main benefit is to avoid creating an ioctl each time we find a
 missing bit.

  
 Note we have to be careful with timers such as the tsc and lapic timer.
 Maybe have a bitmask at the front specifying which elements are active.

  
 ...and the lapic timers are another argument.

 Regarding TSC, which means MSRs: I tend to include only states into the
 this meta state which have fixed sizes. Otherwise things will get very
 hairy. And the GET/SET_MSRS interface is already fairly flexible, the
 question would be again: What can we gain by unifying?


 For MSRs, not much.

 Note we can make it work, by storing an offset/length at a fixed
 location and letting userspace point it into the reserved area.
  
 Hmm, pointers... That makes me think of a meta IOCTL like this:

 struct kvm_vcpu_state {
  int substates;
  void __user *substate[0];
 };


 
 True pointers are no go since we have to deal with compat code (31/32 
 bit userspace calling into a 64 bit kernel).  Offsets into the state 
 structure are easier.

So current KVM_GET_DIRTY_LOG is broken in the compat case...

 
 #define KVM_VCPU_STATE_REGS  0 /* i.e. substate[0] points to kvm_regs */
 #define KVM_VCPU_STATE_SREGS 1
 #define KVM_VCPU_STATE_LAPIC 2
 ...

 We could easily extend the call with more substates just by defining new
 pointer slots. Moreover, user space could define which substates should
 be get/set by simply passing NULL or a valid pointer for substate[n] (or
 by limiting the substates field).

 The only ugliness I see is the missing type safety as we would have to
 deal with void pointers to the substate structures here.

 
 For fixed sized state a feature bitmap is sufficient I think.
 

We'll probably have to deal with both. Therefore, I'm looking for a
unified solution.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Modifying RAM during runtime on guest

2009-10-05 Thread Daniel Bareiro
On Tuesday, 29 September 2009 00:55:02 -0400,
Jim Paris wrote:

  According to the tests that I was doing in guest with kernel with
  support for virtio, shrinking works, but when trying to return to
  the amount of initial memory, seems that it fails and I lose
  connectivity by serial console and ssh.
  
  In the guest:
  
  # uname -a
  Linux central 2.6.26-2-amd64 #1 SMP Sun Jul 26 20:35:48 UTC 2009 x86_64 
  GNU/Linux
  
  # cat config-2.6.26-2-amd64 | grep -i virtio
  CONFIG_NET_9P_VIRTIO=m
  CONFIG_VIRTIO_BLK=m
  CONFIG_VIRTIO_NET=m
  CONFIG_HW_RANDOM_VIRTIO=m
  CONFIG_VIRTIO=m
  CONFIG_VIRTIO_RING=m
  CONFIG_VIRTIO_PCI=m
  CONFIG_VIRTIO_BALLOON=m
  
  In the host:
  
  # telnet localhost 4045
  Trying 127.0.0.1...
  Connected to localhost.
  Escape character is '^]'.
  QEMU 0.10.50 monitor - type 'help' for more information
  (qemu)
  (qemu) info balloon
  balloon: actual=512
  (qemu) balloon 256
  (qemu) info balloon
  balloon: actual=256
  (qemu)
  (qemu) balloon 512
  (qemu) info balloon
  balloon: actual=36
  
  At this moment I'm unable to connect by serial console to the guest
  and the ssh sessions are freeze nevertheless kvm process of the
  guest is running.  Using a VNC client I see several messages of this
  type in the guest:
  
  Out of memory: kill process  (name) score xx or a child
  Killed process  (name)
  
  Finally:
  
  Kernel panic - not syncing - Out of memory and no killable processes...

 Hi Daniel,

Hi, Jim.

 I noticed no-one answered this, and I just ran into the same thing
 myself. As Avi pointed out earlier, it is a guest bug, and upgrading
 the guest to 2.6.27 should fix it:

   http://www.mail-archive.com/kvm@vger.kernel.org/msg10849.html

In this moment I doesn't have Internet connectivity in my house, but, as
soon as it be possible, I shall download the necessary software to
compile 2.6.27 or superior and then I tell you the result of the tests.

Thanks for your reply. 

Regards,
Daniel
-- 
Fingerprint: BFB3 08D6 B4D1 31B2 72B9  29CE 6696 BF1B 14E6 1D37
Powered by Debian GNU/Linux Squeeze - Linux user #188.598


signature.asc
Description: Digital signature


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Avi Kivity

On 10/05/2009 02:18 PM, Jan Kiszka wrote:



True pointers are no go since we have to deal with compat code (31/32
bit userspace calling into a 64 bit kernel).  Offsets into the state
structure are easier.
 

So current KVM_GET_DIRTY_LOG is broken in the compat case...

   


Yes, for big-endian 32/64 and s390.  There are some patches floating around.


We'll probably have to deal with both. Therefore, I'm looking for a
unified solution.

   


array of

struct {
   __u16 type;
   __u16 offset;
} ?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Jan Kiszka
Avi Kivity wrote:
 On 10/05/2009 02:18 PM, Jan Kiszka wrote:
 True pointers are no go since we have to deal with compat code (31/32
 bit userspace calling into a 64 bit kernel).  Offsets into the state
 structure are easier.
  
 So current KVM_GET_DIRTY_LOG is broken in the compat case...


 
 Yes, for big-endian 32/64 and s390.  There are some patches floating around.

Well, that's for fixing up the endianess of the bitmap itself. But the
problem with void * in compat code are their different sizes. And
GET_DIRTY_LOG solves this via padding:

union {
void __user *dirty_bitmap;
__u64 padding2;
};

So this should not make pointers a no-go, should it?

 
 We'll probably have to deal with both. Therefore, I'm looking for a
 unified solution.


 
 array of
 
 struct {
 __u16 type;
 __u16 offset;
 } ?
 

For sure possible, just the setup of such data structure in user space
gets a bit, well, unhandy.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Qemu-devel] Release plan for 0.12.0

2009-10-05 Thread Luiz Capitulino
On Sat, 03 Oct 2009 12:04:57 +0200
Avi Kivity a...@redhat.com wrote:

 On 10/01/2009 11:13 PM, Luiz Capitulino wrote:
  If we're going to support the protocol for 0.12, I'd like to most of the
  code merged by the end of October.
   
Four weeks.. Not so much time, but let's try.
 
There are two major issues that may delay QMP.
 
Firstly, we are still on the infrastructure/design phase, which
  is natural to take time. Maybe when handlers start getting converted
  en masse things will be faster.
 
 
 I sure hope so.  Maybe someone can pitch in if not.

 I've written a TODO list if someone is willing to help:

http://tinyurl.com/ya7l6bo

Secondly: testing. I have a very ugly python script to test the
  already converted handlers. The problem is not only the ugliness,
  the right way to do this would be to use kvm-autotest. So, I was
  planning to take a detailed look at it and perhaps start writing
  tests for QMP right when each handler is converted. Right Thing,
  but takes time.
 
 
 I think this could be done by having autotest use two monitors, one with 
 the machine protocol and one with the human protocol, trying first the 
 machine protocol and falling back if the command is not supported.

 Yes, sounds a good idea.

 Hopefully we can get the autotest people to work on it so we parallelize 
 development.  They'll also give user-oriented feedback which can be 
 valuable.

 I will talk to them about that.

 Are you using a standard json parser with your test script?  That's an 
 additional validation.

 I'm using Python's json module, but I could run one of the checkers
listed in the json's page for each test, before the Python's module
kicks in.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 2/2] KVM: x86: Rework guest single-step flag injection and filtering

2009-10-05 Thread Avi Kivity

On 10/05/2009 01:07 PM, Jan Kiszka wrote:

Push TF and RF injection and filtering on guest single-stepping into the
vender get/set_rflags callbacks. This makes the whole mechanism more
robust /wrt user space IOTCTL order and instruction emulations.
   


Applied, thanks.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Heads up: More user-unaccessible x86 states?

2009-10-05 Thread Avi Kivity

On 10/05/2009 02:42 PM, Jan Kiszka wrote:



Yes, for big-endian 32/64 and s390.  There are some patches floating around.
 

Well, that's for fixing up the endianess of the bitmap itself. But the
problem with void * in compat code are their different sizes. And
GET_DIRTY_LOG solves this via padding:

union {
void __user *dirty_bitmap;
__u64 padding2;
};

So this should not make pointers a no-go, should it?
   


No, it doesn't work.  Big-endian will place the pointer at offset zero 
which is the high-end word when read by the host; and 31-bit s390 needs 
something unsurprisingly strange to be done to the pointer.


The patches I mentioned involve creating a compat_ioctl callback, 
something which I tried to avoid but failed, not having considered 
big-endian and s390.



We'll probably have to deal with both. Therefore, I'm looking for a
unified solution.


   

array of

struct {
 __u16 type;
 __u16 offset;
} ?

 

For sure possible, just the setup of such data structure in user space
gets a bit, well, unhandy.
   


Yes.  Fixed-offset substructures are a lot easier and less error-prone.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: migrate_set_downtime bug

2009-10-05 Thread Glauber Costa
On Mon, Oct 05, 2009 at 02:17:30PM +0200, Avi Kivity wrote:
 On 09/30/2009 08:41 PM, Dietmar Maurer wrote:

 I just think of common scenarios like 'maintanace mode', where all VM should 
 migrate to another host. A endless migrate task can make that fail.

 For me, it is totally unclear what value I should set for 'max_downtime' to 
 avoid that behavior?



 We used to have a heuristic that said 'if an iteration transfers more  
 pages than the previous iteration, we've stopped converging'.  Why  
 wouldn't that work?
Because it seems people agreed that mgmt tools would be the place for those 
heuristics.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: migrate_set_downtime bug

2009-10-05 Thread Avi Kivity

On 10/05/2009 03:04 PM, Glauber Costa wrote:

On Mon, Oct 05, 2009 at 02:17:30PM +0200, Avi Kivity wrote:
   

On 09/30/2009 08:41 PM, Dietmar Maurer wrote:
 

I just think of common scenarios like 'maintanace mode', where all VM should 
migrate to another host. A endless migrate task can make that fail.

For me, it is totally unclear what value I should set for 'max_downtime' to 
avoid that behavior?


   

We used to have a heuristic that said 'if an iteration transfers more
pages than the previous iteration, we've stopped converging'.  Why
wouldn't that work?
 

Because it seems people agreed that mgmt tools would be the place for those 
heuristics.
   


Heuristics like number of pages, maybe.  But since we don't export 
iteration information, we can't expect management tools to stop the 
guest if migration doesn't converge.


I suppose it could issue a 'stop' after some amount of time (constant * 
memory size / bandwidth).


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: migrate_set_downtime bug

2009-10-05 Thread Dietmar Maurer
 We used to have a heuristic that said 'if an iteration transfers more
 pages than the previous iteration, we've stopped converging'.  Why
 wouldn't that work?

This does not protect you from very long migration times.

- Dietmar

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: migrate_set_downtime bug

2009-10-05 Thread Avi Kivity

On 10/05/2009 04:01 PM, Dietmar Maurer wrote:

We used to have a heuristic that said 'if an iteration transfers more
pages than the previous iteration, we've stopped converging'.  Why
wouldn't that work?
 

This does not protect you from very long migration times.

   


Well, if each iteration transfers one page less than the previous one, 
it doesn't.


You can always issue a 'stop' from the monitor.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Qemu-devel] Release plan for 0.12.0

2009-10-05 Thread Avi Kivity

On 10/05/2009 02:43 PM, Luiz Capitulino wrote:

Are you using a standard json parser with your test script?  That's an
additional validation.
 

  I'm using Python's json module, but I could run one of the checkers
listed in the json's page for each test, before the Python's module
kicks in.
   


python-json should be sufficient, I think.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: migrate_set_downtime bug

2009-10-05 Thread Dietmar Maurer
 Heuristics like number of pages, maybe.  But since we don't export
 iteration information, we can't expect management tools to stop the
 guest if migration doesn't converge.
 
 I suppose it could issue a 'stop' after some amount of time (constant *
 memory size / bandwidth).

'bandwidth' is something that changes dynamically (or by user settings), so why 
don't we simply abort after some amount of transferred memory (constant * 
memory size). This can be implemented by the management application without 
problems, although it's much easier inside kvm.

- Dietmar


--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: migrate_set_downtime bug

2009-10-05 Thread Dietmar Maurer


 -Original Message-
 From: Avi Kivity [mailto:a...@redhat.com]
 Sent: Montag, 05. Oktober 2009 16:06
 To: Dietmar Maurer
 Cc: Glauber Costa; Anthony Liguori; kvm
 Subject: Re: migrate_set_downtime bug
 
 On 10/05/2009 04:01 PM, Dietmar Maurer wrote:
  We used to have a heuristic that said 'if an iteration transfers
 more
  pages than the previous iteration, we've stopped converging'.  Why
  wouldn't that work?
 
  This does not protect you from very long migration times.
 
 
 
 Well, if each iteration transfers one page less than the previous one,
 it doesn't.

That approach also depends on bandwidth changes.

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: migrate_set_downtime bug

2009-10-05 Thread Dietmar Maurer
 On 10/05/2009 04:01 PM, Dietmar Maurer wrote:
  We used to have a heuristic that said 'if an iteration transfers
 more
  pages than the previous iteration, we've stopped converging'.  Why
  wouldn't that work?
 
  This does not protect you from very long migration times.
 
 
 
 Well, if each iteration transfers one page less than the previous one,
 it doesn't.

So how long does a migration take in this scenario when you have a VM with 8GB 
RAM?

- dietmar

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: migrate_set_downtime bug

2009-10-05 Thread Avi Kivity

On 10/05/2009 04:08 PM, Dietmar Maurer wrote:

Well, if each iteration transfers one page less than the previous one,
it doesn't.
 

So how long does a migration take in this scenario when you have a VM with 8GB 
RAM?

   


At 1 Gbps, about 2 years.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: migrate_set_downtime bug

2009-10-05 Thread Dietmar Maurer
 We used to have a heuristic that said 'if an iteration transfers more
 pages than the previous iteration, we've stopped converging'.  Why
 wouldn't that work?

I agree that this is the 'right' approach - but it is just too difficult to 
detect that we are not 'converging', and it does not set a limit on migration 
time.

- Dietmar

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


buildbot failure in qemu-kvm on default_x86_64_out_of_tree

2009-10-05 Thread qemu-kvm
The Buildbot has detected a new failure of default_x86_64_out_of_tree on 
qemu-kvm.
Full details are available at:
 
http://buildbot.b1-systems.de/qemu-kvm/builders/default_x86_64_out_of_tree/builds/37

Buildbot URL: http://buildbot.b1-systems.de/qemu-kvm/

Buildslave for this Build: b1_qemu_kvm_1

Build Reason: 
Build Source Stamp: [branch master] HEAD
Blamelist: Aurelien Jarno aurel...@aurel32.net,Blue Swirl 
blauwir...@gmail.com,Bruce Rogers brog...@novell.com,Dominic Evans 
oldma...@gmail.com,Gerd Hoffmann kra...@redhat.com,Glauber Costa 
glom...@redhat.com,Igor V. Kovalenko igor.v.kovale...@gmail.com,Izik Eidus 
iei...@redhat.com,Juan Quintela quint...@redhat.com,Laurent Desnogues 
laurent.desnog...@gmail.com,Luiz Capitulino lcapitul...@redhat.com,Marcelo 
Tosatti mtosa...@redhat.com,Mark McLoughlin mar...@redhat.com,Pierre Riteau 
pierre.rit...@irisa.fr,Stefan Weil w...@mail.berlios.de,malc 
av1...@comtv.ru

BUILD FAILED: failed git

sincerely,
 -The Buildbot

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


buildbot failure in qemu-kvm on disable_kvm_x86_64_debian_5_0

2009-10-05 Thread qemu-kvm
The Buildbot has detected a new failure of disable_kvm_x86_64_debian_5_0 on 
qemu-kvm.
Full details are available at:
 
http://buildbot.b1-systems.de/qemu-kvm/builders/disable_kvm_x86_64_debian_5_0/builds/86

Buildbot URL: http://buildbot.b1-systems.de/qemu-kvm/

Buildslave for this Build: b1_qemu_kvm_1

Build Reason: 
Build Source Stamp: [branch master] HEAD
Blamelist: Aurelien Jarno aurel...@aurel32.net,Blue Swirl 
blauwir...@gmail.com,Bruce Rogers brog...@novell.com,Dominic Evans 
oldma...@gmail.com,Gerd Hoffmann kra...@redhat.com,Glauber Costa 
glom...@redhat.com,Igor V. Kovalenko igor.v.kovale...@gmail.com,Izik Eidus 
iei...@redhat.com,Juan Quintela quint...@redhat.com,Laurent Desnogues 
laurent.desnog...@gmail.com,Luiz Capitulino lcapitul...@redhat.com,Marcelo 
Tosatti mtosa...@redhat.com,Mark McLoughlin mar...@redhat.com,Pierre Riteau 
pierre.rit...@irisa.fr,Stefan Weil w...@mail.berlios.de,malc 
av1...@comtv.ru

BUILD FAILED: failed git

sincerely,
 -The Buildbot

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


buildbot failure in qemu-kvm on default_i386_debian_5_0

2009-10-05 Thread qemu-kvm
The Buildbot has detected a new failure of default_i386_debian_5_0 on qemu-kvm.
Full details are available at:
 
http://buildbot.b1-systems.de/qemu-kvm/builders/default_i386_debian_5_0/builds/98

Buildbot URL: http://buildbot.b1-systems.de/qemu-kvm/

Buildslave for this Build: b1_qemu_kvm_2

Build Reason: 
Build Source Stamp: [branch master] HEAD
Blamelist: Aurelien Jarno aurel...@aurel32.net,Blue Swirl 
blauwir...@gmail.com,Bruce Rogers brog...@novell.com,Dominic Evans 
oldma...@gmail.com,Gerd Hoffmann kra...@redhat.com,Glauber Costa 
glom...@redhat.com,Igor V. Kovalenko igor.v.kovale...@gmail.com,Izik Eidus 
iei...@redhat.com,Juan Quintela quint...@redhat.com,Laurent Desnogues 
laurent.desnog...@gmail.com,Luiz Capitulino lcapitul...@redhat.com,Marcelo 
Tosatti mtosa...@redhat.com,Mark McLoughlin mar...@redhat.com,Pierre Riteau 
pierre.rit...@irisa.fr,Stefan Weil w...@mail.berlios.de,malc 
av1...@comtv.ru

BUILD FAILED: failed git

sincerely,
 -The Buildbot

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


buildbot failure in qemu-kvm on disable_kvm_i386_debian_5_0

2009-10-05 Thread qemu-kvm
The Buildbot has detected a new failure of disable_kvm_i386_debian_5_0 on 
qemu-kvm.
Full details are available at:
 
http://buildbot.b1-systems.de/qemu-kvm/builders/disable_kvm_i386_debian_5_0/builds/87

Buildbot URL: http://buildbot.b1-systems.de/qemu-kvm/

Buildslave for this Build: b1_qemu_kvm_2

Build Reason: 
Build Source Stamp: [branch master] HEAD
Blamelist: Aurelien Jarno aurel...@aurel32.net,Blue Swirl 
blauwir...@gmail.com,Bruce Rogers brog...@novell.com,Dominic Evans 
oldma...@gmail.com,Gerd Hoffmann kra...@redhat.com,Glauber Costa 
glom...@redhat.com,Igor V. Kovalenko igor.v.kovale...@gmail.com,Izik Eidus 
iei...@redhat.com,Juan Quintela quint...@redhat.com,Laurent Desnogues 
laurent.desnog...@gmail.com,Luiz Capitulino lcapitul...@redhat.com,Marcelo 
Tosatti mtosa...@redhat.com,Mark McLoughlin mar...@redhat.com,Pierre Riteau 
pierre.rit...@irisa.fr,Stefan Weil w...@mail.berlios.de,malc 
av1...@comtv.ru

BUILD FAILED: failed git

sincerely,
 -The Buildbot

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


buildbot failure in qemu-kvm on disable_kvm_i386_out_of_tree

2009-10-05 Thread qemu-kvm
The Buildbot has detected a new failure of disable_kvm_i386_out_of_tree on 
qemu-kvm.
Full details are available at:
 
http://buildbot.b1-systems.de/qemu-kvm/builders/disable_kvm_i386_out_of_tree/builds/35

Buildbot URL: http://buildbot.b1-systems.de/qemu-kvm/

Buildslave for this Build: b1_qemu_kvm_2

Build Reason: 
Build Source Stamp: [branch master] HEAD
Blamelist: Aurelien Jarno aurel...@aurel32.net,Blue Swirl 
blauwir...@gmail.com,Bruce Rogers brog...@novell.com,Dominic Evans 
oldma...@gmail.com,Gerd Hoffmann kra...@redhat.com,Glauber Costa 
glom...@redhat.com,Igor V. Kovalenko igor.v.kovale...@gmail.com,Izik Eidus 
iei...@redhat.com,Juan Quintela quint...@redhat.com,Laurent Desnogues 
laurent.desnog...@gmail.com,Luiz Capitulino lcapitul...@redhat.com,Marcelo 
Tosatti mtosa...@redhat.com,Mark McLoughlin mar...@redhat.com,Pierre Riteau 
pierre.rit...@irisa.fr,Stefan Weil w...@mail.berlios.de,malc 
av1...@comtv.ru

BUILD FAILED: failed git

sincerely,
 -The Buildbot

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


buildbot failure in qemu-kvm on default_i386_out_of_tree

2009-10-05 Thread qemu-kvm
The Buildbot has detected a new failure of default_i386_out_of_tree on qemu-kvm.
Full details are available at:
 
http://buildbot.b1-systems.de/qemu-kvm/builders/default_i386_out_of_tree/builds/35

Buildbot URL: http://buildbot.b1-systems.de/qemu-kvm/

Buildslave for this Build: b1_qemu_kvm_2

Build Reason: 
Build Source Stamp: [branch master] HEAD
Blamelist: Aurelien Jarno aurel...@aurel32.net,Blue Swirl 
blauwir...@gmail.com,Bruce Rogers brog...@novell.com,Dominic Evans 
oldma...@gmail.com,Gerd Hoffmann kra...@redhat.com,Glauber Costa 
glom...@redhat.com,Igor V. Kovalenko igor.v.kovale...@gmail.com,Izik Eidus 
iei...@redhat.com,Juan Quintela quint...@redhat.com,Laurent Desnogues 
laurent.desnog...@gmail.com,Luiz Capitulino lcapitul...@redhat.com,Marcelo 
Tosatti mtosa...@redhat.com,Mark McLoughlin mar...@redhat.com,Pierre Riteau 
pierre.rit...@irisa.fr,Stefan Weil w...@mail.berlios.de,malc 
av1...@comtv.ru

BUILD FAILED: failed git

sincerely,
 -The Buildbot

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] Adding NAS Parallel Benchmarks (NPB) as client test

2009-10-05 Thread Lucas Meneghel Rodrigues
This new autotest module implements a benchmark for parallel
processing (SMP) systems. Besides the traditional benchmarking,
some sanity checks are made, in order to test effectively the
SMP implementation of virtual machines. So this module at
the same time is a regular benchmark, but it can be also used
as a functional test for virtualization.

V3: Implemented suggestions made during review of the module.
Binary diff removed, this is for the sake of documentation.

Signed-off-by: Cao, Chen k...@redhat.com
---
 client/tests/kvm/autotest_control/npb.control |   28 +++
 client/tests/kvm/kvm_tests.cfg.sample |4 +
 client/tests/npb/NPB3.3.tar.gz|  Bin 0 - 615658 bytes
 client/tests/npb/control  |   28 +++
 client/tests/npb/enable-all-tests.patch   |  233 +
 client/tests/npb/npb.py   |  145 +++
 6 files changed, 438 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/autotest_control/npb.control
 create mode 100644 client/tests/npb/NPB3.3.tar.gz
 create mode 100644 client/tests/npb/control
 create mode 100644 client/tests/npb/enable-all-tests.patch
 create mode 100644 client/tests/npb/npb.py

diff --git a/client/tests/kvm/autotest_control/npb.control 
b/client/tests/kvm/autotest_control/npb.control
new file mode 100644
index 000..2a8a8bb
--- /dev/null
+++ b/client/tests/kvm/autotest_control/npb.control
@@ -0,0 +1,28 @@
+NAME = NAS Parallel Benchmarks
+AUTHOR = Cao, Chen k...@redhat.com
+TEST_TYPE = CLIENT
+TEST_CLASS = HARDWARE
+TEST_CATEGORY = BENCHMARK
+TIME = MEDIUM
+DOC = \
+Using NPB, OpenMP implementation.
+
+See http://www.nas.nasa.gov/Software/NPB/
+
+
+# Supported tests (benchmarks):
+#   bt.A bt.B bt.C bt.D bt.E bt.S bt.W
+#   cg.A cg.B cg.C cg.S cg.W
+#   dc.A dc.B dc.S dc.W
+#   ep.A ep.B ep.C ep.D ep.E ep.S ep.W
+#   ft.A ft.B ft.S ft.W
+#   is.A is.B is.C is.S is.W
+#   lu.A lu.B lu.C lu.S lu.W
+#   mg.A mg.B mg.S mg.W
+#   sp.A sp.B sp.C sp.D sp.E sp.S sp.W
+#   ua.A ua.B ua.C ua.S ua.W
+#
+# Please refer to npb.py for more infomation about
+# the arguments.
+job.run_test(url='npb', tests='ep.A ep.B')
+
diff --git a/client/tests/kvm/kvm_tests.cfg.sample 
b/client/tests/kvm/kvm_tests.cfg.sample
index ceda7b3..fa4dd12 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -96,6 +96,9 @@ variants:
 # control file and set this timeout appropriately.
 test_timeout = 3900
 test_control_file = cerberus.control
+- npb:
+test_name = npb
+test_control_file = npb.control
 
 - linux_s3: install setup
 type = linux_s3
@@ -639,6 +642,7 @@ linux_s3:
 
 variants:
 - @up:
+no autotest.npb
 - smp2:
 extra_params +=  -smp 2
diff --git a/client/tests/npb/control b/client/tests/npb/control
new file mode 100644
index 000..bebd0e9
--- /dev/null
+++ b/client/tests/npb/control
@@ -0,0 +1,28 @@
+NAME = NAS Parallel Benchmarks
+AUTHOR = Cao, Chen k...@redhat.com
+TEST_TYPE = CLIENT
+TEST_CLASS = HARDWARE
+TEST_CATEGORY = BENCHMARK
+TIME = MEDIUM
+DOC = \
+Using NPB, OpenMP implementation.
+
+See http://www.nas.nasa.gov/Software/NPB/
+
+
+# Supported tests (benchmarks):
+#   bt.A bt.B bt.C bt.D bt.E bt.S bt.W
+#   cg.A cg.B cg.C cg.S cg.W
+#   dc.A dc.B dc.S dc.W
+#   ep.A ep.B ep.C ep.D ep.E ep.S ep.W
+#   ft.A ft.B ft.S ft.W
+#   is.A is.B is.C is.S is.W
+#   lu.A lu.B lu.C lu.S lu.W
+#   mg.A mg.B mg.S mg.W
+#   sp.A sp.B sp.C sp.D sp.E sp.S sp.W
+#   ua.A ua.B ua.C ua.S ua.W
+#
+# Please refer to npb.py for more infomation about
+# the arguments.
+job.run_test(url='npb', tests='ep.A ep.B bt.S bt.W')
+
diff --git a/client/tests/npb/enable-all-tests.patch 
b/client/tests/npb/enable-all-tests.patch
new file mode 100644
index 000..f08a9d3
--- /dev/null
+++ b/client/tests/npb/enable-all-tests.patch
@@ -0,0 +1,233 @@
+diff --git a/NPB3.3-OMP/config/make.def b/NPB3.3-OMP/config/make.def
+new file mode 100644
+index 000..afffe7d
+--- /dev/null
 b/NPB3.3-OMP/config/make.def
+@@ -0,0 +1,161 @@
++#---
++#
++#SITE- AND/OR PLATFORM-SPECIFIC DEFINITIONS. 
++#
++#---
++
++#---
++# Items in this file will need to be changed for each platform.
++#---
++
++#---
++# Parallel Fortran:
++#
++# For CG, EP, FT, MG, LU, SP, BT and UA, which are in Fortran, the following 
++# must be defined:
++#
++# F77- Fortran compiler
++# FFLAGS - Fortran compilation arguments
++# F_INC  - any -I arguments required for compiling Fortran 
++# FLINK  - 

Re: migrate_set_downtime bug

2009-10-05 Thread Glauber Costa
On Mon, Oct 05, 2009 at 04:09:43PM +0200, Dietmar Maurer wrote:
  Heuristics like number of pages, maybe.  But since we don't export
  iteration information, we can't expect management tools to stop the
  guest if migration doesn't converge.
  
  I suppose it could issue a 'stop' after some amount of time (constant *
  memory size / bandwidth).
 
 'bandwidth' is something that changes dynamically (or by user settings), so 
 why don't we simply abort after some amount of transferred memory (constant * 
 memory size). This can be implemented by the management application without 
 problems, although it's much easier inside kvm.
 
Easier, yes.

But then once it is done, people wanting a different behaviour for some valid 
reason are stuck with that.
This is the very reason we expose information about migration in the monitor to 
begin with.

Again, I believe the fix for this convergence problem does not belong here.

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] qemu-kvm: kvm headers update

2009-10-05 Thread Avi Kivity

On 10/05/2009 11:46 AM, Michael S. Tsirkin wrote:

Update headers from linux 2.6.32-rc1, this mainly
adds irqfd which will make it easier to add vhost,
down the line. Also reduce code duplication by including
kvm_types.h instead of copying it.

Signed-off-by: Michael S. Tsirkinm...@redhat.com
---
  kvm/include/linux/kvm.h|  131 +++--
  kvm/include/linux/kvm_para.h   |   56 +---
  kvm/include/x86/asm/kvm.h  |   46 +
  kvm/include/x86/asm/kvm_para.h |  141 +---
  4 files changed, 61 insertions(+), 313 deletions(-)

diff --git a/kvm/include/linux/kvm.h b/kvm/include/linux/kvm.h
index 801865d..2c9a209 100644
--- a/kvm/include/linux/kvm.h
+++ b/kvm/include/linux/kvm.h
@@ -1,43 +1,4 @@
-#ifndef KVM_UNIFDEF_H
-#define KVM_UNIFDEF_H
-
-#ifdef __i386__
-#ifndef CONFIG_X86_32
-#define CONFIG_X86_32 1
-#endif
-#endif
-
-#ifdef __x86_64__
-#ifndef CONFIG_X86_64
-#define CONFIG_X86_64 1
-#endif
-#endif
-
-#if defined(__i386__) || defined (__x86_64__)
-#ifndef CONFIG_X86
-#define CONFIG_X86 1
-#endif
-#endif
-
-#ifdef __ia64__
-#ifndef CONFIG_IA64
-#define CONFIG_IA64 1
-#endif
-#endif
-
-#ifdef __PPC__
-#ifndef CONFIG_PPC
-#define CONFIG_PPC 1
-#endif
-#endif
-
-#ifdef __s390__
-#ifndef CONFIG_S390
-#define CONFIG_S390 1
-#endif
-#endif
-
-#endif
   


Why are you dropping these?  Any dependencies on CONFIG_ will be broken.

Or is this the output of 'make headers_install'?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Autotest] [PATCH] Adding NAS Parallel Benchmarks(NPB) as client test [1/2] V2

2009-10-05 Thread Lucas Meneghel Rodrigues
First of all, thanks for this client side module Chen. I believe it
might be interesting for quite a lot of people doing performance
testing.

Here are my remarks found during the review of your code:

On Mon, Sep 21, 2009 at 7:01 AM, Cao, Chen k...@redhat.com wrote:
 - Using NPB OpenMP implementaion.

 Changes from the original patch:
 Remove the variant 'extra_params += smp 2', and add a restriction
 (no autotest.npb) under variant 'up', according to mgoldish's advice.
 Nevertheless, user can run this test with smp=1, if he/she wants to.

 Signed-off-by: Cao, Chen k...@redhat.com
 ---
  client/tests/kvm/kvm_tests.cfg.sample   |    4 +
  client/tests/npb/control                |   28 
  client/tests/npb/enable-all-tests.patch |  233 
 +++
  client/tests/npb/npb.py                 |   95 +
  4 files changed, 360 insertions(+), 0 deletions(-)
  create mode 100644 client/tests/npb/control
  create mode 100644 client/tests/npb/enable-all-tests.patch
  create mode 100644 client/tests/npb/npb.py

 diff --git a/client/tests/kvm/kvm_tests.cfg.sample 
 b/client/tests/kvm/kvm_tests.cfg.sample
 index 285a38f..b03b7cd 100644
 --- a/client/tests/kvm/kvm_tests.cfg.sample
 +++ b/client/tests/kvm/kvm_tests.cfg.sample
 @@ -90,6 +90,9 @@ variants:
             - disktest:
                 test_name = disktest
                 test_control_file = disktest.control
 +            - npb:
 +                test_name = npb
 +                test_control_file = npb.control

     - linux_s3:     install setup
         type = linux_s3
 @@ -567,6 +570,7 @@ linux_s3:

  variants:
     - @up:
 +        no autotest.npb
     - smp2:
         extra_params +=  -smp 2

 diff --git a/client/tests/npb/control b/client/tests/npb/control
 new file mode 100644
 index 000..d6c2adb
 --- /dev/null
 +++ b/client/tests/npb/control
 @@ -0,0 +1,28 @@
 +NAME = NAS Parallel Benchmarks
 +AUTHOR = Cao, Chen k...@redhat.com
 +TEST_TYPE = CLIENT
 +TEST_CLASS = HARDWARE
 +TEST_CATEGORY = BENCHMARK
 +TIME = MEDIUM
 +DOC = \
 +Using NPB, OpenMP implementation.
 +
 +See http://www.nas.nasa.gov/Software/NPB/
 +
 +
 +# Supported tests (benchmarks):
 +#   bt.A bt.B bt.C bt.D bt.E bt.S bt.W
 +#   cg.A cg.B cg.C cg.S cg.W
 +#   dc.A dc.B dc.S dc.W
 +#   ep.A ep.B ep.C ep.D ep.E ep.S ep.W
 +#   ft.A ft.B ft.S ft.W
 +#   is.A is.B is.C is.S is.W
 +#   lu.A lu.B lu.C lu.S lu.W
 +#   mg.A mg.B mg.S mg.W
 +#   sp.A sp.B sp.C sp.D sp.E sp.S sp.W
 +#   ua.A ua.B ua.C ua.S ua.W
 +#
 +# Please refer to npb.py for more infomation about
 +# the arguments.
 +job.run_test(url='npb', tests='ep.A ep.B', ratio=0.5)
 +
 diff --git a/client/tests/npb/enable-all-tests.patch 
 b/client/tests/npb/enable-all-tests.patch
 new file mode 100644
 index 000..f08a9d3
 --- /dev/null
 +++ b/client/tests/npb/enable-all-tests.patch
 @@ -0,0 +1,233 @@
 +diff --git a/NPB3.3-OMP/config/make.def b/NPB3.3-OMP/config/make.def
 +new file mode 100644
 +index 000..afffe7d
 +--- /dev/null
  b/NPB3.3-OMP/config/make.def
 +@@ -0,0 +1,161 @@
 ++#---
 ++#
 ++#                SITE- AND/OR PLATFORM-SPECIFIC DEFINITIONS.
 ++#
 ++#---
 ++
 ++#---
 ++# Items in this file will need to be changed for each platform.
 ++#---
 ++
 ++#---
 ++# Parallel Fortran:
 ++#
 ++# For CG, EP, FT, MG, LU, SP, BT and UA, which are in Fortran, the following
 ++# must be defined:
 ++#
 ++# F77        - Fortran compiler
 ++# FFLAGS     - Fortran compilation arguments
 ++# F_INC      - any -I arguments required for compiling Fortran
 ++# FLINK      - Fortran linker
 ++# FLINKFLAGS - Fortran linker arguments
 ++# F_LIB      - any -L and -l arguments required for linking Fortran
 ++#
 ++# compilations are done with $(F77) $(F_INC) $(FFLAGS) or
 ++#                            $(F77) $(FFLAGS)
 ++# linking is done with       $(FLINK) $(F_LIB) $(FLINKFLAGS)
 ++#---
 ++
 ++#---
 ++# This is the fortran compiler used for Fortran programs
 ++#---
 ++F77 = gfortran
 ++# This links fortran programs; usually the same as ${F77}
 ++FLINK = $(F77)
 ++
 ++#---
 ++# These macros are passed to the linker
 ++#---
 ++F_LIB  =
 ++
 ++#---
 ++# These macros are passed to the compiler
 

[PATCH] Refactor of setup_modules.py

2009-10-05 Thread Lucas Meneghel Rodrigues
Currently client/setup_modules.py has an undefined variable error
that was spot by Amos Kong ak...@redhat.com, and a simple fix
was proposed. (see http://patchwork.test.kernel.org/patch/1315/)

It seems to me that in a previous refactor of setup_modules, some
code was moved but not all references were properly fixed.

Amos's fix, while fine, introduces global variables to accomplish
the goal and didn't fix another undefined reference that was lying
around. As in general it's advisable to avoid resorting to global
namespacing, I propose another fix: Encapsulate the library
namespace setup into a class, and make setup() just a delegate
of the setup() method inside that class. This way it's possible
to encapsulate code and use class attributes to avoid using
global variables.

Since I was there I modified the docstrings of that file to comply
with the docstring guideline.

John, please let me know what you think about this.

Complexity: Medium (Code refactor, all public API preserved)
Risk: High (Core code executed by the client). Change was tested though :)

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/setup_modules.py |  306 --
 1 files changed, 186 insertions(+), 120 deletions(-)

diff --git a/client/setup_modules.py b/client/setup_modules.py
index dc255c4..188114c 100644
--- a/client/setup_modules.py
+++ b/client/setup_modules.py
@@ -1,4 +1,9 @@
-__author__ = jadman...@google.com (John Admanski)
+
+Utility library used for autotest library namespace configuration.
+
+...@copyright Google 2007-2009
+...@author John Admanski jadman...@google.com
+
 
 import os, sys
 
@@ -12,134 +17,195 @@ check_version.check_python_version()
 
 import new, glob, traceback
 
+class namespace_config(object):
+
+Class to perform namespace configuration for autotest.
 
-def _create_module(name):
-Create a single top-level module
-module = new.module(name)
-sys.modules[name] = module
-return module
-
-
-def _create_module_and_parents(name):
-Create a module, and all the necessary parents
-parts = name.split(.)
-# first create the top-level module
-parent = _create_module(parts[0])
-created_parts = [parts[0]]
-parts.pop(0)
-# now, create any remaining child modules
-while parts:
-child_name = parts.pop(0)
-module = new.module(child_name)
-setattr(parent, child_name, module)
-created_parts.append(child_name)
-sys.modules[..join(created_parts)] = module
-parent = module
-
-
-def _import_children_into_module(parent_module_name, path):
-Import all the packages on a path into a parent module
-# find all the packages at 'path'
-names = []
-for filename in os.listdir(path):
-full_name = os.path.join(path, filename)
-if not os.path.isdir(full_name):
-continue   # skip files
-if . in filename:
-continue   # if . is in the name it's not a valid package name
-if not os.access(full_name, os.R_OK | os.X_OK):
-continue   # need read + exec access to make a dir importable
-if __init__.py in os.listdir(full_name):
-names.append(filename)
-# import all the packages and insert them into 'parent_module'
-sys.path.insert(0, path)
-for name in names:
-module = __import__(name)
-# add the package to the parent
-parent_module = sys.modules[parent_module_name]
-setattr(parent_module, name, module)
-full_name = parent_module_name + . + name
-sys.modules[full_name] = module
-# restore the system path
-sys.path.pop(0)
+This is what makes possible to use statements like
+from autotest_lib.client.bin import package
+
+def __init__(self, base_path, root_module_name=):
+
+Perform all the necessary setup so that all the packages at
+'base_path' can be imported via import root_module_name.package.
+If root_module_name is empty, then all the packages at base_path
+are inserted as top-level packages.
+
+Also, setup all the common.* aliases for modules in the common
+library.
+
+The setup must be different if you are running on an Autotest server
+or on a test machine that just has the client directories installed.
+
+@param base_path: Path used to compute all relative paths needed for
+the setup.
+@param root_module_name: Top level name of the 'virtual' library
+namespace.
+
+# Hack... Any better ideas?
+if (root_module_name == 'autotest_lib.client' and
+os.path.exists(os.path.join(os.path.dirname(__file__),
+'..', 'server'))):
+root_module_name = 'autotest_lib'
+base_path = os.path.abspath(os.path.join(base_path, '..'))
+self.root_module_name = root_module_name
+self.base_path = 

Re: [Autotest] [KVM-AUTOTEST PATCH] fix an error of undefine variable

2009-10-05 Thread Lucas Meneghel Rodrigues
Hi Amos, thanks for the fix! It is fine, but to avoid introducing
global statements, I've worked out another patch to fix that issue.
Also, there was another undefined variable lying around (base_path).
I've finished up the patch and sent it to the mailing list!

Cheers,

On Tue, Sep 22, 2009 at 11:32 PM, Amos Kong ak...@redhat.com wrote:

 Sorry for lost autot...@test.kernel.org. So send again. Thanks.
 --
 fix an error of undefine variable

 09/23 09:50:27 ERROR| traceback:0013|   File 
 /usr/lib64/python2.4/logging/__init__.py, line 744, in emit
 09/23 09:50:27 ERROR| traceback:0013|     self.handleError(record)
 09/23 09:50:27 ERROR| traceback:0013|   File 
 /root/project/autotest/client/setup_modules.py, line 86, in 
 _autotest_logging_handle_error
 09/23 09:50:27 ERROR| traceback:0013|     traceback.print_stack()
 09/23 09:50:27 ERROR|setup_modu:0087| Future logging formatting exceptions 
 disabled.
 09/23 09:50:27 ERROR|       kvm:0067| Test failed: global name 
 'root_module_name' is not defined

 Signed-off-by: Amos Kong ak...@redhat.com
 ---
  client/setup_modules.py |    4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)

 diff --git a/client/setup_modules.py b/client/setup_modules.py
 index dc255c4..8b3200b 100644
 --- a/client/setup_modules.py
 +++ b/client/setup_modules.py
 @@ -86,7 +86,7 @@ def _autotest_logging_handle_error(self, record):
         traceback.print_stack()
         sys.stderr.write('Future logging formatting exceptions disabled.\n')

 -    if root_module_name == 'autotest_lib':
 +    if global_root_module_name == 'autotest_lib':
         # Allow locally installed third party packages to be found
         # before any that are installed on the system itself when not.
         # running as a client.
 @@ -124,6 +124,8 @@ def setup(base_path, root_module_name=):
     The setup must be different if you are running on an Autotest server
     or on a test machine that just has the client directories installed.
     
 +    global global_root_module_name
 +    global_root_module_name = root_module_name
     # Hack... Any better ideas?
     if (root_module_name == 'autotest_lib.client' and
         os.path.exists(os.path.join(os.path.dirname(__file__),
 --
 1.5.5.6

 ___
 Autotest mailing list
 autot...@test.kernel.org
 http://test.kernel.org/cgi-bin/mailman/listinfo/autotest




-- 
Lucas
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] Fixing undefined variable reference error in setup_modules.py

2009-10-05 Thread Lucas Meneghel Rodrigues
Fix an undefined variable reference error inside
client/setup_modules.py - just remove code that wasn't
supposed to be there.

3rd try: After IRC chat and investigation, this is a
best fix than my previous patch that tried to accomplish
the same fix.

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/setup_modules.py |8 
 1 files changed, 0 insertions(+), 8 deletions(-)

diff --git a/client/setup_modules.py b/client/setup_modules.py
index dc255c4..6f2ea30 100644
--- a/client/setup_modules.py
+++ b/client/setup_modules.py
@@ -86,14 +86,6 @@ def _autotest_logging_handle_error(self, record):
 traceback.print_stack()
 sys.stderr.write('Future logging formatting exceptions disabled.\n')
 
-if root_module_name == 'autotest_lib':
-# Allow locally installed third party packages to be found
-# before any that are installed on the system itself when not.
-# running as a client.
-# This is primarily for the benefit of frontend and tko so that they
-# may use libraries other than those available as system packages.
-sys.path.insert(0, os.path.join(base_path, site-packages))
-
 
 def _monkeypatch_logging_handle_error():
 # Hack out logging.py*
-- 
1.6.2.5

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Refactor of setup_modules.py

2009-10-05 Thread John Admanski
Here's what I had in mind.

-- John

On Mon, Oct 5, 2009 at 11:20 AM, John Admanski jadman...@google.com wrote:

 After some investigation and irc chat, it has become clear that this 
 misbehaving chunk of code in _autotest_logging_handle_error is in fact an 
 error; the patch that was applied in rev 3649 in subversion added a chunk of 
 the code from setup to the end of _autotest_logging_handle_error, which 
 doesn't work very well since it references a variable local to setup.
 The simpler solution would be to just delete that chunk of code.

 -- John

 On Mon, Oct 5, 2009 at 10:40 AM, Lucas Meneghel Rodrigues l...@redhat.com 
 wrote:

 Currently client/setup_modules.py has an undefined variable error
 that was spot by Amos Kong ak...@redhat.com, and a simple fix
 was proposed. (see http://patchwork.test.kernel.org/patch/1315/)

 It seems to me that in a previous refactor of setup_modules, some
 code was moved but not all references were properly fixed.

 Amos's fix, while fine, introduces global variables to accomplish
 the goal and didn't fix another undefined reference that was lying
 around. As in general it's advisable to avoid resorting to global
 namespacing, I propose another fix: Encapsulate the library
 namespace setup into a class, and make setup() just a delegate
 of the setup() method inside that class. This way it's possible
 to encapsulate code and use class attributes to avoid using
 global variables.

 Since I was there I modified the docstrings of that file to comply
 with the docstring guideline.

 John, please let me know what you think about this.

 Complexity: Medium (Code refactor, all public API preserved)
 Risk: High (Core code executed by the client). Change was tested though :)

 Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
 ---
  client/setup_modules.py |  306 
 --
  1 files changed, 186 insertions(+), 120 deletions(-)

 diff --git a/client/setup_modules.py b/client/setup_modules.py
 index dc255c4..188114c 100644
 --- a/client/setup_modules.py
 +++ b/client/setup_modules.py
 @@ -1,4 +1,9 @@
 -__author__ = jadman...@google.com (John Admanski)
 +
 +Utility library used for autotest library namespace configuration.
 +
 +...@copyright Google 2007-2009
 +...@author John Admanski jadman...@google.com
 +

  import os, sys

 @@ -12,134 +17,195 @@ check_version.check_python_version()

  import new, glob, traceback

 +class namespace_config(object):
 +    
 +    Class to perform namespace configuration for autotest.

 -def _create_module(name):
 -    Create a single top-level module
 -    module = new.module(name)
 -    sys.modules[name] = module
 -    return module
 -
 -
 -def _create_module_and_parents(name):
 -    Create a module, and all the necessary parents
 -    parts = name.split(.)
 -    # first create the top-level module
 -    parent = _create_module(parts[0])
 -    created_parts = [parts[0]]
 -    parts.pop(0)
 -    # now, create any remaining child modules
 -    while parts:
 -        child_name = parts.pop(0)
 -        module = new.module(child_name)
 -        setattr(parent, child_name, module)
 -        created_parts.append(child_name)
 -        sys.modules[..join(created_parts)] = module
 -        parent = module
 -
 -
 -def _import_children_into_module(parent_module_name, path):
 -    Import all the packages on a path into a parent module
 -    # find all the packages at 'path'
 -    names = []
 -    for filename in os.listdir(path):
 -        full_name = os.path.join(path, filename)
 -        if not os.path.isdir(full_name):
 -            continue   # skip files
 -        if . in filename:
 -            continue   # if . is in the name it's not a valid package name
 -        if not os.access(full_name, os.R_OK | os.X_OK):
 -            continue   # need read + exec access to make a dir importable
 -        if __init__.py in os.listdir(full_name):
 -            names.append(filename)
 -    # import all the packages and insert them into 'parent_module'
 -    sys.path.insert(0, path)
 -    for name in names:
 -        module = __import__(name)
 -        # add the package to the parent
 -        parent_module = sys.modules[parent_module_name]
 -        setattr(parent_module, name, module)
 -        full_name = parent_module_name + . + name
 -        sys.modules[full_name] = module
 -    # restore the system path
 -    sys.path.pop(0)
 +    This is what makes possible to use statements like
 +    from autotest_lib.client.bin import package
 +    
 +    def __init__(self, base_path, root_module_name=):
 +        
 +        Perform all the necessary setup so that all the packages at
 +        'base_path' can be imported via import root_module_name.package.
 +        If root_module_name is empty, then all the packages at base_path
 +        are inserted as top-level packages.
 +
 +        Also, setup all the common.* aliases for modules in the common
 +        library.
 +
 +        The setup must be different if you are 

Re: [Autotest] [PATCH] Fix a bug in function create in kvm_vm

2009-10-05 Thread Lucas Meneghel Rodrigues
Hi Yolkfull! I've checked your patch, but it turns out that the comma
is valid syntax for the logging module. By any chance you actually had
an error with it?

On Mon, Sep 28, 2009 at 4:45 AM, Yolkfull Chow yz...@redhat.com wrote:

 Signed-off-by: Yolkfull Chow yz...@redhat.com
 ---
  client/tests/kvm/kvm_vm.py |    2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

 diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
 index 55220f9..8ae 100755
 --- a/client/tests/kvm/kvm_vm.py
 +++ b/client/tests/kvm/kvm_vm.py
 @@ -406,7 +406,7 @@ class VM:
                               self.process.get_output()))
                 return False

 -            logging.debug(VM appears to be alive with PID %d,
 +            logging.debug(VM appears to be alive with PID %d %
                           self.process.get_pid())
             return True

 --
 1.6.2.5

 ___
 Autotest mailing list
 autot...@test.kernel.org
 http://test.kernel.org/cgi-bin/mailman/listinfo/autotest




-- 
Lucas
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Autotest] [KVM-AUTOTEST PATCH 1/6] KVM test: kvm_subprocess: use read_nonblocking(0) instead of read_nonblocking (0.1)

2009-10-05 Thread Lucas Meneghel Rodrigues
Patchset applied!

On Tue, Sep 29, 2009 at 5:04 PM, Michael Goldish mgold...@redhat.com wrote:
 In get_command_status_output() and is_responsive() use read_nonblocking(0) to
 read the unread output before sending input (e.g. a command).
 The timeout is currently 0.1 because theoretically it should help if the guest
 still produces output when the function is called, but in practice there's no
 guarantee that a value of 0.1 will suffice.  Therefore, it is be the user's
 responsibility to make sure the guest stopped producing output before
 get_command_status_output() is called.  This can be guaranteed (in most cases)
 by using get_command_status_output() and friends instead of sendline() to send
 commands (because the former waits for the prompt to return, whereas the 
 latter
 returns immediately).

 Signed-off-by: Michael Goldish mgold...@redhat.com
 ---
  client/tests/kvm/kvm_subprocess.py |    4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)

 diff --git a/client/tests/kvm/kvm_subprocess.py 
 b/client/tests/kvm/kvm_subprocess.py
 index 424c801..730f20e 100755
 --- a/client/tests/kvm/kvm_subprocess.py
 +++ b/client/tests/kvm/kvm_subprocess.py
 @@ -1028,7 +1028,7 @@ class kvm_shell_session(kvm_expect):
         
         # Read all output that's waiting to be read, to make sure the output
         # we read next is in response to the newline sent
 -        self.read_nonblocking(timeout=0.1)
 +        self.read_nonblocking(timeout=0)
         # Send a newline
         self.sendline()
         # Wait up to timeout seconds for some output from the child
 @@ -1095,7 +1095,7 @@ class kvm_shell_session(kvm_expect):
         logging.debug(Sending command: %s % command)

         # Read everything that's waiting to be read
 -        self.read_nonblocking(0.1)
 +        self.read_nonblocking(timeout=0)

         # Send the command and get its output
         self.sendline(command)
 --
 1.5.4.1

 ___
 Autotest mailing list
 autot...@test.kernel.org
 http://test.kernel.org/cgi-bin/mailman/listinfo/autotest




-- 
Lucas
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Autotest] [KVM-AUTOTEST PATCH 1/4] KVM test: allow setting shell line separator string in the config file

2009-10-05 Thread Lucas Meneghel Rodrigues
Patchset applied!

On Sun, Sep 20, 2009 at 12:16 PM, Michael Goldish mgold...@redhat.com wrote:
 The shell line separator string is appended to strings sent by sendline().

 The string is controlled by the parameter shell_linesep.  It defaults to \n.

 Signed-off-by: Michael Goldish mgold...@redhat.com
 ---
  client/tests/kvm/kvm_tests.cfg.sample |    1 +
  client/tests/kvm/kvm_utils.py         |   12 ++--
  client/tests/kvm/kvm_vm.py            |    7 ---
  3 files changed, 11 insertions(+), 9 deletions(-)

 diff --git a/client/tests/kvm/kvm_tests.cfg.sample 
 b/client/tests/kvm/kvm_tests.cfg.sample
 index 38f5a5a..540d0a2 100644
 --- a/client/tests/kvm/kvm_tests.cfg.sample
 +++ b/client/tests/kvm/kvm_tests.cfg.sample
 @@ -364,6 +364,7 @@ variants:
         shell_prompt = ^\w:\\.*\s*$
         username = Administrator
         password = 123456
 +        shell_linesep = \r\n
         shell_client = nc
         shell_port = 22
         # File transfers are currently unsupported
 diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
 index 88299be..53b664a 100644
 --- a/client/tests/kvm/kvm_utils.py
 +++ b/client/tests/kvm/kvm_utils.py
 @@ -576,7 +576,7 @@ def scp_from_remote(host, port, username, password, 
 remote_path, local_path,
     return remote_scp(command, password, timeout)


 -def ssh(host, port, username, password, prompt, timeout=10):
 +def ssh(host, port, username, password, prompt, linesep=\n, timeout=10):
     
     Log into a remote host (guest) using SSH.

 @@ -591,10 +591,10 @@ def ssh(host, port, username, password, prompt, 
 timeout=10):
     
     command = (ssh -o UserKnownHostsFile=/dev/null -p %s %...@%s %
                (port, username, host))
 -    return remote_login(command, password, prompt, \n, timeout)
 +    return remote_login(command, password, prompt, linesep, timeout)


 -def telnet(host, port, username, password, prompt, timeout=10):
 +def telnet(host, port, username, password, prompt, linesep=\n, timeout=10):
     
     Log into a remote host (guest) using Telnet.

 @@ -608,10 +608,10 @@ def telnet(host, port, username, password, prompt, 
 timeout=10):
     @return: kvm_spawn object on success and None on failure.
     
     command = telnet -l %s %s %s % (username, host, port)
 -    return remote_login(command, password, prompt, \r\n, timeout)
 +    return remote_login(command, password, prompt, linesep, timeout)


 -def netcat(host, port, username, password, prompt, timeout=10):
 +def netcat(host, port, username, password, prompt, linesep=\n, timeout=10):
     
     Log into a remote host (guest) using Netcat.

 @@ -625,7 +625,7 @@ def netcat(host, port, username, password, prompt, 
 timeout=10):
     @return: kvm_spawn object on success and None on failure.
     
     command = nc %s %s % (host, port)
 -    return remote_login(command, password, prompt, \n, timeout)
 +    return remote_login(command, password, prompt, linesep, timeout)


  # The following are utility functions related to ports.
 diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
 index 55220f9..07ceb6d 100755
 --- a/client/tests/kvm/kvm_vm.py
 +++ b/client/tests/kvm/kvm_vm.py
 @@ -669,6 +669,7 @@ class VM:
         username = self.params.get(username, )
         password = self.params.get(password, )
         prompt = self.params.get(shell_prompt, [\#\$])
 +        linesep = eval('%s' % self.params.get(shell_linesep, r\n))
         client = self.params.get(shell_client)
         address = self.get_address(nic_index)
         port = self.get_port(int(self.params.get(shell_port)))
 @@ -679,13 +680,13 @@ class VM:

         if client == ssh:
             session = kvm_utils.ssh(address, port, username, password,
 -                                    prompt, timeout)
 +                                    prompt, linesep, timeout)
         elif client == telnet:
             session = kvm_utils.telnet(address, port, username, password,
 -                                       prompt, timeout)
 +                                       prompt, linesep, timeout)
         elif client == nc:
             session = kvm_utils.netcat(address, port, username, password,
 -                                       prompt, timeout)
 +                                       prompt, linesep, timeout)

         if session:
             session.set_status_test_command(self.params.get(status_test_
 --
 1.5.4.1

 ___
 Autotest mailing list
 autot...@test.kernel.org
 http://test.kernel.org/cgi-bin/mailman/listinfo/autotest




-- 
Lucas
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] qemu-kvm: kvm headers update

2009-10-05 Thread Michael S. Tsirkin
On Mon, Oct 05, 2009 at 05:38:35PM +0200, Avi Kivity wrote:
 On 10/05/2009 11:46 AM, Michael S. Tsirkin wrote:
 Update headers from linux 2.6.32-rc1, this mainly
 adds irqfd which will make it easier to add vhost,
 down the line. Also reduce code duplication by including
 kvm_types.h instead of copying it.

 Signed-off-by: Michael S. Tsirkinm...@redhat.com
 ---
   kvm/include/linux/kvm.h|  131 +++--
   kvm/include/linux/kvm_para.h   |   56 +---
   kvm/include/x86/asm/kvm.h  |   46 +
   kvm/include/x86/asm/kvm_para.h |  141 
 +---
   4 files changed, 61 insertions(+), 313 deletions(-)

 diff --git a/kvm/include/linux/kvm.h b/kvm/include/linux/kvm.h
 index 801865d..2c9a209 100644
 --- a/kvm/include/linux/kvm.h
 +++ b/kvm/include/linux/kvm.h
 @@ -1,43 +1,4 @@
 -#ifndef KVM_UNIFDEF_H
 -#define KVM_UNIFDEF_H
 -
 -#ifdef __i386__
 -#ifndef CONFIG_X86_32
 -#define CONFIG_X86_32 1
 -#endif
 -#endif
 -
 -#ifdef __x86_64__
 -#ifndef CONFIG_X86_64
 -#define CONFIG_X86_64 1
 -#endif
 -#endif
 -
 -#if defined(__i386__) || defined (__x86_64__)
 -#ifndef CONFIG_X86
 -#define CONFIG_X86 1
 -#endif
 -#endif
 -
 -#ifdef __ia64__
 -#ifndef CONFIG_IA64
 -#define CONFIG_IA64 1
 -#endif
 -#endif
 -
 -#ifdef __PPC__
 -#ifndef CONFIG_PPC
 -#define CONFIG_PPC 1
 -#endif
 -#endif
 -
 -#ifdef __s390__
 -#ifndef CONFIG_S390
 -#define CONFIG_S390 1
 -#endif
 -#endif
 -
 -#endif


 Why are you dropping these?  Any dependencies on CONFIG_ will be broken.

Instead I included kvm_types which has these.


 Or is this the output of 'make headers_install'?

yes. I just added include to kvm_types to add the CONFIG_ stuff.
Or maybe instead we should fix users to depend on the correct ifdefs.

 -- 
 error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[patch 0/2] kvm: test: parallel vmexit (v2)

2009-10-05 Thread Marcelo Tosatti
Addressing comments.


--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[patch 2/2] test: vmexit: run parallel tests on each cpu

2009-10-05 Thread Marcelo Tosatti
So one can measure SMP overhead.

Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

Index: qemu-kvm-parallel-vmexit/kvm/user/test/x86/vmexit.c
===
--- qemu-kvm-parallel-vmexit.orig/kvm/user/test/x86/vmexit.c
+++ qemu-kvm-parallel-vmexit/kvm/user/test/x86/vmexit.c
@@ -80,22 +80,38 @@ static struct test {
void (*func)(void);
const char *name;
int (*valid)(void);
+   int parallel;
 } tests[] = {
-   { cpuid, cpuid, },
-   { vmcall, vmcall, },
-   { mov_from_cr8, mov_from_cr8 },
-   { mov_to_cr8, mov_to_cr8 },
-   { ipi, ipi, is_smp },
-   { ipi_halt, ipi+halt, is_smp },
+   { cpuid, cpuid, .parallel = 1,  },
+   { vmcall, vmcall, .parallel = 1, },
+   { mov_from_cr8, mov_from_cr8, .parallel = 1, },
+   { mov_to_cr8, mov_to_cr8 , .parallel = 1, },
+   { ipi, ipi, is_smp, .parallel = 0, },
+   { ipi_halt, ipi+halt, is_smp, .parallel = 0, },
 };
 
+unsigned iterations;
+volatile int nr_cpus_done;
+
+static void run_test(void *_func)
+{
+int i;
+void (*func)(void) = _func;
+
+for (i = 0; i  iterations; ++i)
+func();
+
+nr_cpus_done++;
+}
+
 static void do_test(struct test *test)
 {
int i;
unsigned long long t1, t2;
-   unsigned iterations = 32;
 void (*func)(void) = test-func;
 
+iterations = 32;
+
 if (test-valid  !test-valid()) {
printf(%s (skipped)\n, test-name);
return;
@@ -104,8 +120,17 @@ static void do_test(struct test *test)
do {
iterations *= 2;
t1 = rdtsc();
-   for (i = 0; i  iterations; ++i)
-   func();
+
+   if (!test-parallel) {
+   for (i = 0; i  iterations; ++i)
+   func();
+   } else {
+   nr_cpus_done = 0;
+   for (i = cpu_count(); i  0; i--)
+   on_cpu_async(i-1, run_test, func);
+   while (nr_cpus_done  cpu_count())
+   ;
+   }
t2 = rdtsc();
} while ((t2 - t1)  GOAL);
printf(%s %d\n, test-name, (int)((t2 - t1) / iterations));


--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[patch 1/2] test: add on_cpu_async

2009-10-05 Thread Marcelo Tosatti
Non-wait version of on_cpu.

Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

Index: qemu-kvm-parallel-vmexit/kvm/user/test/lib/x86/smp.c
===
--- qemu-kvm-parallel-vmexit.orig/kvm/user/test/lib/x86/smp.c
+++ qemu-kvm-parallel-vmexit/kvm/user/test/lib/x86/smp.c
@@ -82,24 +82,38 @@ static void setup_smp_id(void *data)
 asm (mov %0, %%gs:0 : : r(apic_id()) : memory);
 }
 
-void on_cpu(int cpu, void (*function)(void *data), void *data)
+static void __on_cpu(int cpu, void (*function)(void *data), void *data,
+ int wait)
 {
 spin_lock(ipi_lock);
 if (cpu == smp_id())
function(data);
 else {
+   ipi_done = 0;
ipi_function = function;
ipi_data = data;
apic_icr_write(APIC_INT_ASSERT | APIC_DEST_PHYSICAL | APIC_DM_FIXED
| IPI_VECTOR,
cpu);
-   while (!ipi_done)
-   ;
-   ipi_done = 0;
+   if (wait) {
+   while (!ipi_done)
+   ;
+   }
 }
 spin_unlock(ipi_lock);
 }
 
+void on_cpu(int cpu, void (*function)(void *data), void *data)
+{
+__on_cpu(cpu, function, data, 1);
+}
+
+void on_cpu_async(int cpu, void (*function)(void *data), void *data)
+{
+__on_cpu(cpu, function, data, 0);
+}
+
+
 void smp_init(void)
 {
 int i;
Index: qemu-kvm-parallel-vmexit/kvm/user/test/lib/x86/smp.h
===
--- qemu-kvm-parallel-vmexit.orig/kvm/user/test/lib/x86/smp.h
+++ qemu-kvm-parallel-vmexit/kvm/user/test/lib/x86/smp.h
@@ -10,6 +10,7 @@ void smp_init(void);
 int cpu_count(void);
 int smp_id(void);
 void on_cpu(int cpu, void (*function)(void *data), void *data);
+void on_cpu_async(int cpu, void (*function)(void *data), void *data);
 void spin_lock(struct spinlock *lock);
 void spin_unlock(struct spinlock *lock);
 


--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Some KVM benchmarks/ IO question

2009-10-05 Thread RW
I've currently a HP DL 380 G6 server which I can use for some benchmarks
I want to share with you. Maybe someone find it interesting or usefull.
Both host and guest running Gentoo with kernel 2.6.31.1 (which is stable
2.6.31 update 1). The host have the following components:

2 x Intel Xeon CPU L5520 - 2 Quad-Processors (static performance, VT-d,
Hyperthreading enabled in BIOS)
8 x 300 GB SAS 10k hard drives (RAID 10)
24 GB RAM

(Some) Host (kernel) settings:
I/O scheduler: deadline
Filesystem: xfs
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
CONFIG_VIRTIO_BLK=m
CONFIG_VIRTIO_NET=m
CONFIG_VIRTIO_CONSOLE=m
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_VIRTIO=m
CONFIG_VIRTIO_RING=m
CONFIG_VIRTIO_PCI=m
CONFIG_VIRTIO_BALLOON=m

(Some) Guest (kernel) settings:
I/O scheduler: deadline/cfq (see below)
Filesystem: ext3 (datamode=ordered/writeback [see below))
VIRTIO network and block driver used
Guest is a qcow2-image (which was expanded with dd to be big
enough for the IO tests so that growing the image doesn't happen
during the IO test.
CONFIG_KVM_CLOCK=y
CONFIG_KVM_GUEST=y
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=m
CONFIG_VIRTIO_CONSOLE=y
CONFIG_HW_RANDOM_VIRTIO=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_PARAVIRT_SPINLOCKS=y

KVM startup options (the imported ones):
-m variable-see_below
-smp variable-see_below
-cpu host
-daemonize
-drive file=/data/kvm/kvmimages/gfs1.qcow2,if=virtio,boot=on
-net nic,vlan=104,model=virtio,macaddr=00:ff:48:23:45:4b
-net tap,vlan=104,ifname=tap.b.gfs1,script=no
-net nic,vlan=96,model=virtio,macaddr=00:ff:48:23:45:4d
-net tap,vlan=96,ifname=tap.f.gfs1,script=no

Since we use KVM mostly for webserver stuff I've run tests with
Apachebench, Graphics Magick and IOzone to simulate our workload as good
as we can.

Apachebench with 4 GB RAM, CFQ scheduler, 1/2/4/8 vProcs compared with
the host (the hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-3428/
Apachebench with 2 GB RAM, CFQ scheduler, 1/2/4/8 vProcs compared with
the host (the hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-6053/
I was interested if more RAM will give more throughput and if more
vProcs will scale. As you can see 8 vProcs (compared to 2xQuadCore+HT)
will give almost the same requests per second in KVM as the host alone
can deliver. RAM doesn't matter in this case.

Graphics Magick resize with 4 GB RAM, CFQ scheduler, 1/2/4/8 vProcs
compared with the host (the hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-5214/
Graphics Magick resize with 2 GB RAM, CFQ scheduler, 1/2/4/8 vProcs
compared with the host (the hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-7186/
With 8 vProcs the KVM runs about 10% slower. RAM doesn't seem to matter
again here.

The following IO tests run with the KVM option cache=none:
IOzone write test with 2 GB RAM, CFQ scheduler, datamode=ordered,
1/2/4/8 vProcs compared with the host (the hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-3564/
IOzone write test with 2 GB RAM, deadline scheduler, datamode=ordered,
1/2/4/8 vProcs compared with the host (the hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-4533/
It's interesting to see that the deadline scheduler gives you about
10-15 MByte/s more throughput. Please note that I haven't checked the
CPU usage during the IO tests.

The following IO tests run with no cache option set so this defaults
to writethrough:
IOzone write test with 2 GB RAM, deadline scheduler, datamode ordered
and writeback (see testpage), 8 vProcs compared with the hoste (the
hp380g6 graph):
http://www.tauceti.net/kvm-benchmarks/merge-7526/

All in all for this tests it seems that a KVM can perform almost as good
as the host. I haven't tested the network throughput but it works quite
well if I copy big files with ssh and/or rsync but I have no numbers yet.

I'm using KVM for 2 years now and 1 year in production and I'm satisfied
with it. Thanks to all developers for their work! But what is still
confusing to me is the IO thing. I've had two times where new releases
due to a bug destroyed some data but it was not obvious to me that it
was a bug and if it was in KVM (with new kernel release) or Qemu. So
I've decided to start KVM's always with cache=none and use ext3 as
filesystem with datamode=ordered. Now the benchmark above shows quite
impressive results with the KVM default option cache=writethrough but
due to the things happend to me in the past I really fear very much to
switch from cache=none to cache=writethrough because of data integrity.

I know that there was some discussion some time ago but can some of the
developers confirm that the following combination saves the data
integrity of the guest filesystems and the integrity of the qcow2 image
as a whole in case of a crash e.g.:

Host: Filessystem xfs for KVM qcow2 images
AND Guest: cache=writethrough, virtio block driver, Filesystems ext3
with 

KVM Support for windows

2009-10-05 Thread Saksena, Abhishek

I would like to know is there a plan for KVM to be supported under Windows(host 
OS)? If yes, by when?


-Thanks
Abhishek
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


bad? encoding for KVM_SET_IRQCHIP

2009-10-05 Thread Robert Henry
I am concerned about an apparent misencoding for ioctl KVM_SET_IRQCHIP

kvm/include/linux/kvm.h defines KVM_SET_IRQCHIP as

#define KVM_SET_IRQCHIP   _IOR(KVMIO,  0x63, struct kvm_irqchip)

But the verb SET in the ioctl name would imply that the ioctl
encoding constructor should be _IOW, not _IOR.

Note also that KVM_SET_PIT seems to have the same encoding issue.

Robert Henry
r...@google.com
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 2/4] KVM: introduce xinterface API for external interaction with guests

2009-10-05 Thread Gregory Haskins
Hi Marcelo!

Marcelo Tosatti wrote:
 On Fri, Oct 02, 2009 at 04:19:27PM -0400, Gregory Haskins wrote:
 What: xinterface is a mechanism that allows kernel modules external to
 the kvm.ko proper to interface with a running guest.  It accomplishes
 this by creating an abstracted interface which does not expose any
 private details of the guest or its related KVM structures, and provides
 a mechanism to find and bind to this interface at run-time.

 Why: There are various subsystems that would like to interact with a KVM
 guest which are ideally suited to exist outside the domain of the kvm.ko
 core logic. For instance, external pci-passthrough, virtual-bus, and
 virtio-net modules are currently under development.  In order for these
 modules to successfully interact with the guest, they need, at the very
 least, various interfaces for signaling IO events, pointer translation,
 and possibly memory mapping.

 The signaling case is covered by the recent introduction of the
 irqfd/ioeventfd mechanisms.  This patch provides a mechanism to cover the
 other cases.  Note that today we only expose pointer-translation related
 functions, but more could be added at a future date as needs arise.

 Example usage: QEMU instantiates a guest, and an external module foo
 that desires the ability to interface with the guest (say via
 open(/dev/foo)).  QEMU may then pass the kvmfd to foo via an
 ioctl, such as: ioctl(foofd, FOO_SET_VMID, kvmfd).  Upon receipt, the
 foo module can issue kvm_xinterface_bind(kvmfd) to acquire
 the proper context.  Internally, the struct kvm* and associated
 struct module* will remain pinned at least until the foo module calls
 kvm_xinterface_put().
 
 --- /dev/null
 +++ b/virt/kvm/xinterface.c
 @@ -0,0 +1,409 @@
 +/*
 + * KVM module interface - Allows external modules to interface with a guest
 + *
 + * Copyright 2009 Novell.  All Rights Reserved.
 + *
 + * Author:
 + *  Gregory Haskins ghask...@novell.com
 + *
 + * This file is free software; you can redistribute it and/or modify
 + * it under the terms of version 2 of the GNU General Public License
 + * as published by the Free Software Foundation.
 + *
 + * This program is distributed in the hope that it will be useful,
 + * but WITHOUT ANY WARRANTY; without even the implied warranty of
 + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 + * GNU General Public License for more details.
 + *
 + * You should have received a copy of the GNU General Public License
 + * along with this program; if not, write to the Free Software Foundation,
 + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
 + */
 +
 +#include linux/mm.h
 +#include linux/vmalloc.h
 +#include linux/highmem.h
 +#include linux/module.h
 +#include linux/mmu_context.h
 +#include linux/kvm_host.h
 +#include linux/kvm_xinterface.h
 +
 +struct _xinterface {
 +struct kvm *kvm;
 +struct task_struct *task;
 +struct mm_struct   *mm;
 +struct kvm_xinterface   intf;
 +struct kvm_memory_slot *slotcache[NR_CPUS];
 +};
 +
 +struct _xvmap {
 +struct kvm_memory_slot*memslot;
 +unsigned long  npages;
 +struct kvm_xvmap   vmap;
 +};
 +
 +static struct _xinterface *
 +to_intf(struct kvm_xinterface *intf)
 +{
 +return container_of(intf, struct _xinterface, intf);
 +}
 +
 +#define _gfn_to_hva(gfn, memslot) \
 +(memslot-userspace_addr + (gfn - memslot-base_gfn) * PAGE_SIZE)
 +
 +/*
 + * gpa_to_hva() - translate a guest-physical to host-virtual using
 + * a per-cpu cache of the memslot.
 + *
 + * The gfn_to_memslot() call is relatively expensive, and the gpa access
 + * patterns exhibit a high degree of locality.  Therefore, lets cache
 + * the last slot used on a per-cpu basis to optimize the lookup
 + *
 + * assumes slots_lock held for read
 + */
 +static unsigned long
 +gpa_to_hva(struct _xinterface *_intf, unsigned long gpa)
 +{
 +int cpu = get_cpu();
 +unsigned long   gfn = gpa  PAGE_SHIFT;
 +struct kvm_memory_slot *memslot = _intf-slotcache[cpu];
 +unsigned long   addr= 0;
 +
 +if (!memslot
 +|| gfn  memslot-base_gfn
 +|| gfn = memslot-base_gfn + memslot-npages) {
 +
 +memslot = gfn_to_memslot(_intf-kvm, gfn);
 +if (!memslot)
 +goto out;
 +
 +_intf-slotcache[cpu] = memslot;
 +}
 +
 +addr = _gfn_to_hva(gfn, memslot) + offset_in_page(gpa);
 +
 +out:
 +put_cpu();
 +
 +return addr;
 
 Please optimize gfn_to_memslot() instead, so everybody benefits. It
 shows very often on profiles.

Yeah, its not a bad idea.  The reason why I did it here is because the
requirements for sync (kvm-vcpu) vs async (xinterface) access is
slightly different.  Sync is probably optimal with per-vcpu caching,
whereas async is optimal with per-cpu.

That said, we could probably build the entire algorithm to be per-cpu as
a compromise and still gain benefits.  

Re: [PATCH v2 2/4] KVM: introduce xinterface API for external interaction with guests

2009-10-05 Thread Gregory Haskins
Avi Kivity wrote:
 On 10/02/2009 10:19 PM, Gregory Haskins wrote:
 What: xinterface is a mechanism that allows kernel modules external to
 the kvm.ko proper to interface with a running guest.  It accomplishes
 this by creating an abstracted interface which does not expose any
 private details of the guest or its related KVM structures, and provides
 a mechanism to find and bind to this interface at run-time.

 
 If this is needed, it should be done as a virt_address_space to which
 kvm and other modules bind, instead of as something that kvm exports and
 other modules import.  The virt_address_space can be identified by an fd
 and passed around to kvm and other modules.

IIUC, what you are proposing is something similar to generalizing the
vbus::memctx object.  I had considered doing something like that in the
early design phase of vbus, but then decided it would be a hard-sell to
the mm crowd, and difficult to generalize.

What do you propose as the interface to program the object?

 
 Why: There are various subsystems that would like to interact with a KVM
 guest which are ideally suited to exist outside the domain of the kvm.ko
 core logic. For instance, external pci-passthrough, virtual-bus, and
 virtio-net modules are currently under development.  In order for these
 modules to successfully interact with the guest, they need, at the very
 least, various interfaces for signaling IO events, pointer translation,
 and possibly memory mapping.

 The signaling case is covered by the recent introduction of the
 irqfd/ioeventfd mechanisms.  This patch provides a mechanism to cover the
 other cases.  Note that today we only expose pointer-translation related
 functions, but more could be added at a future date as needs arise.

 Example usage: QEMU instantiates a guest, and an external module foo
 that desires the ability to interface with the guest (say via
 open(/dev/foo)).  QEMU may then pass the kvmfd to foo via an
 ioctl, such as: ioctl(foofd, FOO_SET_VMID,kvmfd).  Upon receipt, the
 foo module can issue kvm_xinterface_bind(kvmfd) to acquire
 the proper context.  Internally, the struct kvm* and associated
 struct module* will remain pinned at least until the foo module calls
 kvm_xinterface_put().


 
 So, under my suggestion above, you'd call
 sys_create_virt_address_space(), populate it, and pass the result to kvm
 and to foo.  This allows the use of virt_address_space without kvm and
 doesn't require foo to interact with kvm.

The problem I see here is that the only way I can think to implement
this generally is something that looks very kvm-esque (slots-to-pages
kind of translation).  Is there a way you can think of that does not
involve a kvm.ko originated vtable that is also not kvm centric?

 
 +struct kvm_xinterface_ops {
 +unsigned long (*copy_to)(struct kvm_xinterface *intf,
 + unsigned long gpa, const void *src,
 + unsigned long len);
 +unsigned long (*copy_from)(struct kvm_xinterface *intf, void *dst,
 +   unsigned long gpa, unsigned long len);
 +struct kvm_xvmap* (*vmap)(struct kvm_xinterface *intf,
 +  unsigned long gpa,
 +  unsigned long len);

 
 How would vmap() work with live migration?

vmap represents shmem regions, and is a per-guest-instance resource.  So
my plan there is that the new and old guest instance would each have the
vmap region instated at the same GPA location (assumption: gpas are
stable across migration), and any state relevant data local to the shmem
(like ring head/tail position) is conveyed in the serialized stream for
the device model.

 
 +
 +static inline void
 +_kvm_xinterface_release(struct kref *kref)
 +{
 +struct kvm_xinterface *intf;
 +struct module *owner;
 +
 +intf = container_of(kref, struct kvm_xinterface, kref);
 +
 +owner = intf-owner;
 +rmb();

 
 Why rmb?

the intf-ops-release() line may invalidate the intf pointer, so we
want to ensure that the read completes before the release() is called.

TBH: I'm not 100% its needed, but I was being conservative.

 
 +
 +intf-ops-release(intf);
 +module_put(owner);
 +}
 +

 +
 +/*
 + * gpa_to_hva() - translate a guest-physical to host-virtual using
 + * a per-cpu cache of the memslot.
 + *
 + * The gfn_to_memslot() call is relatively expensive, and the gpa access
 + * patterns exhibit a high degree of locality.  Therefore, lets cache
 + * the last slot used on a per-cpu basis to optimize the lookup
 + *
 + * assumes slots_lock held for read
 + */
 +static unsigned long
 +gpa_to_hva(struct _xinterface *_intf, unsigned long gpa)
 +{
 +int cpu = get_cpu();
 +unsigned long   gfn = gpa  PAGE_SHIFT;
 +struct kvm_memory_slot *memslot = _intf-slotcache[cpu];
 +unsigned long   addr= 0;
 +
 +if (!memslot
 +|| gfn  memslot-base_gfn
 +|| gfn= memslot-base_gfn + memslot-npages) {
 +
 +memslot = 

Re: [PATCH v2 4/4] KVM: add scatterlist support to xinterface

2009-10-05 Thread Gregory Haskins
Avi Kivity wrote:
 On 10/02/2009 10:19 PM, Gregory Haskins wrote:
 This allows a scatter-gather approach to IO, which will be useful for
 building high performance interfaces, like zero-copy and low-latency
 copy (avoiding multiple calls to copy_to/from).

 The interface is based on the existing scatterlist infrastructure.  The
 caller is expected to pass in a scatterlist with its dma field
 populated with valid GPAs.  The xinterface will then populate each
 entry by translating the GPA to a page*.

 The caller signifies completion by simply performing a put_page() on
 each page returned in the list.

 Signed-off-by: Gregory Haskinsghask...@novell.com
 ---

   include/linux/kvm_xinterface.h |4 ++
   virt/kvm/xinterface.c  |   72
 
   2 files changed, 76 insertions(+), 0 deletions(-)

 diff --git a/include/linux/kvm_xinterface.h
 b/include/linux/kvm_xinterface.h
 index 684b6f8..eefb575 100644
 --- a/include/linux/kvm_xinterface.h
 +++ b/include/linux/kvm_xinterface.h
 @@ -9,6 +9,7 @@
   #includelinux/kref.h
   #includelinux/module.h
   #includelinux/file.h
 +#includelinux/scatterlist.h

   struct kvm_xinterface;
   struct kvm_xvmap;
 @@ -36,6 +37,9 @@ struct kvm_xinterface_ops {
   u64 addr,
   unsigned long len,
   unsigned long flags);
 +unsigned long (*sgmap)(struct kvm_xinterface *intf,
 +   struct scatterlist *sgl, int nents,
 +   unsigned long flags);
   void (*release)(struct kvm_xinterface *);
   };

 diff --git a/virt/kvm/xinterface.c b/virt/kvm/xinterface.c
 index c356835..16729f6 100644
 --- a/virt/kvm/xinterface.c
 +++ b/virt/kvm/xinterface.c
 @@ -467,6 +467,77 @@ fail:

   }

 +static unsigned long
 +xinterface_sgmap(struct kvm_xinterface *intf,
 + struct scatterlist *sgl, int nents,
 + unsigned long flags)
 +{
 +struct _xinterface *_intf   = to_intf(intf);
 +struct task_struct *p   = _intf-task;
 +struct mm_struct   *mm  = _intf-mm;
 +struct kvm *kvm = _intf-kvm;
 +struct kvm_memory_slot *memslot = NULL;
 +boolkthread = !current-mm;
 +int ret;
 +struct scatterlist *sg;
 +int i;
 +
 +down_read(kvm-slots_lock);
 +
 +if (kthread)
 +use_mm(_intf-mm);
 +
 +for_each_sg(sgl, sg, nents, i) {
 +unsigned long   gpa= sg_dma_address(sg);
 +unsigned long   len= sg_dma_len(sg);
 +unsigned long   gfn= gpa  PAGE_SHIFT;
 +off_t   offset = offset_in_page(gpa);
 +unsigned long   hva;
 +struct page*pg;
 +
 +/* ensure that we do not have more than one page per entry */
 +if ((PAGE_ALIGN(len + offset)  PAGE_SHIFT) != 1) {
 +ret = -EINVAL;
 +break;
 +}
 +
 +/* check for a memslot-cache miss */
 +if (!memslot
 +|| gfn  memslot-base_gfn
 +|| gfn= memslot-base_gfn + memslot-npages) {
 +memslot = gfn_to_memslot(kvm, gfn);
 +if (!memslot) {
 +ret = -EFAULT;
 +break;
 +}
 +}
 +
 +hva = (memslot-userspace_addr +
 +   (gfn - memslot-base_gfn) * PAGE_SIZE);
 +
 +if (kthread || current-mm == mm)
 +ret = get_user_pages_fast(hva, 1, 1,pg);
 +else
 +ret = get_user_pages(p, mm, hva, 1, 1, 0,pg, NULL);

 
 One of these needs the mm semaphore.

Indeed.  Good catch.

 
 +
 +if (ret != 1) {
 +if (ret= 0)
 +ret = -EFAULT;
 +break;
 +}
 +
 +sg_set_page(sg, pg, len, offset);
 +ret = 0;
 +}
 +
 +if (kthread)
 +unuse_mm(_intf-mm);
 +
 +up_read(kvm-slots_lock);
 +
 +return ret;
 +}
 +
   static void
   xinterface_release(struct kvm_xinterface *intf)
   {
 @@ -483,6 +554,7 @@ struct kvm_xinterface_ops _xinterface_ops = {
   .copy_from   = xinterface_copy_from,
   .vmap= xinterface_vmap,
   .ioevent = xinterface_ioevent,
 +.sgmap   = xinterface_sgmap,
   .release = xinterface_release,
   };


 -- 
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html

 
 




signature.asc
Description: OpenPGP digital signature


[PATCH 1/6] KVM test: Added floppy and tftp options to qemu command

2009-10-05 Thread Lucas Meneghel Rodrigues
This allows you to not specify the full path in the
config file, appends to the image dir.

Signed-off-by: David Huff dh...@redhat.com
---
 client/tests/kvm/kvm_vm.py |   12 
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index f37c6f5..595b749 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -264,6 +264,18 @@ class VM:
 iso = kvm_utils.get_path(root_dir, iso)
 qemu_cmd +=  -cdrom %s % iso
 
+# We may want to add {floppy_otps} parameter for -fda
+# {fat:floppy:}/path/. However vvfat is not usually recommended
+floppy = params.get(floppy)
+if floppy:
+floppy = os.path.join(image_dir, floppy)
+qemu_cmd +=  -fda %s % floppy
+
+tftp = params.get(tftp)
+if tftp:
+tftp = os.path.join(image_dir, tftp)
+qemu_cmd +=  -tftp %s % tftp
+
 extra_params = params.get(extra_params)
 if extra_params:
 qemu_cmd +=  %s % extra_params
-- 
1.6.2.5

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/6] KVM test: Introducing unattended install subtest

2009-10-05 Thread Lucas Meneghel Rodrigues
In order to resolve the question, 'how will the guest
operating system tell the host operating system that
the unattended install process finish', we took the
simple approach and created a simple socket communication
ack process: The test instantiates a server tcp socket
on port 12323, and waits during a specified amount of time.

For guests, the vast majority of the unattended install
processes can deal with executing commands at the end of
the install process. Let's take advantage of that and
make clients tell the server about the end of the process
using simple programs that can do that. The implementation
of that strategy varies trough different operating systems.

This is the kvm test implementation code, client programs
will follow on later patches.

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/tests/unattended_install.py |   45 ++
 1 files changed, 45 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/tests/unattended_install.py

diff --git a/client/tests/kvm/tests/unattended_install.py 
b/client/tests/kvm/tests/unattended_install.py
new file mode 100644
index 000..60e2a35
--- /dev/null
+++ b/client/tests/kvm/tests/unattended_install.py
@@ -0,0 +1,45 @@
+import logging, time, socket
+from autotest_lib.client.common_lib import error
+import kvm_utils, kvm_test_utils
+
+
+def run_unattended_install(test, params, env):
+
+Unattended install test:
+1) Starts a VM with an appropriated setup to start an unattended OS 
install.
+2) Wait until the install reports to the install watcher its end.
+
+@param test: KVM test object.
+@param params: Dictionary with the test parameters.
+@param env: Dictionary with test environment.
+
+vm = kvm_utils.env_get_vm(env, params.get(main_vm))
+if not vm:
+raise error.TestError(VM object not found in environment)
+if not vm.is_alive():
+raise error.TestError(VM seems to be dead; Test requires a living VM)
+
+logging.info(Starting unattended install watch process)
+server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+server.bind(('', 12323))
+server.listen(1)
+
+end_time = time.time() + float(params.get(timeout, 3000))
+
+while True:
+server.settimeout(end_time - time.time())
+try:
+(client, addr) = server.accept()
+except socket.timeout:
+server.close()
+raise error.TestFail('Timeout elapsed while waiting for install to 
'
+ 'finish.')
+msg = client.recv(1024)
+logging.debug(Received '%s' from %s, msg, addr)
+if msg == 'done':
+logging.info('Guest reported successful installation')
+server.close()
+break
+else:
+logging.error('Got invalid string from client: %s.' % msg)
+
-- 
1.6.2.5

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[patch 3/3] test: vmexit: inl from pmtimer

2009-10-05 Thread Marcelo Tosatti

Add inl(ACPI_PMTIMER_PORT) test.

Signed-off-by: Marcelo Tosatti mtosa...@redhat.com

Index: qemu-kvm-parallel-vmexit/kvm/user/test/x86/vmexit.c
===
--- qemu-kvm-parallel-vmexit.orig/kvm/user/test/x86/vmexit.c
+++ qemu-kvm-parallel-vmexit/kvm/user/test/x86/vmexit.c
@@ -17,6 +17,13 @@ static inline unsigned long long rdtsc()
return r;
 }
 
+static unsigned int inl(unsigned short port)
+{
+unsigned int val;
+asm volatile(inl %w1, %0 : =a(val) : Nd(port));
+return val;
+}
+
 #define GOAL (1ull  30)
 
 #ifdef __x86_64__
@@ -76,6 +83,11 @@ static void ipi_halt(void)
;
 }
 
+static void inl_pmtimer(void)
+{
+inl(0xb008);
+}
+
 static struct test {
void (*func)(void);
const char *name;
@@ -86,6 +98,7 @@ static struct test {
{ vmcall, vmcall, .parallel = 1, },
{ mov_from_cr8, mov_from_cr8, .parallel = 1, },
{ mov_to_cr8, mov_to_cr8 , .parallel = 1, },
+   { inl_pmtimer, inl_from_pmtimer, .parallel = 1, },
{ ipi, ipi, is_smp, .parallel = 0, },
{ ipi_halt, ipi+halt, is_smp, .parallel = 0, },
 };
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/6] KVM test: Add unattended install files for F11 and WinXP

2009-10-05 Thread Lucas Meneghel Rodrigues
For the purpose of developing the test, 2 operating systems
were used, Fedora 11 and Windows XP 32 bit. This patch adds
the unattended install files for each OS:

 * Fedora 11: Fedora-11.ks
 * Win XP: winnt.bat and winxp32.sif

Attention: Those files work great in case we use user mode
networking. For TAP mode networking, kickstart templating
needs to be implemented, and I will leave that for a later
patch

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/unattended/Fedora-11.ks |   36 +++
 client/tests/kvm/unattended/winnt.bat|   12 +
 client/tests/kvm/unattended/winxp32.sif  |   72 ++
 3 files changed, 120 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/unattended/Fedora-11.ks
 create mode 100644 client/tests/kvm/unattended/winnt.bat
 create mode 100644 client/tests/kvm/unattended/winxp32.sif

diff --git a/client/tests/kvm/unattended/Fedora-11.ks 
b/client/tests/kvm/unattended/Fedora-11.ks
new file mode 100644
index 000..2e51861
--- /dev/null
+++ b/client/tests/kvm/unattended/Fedora-11.ks
@@ -0,0 +1,36 @@
+install
+cdrom
+text
+reboot
+lang en_US
+keyboard us
+network --bootproto dhcp
+rootpw 123456
+firewall --enabled --ssh
+selinux --enforcing
+timezone --utc America/New_York
+firstboot --disable
+bootloader --location=mbr
+zerombr
+
+clearpart --all --initlabel
+part /boot  --fstype=ext3 --size=100
+part /  --fstype=ext3 --size=2000
+part swap   --fstype=swap --size=512
+
+%packages
+@ Base
+%end
+
+%post --interpreter /usr/bin/python
+import socket, os
+os.system('iptables -F')
+os.system('echo 0  /selinux/enforce')
+port = 12323
+buf = 1024
+addr = ('10.0.0.2', port)
+client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+client.connect(addr)
+client.sendto('done', addr)
+client.close()
+%end
\ No newline at end of file
diff --git a/client/tests/kvm/unattended/winnt.bat 
b/client/tests/kvm/unattended/winnt.bat
new file mode 100644
index 000..e3e0645
--- /dev/null
+++ b/client/tests/kvm/unattended/winnt.bat
@@ -0,0 +1,12 @@
+...@rem SetupMgrTag
+...@echo off
+
+rem
+rem This is a SAMPLE batch script generated by Setup Manager.
+rem If this script is moved from the location where it was generated, it may 
have to be modified.
+rem
+
+set AnswerFile=.\winnt.sif
+set SetupFiles=D:\i386
+
+D:\i386\winnt32 /s:%SetupFiles% /unattend:%AnswerFile%
diff --git a/client/tests/kvm/unattended/winxp32.sif 
b/client/tests/kvm/unattended/winxp32.sif
new file mode 100644
index 000..dc0c106
--- /dev/null
+++ b/client/tests/kvm/unattended/winxp32.sif
@@ -0,0 +1,72 @@
+;SetupMgrTag
+[Data]
+AutoPartition=1
+MsDosInitiated=0
+UnattendedInstall=Yes
+
+[Unattended]
+Repartition=Yes
+UnattendMode=FullUnattended
+OemSkipEula=Yes
+OemPreinstall=No
+TargetPath=\WINDOWS
+UnattendSwitch=Yes
+CrashDumpSetting=1
+DriverSigningPolicy=ignore
+WaitForReboot=no
+
+[GuiUnattended]
+AdminPassword=123456
+EncryptedAdminPassword=NO
+TimeZone=85
+OemSkipWelcome=1
+AutoLogon=Yes
+AutoLogonCount=1
+OEMSkipRegional=1
+
+[UserData]
+ProductKey=[replace-with-your-prodkey]
+FullName=Autotest Mindless Drone
+OrgName=Autotest
+ComputerName=*
+
+[Identification]
+JoinWorkgroup=WORKGROUP
+
+[Networking]
+InstallDefaultComponents=Yes
+
+[Proxy]
+Proxy_Enable=0
+Use_Same_Proxy=0
+
+[Components]
+dialer=off
+media_clips=off
+media_utopia=off
+msnexplr=off
+netoc=off
+OEAccess=off
+templates=off
+WMAccess=off
+zonegames=off
+
+[TerminalServices]
+AllowConnections=1
+
+[WindowsFirewall]
+Profiles=WindowsFirewall.TurnOffFirewall
+
+[WindowsFirewall.TurnOffFirewall]
+Mode=0
+
+[Branding]
+BrandIEUsingUnattended=Yes
+
+[GuiRunOnce]
+Command0=cmd /c A:\finish.exe 10.0.0.2
+
+[Display]
+Xresolution=1024
+YResolution=768
+
-- 
1.6.2.5

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 4/6] KVM test: Add unattended install script

2009-10-05 Thread Lucas Meneghel Rodrigues
In order to make it possible to prepare the environment
for the guests installation, we have to:

 * Prepare a boot floppy for both windows and linux.
The boot floppy contains the unattended file, and in
case of windows, it will also hold a program that
can send to the server a ACK message. For Kickstart,
we just built the client program that sends the
message into the kickstart file.

 * Prepare PXE boot for Linux guests. Will prepare a
location with the pxelinux.0 boot loader + the vmlinuz
and initrd.img images used to do network boot.

As I pointed out, the script is in python due to project
policy. At this point I strongly believe the environment
scripts should be rather libraries that can use full
use of the autotest API, but for a first pass I believe
the script is good enough.

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/scripts/unattended.py |  223 
 1 files changed, 223 insertions(+), 0 deletions(-)
 create mode 100755 client/tests/kvm/scripts/unattended.py

diff --git a/client/tests/kvm/scripts/unattended.py 
b/client/tests/kvm/scripts/unattended.py
new file mode 100755
index 000..5d730d2
--- /dev/null
+++ b/client/tests/kvm/scripts/unattended.py
@@ -0,0 +1,223 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+import os, sys, shutil, tempfile, re
+import common
+
+
+Simple script to setup unattended installs on KVM guests.
+
+
+class SetupError(Exception):
+
+Simple wrapper for the builtin Exception class.
+
+pass
+
+
+class UnattendedInstall:
+
+Creates a floppy disk image that will contain a config file for unattended
+OS install. Optionally, sets up a PXE install server using qemu built in
+TFTP and DHCP servers to install a particular operating system. The
+parameters to the script are retrieved from environment variables.
+
+def __init__(self):
+
+Gets params from environment variables and sets class attributes.
+
+script_dir = os.path.dirname(sys.modules[__name__].__file__)
+kvm_test_dir = os.path.abspath(os.path.join(script_dir, ..))
+images_dir = os.path.join(kvm_test_dir, 'images')
+isos_dir = os.path.join(kvm_test_dir, 'isos')
+self.deps_dir = os.path.join(kvm_test_dir, 'deps')
+self.unattended_dir = os.path.join(kvm_test_dir, 'unattended')
+
+try:
+tftp_root = os.environ['KVM_TEST_tftp']
+self.tftp_root = os.path.join(images_dir, tftp_root)
+if not os.path.isdir(self.tftp_root):
+os.makedirs(self.tftp_root)
+except KeyError:
+self.tftp_root = ''
+
+try:
+self.kernel_args = os.environ['KVM_TEST_kernel_args']
+except KeyError:
+self.kernel_args = ''
+
+try:
+self.finish_program= os.environ['KVM_TEST_finish_program']
+except:
+self.finish_program = None
+
+
+cdrom_iso = os.environ['KVM_TEST_cdrom']
+self.unattended_file = os.environ['KVM_TEST_unattended_file']
+
+self.qemu_img_bin = os.path.join(kvm_test_dir, 'qemu-img')
+self.cdrom_iso = os.path.join(isos_dir, cdrom_iso)
+self.floppy_mount = tempfile.mkdtemp(prefix='floppy_', dir='/tmp')
+self.cdrom_mount = tempfile.mkdtemp(prefix='cdrom_', dir='/tmp')
+self.floppy_img = os.path.join(images_dir, 'floppy.img')
+
+
+def create_boot_floppy(self):
+
+Prepares a boot floppy by creating a floppy image file, mounting it and
+copying an answer file (kickstarts for RH based distros, answer files
+for windows) to it. After that the image is umounted.
+
+print Creating boot floppy
+
+if self.floppy_img:
+os.remove(self.floppy_img)
+#c_cmd = 'bximage -q -fd -size=1.44 %s' % (self.floppy_img)
+c_cmd = '%s create -f raw %s 1440k' % (self.qemu_img_bin, 
self.floppy_img)
+if os.system(c_cmd):
+raise SetupError('Could not create floppy image.')
+
+f_cmd = 'mkfs.msdos -s 1 %s' % self.floppy_img
+if os.system(f_cmd):
+raise SetupError('Error formatting floppy image.')
+
+m_cmd = 'mount -o loop %s %s' % (self.floppy_img, self.floppy_mount)
+if os.system(m_cmd):
+raise SetupError('Could not mount floppy image.')
+
+if self.unattended_file.endswith('.sif'):
+dest_fname = 'winnt.sif'
+setup_file = 'winnt.bat'
+setup_file_path = os.path.join(self.unattended_dir, setup_file)
+setup_file_dest = os.path.join(self.floppy_mount, setup_file)
+shutil.copyfile(setup_file_path, setup_file_dest)
+elif self.unattended_file.endswith('.ks'):
+dest_fname = 'ks.cfg'
+
+dest = os.path.join(self.floppy_mount, dest_fname)
+shutil.copyfile(self.unattended_file, dest)
+
+if self.finish_program:
+

[PATCH 6/6] KVM test: kvm_tests.cfg.sample changes for unattended

2009-10-05 Thread Lucas Meneghel Rodrigues
In order to accomodate the unattended install test,
some changes had to be made:

 * Now cd isos and md5 sum values don't belong only
to the step file install only
 * introduced the needed variables for the unattended
install test
 * Created 2 test sets, winxp_unattended and fc11_unattended.

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/kvm_tests.cfg.sample |  152 +++-
 1 files changed, 109 insertions(+), 43 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample 
b/client/tests/kvm/kvm_tests.cfg.sample
index eb8eefd..7b49edb 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -49,6 +49,14 @@ variants:
 kill_vm_timeout = 60
 kill_vm_timeout_on_error = 0
 
+- unattended_install:
+type = unattended_install
+kill_vm_on_error = yes
+force_create_image = yes
+pre_command = scripts/unattended.py
+floppy = floppy.img
+extra_params = -boot d
+
 - setup:install
 type = steps
 fail_if_stuck_for = 300
@@ -244,36 +252,74 @@ variants:
 - 8.32:
 no setup
 image_name = fc8-32
+cdrom = linux/Fedora-8-i386-DVD.iso
+md5sum = dd6c79fddfff36d409d02242e7b10189
+md5sum_1m = dabae451bb69fbbad0e505b25144b1f9
 install:
 steps = Fedora-8-i386.steps
-cdrom = linux/Fedora-8-i386-DVD.iso
-md5sum = dd6c79fddfff36d409d02242e7b10189
-md5sum_1m = dabae451bb69fbbad0e505b25144b1f9
 
 - 8.64:
 no setup
 image_name = fc8-64
+cdrom = linux/Fedora-8-x86_64-DVD.iso
+md5sum = 2cb231a86709dec413425fd2f8bf5295
+md5sum_1m = 145f6414e19492649a56c89f0a45e719
 install:
 steps = Fedora-8-64.steps
-cdrom = linux/Fedora-8-x86_64-DVD.iso
-md5sum = 2cb231a86709dec413425fd2f8bf5295
-md5sum_1m = 145f6414e19492649a56c89f0a45e719
 
 - 9.32:
 image_name = fc9-32
+cdrom = linux/Fedora-9-i386-DVD.iso
+md5sum = 72601f685ea8c808c303353d8bf4d307
+md5sum_1m = f24fa25689e5863f1b99984c6feb787f
 install:
 steps = Fedora-9-i386.steps
-cdrom = linux/Fedora-9-i386-DVD.iso
-md5sum = 72601f685ea8c808c303353d8bf4d307
-md5sum_1m = f24fa25689e5863f1b99984c6feb787f
 
 - 9.64:
 image_name = fc9-64
+cdrom = linux/Fedora-9-x86_64-DVD.iso
+md5sum = 05b2ebeed273ec54d6f9ed3d61ea4c96
+md5sum_1m = 9822ab5097e37e8fe306ef2192727db4
 install:
 steps = Fedora-9-64.steps
-cdrom = linux/Fedora-9-x86_64-DVD.iso
-md5sum = 05b2ebeed273ec54d6f9ed3d61ea4c96
-md5sum_1m = 9822ab5097e37e8fe306ef2192727db4
+
+- 10.32:
+image_name = fc10-32
+cdrom = linux/Fedora-10-i386-DVD.iso
+md5sum = 27e581edb392728c4a07d00d3fc5ced0
+md5sum_1m = bd67c68bdf595e4ba7131ec702159181
+install:
+steps =
+unattended_install:
+tftp = tftpboot
+extra_params = -bootp /pxelinux.0 -boot n
+kernel_args = ks=floppy nicdelay=60
+unattended_file = unattended/Fedora-10.ks
+
+- 11.32:
+image_name = fc11-32
+cdrom = linux/Fedora-11-i386-DVD.iso
+md5sum = e3b1e2d1ba42aa4705fa5f41771b3927
+md5sum_1m = dc8ddf90648c247339c721395aa49714
+install:
+steps =
+unattended_install:
+tftp = tftpboot
+extra_params = -bootp /pxelinux.0 -boot n
+kernel_args = ks=floppy nicdelay=60
+unattended_file = unattended/Fedora-11.ks
+
+- 11.64:
+image_name = fc11-64
+cdrom = linux/Fedora-11-x86_64-DVD.iso
+md5sum = 

[PATCH 5/6] KVM test: Utility to make windows guests ACK end of install

2009-10-05 Thread Lucas Meneghel Rodrigues
In order to make windows guests to tell the unattended install
server that the install process ended, a simple C++ program
using the winsock 2 API had to be developed. The program was
based on an MSDN article. Its usage is

finish.exe [host-ip]

The test expect this to be compiled under deps. This can be
accomplished on linux hosts that support the mingw cross
compiling environment (or you can just compile it under your
windows guest). For Fedora 11, the instructions are:

yum install mingw32-w32api mingw32-gcc-c++

i686-pc-mingw32-g++ finish.cpp -lws2_32 -o finish.exe

Binaries produced by this process work just fine on my tests.
I still haven't made my mind about shipping binaries, with
the test, so in the meantime you want to check this compile
the code as you wish and put it under the same deps dir.

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/deps/finish.cpp |  121 ++
 1 files changed, 121 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/deps/finish.cpp

diff --git a/client/tests/kvm/deps/finish.cpp b/client/tests/kvm/deps/finish.cpp
new file mode 100644
index 000..9c2867c
--- /dev/null
+++ b/client/tests/kvm/deps/finish.cpp
@@ -0,0 +1,121 @@
+// Simple app that only sends an ack string to the KVM unattended install
+// watch code.
+//
+// You must link this code with Ws2_32.lib, Mswsock.lib, and Advapi32.lib
+//
+// Author: Lucas Meneghel Rodrigues l...@redhat.com
+// Code was adapted from an MSDN sample.
+
+// Usage: finish.exe [Host OS IP]
+
+// MinGW's ws2tcpip.h only defines getaddrinfo and other functions only for
+// the case _WIN32_WINNT = 0x0501.
+#ifdef __MINGW32__
+#undef _WIN32_WINNT
+#define _WIN32_WINNT 0x501
+#endif
+
+#include windows.h
+#include winsock2.h
+#include ws2tcpip.h
+#include stdlib.h
+#include stdio.h
+
+#define DEFAULT_BUFLEN 512
+#define DEFAULT_PORT 12323
+
+int main(int argc, char **argv)
+{
+WSADATA wsaData;
+SOCKET ConnectSocket = INVALID_SOCKET;
+struct addrinfo *result = NULL,
+*ptr = NULL,
+hints;
+char *sendbuf = done;
+char recvbuf[DEFAULT_BUFLEN];
+int iResult;
+int recvbuflen = DEFAULT_BUFLEN;
+
+// Validate the parameters
+if (argc != 2) {
+printf(usage: %s server-name\n, argv[0]);
+return 1;
+}
+
+// Initialize Winsock
+iResult = WSAStartup(MAKEWORD(2,2), wsaData);
+if (iResult != 0) {
+printf(WSAStartup failed: %d\n, iResult);
+return 1;
+}
+
+ZeroMemory( hints, sizeof(hints) );
+hints.ai_family = AF_UNSPEC;
+hints.ai_socktype = SOCK_STREAM;
+hints.ai_protocol = IPPROTO_TCP;
+
+// Resolve the server address and port
+iResult = getaddrinfo(argv[1], DEFAULT_PORT, hints, result);
+if ( iResult != 0 ) {
+printf(getaddrinfo failed: %d\n, iResult);
+WSACleanup();
+return 1;
+}
+
+// Attempt to connect to an address until one succeeds
+for(ptr=result; ptr != NULL ;ptr=ptr-ai_next) {
+
+// Create a SOCKET for connecting to server
+ConnectSocket = socket(ptr-ai_family, ptr-ai_socktype,
+ptr-ai_protocol);
+if (ConnectSocket == INVALID_SOCKET) {
+printf(Error at socket(): %ld\n, WSAGetLastError());
+freeaddrinfo(result);
+WSACleanup();
+return 1;
+}
+
+// Connect to server.
+iResult = connect( ConnectSocket, ptr-ai_addr, (int)ptr-ai_addrlen);
+if (iResult == SOCKET_ERROR) {
+closesocket(ConnectSocket);
+ConnectSocket = INVALID_SOCKET;
+continue;
+}
+break;
+}
+
+freeaddrinfo(result);
+
+if (ConnectSocket == INVALID_SOCKET) {
+printf(Unable to connect to server!\n);
+WSACleanup();
+return 1;
+}
+
+// Send the ACK buffer
+iResult = send( ConnectSocket, sendbuf, (int)strlen(sendbuf), 0 );
+if (iResult == SOCKET_ERROR) {
+printf(send failed: %d\n, WSAGetLastError());
+closesocket(ConnectSocket);
+WSACleanup();
+return 1;
+}
+
+printf(Bytes Sent: %ld\n, iResult);
+
+// shutdown the connection since no more data will be sent
+iResult = shutdown(ConnectSocket, SD_SEND);
+if (iResult == SOCKET_ERROR) {
+printf(shutdown failed: %d\n, WSAGetLastError());
+closesocket(ConnectSocket);
+WSACleanup();
+return 1;
+}
+
+// cleanup
+closesocket(ConnectSocket);
+WSACleanup();
+
+return 0;
+}
-- 
1.6.2.5

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[ kvm-Bugs-2873263 ] Segfaults after resize window in monitor virtual console

2009-10-05 Thread SourceForge.net
Bugs item #2873263, was opened at 2009-10-06 10:30
Message generated for change (Tracker Item Submitted) made by kolobrod
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detailatid=893831aid=2873263group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Sergey _ (kolobrod)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfaults after resize window in monitor virtual console

Initial Comment:
2.6.30-gentoo-r5 #3 SMP Mon Oct 5 15:31:57 YEKST 2009 x86_64 Intel(R) Core(TM)2 
CPU 6300 @ 1.86GHz GenuineIntel GNU/Linux

app-emulation/kvm-88-r1  USE=alsa esd gnutls ncurses sdl vde -bluetooth 
-havekernel -modules -pulseaudio

kvm -drive file=/home/sergey/virtual/test.img,if=virtio,media=disk,boot=on -net 
nic,model=virtio -boot c -m 384 -vga std 

Window resize causes segmentation fault while in monitor or serial virtual 
console (Ctrl+Alt+1, Ctrl+Alt+2). This problem appears even before any OS is 
booted.
It seems that after resizing window in target system display virtual console 
other consoles begin to resize normally.
-no-kvm-irqchip or -no-kvm-pit or -no-kvm does not help.

--

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detailatid=893831aid=2873263group_id=180599
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: linux-next: tree build failure

2009-10-05 Thread Jan Beulich
 Hollis Blanchard holl...@us.ibm.com 02.10.09 17:48 
On Wed, 2009-09-30 at 07:35 +0100, Jan Beulich wrote:
 The one Rusty suggested the other day may help here. I don't like it
 as a drop-in replacement for BUILD_BUG_ON() though (due to it
 deferring the error generated to the linking stage), I'd rather view
 this as an improvement to MAYBE_BUILD_BUG_ON() (which should
 then be used here).

Can you be more specific?

I have no idea what Rusty suggested where. I can't even guess what

I'm attaching Rusty's response I was referring to.

MAYBE_BUILD_BUG_ON() is supposed to do (sounds like a terrible name).

Agreed - but presumably better than just deleting the bogus instances
altogether...

Jan
---BeginMessage---
On Wed, 19 Aug 2009 01:29:25 am Jan Beulich wrote:
 gcc permitting variable length arrays makes the current construct
 used for BUILD_BUG_ON() useless, as that doesn't produce any diagnostic
 if the controlling expression isn't really constant. Instead, this
 patch makes it so that a bit field gets used here. Consequently, those
 uses where the condition isn't really constant now also need fixing.
 
 Note that in the gfp.h, kmemcheck.h, and virtio_config.h cases
 MAYBE_BUILD_BUG_ON() really just serves documentation purposes - even
 if the expression is compile time constant (__builtin_constant_p()
 yields true), the array is still deemed of variable length by gcc, and
 hence the whole expression doesn't have the intended effect.
 
 Signed-off-by: Jan Beulich jbeul...@novell.com

We used to use an undefined symbol here; diagnostics are worse but it catches
more stuff.

Perhaps a hybrid is the way to go?

#ifndef __OPTIMIZE__
#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
#else
/* If it's a constant, catch it at compile time, otherwise at link time. */
extern int __build_bug_on_failed;
#define BUILD_BUG_ON(condition) \
do {\
((void)sizeof(char[1 - 2*!!(condition)]));  \
if (condition) __build_bug_on_failed = 1;   \
} while(0)
#endif

Thanks,
Rusty.
---End Message---