Re: GSOC 2011 (KVM for MIPS)

2011-02-21 Thread Jan Kiszka
On 2011-02-21 02:09, yajin wrote:
 On Sun, Feb 20, 2011 at 9:16 AM, Jan Kiszka jan.kis...@web.de wrote:
 On 2011-02-20 14:08, Aurelien Jarno wrote:
 On Wed, Feb 16, 2011 at 07:32:31PM -0500, yajin wrote:
 Hi all,

 Hi,

 I have proposed an idea of GSOC 2011 about adding KVM support to MIPS
 architecture. I have solid experience on MIPS architecture and
 familiar with qemu. I am very interested in this project.

 Having KVM support for MIPS would be really great. AFAIK there is not
 hardware virtualization support on MIPS (except if you plan to use the
 supervisor mode for that), so how do you plan to proceed here? Executing
 everything in user mode and emulating the trapped instructions? MIPS is
 using a split address space depending on the mode, so that may not be
 that easy.

 All of that to say it would be nice to have a more detailed description
 about such a GSOC, describing the your exact plans with a rough
 timeline. This is especially true, given that adding support for a new
 architecture in KVM seems to be a huge task for a GSOC.

 MontaVista has KVM for MIPS on their product slides [1]. Maybe someone
 has a good contact to them and can poke for more details. They probably
 won't offer it for direct download but need to distribute it to their
 customers as open source anyway.

 
 Hi Jan,
 
 Thanks very much for your information. I am not sure how they
 distribute their source code. Anyone knows about that?
 
 Jan, would you like to be one of the mentors of this project? Your
 experience of KVM would be very helpful. Thanks

Your project is really interesting. But given that my bandwidth is
limited and I've already a different project proposal filed, I can't
commit myself to this yet. I may serve as a backup mentor, but no
promises at this point.

Jan



signature.asc
Description: OpenPGP digital signature


Re: [Qemu-devel] Re: [PATCH 07/18] Introduce fault tolerant VM transaction QEMUFile and ft_mode.

2011-02-21 Thread Yoshiaki Tamura
Hi Green,

2011/2/21 ya su suya94...@gmail.com:
 Yoshiaki:

     I have one question about ram_save_live, during migration 3
 stage(completation stage), it will call
 cpu_physical_memory_set_dirty_tracking(0) to stop recording ram dirty pages.
 at the end of migrate_ft_trans_connect function, it will invoke vm_start(),
 at this time, cpu_physical_memory_set_dirty_tracking(1) is not called yet,
 so there may have some ram pages not recorded when qemu_savevm_trans_begin
 is called.  I think you need calll
 cpu_physical_memory_set_dirty_tracking(1) in migrate_ft_trans_connect
 function, Am I right?

Thank you for taking a look.
When qemu_savevm_trans_begin is called for the first time, it
calls ram_save_live with stage 1, that sends all pages and sets
dirty tracking, so there won't be missing pages.  Note that
event-tap is turned on by then, meaning no outputs are sent before
finishing the first transaction.  I understand that this
implementation is inefficient, and planning to introduce a new
stage that is almost same as stage 3 but keeps dirty tracking in
the future.

Thanks,

Yoshi


 BR

 Green.


 2011/2/10 Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp

 This code implements VM transaction protocol.  Like buffered_file, it
 sits between savevm and migration layer.  With this architecture, VM
 transaction protocol is implemented mostly independent from other
 existing code.

 Signed-off-by: Yoshiaki Tamura tamura.yoshi...@lab.ntt.co.jp
 Signed-off-by: OHMURA Kei ohmura@lab.ntt.co.jp
 ---
  Makefile.objs   |    1 +
  ft_trans_file.c |  624
 +++
  ft_trans_file.h |   72 +++
  migration.c     |    3 +
  trace-events    |   15 ++
  5 files changed, 715 insertions(+), 0 deletions(-)
  create mode 100644 ft_trans_file.c
  create mode 100644 ft_trans_file.h

 diff --git a/Makefile.objs b/Makefile.objs
 index 353b1a8..04148b5 100644
 --- a/Makefile.objs
 +++ b/Makefile.objs
 @@ -100,6 +100,7 @@ common-obj-y += msmouse.o ps2.o
  common-obj-y += qdev.o qdev-properties.o
  common-obj-y += block-migration.o
  common-obj-y += pflib.o
 +common-obj-y += ft_trans_file.o

  common-obj-$(CONFIG_BRLAPI) += baum.o
  common-obj-$(CONFIG_POSIX) += migration-exec.o migration-unix.o
 migration-fd.o
 diff --git a/ft_trans_file.c b/ft_trans_file.c
 new file mode 100644
 index 000..2b42b95
 --- /dev/null
 +++ b/ft_trans_file.c
 @@ -0,0 +1,624 @@
 +/*
 + * Fault tolerant VM transaction QEMUFile
 + *
 + * Copyright (c) 2010 Nippon Telegraph and Telephone Corporation.
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2.  See
 + * the COPYING file in the top-level directory.
 + *
 + * This source code is based on buffered_file.c.
 + * Copyright IBM, Corp. 2008
 + * Authors:
 + *  Anthony Liguori        aligu...@us.ibm.com
 + */
 +
 +#include qemu-common.h
 +#include qemu-error.h
 +#include hw/hw.h
 +#include qemu-timer.h
 +#include sysemu.h
 +#include qemu-char.h
 +#include trace.h
 +#include ft_trans_file.h
 +
 +typedef struct FtTransHdr
 +{
 +    uint16_t cmd;
 +    uint16_t id;
 +    uint32_t seq;
 +    uint32_t payload_len;
 +} FtTransHdr;
 +
 +typedef struct QEMUFileFtTrans
 +{
 +    FtTransPutBufferFunc *put_buffer;
 +    FtTransGetBufferFunc *get_buffer;
 +    FtTransPutReadyFunc *put_ready;
 +    FtTransGetReadyFunc *get_ready;
 +    FtTransWaitForUnfreezeFunc *wait_for_unfreeze;
 +    FtTransCloseFunc *close;
 +    void *opaque;
 +    QEMUFile *file;
 +
 +    enum QEMU_VM_TRANSACTION_STATE state;
 +    uint32_t seq;
 +    uint16_t id;
 +
 +    int has_error;
 +
 +    bool freeze_output;
 +    bool freeze_input;
 +    bool rate_limit;
 +    bool is_sender;
 +    bool is_payload;
 +
 +    uint8_t *buf;
 +    size_t buf_max_size;
 +    size_t put_offset;
 +    size_t get_offset;
 +
 +    FtTransHdr header;
 +    size_t header_offset;
 +} QEMUFileFtTrans;
 +
 +#define IO_BUF_SIZE 32768
 +
 +static void ft_trans_append(QEMUFileFtTrans *s,
 +                            const uint8_t *buf, size_t size)
 +{
 +    if (size  (s-buf_max_size - s-put_offset)) {
 +        trace_ft_trans_realloc(s-buf_max_size, size + 1024);
 +        s-buf_max_size += size + 1024;
 +        s-buf = qemu_realloc(s-buf, s-buf_max_size);
 +    }
 +
 +    trace_ft_trans_append(size);
 +    memcpy(s-buf + s-put_offset, buf, size);
 +    s-put_offset += size;
 +}
 +
 +static void ft_trans_flush(QEMUFileFtTrans *s)
 +{
 +    size_t offset = 0;
 +
 +    if (s-has_error) {
 +        error_report(flush when error %d, bailing, s-has_error);
 +        return;
 +    }
 +
 +    while (offset  s-put_offset) {
 +        ssize_t ret;
 +
 +        ret = s-put_buffer(s-opaque, s-buf + offset, s-put_offset -
 offset);
 +        if (ret == -EAGAIN) {
 +            break;
 +        }
 +
 +        if (ret = 0) {
 +            error_report(error flushing data, %s, strerror(errno));
 +            s-has_error = FT_TRANS_ERR_FLUSH;
 +            break;
 +        } else {
 +            offset += 

Re: [PATCH] kvm: allow RO page when atomic !write_fault

2011-02-21 Thread Gleb Natapov
On Mon, Feb 21, 2011 at 11:47:36AM +0800, Lai Jiangshan wrote:
 
 Atomic-able hva_to_pfn() patches and allow-read-only-page patches
 are merged almost the same time. But hva_to_pfn() does not handle
 these two issues well together.
 
 When @atomic  !@write_fault  host-is-read-only-page-mapped
 the code will return fault_page, actually, it will be better
 if we return the pfn of the read-only-page.
 
This patch will conflict with not yet applied https://lkml.org/lkml/2011/2/1/94
What it the status of that patch?

 Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com
 ---
 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
 index 1fa0d29..f49cfc0 100644
 --- a/virt/kvm/kvm_main.c
 +++ b/virt/kvm/kvm_main.c
 @@ -1060,31 +1060,27 @@ static pfn_t hva_to_pfn(struct kvm *kvm, unsigned 
 long addr, bool atomic,
   BUG_ON(!write_fault  !writable);
  
   if (writable)
 - *writable = true;
 + *writable = write_fault;
  
   if (atomic || async)
 - npages = __get_user_pages_fast(addr, 1, 1, page);
 + npages = __get_user_pages_fast(addr, 1, write_fault, page);
  
   if (unlikely(npages != 1)  !atomic) {
   might_sleep();
 -
 - if (writable)
 - *writable = write_fault;
 -
   npages = get_user_pages_fast(addr, 1, write_fault, page);
 + }
  
 - /* map read fault as writable if possible */
 - if (unlikely(!write_fault)  npages == 1) {
 - struct page *wpage[1];
 + /* map read fault as writable if possible */
 + if (unlikely(!write_fault)  npages == 1) {
 + struct page *wpage[1];
  
 - npages = __get_user_pages_fast(addr, 1, 1, wpage);
 - if (npages == 1) {
 - *writable = true;
 - put_page(page[0]);
 - page[0] = wpage[0];
 - }
 - npages = 1;
 + npages = __get_user_pages_fast(addr, 1, 1, wpage);
 + if (npages == 1) {
 + *writable = true;
 + put_page(page[0]);
 + page[0] = wpage[0];
   }
 + npages = 1;
   }
  
   if (unlikely(npages != 1)) {
 --
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/

--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/2] KVM: VMX: write new TR selector value into vmcs immediately if it changes during vm86 mode.

2011-02-21 Thread Gleb Natapov
When vm86 is active TR descriptor is updated with vm86 task values,
but selector is left intact. vmx_set_segment() makes sure that if TR
register is written into while vm86 is active the new values are saved
for use after vm86 is deactivated, but since selector is not updated on
vm86 activation/deactivation new value is lost. Fix this by writing new
selector into vmcs immediately.

Signed-off-by: Gleb Natapov g...@redhat.com
---
 arch/x86/kvm/vmx.c |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a61ed93..dafb67e 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2138,6 +2138,7 @@ static void vmx_set_segment(struct kvm_vcpu *vcpu,
u32 ar;
 
if (vmx-rmode.vm86_active  seg == VCPU_SREG_TR) {
+   vmcs_write16(sf-selector, var-selector);
vmx-rmode.tr.selector = var-selector;
vmx-rmode.tr.base = var-base;
vmx-rmode.tr.limit = var-limit;
-- 
1.7.2.3

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/2] KVM: VMX: Initialize vm86 TSS only once.

2011-02-21 Thread Gleb Natapov
Currently vm86 task is initialized on each real mode entry and vcpu
reset. Initialization is done by zeroing TSS and updating relevant
fields. But since all vcpus are using the same TSS there is a race where
one vcpu may use TSS while other vcpu is initializing it, so the vcpu
that uses TSS will see wrong TSS content and will behave incorrectly.
Fix that by initializing TSS only once.

Signed-off-by: Gleb Natapov g...@redhat.com
---
 arch/x86/kvm/vmx.c |   28 ++--
 1 files changed, 6 insertions(+), 22 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index dafb67e..e2b8c6b 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -176,7 +176,6 @@ static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
return container_of(vcpu, struct vcpu_vmx, vcpu);
 }
 
-static int init_rmode(struct kvm *kvm);
 static u64 construct_eptp(unsigned long root_hpa);
 static void kvm_cpu_vmxon(u64 addr);
 static void kvm_cpu_vmxoff(void);
@@ -1802,7 +1801,6 @@ static void enter_rmode(struct kvm_vcpu *vcpu)
 
 continue_rmode:
kvm_mmu_reset_context(vcpu);
-   init_rmode(vcpu-kvm);
 }
 
 static void vmx_set_efer(struct kvm_vcpu *vcpu, u64 efer)
@@ -2737,22 +2735,6 @@ static int vmx_vcpu_setup(struct vcpu_vmx *vmx)
return 0;
 }
 
-static int init_rmode(struct kvm *kvm)
-{
-   int idx, ret = 0;
-
-   idx = srcu_read_lock(kvm-srcu);
-   if (!init_rmode_tss(kvm))
-   goto exit;
-   if (!init_rmode_identity_map(kvm))
-   goto exit;
-
-   ret = 1;
-exit:
-   srcu_read_unlock(kvm-srcu, idx);
-   return ret;
-}
-
 static int vmx_vcpu_reset(struct kvm_vcpu *vcpu)
 {
struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -2760,10 +2742,6 @@ static int vmx_vcpu_reset(struct kvm_vcpu *vcpu)
int ret;
 
vcpu-arch.regs_avail = ~((1  VCPU_REGS_RIP) | (1  VCPU_REGS_RSP));
-   if (!init_rmode(vmx-vcpu.kvm)) {
-   ret = -ENOMEM;
-   goto out;
-   }
 
vmx-rmode.vm86_active = 0;
 
@@ -3009,6 +2987,9 @@ static int vmx_set_tss_addr(struct kvm *kvm, unsigned int 
addr)
if (ret)
return ret;
kvm-arch.tss_addr = addr;
+   if (!init_rmode_tss(kvm))
+   return  -ENOMEM;
+
return 0;
 }
 
@@ -4224,8 +4205,11 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm *kvm, 
unsigned int id)
if (!kvm-arch.ept_identity_map_addr)
kvm-arch.ept_identity_map_addr =
VMX_EPT_IDENTITY_PAGETABLE_ADDR;
+   err = -ENOMEM;
if (alloc_identity_pagetable(kvm) != 0)
goto free_vmcs;
+   if (!init_rmode_identity_map(kvm))
+   goto free_vmcs;
}
 
return vmx-vcpu;
-- 
1.7.2.3

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] stop and show registers on error.

2011-02-21 Thread Gleb Natapov

Signed-off-by: Gleb Natapov g...@redhat.com
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 49cd683..2f3f683 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -644,8 +644,7 @@ int kvm_run(CPUState *env)
 break;
 #endif
case KVM_EXIT_INTERNAL_ERROR:
-kvm_handle_internal_error(env, run);
-r = 1;
+r = kvm_handle_internal_error(env, run);
break;
 default:
 if (kvm_arch_run(env)) {
@@ -1233,6 +1232,7 @@ int kvm_cpu_exec(CPUState *env)
 r = kvm_run(env);
 if (r  0) {
 printf(kvm_run returned %d\n, r);
+kvm_show_regs(env);
 vm_stop(0);
 }
 
--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[KVM-AUTOTEST PATCH 1/3] KVM test: kvm_config.py: support one-line conditional blocks

2011-02-21 Thread Michael Goldish
Support statements such as:

ide: only unattended_install

which is just a short form for

ide:
only unattended_install

Currently the short form is supported only for assignment statements.
This patch allows it to be used for all types of statements.

Note: multiple filters can be combined on the same line, e.g.

ide: qcow2: Fedora: only unattended_install

is equivalent to:

ide..qcow2..Fedora: only unattended_install

Signed-off-by: Michael Goldish mgold...@redhat.com
---
 client/tests/kvm/kvm_config.py |   79 ++--
 1 files changed, 44 insertions(+), 35 deletions(-)

diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py
index a125129..e124e27 100755
--- a/client/tests/kvm/kvm_config.py
+++ b/client/tests/kvm/kvm_config.py
@@ -330,7 +330,7 @@ class Parser(object):
 self._debug(reached leaf, returning it)
 d = {name: name, dep: dep, shortname: ..join(shortname)}
 for filename, linenum, op in new_content:
-op.apply_to_dict(d, ctx, ctx_set)
+op.apply_to_dict(d)
 yield d
 # If this node did not produce any dicts, remember the failed filters
 # of its descendants
@@ -470,28 +470,31 @@ class Parser(object):
 node.content += [(cr.filename, linenum, f)]
 continue
 
+# Look for operators
+op_match = _ops_exp.search(line)
+
 # Parse conditional blocks
-if line.endswith(:):
-try:
-cond = Condition(line)
-except ParserError, e:
-e.line = line
-e.filename = cr.filename
-e.linenum = linenum
-raise
-self._parse(cr, cond, prev_indent=indent)
-node.content += [(cr.filename, linenum, cond)]
-continue
+if : in line:
+index = line.index(:)
+if not op_match or index  op_match.start():
+index += 1
+cr.set_next_line(line[index:], indent, linenum)
+line = line[:index]
+try:
+cond = Condition(line)
+except ParserError, e:
+e.line = line
+e.filename = cr.filename
+e.linenum = linenum
+raise
+self._parse(cr, cond, prev_indent=indent)
+node.content += [(cr.filename, linenum, cond)]
+continue
 
 # Parse regular operators
-try:
-op = Op(line)
-except ParserError, e:
-e.line = line
-e.filename = cr.filename
-e.linenum = linenum
-raise
-node.content += [(cr.filename, linenum, op)]
+if not op_match:
+raise ParserError(Syntax error, line, cr.filename, linenum)
+node.content += [(cr.filename, linenum, Op(line, op_match))]
 
 return node
 
@@ -556,26 +559,17 @@ _ops_exp = re.compile(|.join([op[0] for op in 
_ops.values()]))
 
 
 class Op(object):
-def __init__(self, line):
-m = re.search(_ops_exp, line)
-if not m:
-raise ParserError(Syntax error: missing operator)
-left = line[:m.start()].strip()
+def __init__(self, line, m):
+self.func = _ops[m.group()][1]
+self.key = line[:m.start()].strip()
 value = line[m.end():].strip()
-if value and ((value[0] == '' and value[-1] == '') or
-  (value[0] == ' and value[-1] == ')):
+if value and (value[0] == value[-1] == '' or
+  value[0] == value[-1] == '):
 value = value[1:-1]
-filters_and_key = map(str.strip, left.split(:))
-self.filters = [Filter(f) for f in filters_and_key[:-1]]
-self.key = filters_and_key[-1]
 self.value = value
-self.func = _ops[m.group()][1]
 
 
-def apply_to_dict(self, d, ctx, ctx_set):
-for f in self.filters:
-if not f.match(ctx, ctx_set):
-return
+def apply_to_dict(self, d):
 self.func(d, self.key, self.value)
 
 
@@ -594,6 +588,7 @@ class StrReader(object):
 self.filename = string
 self._lines = []
 self._line_index = 0
+self._stored_line = None
 for linenum, line in enumerate(s.splitlines()):
 line = line.rstrip().expandtabs()
 stripped_line = line.lstrip()
@@ -614,6 +609,10 @@ class StrReader(object):
 indentation level.  If no line is available, (None, -1, -1) is
 returned.
 
+if self._stored_line:
+ret = self._stored_line
+self._stored_line = None
+return ret
 if self._line_index = 

[KVM-AUTOTEST PATCH 2/3] KVM test: kvm_config.py: process_content(): look for Op instead of str

2011-02-21 Thread Michael Goldish
Looking for str is a mistake because content should only contains Ops and
Filters.  This is a harmless bug but it should be fixed.

Signed-off-by: Michael Goldish mgold...@redhat.com
---
 client/tests/kvm/kvm_config.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py
index e124e27..b6e089b 100755
--- a/client/tests/kvm/kvm_config.py
+++ b/client/tests/kvm/kvm_config.py
@@ -226,7 +226,7 @@ class Parser(object):
 #filters first.
 for t in content:
 filename, linenum, obj = t
-if type(obj) is str:
+if type(obj) is Op:
 new_content.append(t)
 continue
 elif type(obj) is OnlyFilter:
-- 
1.7.3.4

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[KVM-AUTOTEST PATCH 3/3] KVM test: kvm_config.py: support negative conditional blocks

2011-02-21 Thread Michael Goldish
If the filter starting a conditional block is preceded by '!', the condition is
negative.  For example:

!Fedora.14, RHEL.6.0:
only qcow2

means for everything except Fedora.14 and RHEL.6.0, allow only qcow2.

Signed-off-by: Michael Goldish mgold...@redhat.com
---
 client/tests/kvm/kvm_config.py |   59 +++-
 1 files changed, 40 insertions(+), 19 deletions(-)

diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py
index b6e089b..4dbb1d4 100755
--- a/client/tests/kvm/kvm_config.py
+++ b/client/tests/kvm/kvm_config.py
@@ -137,6 +137,14 @@ class NoOnlyFilter(Filter):
 
 
 class OnlyFilter(NoOnlyFilter):
+def is_irrelevant(self, ctx, ctx_set, descendant_labels):
+return self.match(ctx, ctx_set)
+
+
+def requires_action(self, ctx, ctx_set, descendant_labels):
+return not self.might_match(ctx, ctx_set, descendant_labels)
+
+
 def might_pass(self, failed_ctx, failed_ctx_set, ctx, ctx_set,
descendant_labels):
 for word in self.filter:
@@ -148,6 +156,14 @@ class OnlyFilter(NoOnlyFilter):
 
 
 class NoFilter(NoOnlyFilter):
+def is_irrelevant(self, ctx, ctx_set, descendant_labels):
+return not self.might_match(ctx, ctx_set, descendant_labels)
+
+
+def requires_action(self, ctx, ctx_set, descendant_labels):
+return self.match(ctx, ctx_set)
+
+
 def might_pass(self, failed_ctx, failed_ctx_set, ctx, ctx_set,
descendant_labels):
 for word in self.filter:
@@ -165,6 +181,13 @@ class Condition(NoFilter):
 self.content = []
 
 
+class NegativeCondition(OnlyFilter):
+def __init__(self, line):
+Filter.__init__(self, line.lstrip(!).rstrip(:))
+self.line = line
+self.content = []
+
+
 class Parser(object):
 
 Parse an input file or string that follows the KVM Test Config File format
@@ -229,24 +252,15 @@ class Parser(object):
 if type(obj) is Op:
 new_content.append(t)
 continue
-elif type(obj) is OnlyFilter:
-if not obj.might_match(ctx, ctx_set, labels):
-self._debug(filter did not pass: %r (%s:%s),
-obj.line, filename, linenum)
-failed_filters.append(t)
-return False
-elif obj.match(ctx, ctx_set):
-continue
-elif type(obj) is NoFilter:
-if obj.match(ctx, ctx_set):
+# obj is an OnlyFilter/NoFilter/Condition/NegativeCondition
+if obj.requires_action(ctx, ctx_set, labels):
+# This filter requires action now
+if type(obj) is OnlyFilter or type(obj) is NoFilter:
 self._debug(filter did not pass: %r (%s:%s),
 obj.line, filename, linenum)
 failed_filters.append(t)
 return False
-elif not obj.might_match(ctx, ctx_set, labels):
-continue
-elif type(obj) is Condition:
-if obj.match(ctx, ctx_set):
+else:
 self._debug(conditional block matches: %r 
(%s:%s),
 obj.line, filename, linenum)
 # Check and unpack the content inside this Condition
@@ -259,9 +273,12 @@ class Parser(object):
 failed_filters.append(t)
 return False
 continue
-elif not obj.might_match(ctx, ctx_set, labels):
-continue
-new_content.append(t)
+elif obj.is_irrelevant(ctx, ctx_set, labels):
+# This filter is no longer relevant and can be removed
+continue
+else:
+# Keep the filter and check it again later
+new_content.append(t)
 return True
 
 def might_pass(failed_ctx,
@@ -429,7 +446,8 @@ class Parser(object):
 # Parse 'variants'
 if line == variants::
 # 'variants' is not allowed inside a conditional block
-if isinstance(node, Condition):
+if (isinstance(node, Condition) or
+isinstance(node, NegativeCondition)):
 raise ParserError('variants' is not allowed inside a 
   conditional block,
   None, cr.filename, linenum)
@@ -481,7 +499,10 @@ class Parser(object):
 cr.set_next_line(line[index:], indent, linenum)
 line = line[:index]
 try:
-cond = Condition(line)
+

[PATCH] qemu-kvm: Mark VCPU state dirty on creation

2011-02-21 Thread Jan Kiszka
This avoids that early cpu_synchronize_state calls try to retrieve an
uninitialized state from the kernel, which even causes a deadlock.

Signed-off-by: Jan Kiszka jan.kis...@siemens.com
---

Corresponding upstream patch will come with next uq/master series.

 qemu-kvm.c |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/qemu-kvm.c b/qemu-kvm.c
index 15e552a..106f148 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -259,6 +259,7 @@ static void kvm_create_vcpu(CPUState *env, int id)
 
 env-kvm_fd = r;
 env-kvm_state = kvm_state;
+env-kvm_vcpu_dirty = 1;
 
 mmap_size = kvm_ioctl(kvm_state, KVM_GET_VCPU_MMAP_SIZE, 0);
 if (mmap_size  0) {
-- 
1.7.1
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


End-User Question: XP-Guest produces high load for no obvious reason

2011-02-21 Thread Andre Tann
Hello everyone,

I'm not sure if this is the correct mailing list with my topic. But the
website says that end user's questions are welcome. So please give me a
hint if this is the wrong place.


On the host:

# rpm -qa | grep kvm
kvm-0.12.5-1.2.1.x86_64

# uname -a
Linux alphawkst 2.6.34.7-0.7-desktop #1 SMP PREEMPT 2010-12-13 11:13:53
+0100 x86_64 x86_64 x86_64 GNU/Linux



The guest is a Windows XP 32 Bit, and everything was fine until
yesterday. For no obvious reason, as soon as the guest starts one of the
two cpu cores has 100% load if started with -smp 1, and both cpu cores
have 100% load if started with -smp 2. There is no process inside the
guest that generates this load. Instead, the guest is idling at 0-5%
(that's what the windows task manager gives in the status line).


The guest is started with:

# qemu-kvm -M pc-0.12 -enable-kvm -smp 2 -m 1024 -name XP2Use -uuid
ad148b2d-b99e-c9de-7c0c-4ab6b716f3ec -rtc base=localtime -boot c -drive
file=/var/lib/libvirt/images/XP2Use.img,if=ide,index=0,boot=on,format=raw,cache=unsafe
-net nic,model=rtl8139,vlan=0,macaddr=52:54:00:5d:a2:a5 -net tap -vnc
127.0.0.1:0 -k de -vga std -usbdevice tablet


The same happens if the machine is started via virt-manager, in which
case I find the following in the process list:

# ps aux | grep qemu
root 12909  100 13.1 1231668 1076428 ? Sl   12:36   0:20
/usr/bin/qemu-kvm -S -M pc-0.12 -enable-kvm -m 1024 -smp
1,sockets=1,cores=1,threads=1 -name XP2Use -uuid
ad148b2d-b99e-c9de-7c0c-4ab6b716f3ec -nodefaults -chardev
socket,id=monitor,path=/var/lib/libvirt/qemu/XP2Use.monitor,server,nowait
-mon chardev=monitor,mode=readline -rtc base=localtime -boot dc -drive
if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device
ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
file=/var/lib/libvirt/images/XP2Use.img,if=none,id=drive-
ide0-0-0,boot=on,format=raw
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0
-device rtl8139,vlan=0,id=net0,mac=52:54:00:5d:a2:a5,bus=pci.0,addr=0x4
-net tap,fd=38,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device
isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc
127.0.0.1:0 -k en-us -vga std -device virtio-balloon-
pci,id=balloon0,bus=pci.0,addr=0x3


Can anyone see a reason for this behavior?


And again, if I'm wrong here, please point me to the right place.

Thanks!

-- 
Andre Tann

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] stop and show registers on error.

2011-02-21 Thread Avi Kivity

On 02/21/2011 12:16 PM, Gleb Natapov wrote:

Signed-off-by: Gleb Natapovg...@redhat.com
diff --git a/qemu-kvm.c b/qemu-kvm.c
index 49cd683..2f3f683 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -644,8 +644,7 @@ int kvm_run(CPUState *env)
  break;
  #endif
case KVM_EXIT_INTERNAL_ERROR:
-kvm_handle_internal_error(env, run);
-r = 1;
+r = kvm_handle_internal_error(env, run);
break;
  default:
  if (kvm_arch_run(env)) {
@@ -1233,6 +1232,7 @@ int kvm_cpu_exec(CPUState *env)
  r = kvm_run(env);
  if (r  0) {
  printf(kvm_run returned %d\n, r);
+kvm_show_regs(env);
  vm_stop(0);
  }


'info registers'.

--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] stop and show registers on error.

2011-02-21 Thread Gleb Natapov
On Mon, Feb 21, 2011 at 02:24:19PM +0200, Avi Kivity wrote:
 On 02/21/2011 12:16 PM, Gleb Natapov wrote:
 Signed-off-by: Gleb Natapovg...@redhat.com
 diff --git a/qemu-kvm.c b/qemu-kvm.c
 index 49cd683..2f3f683 100644
 --- a/qemu-kvm.c
 +++ b/qemu-kvm.c
 @@ -644,8 +644,7 @@ int kvm_run(CPUState *env)
   break;
   #endif
  case KVM_EXIT_INTERNAL_ERROR:
 -kvm_handle_internal_error(env, run);
 -r = 1;
 +r = kvm_handle_internal_error(env, run);
  break;
   default:
   if (kvm_arch_run(env)) {
 @@ -1233,6 +1232,7 @@ int kvm_cpu_exec(CPUState *env)
   r = kvm_run(env);
   if (r  0) {
   printf(kvm_run returned %d\n, r);
 +kvm_show_regs(env);
   vm_stop(0);
   }
 
 'info registers'.
 
That is if you can reproduce. For useful bug reports it is better to
have them printed for user.

--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] KVM test: encapsulate unittests build and move it to parent class

2011-02-21 Thread Lucas Meneghel Rodrigues
In installer.py, we realized that we could benefit from having
the unittests build in pretty much all build types, after all,
if the userspace is sufficiently new we can run the unittests,
regardless of the way the binaries were build.

Encapsulate the unittests build and install to a method and
move that method to the base installer, making all install
methods benefit from it. Just take extra care to certify
the unittests are properly linked.

Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/installer.py |   78 
 1 files changed, 47 insertions(+), 31 deletions(-)

diff --git a/client/tests/kvm/installer.py b/client/tests/kvm/installer.py
index a757223..6b2a6fe 100644
--- a/client/tests/kvm/installer.py
+++ b/client/tests/kvm/installer.py
@@ -231,6 +231,42 @@ class BaseInstaller(object):
 self._full_module_list = list(self._module_list())
 
 
+def install_unittests(self):
+userspace_srcdir = os.path.join(self.srcdir, kvm_userspace)
+test_repo = self.params.get(test_git_repo)
+test_branch = self.params.get(test_branch, master)
+test_commit = self.params.get(test_commit, None)
+test_lbranch = self.params.get(test_lbranch, master)
+
+if test_repo:
+test_srcdir = os.path.join(self.srcdir, kvm-unit-tests)
+kvm_utils.get_git_branch(test_repo, test_branch, test_srcdir,
+ test_commit, test_lbranch)
+unittest_cfg = os.path.join(test_srcdir, 'x86',
+'unittests.cfg')
+self.test_srcdir = test_srcdir
+else:
+unittest_cfg = os.path.join(userspace_srcdir, 'kvm', 'test', 'x86',
+'unittests.cfg')
+self.unittest_cfg = None
+if os.path.isfile(unittest_cfg):
+self.unittest_cfg = unittest_cfg
+else:
+if test_repo:
+logging.error(No unittest config file %s found, skipping 
+  unittest build, self.unittest_cfg)
+
+self.unittest_prefix = None
+if self.unittest_cfg:
+logging.info(Building and installing unittests)
+os.chdir(os.path.dirname(os.path.dirname(self.unittest_cfg)))
+utils.system('./configure --prefix=%s' % self.prefix)
+utils.system('make')
+utils.system('make install')
+self.unittest_prefix = os.path.join(self.prefix, 'share', 'qemu',
+'tests')
+
+
 def full_module_list(self):
 Return the module list used by the installer
 
@@ -341,11 +377,13 @@ class YumInstaller(BaseInstaller):
 
 
 def install(self):
+self.install_unittests()
 self._clean_previous_installs()
 self._get_packages()
 self._install_packages()
 create_symlinks(test_bindir=self.test_bindir,
-bin_list=self.qemu_bin_paths)
+bin_list=self.qemu_bin_paths,
+unittest=self.unittest_prefix)
 self.reload_modules_if_needed()
 if self.save_results:
 save_build(self.srcdir, self.results_dir)
@@ -386,8 +424,10 @@ class KojiInstaller(YumInstaller):
 super(KojiInstaller, self)._clean_previous_installs()
 self._get_packages()
 super(KojiInstaller, self)._install_packages()
+self.install_unittests()
 create_symlinks(test_bindir=self.test_bindir,
-bin_list=self.qemu_bin_paths)
+bin_list=self.qemu_bin_paths,
+unittest=self.unittest_prefix)
 self.reload_modules_if_needed()
 if self.save_results:
 save_build(self.srcdir, self.results_dir)
@@ -532,7 +572,10 @@ class SourceDirInstaller(BaseInstaller):
 utils.system(make install)
 if self.path_to_roms:
 install_roms(self.path_to_roms, self.prefix)
-create_symlinks(self.test_bindir, self.prefix)
+self.install_unittests()
+create_symlinks(test_bindir=self.test_bindir,
+prefix=self.prefix,
+unittest=self.unittest_prefix)
 
 
 def _load_modules(self, mod_list):
@@ -564,22 +607,18 @@ class GitInstaller(SourceDirInstaller):
 kernel_repo = params.get(git_repo)
 user_repo = params.get(user_git_repo)
 kmod_repo = params.get(kmod_repo)
-test_repo = params.get(test_git_repo)
 
 kernel_branch = params.get(kernel_branch, master)
 user_branch = params.get(user_branch, master)
 kmod_branch = params.get(kmod_branch, master)
-test_branch = params.get(test_branch, master)
 
 kernel_lbranch = params.get(kernel_lbranch, master)
 user_lbranch = params.get(user_lbranch, master)
 kmod_lbranch = params.get(kmod_lbranch, master)
-

Re: [PATCH] stop and show registers on error.

2011-02-21 Thread Jan Kiszka
On 2011-02-21 13:25, Gleb Natapov wrote:
 On Mon, Feb 21, 2011 at 02:24:19PM +0200, Avi Kivity wrote:
 On 02/21/2011 12:16 PM, Gleb Natapov wrote:
 Signed-off-by: Gleb Natapovg...@redhat.com
 diff --git a/qemu-kvm.c b/qemu-kvm.c
 index 49cd683..2f3f683 100644
 --- a/qemu-kvm.c
 +++ b/qemu-kvm.c
 @@ -644,8 +644,7 @@ int kvm_run(CPUState *env)
  break;
  #endif
 case KVM_EXIT_INTERNAL_ERROR:
 -kvm_handle_internal_error(env, run);
 -r = 1;
 +r = kvm_handle_internal_error(env, run);
 break;
  default:
  if (kvm_arch_run(env)) {
 @@ -1233,6 +1232,7 @@ int kvm_cpu_exec(CPUState *env)
  r = kvm_run(env);
  if (r  0) {
  printf(kvm_run returned %d\n, r);
 +kvm_show_regs(env);
  vm_stop(0);
  }

 'info registers'.

 That is if you can reproduce. For useful bug reports it is better to
 have them printed for user.

FWIW, this patch just aligns qemu-kvm to what upstream already does and
what qemu-kvm will soon or later do as well when it starts using the
upstream loop. I'm neutral /wrt applying this patch before the cleanup
or achieving the same by consolidating the code.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Slow disk IO on virtio kvm guests with Centos 5.5 as hypervisor

2011-02-21 Thread Thomas Broda
On Wed, 16 Feb 2011 16:07:54 +0100, Thomas Broda tho...@bassfimass.de
wrote:

 I will try Fedora then.

I've installed Fedora 14 on the hypervisor last weekend, and the disk
performance is as bad as it used to be under Centos 5.5.

I think this related to the RAID controller. A Google search for p400
slow raid5 actually says it all :-)

I think I'll get another disk and switch to RAID10 instead.

-- 
Thomas
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] stop and show registers on error.

2011-02-21 Thread Gleb Natapov
On Mon, Feb 21, 2011 at 01:53:15PM +0100, Jan Kiszka wrote:
 On 2011-02-21 13:25, Gleb Natapov wrote:
  On Mon, Feb 21, 2011 at 02:24:19PM +0200, Avi Kivity wrote:
  On 02/21/2011 12:16 PM, Gleb Natapov wrote:
  Signed-off-by: Gleb Natapovg...@redhat.com
  diff --git a/qemu-kvm.c b/qemu-kvm.c
  index 49cd683..2f3f683 100644
  --- a/qemu-kvm.c
  +++ b/qemu-kvm.c
  @@ -644,8 +644,7 @@ int kvm_run(CPUState *env)
   break;
   #endif
case KVM_EXIT_INTERNAL_ERROR:
  -kvm_handle_internal_error(env, run);
  -r = 1;
  +r = kvm_handle_internal_error(env, run);
break;
   default:
   if (kvm_arch_run(env)) {
  @@ -1233,6 +1232,7 @@ int kvm_cpu_exec(CPUState *env)
   r = kvm_run(env);
   if (r  0) {
   printf(kvm_run returned %d\n, r);
  +kvm_show_regs(env);
   vm_stop(0);
   }
 
  'info registers'.
 
  That is if you can reproduce. For useful bug reports it is better to
  have them printed for user.
 
 FWIW, this patch just aligns qemu-kvm to what upstream already does and
 what qemu-kvm will soon or later do as well when it starts using the
 upstream loop. I'm neutral /wrt applying this patch before the cleanup
 or achieving the same by consolidating the code.
 
Indeed. When I got no register dump on error I stared at the wrong code
for 20 minutes trying to understand how it could happen. Having two kvm
implementations in the same tree is such fun! 

--
Gleb.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] KVM test: Move enumerate test dicts code to kvm_utils.run_tests()

2011-02-21 Thread Lucas Meneghel Rodrigues
Signed-off-by: Lucas Meneghel Rodrigues l...@redhat.com
---
 client/tests/kvm/control  |7 ++-
 client/tests/kvm/kvm_utils.py |9 ++---
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/client/tests/kvm/control b/client/tests/kvm/control
index be37678..d9ff70c 100644
--- a/client/tests/kvm/control
+++ b/client/tests/kvm/control
@@ -39,7 +39,7 @@ str = 
 parser = kvm_config.Parser()
 parser.parse_file(os.path.join(kvm_test_dir, build.cfg))
 parser.parse_string(str)
-if not kvm_utils.run_tests(parser.get_dicts(), job):
+if not kvm_utils.run_tests(parser, job):
 logging.error(KVM build step failed, exiting.)
 sys.exit(1)
 
@@ -68,10 +68,7 @@ if args:
 pass
 parser.parse_string(str)
 
-logging.info(Selected tests:)
-for i, d in enumerate(parser.get_dicts()):
-logging.info(Test %4d:  %s % (i + 1, d[shortname]))
-kvm_utils.run_tests(parser.get_dicts(), job)
+kvm_utils.run_tests(parser, job)
 
 # Generate a nice HTML report inside the job's results dir
 kvm_utils.create_report(kvm_test_dir, job.resultdir)
diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index 9e25a0a..de52b65 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -1084,20 +1084,23 @@ def get_hash_from_file(hash_path, dvd_basename):
 return line.split()[0]
 
 
-def run_tests(test_list, job):
+def run_tests(parser, job):
 
 Runs the sequence of KVM tests based on the list of dictionaries
 generated by the configuration system, handling dependencies.
 
-@param test_list: List with all dictionary test parameters.
+@param parser: Config parser object.
 @param job: Autotest job object.
 
 @return: True, if all tests ran passed, False if any of them failed.
 
+for i, d in enumerate(parser.get_dicts()):
+logging.info(Test %4d:  %s % (i + 1, d[shortname]))
+
 status_dict = {}
 failed = False
 
-for dict in test_list:
+for dict in parser.get_dicts():
 if dict.get(skip) == yes:
 continue
 dependencies_satisfied = True
-- 
1.7.4

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


KVM Agenda for Feb 22

2011-02-21 Thread Juan Quintela

please send in any agenda items you are interested in covering.

thanks, Juan.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 5/6] KVM: X86: Delegate tsc-offset calculation to architecture code

2011-02-21 Thread Roedel, Joerg
(Sorry for the delay, I had to spend some days sick at home :-( )

On Fri, Feb 11, 2011 at 05:12:29PM -0500, Zachary Amsden wrote:
 On 02/09/2011 12:29 PM, Joerg Roedel wrote:

 So I've gone over this series and the only issue I see so far is with 
 this patch, and it doesn't have to do with upstream code, rather with 
 code I was about to send.
 
 Logically, the compensation done by adjust_tsc_offset should also be 
 scaled; currently, this happens only for reasons, both of which are 
 meant to deal with unstable TSCs; since TSC scaling won't happen on 
 those processors with unstable TSCs, we don't need to worry about it there.

The tsc_offset is applied after the TSC is scaled so there is no good
way to scale the offset with the TSC value itself.
What we can do is to use guest-tsc values only when we calculate an
adjustment. So any tsc-offset adjustment made with adjust_tsc_offset()
needs to be a function of guest-tsc values. One call-place of the
function already does this and the other one can be converted easily.
I'll do that in the next version of this patch-set.
From what I understand of your upcoming patch the accumulation of
tsc-offsets could also be calculated from guest-tsc values instead of
native_read_tsc() values, no?

Regards,

Joerg

-- 
AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 6/6] KVM: X86: Implement userspace interface to set virtual_tsc_khz

2011-02-21 Thread Roedel, Joerg
On Sun, Feb 13, 2011 at 10:12:14AM -0500, Avi Kivity wrote:
 On 02/09/2011 07:29 PM, Joerg Roedel wrote:
  This patch implements two new vm-ioctls to get and set the
  virtual_tsc_khz if the machine supports tsc-scaling. Setting
  the tsc-frequency is only possible before userspace creates
  any vcpu.
 
  Signed-off-by: Joerg Roedeljoerg.roe...@amd.com
  ---
arch/x86/include/asm/kvm_host.h |1 +
arch/x86/kvm/svm.c  |3 +++
arch/x86/kvm/x86.c  |   38 
  ++
include/linux/kvm.h |4 
4 files changed, 46 insertions(+), 0 deletions(-)
 
 
 Documentation/kvm/api.txt +
 
  @@ -633,6 +633,7 @@ int kvm_pv_mmu_op(struct kvm_vcpu *vcpu, unsigned long 
  bytes,
u8 kvm_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn);
 
extern bool tdp_enabled;
  +extern int kvm_has_tsc_control;
 
 bool
 
 
  +   case KVM_SET_TSC_KHZ: {
  +   u32 user_tsc_khz;
  +
  +   if (!kvm_has_tsc_control)
  +   break;
  +
  +   /*
  +* We force the tsc frequency to be set before any
  +* vcpu is created
  +*/
  +   if (atomic_read(kvm-online_vcpus)  0)
  +   goto out;
 
 What if a vcpu is created here?  No locking AFAICS.
 
  +
  +   user_tsc_khz = arg;
  +
  +   kvm_arch_set_tsc_khz(kvm, user_tsc_khz);
  +
 
 Error check for impossible values (0, values the tsc multiplier can't 
 reach)?
 
  +   r = 0;
  +   goto out;
  +   }
  +   case KVM_GET_TSC_KHZ: {
  +
  +   if (!kvm_has_tsc_control)
  +   break;
  +
  +   r = -EFAULT;
  +   if (copy_to_user(argp,kvm-arch.virtual_tsc_khz, 
  sizeof(__u32)))
  +   goto out;
 
 Should be the return value, no?
 
  +
  +   r = 0;
  +   goto out;
  +   }
 
  default:
  ;
 
 
  @@ -677,6 +678,9 @@ struct kvm_clock_data {
#define KVM_SET_PIT2  _IOW(KVMIO,  0xa0, struct 
  kvm_pit_state2)
/* Available with KVM_CAP_PPC_GET_PVINFO */
#define KVM_PPC_GET_PVINFO  _IOW(KVMIO,  0xa1, struct 
  kvm_ppc_pvinfo)
  +/* Available with KVM_CAP_TSC_CONTROL */
  +#define KVM_SET_TSC_KHZ   _IOW(KVMIO,  0xa2, __u32)
  +#define KVM_GET_TSC_KHZ   _IOR(KVMIO,  0xa3, __u32)

 
 _IO() - use arg or return value
 _IOW/_IOR - copy_to/from_user()
 
 pick one, but don't mix.

Thanks, I'll fix these issued in the next version.

Regards,

Joerg

-- 
AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 0/6] KVM support for TSC scaling

2011-02-21 Thread Roedel, Joerg
On Sun, Feb 13, 2011 at 10:19:19AM -0500, Avi Kivity wrote:
 On 02/09/2011 07:29 PM, Joerg Roedel wrote:
  Hi Avi, Marcelo,
 
  here is the patch-set to implement the TSC-scaling feature of upcoming
  AMD CPUs. When this feature is supported the CPU provides a new MSR
  which holds a multiplier for the hardware TSC which is applied on the
  value rdtsc[p] and reads of MSR 0x10. This feature can be used to
  emulate a given tsc frequency for the guest.
  Patch 1 is not directly related to this patch-set because it only fixes
  a bug which prevented me from testing these patches. In fact it fixes
  the same bug Andre sent a patch for. But after the discussion about his
  patch he told me to just post my patch and thus here it is.
 
 
 Questions:
 - the tsc multiplier really is a multiplier, right?  Not an addend that 
 is added every cycle.

Yes, it is a real multiplier. But writes to the TSC-MSR will change the
unscaled TSC value.

 
 So
 
  wrmsr(TSC, 1e9)
  wrmsr(TSC_MULT, 2.)
  t = rdtsc()
 
 will return about 2e9, not 1e9 + 2*(time to execute the code snippet) ?

Right. And if you exchange the two wrmsr calls it will still give you
the same result.

 - what's the cost of wrmsr(TSC_MULT)?

Hard to tell by now because I only have numbers for pre-production
hardware. 

 There are really two ways to implement this feature.  One is fully 
 generic, like you did.  The other is to implement it at the host level - 
 have a sysfs file and/or kernel parameter for the desired tsc frequency, 
 write it once, and forget about it.  Trust management to set the host 
 tsc frequency to the same value on all hosts in a migration cluster.

The motivation here is mostly the flexibility. Scale the TSC for the
whole migration cluster only makes sense if all hosts there support the
feature. But the most likely scenario is that existing migration
clusters will be extended by new machines and guests will be migrated
there. And these guests should be able to see the same TSC frequency on
the new host as the had on the old one. The older machines in the
cluster may even have different TSC frequencys. With this flexible
implementation those scenarios are possible. A host-wide setting for the
scaling will make the feature useless in those (common) scenarios.

Regards,

Joerg

-- 
AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Qemu-devel] KVM Agenda for Feb 22

2011-02-21 Thread Anthony Liguori

On 02/21/2011 11:12 AM, Juan Quintela wrote:

please send in any agenda items you are interested in covering.
   


 - 0.14.0 release is out, thanks to everyone that participated!  Let's 
discuss what worked well, what could be improved.

 - 0.15 planning
 - Should we do a bump to 1.0?

 - Probably should have a placeholder for something like criteria for 
block format acceptance


Regards,

Anthony Liguori


thanks, Juan.

   


--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread James Neave
On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson
alex.william...@redhat.com wrote:
 On Sat, 2011-02-12 at 16:04 +, James Neave wrote:
 On Tue, Feb 8, 2011 at 10:17 AM, James Neave robo...@gmail.com wrote:
  On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund ke...@kelu.dk wrote:
  2011/2/7 Daniel P. Berrange berra...@redhat.com:
  On Sat, Feb 05, 2011 at 04:34:01PM +, James Neave wrote:
  Hi,
 
  I'm trying to pass a NOVA-T-500 TV Tuner card through to a gust VM.
  I'm getting the error The driver 'pci-stub' is occupying your device
  :08:06.2
 
  This is a rather misleading error message. It is *expected* that
  pci-stub will occupy the device. Unfortunately the rest of the
  error messages QEMU is printing aren't much help either, but
  ultimately something is returning -EBUSY in the PCI device assign
  step
 
  James, as far as I remember, I had the same issue when I set up my
  system. Looking at my current (working) boot-script, apparently I've
  added a 4th line which removes the pci-stub again as a
  workaroundand it works:
 
  echo  0016  /sys/bus/pci/drivers/pci-stub/new_id
  echo :04:08.0  /sys/bus/pci/drivers/ivtv/unbind
  echo :04:08.0  /sys/bus/pci/drivers/pci-stub/bind
  echo  0016  /sys/bus/pci/drivers/pci-stub/remove_id
 
  Best regards
  Kenni
 
 
  Hi Kenni,
 
  Can I get a bit more information on boot-script please? Which file
  exaclty have you put this in? Did you write your own service script
  and put it in init.d?
 
  I've tried this:
 
  echo 8086 10b9  /sys/bus/pci/drivers/pci-stub/new_id
  echo :01:00.0  /sys/bus/pci/devices/:01:00.0/driver/unbind
  echo :01:00.0  /sys/bus/pci/drivers/pci-stub/bind
 
  I'll try it again with the fourth line added, manually before I start the 
  VM.
 
  How come yours is 'echo PCI  /sys/bus/drivers/DRIVERNAME/unbind'
  and mine is echo PCI  /sys/bus/pci/devices/PCI/driver/unbind
 
  Obviously one looks up which driver is being used by the PCI id, but
  how do I look up which driver my PCI card is using?
 
  Thanks,
 
  James.
 

 Hi,

 OK, adding the fourth line does nothing, changing the second line to
 driver rather than device does nothing, including in combination
 with fourth line there/not there.

 Yep, the last line is just removing the id from pci-stub, it doesn't
 change anything about devices that are already bound to it.  driver vs
 device are just different ways to get to the same thing.

 So I'm still stuck, can anybody else help?
 Perhaps point me to a guide on how to compile the latest qemu-kvm
 against my kernel?

 I don't know why you're getting -EBUSY for this device, but maybe we can
 start from a clean slate and see if it helps.  Here's what I would
 suggest:

 echo :08:06.0  /sys/bus/pci/devices/:08:06.0/driver/unbind
 echo :08:06.1  /sys/bus/pci/devices/:08:06.1/driver/unbind
 echo :08:06.2  /sys/bus/pci/devices/:08:06.2/driver/unbind
 echo :08:0e.0  /sys/bus/pci/devices/:08:0e.0/driver/unbind

 Note we have to knock out the firewire because it shares an interrupt
 with the ehci device you're trying to assign.  We want to remove the USB
 controller entirely from the host.  Your dmesg indicates the host is
 still seeing the device via the uhci ports, and isn't happy about it.
 You can ignore pci-stub for the moment, it's just a way to keep drivers
 from claiming the device, it's not required for device assignment.  Now,
 instead of only trying to assign the ehci, let's move the whole usb
 controller to the guest:

 -device pci-assign,host=08:06.0,addr=5.0 \
 -device pci-assign,host=08:06.1,addr=5.1 \
 -device pci-assign,host=08:06.2,addr=5.2

 (slot 5 on the guest is arbitrary, pick something else if you need to)
 If that works, then you can bind all those devices to pci-stub and it
 should still work.

 Alex

Hi,

Sorry about the slow reply, I hosed all of my PCs fiddling with
compiling the latest qemu, took me a while to put it all back together
again in between work!

I'm afraid I use virtmanager, although I guess using the add pci
device function is the same as -device pci-assign? It does seem to
add it to the command line that gets written out to the log files in
/var/log/libvirt/qemu.

Nevertheless, I've tried that and still not luck, the log output is
similar, with one extra line:

http://pastebin.com/MJ6aqjNq

Using raw in/out ioport access (sysfs - Input/output error)

Here's the dmesg output:
http://pastebin.com/AE1euUN1

I'm going to google that new line and consider trying the libvirt ppa
that accompanies the qemu-kvm ppa I'm using.

Many Thanks,

James.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread Alex Williamson
On Mon, 2011-02-21 at 20:31 +, James Neave wrote:
 On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson
 alex.william...@redhat.com wrote:
  On Sat, 2011-02-12 at 16:04 +, James Neave wrote:
  On Tue, Feb 8, 2011 at 10:17 AM, James Neave robo...@gmail.com wrote:
   On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund ke...@kelu.dk wrote:
   2011/2/7 Daniel P. Berrange berra...@redhat.com:
   On Sat, Feb 05, 2011 at 04:34:01PM +, James Neave wrote:
   Hi,
  
   I'm trying to pass a NOVA-T-500 TV Tuner card through to a gust VM.
   I'm getting the error The driver 'pci-stub' is occupying your device
   :08:06.2
  
   This is a rather misleading error message. It is *expected* that
   pci-stub will occupy the device. Unfortunately the rest of the
   error messages QEMU is printing aren't much help either, but
   ultimately something is returning -EBUSY in the PCI device assign
   step
  
   James, as far as I remember, I had the same issue when I set up my
   system. Looking at my current (working) boot-script, apparently I've
   added a 4th line which removes the pci-stub again as a
   workaroundand it works:
  
   echo  0016  /sys/bus/pci/drivers/pci-stub/new_id
   echo :04:08.0  /sys/bus/pci/drivers/ivtv/unbind
   echo :04:08.0  /sys/bus/pci/drivers/pci-stub/bind
   echo  0016  /sys/bus/pci/drivers/pci-stub/remove_id
  
   Best regards
   Kenni
  
  
   Hi Kenni,
  
   Can I get a bit more information on boot-script please? Which file
   exaclty have you put this in? Did you write your own service script
   and put it in init.d?
  
   I've tried this:
  
   echo 8086 10b9  /sys/bus/pci/drivers/pci-stub/new_id
   echo :01:00.0  /sys/bus/pci/devices/:01:00.0/driver/unbind
   echo :01:00.0  /sys/bus/pci/drivers/pci-stub/bind
  
   I'll try it again with the fourth line added, manually before I start 
   the VM.
  
   How come yours is 'echo PCI  /sys/bus/drivers/DRIVERNAME/unbind'
   and mine is echo PCI  /sys/bus/pci/devices/PCI/driver/unbind
  
   Obviously one looks up which driver is being used by the PCI id, but
   how do I look up which driver my PCI card is using?
  
   Thanks,
  
   James.
  
 
  Hi,
 
  OK, adding the fourth line does nothing, changing the second line to
  driver rather than device does nothing, including in combination
  with fourth line there/not there.
 
  Yep, the last line is just removing the id from pci-stub, it doesn't
  change anything about devices that are already bound to it.  driver vs
  device are just different ways to get to the same thing.
 
  So I'm still stuck, can anybody else help?
  Perhaps point me to a guide on how to compile the latest qemu-kvm
  against my kernel?
 
  I don't know why you're getting -EBUSY for this device, but maybe we can
  start from a clean slate and see if it helps.  Here's what I would
  suggest:
 
  echo :08:06.0  /sys/bus/pci/devices/:08:06.0/driver/unbind
  echo :08:06.1  /sys/bus/pci/devices/:08:06.1/driver/unbind
  echo :08:06.2  /sys/bus/pci/devices/:08:06.2/driver/unbind
  echo :08:0e.0  /sys/bus/pci/devices/:08:0e.0/driver/unbind
 
  Note we have to knock out the firewire because it shares an interrupt
  with the ehci device you're trying to assign.  We want to remove the USB
  controller entirely from the host.  Your dmesg indicates the host is
  still seeing the device via the uhci ports, and isn't happy about it.
  You can ignore pci-stub for the moment, it's just a way to keep drivers
  from claiming the device, it's not required for device assignment.  Now,
  instead of only trying to assign the ehci, let's move the whole usb
  controller to the guest:
 
  -device pci-assign,host=08:06.0,addr=5.0 \
  -device pci-assign,host=08:06.1,addr=5.1 \
  -device pci-assign,host=08:06.2,addr=5.2
 
  (slot 5 on the guest is arbitrary, pick something else if you need to)
  If that works, then you can bind all those devices to pci-stub and it
  should still work.
 
  Alex
 
 Hi,
 
 Sorry about the slow reply, I hosed all of my PCs fiddling with
 compiling the latest qemu, took me a while to put it all back together
 again in between work!
 
 I'm afraid I use virtmanager, although I guess using the add pci
 device function is the same as -device pci-assign? It does seem to
 add it to the command line that gets written out to the log files in
 /var/log/libvirt/qemu.
 
 Nevertheless, I've tried that and still not luck, the log output is
 similar, with one extra line:
 
 http://pastebin.com/MJ6aqjNq

You actually ended up with:

-device pci-assign,host=08:06.0,id=hostdev0,configfd=58,bus=pci.0,addr=0x6 \
-device pci-assign,host=08:06.1,id=hostdev1,configfd=59,bus=pci.0,addr=0x7 \
-device pci-assign,host=08:06.2,id=hostdev2,configfd=60,bus=pci.0,addr=0x8

Which isn't quite what I was suggesting.  You probably have xml that
looks like this:

hostdev mode='subsystem' type='pci' managed='yes'
  source
address domain='0x' bus='0x08' slot='0x06' function='0x0'/
 

Re: [PATCH 0/6] KVM support for TSC scaling

2011-02-21 Thread Zachary Amsden

On 02/21/2011 12:28 PM, Roedel, Joerg wrote:

On Sun, Feb 13, 2011 at 10:19:19AM -0500, Avi Kivity wrote:
   

On 02/09/2011 07:29 PM, Joerg Roedel wrote:
 

Hi Avi, Marcelo,

here is the patch-set to implement the TSC-scaling feature of upcoming
AMD CPUs. When this feature is supported the CPU provides a new MSR
which holds a multiplier for the hardware TSC which is applied on the
value rdtsc[p] and reads of MSR 0x10. This feature can be used to
emulate a given tsc frequency for the guest.
Patch 1 is not directly related to this patch-set because it only fixes
a bug which prevented me from testing these patches. In fact it fixes
the same bug Andre sent a patch for. But after the discussion about his
patch he told me to just post my patch and thus here it is.

   

Questions:
- the tsc multiplier really is a multiplier, right?  Not an addend that
is added every cycle.
 

Yes, it is a real multiplier. But writes to the TSC-MSR will change the
unscaled TSC value.

   

So

  wrmsr(TSC, 1e9)
  wrmsr(TSC_MULT, 2.)
  t = rdtsc()

will return about 2e9, not 1e9 + 2*(time to execute the code snippet) ?
 

Right. And if you exchange the two wrmsr calls it will still give you
the same result.

   

- what's the cost of wrmsr(TSC_MULT)?
 

Hard to tell by now because I only have numbers for pre-production
hardware.

   

There are really two ways to implement this feature.  One is fully
generic, like you did.  The other is to implement it at the host level -
have a sysfs file and/or kernel parameter for the desired tsc frequency,
write it once, and forget about it.  Trust management to set the host
tsc frequency to the same value on all hosts in a migration cluster.
 

The motivation here is mostly the flexibility. Scale the TSC for the
whole migration cluster only makes sense if all hosts there support the
feature. But the most likely scenario is that existing migration
clusters will be extended by new machines and guests will be migrated
there. And these guests should be able to see the same TSC frequency on
the new host as the had on the old one. The older machines in the
cluster may even have different TSC frequencys. With this flexible
implementation those scenarios are possible. A host-wide setting for the
scaling will make the feature useless in those (common) scenarios.
   


It's also possible to scale the TSCs of the cluster to be matching 
outside of the framework of KVM.  In that case, the VCPU client (qemu) 
simply needs to be smart enough to not request the TSC rate be scaled.  
That approach is completely compatible with this implementation.


If you do indeed want to have mixed speed VMs running on a single host, 
that can also be done with the approach here.


Combining the two - supporting a standard cluster rate via host scaling, 
plus a variable rate for martian VMs (those not conforming to the 
standard cluster rate) would require some more work, as the multiplier 
written back on exit from a martian would not be 1.0, rather something 
else.  Everything else should work as long as tsc_khz still expresses 
the natural rate of the TSC, even when scaled to a standard cluster 
rate.  In that case, you can also pursue Avi's suggestion of skipping 
the MSR loads for VMs where the rate matches the host rate.


Adding an export to the kernel indicating the currently applied scaling 
rate may not be a bad idea if you want to support such an implementation 
in the future.


I did have one slight concern about scaling in general.  What happens 
when the CPU khz rate is not uniformly detected across machines or 
clusters?  In general, it does vary a bit, I see differences out to the 
5th digit of precision on the same machine.  This is close enough to be 
within the range of NTP correction (500 ppm), but also small enough to 
represent real clock differences (and of course, there is some 
measurement error).


If you are within the threshold where NTP can correct the time, you may 
not want to apply a multiplier to the TSC at all.  Again, this decision 
can be made in the userspace component, but it's an important 
consideration to bring up for the qemu patches that will be required to 
support this.


Zach
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread James Neave
On Mon, Feb 21, 2011 at 10:49 PM, James Neave robo...@gmail.com wrote:
 On Mon, Feb 21, 2011 at 10:25 PM, James Neave robo...@gmail.com wrote:
 On Mon, Feb 21, 2011 at 9:01 PM, Alex Williamson
 alex.william...@redhat.com wrote:
 On Mon, 2011-02-21 at 20:31 +, James Neave wrote:
 On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson
 alex.william...@redhat.com wrote:
  On Sat, 2011-02-12 at 16:04 +, James Neave wrote:
  On Tue, Feb 8, 2011 at 10:17 AM, James Neave robo...@gmail.com wrote:
   On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund ke...@kelu.dk wrote:
   2011/2/7 Daniel P. Berrange berra...@redhat.com:
   On Sat, Feb 05, 2011 at 04:34:01PM +, James Neave wrote:
   Hi,
  
   I'm trying to pass a NOVA-T-500 TV Tuner card through to a gust VM.
   I'm getting the error The driver 'pci-stub' is occupying your 
   device
   :08:06.2
  
   This is a rather misleading error message. It is *expected* that
   pci-stub will occupy the device. Unfortunately the rest of the
   error messages QEMU is printing aren't much help either, but
   ultimately something is returning -EBUSY in the PCI device assign
   step
  
   James, as far as I remember, I had the same issue when I set up my
   system. Looking at my current (working) boot-script, apparently I've
   added a 4th line which removes the pci-stub again as a
   workaroundand it works:
  
   echo  0016  /sys/bus/pci/drivers/pci-stub/new_id
   echo :04:08.0  /sys/bus/pci/drivers/ivtv/unbind
   echo :04:08.0  /sys/bus/pci/drivers/pci-stub/bind
   echo  0016  /sys/bus/pci/drivers/pci-stub/remove_id
  
   Best regards
   Kenni
  
  
   Hi Kenni,
  
   Can I get a bit more information on boot-script please? Which file
   exaclty have you put this in? Did you write your own service script
   and put it in init.d?
  
   I've tried this:
  
   echo 8086 10b9  /sys/bus/pci/drivers/pci-stub/new_id
   echo :01:00.0  /sys/bus/pci/devices/:01:00.0/driver/unbind
   echo :01:00.0  /sys/bus/pci/drivers/pci-stub/bind
  
   I'll try it again with the fourth line added, manually before I start 
   the VM.
  
   How come yours is 'echo PCI  /sys/bus/drivers/DRIVERNAME/unbind'
   and mine is echo PCI  /sys/bus/pci/devices/PCI/driver/unbind
  
   Obviously one looks up which driver is being used by the PCI id, but
   how do I look up which driver my PCI card is using?
  
   Thanks,
  
   James.
  
 
  Hi,
 
  OK, adding the fourth line does nothing, changing the second line to
  driver rather than device does nothing, including in combination
  with fourth line there/not there.
 
  Yep, the last line is just removing the id from pci-stub, it doesn't
  change anything about devices that are already bound to it.  driver vs
  device are just different ways to get to the same thing.
 
  So I'm still stuck, can anybody else help?
  Perhaps point me to a guide on how to compile the latest qemu-kvm
  against my kernel?
 
  I don't know why you're getting -EBUSY for this device, but maybe we can
  start from a clean slate and see if it helps.  Here's what I would
  suggest:
 
  echo :08:06.0  /sys/bus/pci/devices/:08:06.0/driver/unbind
  echo :08:06.1  /sys/bus/pci/devices/:08:06.1/driver/unbind
  echo :08:06.2  /sys/bus/pci/devices/:08:06.2/driver/unbind
  echo :08:0e.0  /sys/bus/pci/devices/:08:0e.0/driver/unbind
 
  Note we have to knock out the firewire because it shares an interrupt
  with the ehci device you're trying to assign.  We want to remove the USB
  controller entirely from the host.  Your dmesg indicates the host is
  still seeing the device via the uhci ports, and isn't happy about it.
  You can ignore pci-stub for the moment, it's just a way to keep drivers
  from claiming the device, it's not required for device assignment.  Now,
  instead of only trying to assign the ehci, let's move the whole usb
  controller to the guest:
 
  -device pci-assign,host=08:06.0,addr=5.0 \
  -device pci-assign,host=08:06.1,addr=5.1 \
  -device pci-assign,host=08:06.2,addr=5.2
 
  (slot 5 on the guest is arbitrary, pick something else if you need to)
  If that works, then you can bind all those devices to pci-stub and it
  should still work.
 
  Alex

 Hi,

 Sorry about the slow reply, I hosed all of my PCs fiddling with
 compiling the latest qemu, took me a while to put it all back together
 again in between work!

 I'm afraid I use virtmanager, although I guess using the add pci
 device function is the same as -device pci-assign? It does seem to
 add it to the command line that gets written out to the log files in
 /var/log/libvirt/qemu.

 Nevertheless, I've tried that and still not luck, the log output is
 similar, with one extra line:

 http://pastebin.com/MJ6aqjNq

 You actually ended up with:

 -device pci-assign,host=08:06.0,id=hostdev0,configfd=58,bus=pci.0,addr=0x6 \
 -device pci-assign,host=08:06.1,id=hostdev1,configfd=59,bus=pci.0,addr=0x7 \
 -device 

Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread James Neave
On Mon, Feb 21, 2011 at 10:55 PM, James Neave robo...@gmail.com wrote:
 On Mon, Feb 21, 2011 at 10:49 PM, James Neave robo...@gmail.com wrote:
 On Mon, Feb 21, 2011 at 10:25 PM, James Neave robo...@gmail.com wrote:
 On Mon, Feb 21, 2011 at 9:01 PM, Alex Williamson
 alex.william...@redhat.com wrote:
 On Mon, 2011-02-21 at 20:31 +, James Neave wrote:
 On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson
 alex.william...@redhat.com wrote:
  On Sat, 2011-02-12 at 16:04 +, James Neave wrote:
  On Tue, Feb 8, 2011 at 10:17 AM, James Neave robo...@gmail.com wrote:
   On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund ke...@kelu.dk wrote:
   2011/2/7 Daniel P. Berrange berra...@redhat.com:
   On Sat, Feb 05, 2011 at 04:34:01PM +, James Neave wrote:
   Hi,
  
   I'm trying to pass a NOVA-T-500 TV Tuner card through to a gust 
   VM.
   I'm getting the error The driver 'pci-stub' is occupying your 
   device
   :08:06.2
  
   This is a rather misleading error message. It is *expected* that
   pci-stub will occupy the device. Unfortunately the rest of the
   error messages QEMU is printing aren't much help either, but
   ultimately something is returning -EBUSY in the PCI device assign
   step
  
   James, as far as I remember, I had the same issue when I set up my
   system. Looking at my current (working) boot-script, apparently I've
   added a 4th line which removes the pci-stub again as a
   workaroundand it works:
  
   echo  0016  /sys/bus/pci/drivers/pci-stub/new_id
   echo :04:08.0  /sys/bus/pci/drivers/ivtv/unbind
   echo :04:08.0  /sys/bus/pci/drivers/pci-stub/bind
   echo  0016  /sys/bus/pci/drivers/pci-stub/remove_id
  
   Best regards
   Kenni
  
  
   Hi Kenni,
  
   Can I get a bit more information on boot-script please? Which file
   exaclty have you put this in? Did you write your own service script
   and put it in init.d?
  
   I've tried this:
  
   echo 8086 10b9  /sys/bus/pci/drivers/pci-stub/new_id
   echo :01:00.0  /sys/bus/pci/devices/:01:00.0/driver/unbind
   echo :01:00.0  /sys/bus/pci/drivers/pci-stub/bind
  
   I'll try it again with the fourth line added, manually before I 
   start the VM.
  
   How come yours is 'echo PCI  /sys/bus/drivers/DRIVERNAME/unbind'
   and mine is echo PCI  /sys/bus/pci/devices/PCI/driver/unbind
  
   Obviously one looks up which driver is being used by the PCI id, but
   how do I look up which driver my PCI card is using?
  
   Thanks,
  
   James.
  
 
  Hi,
 
  OK, adding the fourth line does nothing, changing the second line to
  driver rather than device does nothing, including in combination
  with fourth line there/not there.
 
  Yep, the last line is just removing the id from pci-stub, it doesn't
  change anything about devices that are already bound to it.  driver vs
  device are just different ways to get to the same thing.
 
  So I'm still stuck, can anybody else help?
  Perhaps point me to a guide on how to compile the latest qemu-kvm
  against my kernel?
 
  I don't know why you're getting -EBUSY for this device, but maybe we can
  start from a clean slate and see if it helps.  Here's what I would
  suggest:
 
  echo :08:06.0  /sys/bus/pci/devices/:08:06.0/driver/unbind
  echo :08:06.1  /sys/bus/pci/devices/:08:06.1/driver/unbind
  echo :08:06.2  /sys/bus/pci/devices/:08:06.2/driver/unbind
  echo :08:0e.0  /sys/bus/pci/devices/:08:0e.0/driver/unbind
 
  Note we have to knock out the firewire because it shares an interrupt
  with the ehci device you're trying to assign.  We want to remove the USB
  controller entirely from the host.  Your dmesg indicates the host is
  still seeing the device via the uhci ports, and isn't happy about it.
  You can ignore pci-stub for the moment, it's just a way to keep drivers
  from claiming the device, it's not required for device assignment.  Now,
  instead of only trying to assign the ehci, let's move the whole usb
  controller to the guest:
 
  -device pci-assign,host=08:06.0,addr=5.0 \
  -device pci-assign,host=08:06.1,addr=5.1 \
  -device pci-assign,host=08:06.2,addr=5.2
 
  (slot 5 on the guest is arbitrary, pick something else if you need to)
  If that works, then you can bind all those devices to pci-stub and it
  should still work.
 
  Alex

 Hi,

 Sorry about the slow reply, I hosed all of my PCs fiddling with
 compiling the latest qemu, took me a while to put it all back together
 again in between work!

 I'm afraid I use virtmanager, although I guess using the add pci
 device function is the same as -device pci-assign? It does seem to
 add it to the command line that gets written out to the log files in
 /var/log/libvirt/qemu.

 Nevertheless, I've tried that and still not luck, the log output is
 similar, with one extra line:

 http://pastebin.com/MJ6aqjNq

 You actually ended up with:

 -device pci-assign,host=08:06.0,id=hostdev0,configfd=58,bus=pci.0,addr=0x6 
 \
 -device 

Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread James Neave
On Mon, Feb 21, 2011 at 10:25 PM, James Neave robo...@gmail.com wrote:
 On Mon, Feb 21, 2011 at 9:01 PM, Alex Williamson
 alex.william...@redhat.com wrote:
 On Mon, 2011-02-21 at 20:31 +, James Neave wrote:
 On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson
 alex.william...@redhat.com wrote:
  On Sat, 2011-02-12 at 16:04 +, James Neave wrote:
  On Tue, Feb 8, 2011 at 10:17 AM, James Neave robo...@gmail.com wrote:
   On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund ke...@kelu.dk wrote:
   2011/2/7 Daniel P. Berrange berra...@redhat.com:
   On Sat, Feb 05, 2011 at 04:34:01PM +, James Neave wrote:
   Hi,
  
   I'm trying to pass a NOVA-T-500 TV Tuner card through to a gust VM.
   I'm getting the error The driver 'pci-stub' is occupying your 
   device
   :08:06.2
  
   This is a rather misleading error message. It is *expected* that
   pci-stub will occupy the device. Unfortunately the rest of the
   error messages QEMU is printing aren't much help either, but
   ultimately something is returning -EBUSY in the PCI device assign
   step
  
   James, as far as I remember, I had the same issue when I set up my
   system. Looking at my current (working) boot-script, apparently I've
   added a 4th line which removes the pci-stub again as a
   workaroundand it works:
  
   echo  0016  /sys/bus/pci/drivers/pci-stub/new_id
   echo :04:08.0  /sys/bus/pci/drivers/ivtv/unbind
   echo :04:08.0  /sys/bus/pci/drivers/pci-stub/bind
   echo  0016  /sys/bus/pci/drivers/pci-stub/remove_id
  
   Best regards
   Kenni
  
  
   Hi Kenni,
  
   Can I get a bit more information on boot-script please? Which file
   exaclty have you put this in? Did you write your own service script
   and put it in init.d?
  
   I've tried this:
  
   echo 8086 10b9  /sys/bus/pci/drivers/pci-stub/new_id
   echo :01:00.0  /sys/bus/pci/devices/:01:00.0/driver/unbind
   echo :01:00.0  /sys/bus/pci/drivers/pci-stub/bind
  
   I'll try it again with the fourth line added, manually before I start 
   the VM.
  
   How come yours is 'echo PCI  /sys/bus/drivers/DRIVERNAME/unbind'
   and mine is echo PCI  /sys/bus/pci/devices/PCI/driver/unbind
  
   Obviously one looks up which driver is being used by the PCI id, but
   how do I look up which driver my PCI card is using?
  
   Thanks,
  
   James.
  
 
  Hi,
 
  OK, adding the fourth line does nothing, changing the second line to
  driver rather than device does nothing, including in combination
  with fourth line there/not there.
 
  Yep, the last line is just removing the id from pci-stub, it doesn't
  change anything about devices that are already bound to it.  driver vs
  device are just different ways to get to the same thing.
 
  So I'm still stuck, can anybody else help?
  Perhaps point me to a guide on how to compile the latest qemu-kvm
  against my kernel?
 
  I don't know why you're getting -EBUSY for this device, but maybe we can
  start from a clean slate and see if it helps.  Here's what I would
  suggest:
 
  echo :08:06.0  /sys/bus/pci/devices/:08:06.0/driver/unbind
  echo :08:06.1  /sys/bus/pci/devices/:08:06.1/driver/unbind
  echo :08:06.2  /sys/bus/pci/devices/:08:06.2/driver/unbind
  echo :08:0e.0  /sys/bus/pci/devices/:08:0e.0/driver/unbind
 
  Note we have to knock out the firewire because it shares an interrupt
  with the ehci device you're trying to assign.  We want to remove the USB
  controller entirely from the host.  Your dmesg indicates the host is
  still seeing the device via the uhci ports, and isn't happy about it.
  You can ignore pci-stub for the moment, it's just a way to keep drivers
  from claiming the device, it's not required for device assignment.  Now,
  instead of only trying to assign the ehci, let's move the whole usb
  controller to the guest:
 
  -device pci-assign,host=08:06.0,addr=5.0 \
  -device pci-assign,host=08:06.1,addr=5.1 \
  -device pci-assign,host=08:06.2,addr=5.2
 
  (slot 5 on the guest is arbitrary, pick something else if you need to)
  If that works, then you can bind all those devices to pci-stub and it
  should still work.
 
  Alex

 Hi,

 Sorry about the slow reply, I hosed all of my PCs fiddling with
 compiling the latest qemu, took me a while to put it all back together
 again in between work!

 I'm afraid I use virtmanager, although I guess using the add pci
 device function is the same as -device pci-assign? It does seem to
 add it to the command line that gets written out to the log files in
 /var/log/libvirt/qemu.

 Nevertheless, I've tried that and still not luck, the log output is
 similar, with one extra line:

 http://pastebin.com/MJ6aqjNq

 You actually ended up with:

 -device pci-assign,host=08:06.0,id=hostdev0,configfd=58,bus=pci.0,addr=0x6 \
 -device pci-assign,host=08:06.1,id=hostdev1,configfd=59,bus=pci.0,addr=0x7 \
 -device pci-assign,host=08:06.2,id=hostdev2,configfd=60,bus=pci.0,addr=0x8

 Which isn't quite what I was suggesting.  You probably have xml 

Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread James Neave
On Mon, Feb 21, 2011 at 9:01 PM, Alex Williamson
alex.william...@redhat.com wrote:
 On Mon, 2011-02-21 at 20:31 +, James Neave wrote:
 On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson
 alex.william...@redhat.com wrote:
  On Sat, 2011-02-12 at 16:04 +, James Neave wrote:
  On Tue, Feb 8, 2011 at 10:17 AM, James Neave robo...@gmail.com wrote:
   On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund ke...@kelu.dk wrote:
   2011/2/7 Daniel P. Berrange berra...@redhat.com:
   On Sat, Feb 05, 2011 at 04:34:01PM +, James Neave wrote:
   Hi,
  
   I'm trying to pass a NOVA-T-500 TV Tuner card through to a gust VM.
   I'm getting the error The driver 'pci-stub' is occupying your device
   :08:06.2
  
   This is a rather misleading error message. It is *expected* that
   pci-stub will occupy the device. Unfortunately the rest of the
   error messages QEMU is printing aren't much help either, but
   ultimately something is returning -EBUSY in the PCI device assign
   step
  
   James, as far as I remember, I had the same issue when I set up my
   system. Looking at my current (working) boot-script, apparently I've
   added a 4th line which removes the pci-stub again as a
   workaroundand it works:
  
   echo  0016  /sys/bus/pci/drivers/pci-stub/new_id
   echo :04:08.0  /sys/bus/pci/drivers/ivtv/unbind
   echo :04:08.0  /sys/bus/pci/drivers/pci-stub/bind
   echo  0016  /sys/bus/pci/drivers/pci-stub/remove_id
  
   Best regards
   Kenni
  
  
   Hi Kenni,
  
   Can I get a bit more information on boot-script please? Which file
   exaclty have you put this in? Did you write your own service script
   and put it in init.d?
  
   I've tried this:
  
   echo 8086 10b9  /sys/bus/pci/drivers/pci-stub/new_id
   echo :01:00.0  /sys/bus/pci/devices/:01:00.0/driver/unbind
   echo :01:00.0  /sys/bus/pci/drivers/pci-stub/bind
  
   I'll try it again with the fourth line added, manually before I start 
   the VM.
  
   How come yours is 'echo PCI  /sys/bus/drivers/DRIVERNAME/unbind'
   and mine is echo PCI  /sys/bus/pci/devices/PCI/driver/unbind
  
   Obviously one looks up which driver is being used by the PCI id, but
   how do I look up which driver my PCI card is using?
  
   Thanks,
  
   James.
  
 
  Hi,
 
  OK, adding the fourth line does nothing, changing the second line to
  driver rather than device does nothing, including in combination
  with fourth line there/not there.
 
  Yep, the last line is just removing the id from pci-stub, it doesn't
  change anything about devices that are already bound to it.  driver vs
  device are just different ways to get to the same thing.
 
  So I'm still stuck, can anybody else help?
  Perhaps point me to a guide on how to compile the latest qemu-kvm
  against my kernel?
 
  I don't know why you're getting -EBUSY for this device, but maybe we can
  start from a clean slate and see if it helps.  Here's what I would
  suggest:
 
  echo :08:06.0  /sys/bus/pci/devices/:08:06.0/driver/unbind
  echo :08:06.1  /sys/bus/pci/devices/:08:06.1/driver/unbind
  echo :08:06.2  /sys/bus/pci/devices/:08:06.2/driver/unbind
  echo :08:0e.0  /sys/bus/pci/devices/:08:0e.0/driver/unbind
 
  Note we have to knock out the firewire because it shares an interrupt
  with the ehci device you're trying to assign.  We want to remove the USB
  controller entirely from the host.  Your dmesg indicates the host is
  still seeing the device via the uhci ports, and isn't happy about it.
  You can ignore pci-stub for the moment, it's just a way to keep drivers
  from claiming the device, it's not required for device assignment.  Now,
  instead of only trying to assign the ehci, let's move the whole usb
  controller to the guest:
 
  -device pci-assign,host=08:06.0,addr=5.0 \
  -device pci-assign,host=08:06.1,addr=5.1 \
  -device pci-assign,host=08:06.2,addr=5.2
 
  (slot 5 on the guest is arbitrary, pick something else if you need to)
  If that works, then you can bind all those devices to pci-stub and it
  should still work.
 
  Alex

 Hi,

 Sorry about the slow reply, I hosed all of my PCs fiddling with
 compiling the latest qemu, took me a while to put it all back together
 again in between work!

 I'm afraid I use virtmanager, although I guess using the add pci
 device function is the same as -device pci-assign? It does seem to
 add it to the command line that gets written out to the log files in
 /var/log/libvirt/qemu.

 Nevertheless, I've tried that and still not luck, the log output is
 similar, with one extra line:

 http://pastebin.com/MJ6aqjNq

 You actually ended up with:

 -device pci-assign,host=08:06.0,id=hostdev0,configfd=58,bus=pci.0,addr=0x6 \
 -device pci-assign,host=08:06.1,id=hostdev1,configfd=59,bus=pci.0,addr=0x7 \
 -device pci-assign,host=08:06.2,id=hostdev2,configfd=60,bus=pci.0,addr=0x8

 Which isn't quite what I was suggesting.  You probably have xml that
 looks like this:

    hostdev mode='subsystem' type='pci' managed='yes'

Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread Chris Wright
* Alex Williamson (alex.william...@redhat.com) wrote:
 I don't know why you're getting -EBUSY for this device, but maybe we can
 start from a clean slate and see if it helps.  Here's what I would
 suggest:

I bet this is an AMD IOMMU box.  Can we get full dmesg?
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread James Neave
On Mon, Feb 21, 2011 at 11:28 PM, Chris Wright chr...@sous-sol.org wrote:
 * Alex Williamson (alex.william...@redhat.com) wrote:
 I don't know why you're getting -EBUSY for this device, but maybe we can
 start from a clean slate and see if it helps.  Here's what I would
 suggest:

 I bet this is an AMD IOMMU box.  Can we get full dmesg?


Hi!

Good guess, it is indeed an 890FX based board.

Here's the latest dmesg:
http://pastebin.com/0ZSP31uf

Lots and other dumps are available further up this thread.

fwiw, I've somehow managed to get it to do something different.
After passing though the soundcard, installing Ubuntu 10.10 on the VM
and trying some of the other 14.x PCI devices I managed to really
upset libvirt.
After restarting it on the host server, I added 08:06.0, 1 and 2 again
and got slightly further:

[ 5805.521230] firewire_ohci: Removed fw-ohci device.
[ 5812.092791] pci-stub :08:06.0: claimed by stub
[ 5812.092861] pci-stub :08:06.1: claimed by stub
[ 5812.093107] pci-stub :08:06.0: claimed by stub
[ 5812.099889] pci-stub :08:06.1: claimed by stub
[ 5812.102623] pci-stub :08:06.2: claimed by stub
[ 5812.102723] pci-stub :08:06.2: claimed by stub
[ 5812.265948] type=1400 audit(1298331554.277:41): apparmor=STATUS
operation=profile_load
name=libvirt-307bfcd2-9dec-29b7-1b4d-c46cd9d7cdbc pid=3227
comm=apparmor_parser
[ 5812.653784] device vnet0 entered promiscuous mode
[ 5812.655088] virbr0: topology change detected, propagating
[ 5812.655097] virbr0: port 1(vnet0) entering forwarding state
[ 5812.655103] virbr0: port 1(vnet0) entering forwarding state
[ 5812.781203] pci-stub :08:06.0: PCI INT A - GSI 20 (level, low) - IRQ 20
[ 5812.820087] pci-stub :08:06.0: restoring config space at offset
0x1 (was 0x210, writing 0x211)
[ 5813.048427] assign device 0:8:6.0
[ 5813.048463] type=1400 audit(1298331555.057:42): apparmor=DENIED
operation=capable parent=1
profile=libvirt-307bfcd2-9dec-29b7-1b4d-c46cd9d7cdbc pid=3236
comm=kvm capability=17  capname=sys_rawio
[ 5813.048505] deassign device 0:8:6.0
[ 5813.080089] pci-stub :08:06.0: restoring config space at offset
0x1 (was 0x210, writing 0x211)
[ 5813.080116] pci-stub :08:06.0: PCI INT A disabled
[ 5813.277511] type=1400 audit(1298331555.287:43): apparmor=STATUS
operation=profile_remove
name=libvirt-307bfcd2-9dec-29b7-1b4d-c46cd9d7cdbc pid=3251
comm=apparmor_parser
[ 5813.340516] uhci_hcd :08:06.0: PCI INT A - GSI 20 (level, low) - IRQ 20
[ 5813.340534] uhci_hcd :08:06.0: UHCI Host Controller
[ 5813.340660] uhci_hcd :08:06.0: new USB bus registered, assigned
bus number 4
[ 5813.340708] uhci_hcd :08:06.0: irq 20, io base 0x7f00
[ 5813.340717] uhci_hcd :08:06.0: unable to allocate consistent
memory for frame list
[ 5813.340858] uhci_hcd :08:06.0: startup error -16
[ 5813.340950] uhci_hcd :08:06.0: USB bus 4 deregistered
[ 5813.341044] uhci_hcd :08:06.0: PCI INT A disabled
[ 5813.341049] uhci_hcd :08:06.0: init :08:06.0 fail, -16

'apparmor=DENIED', I've fixed those before by adding to
/etc/apparmor/ profiles for libvirt.
I guess that means I have to add rw access for sys_rawio?

That's all for tonight, probably won't get any more time on this until
Wednesday 19:00 GMT.

Many Thanks,

James.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2

2011-02-21 Thread Chris Wright
* James Neave (robo...@gmail.com) wrote:
 Finally, here is the very latest dmesg:
 http://pastebin.com/9HE61K62

OK, this is an AMD IOMMU box.

[0.00] ACPI: IVRS cfcf9830 000E0 (v01  AMD RD890S 00202031 
AMD  )

It's discovered and enalbed properly:

[0.698992] AMD-Vi: Enabling IOMMU at :00:00.2 cap 0x40
[0.710287] AMD-Vi: Lazy IO/TLB flushing enabled

 Does anybody know the debug kernel switches for iommu?

Two helpful kernel commandline options are:

amd_iommu_dump debug (and drop quiet)

The problem is when you attach the device (function) you're getting
stuck up in conflicts with the existing domain for that function.

My guess is that all the functions are behind a PCI to PCI bridge, so the alias
lookup is finding a conflict.

thanks,
-chris
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html