Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Wanpeng Li]

2017-08-10 1:07 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
>
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530

Please have a try. https://lkml.org/lkml/2017/8/10/27

Regards,
Wanpeng Li

>
>
> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>  panic+0x1e4/0x417 kernel/panic.c:180
>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>  report_bug+0x211/0x2d0 lib/bug.c:183
>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
> RSP: 0018:8800627cf670 EFLAGS: 00010297
> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
> RDX:  RSI:  RDI: 880067d08260
> RBP: 8800627cfa40 R08: 0001 R09: 
> R10: 8800627cfa58 R11:  R12: 0001
> R13:  R14: 88006b934880 R15: 880067d08040
>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>  vfs_ioctl fs/ioctl.c:45 [inline]
>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe
> RIP: 0033:0x44ccc9
> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 0044ccc9
> RDX:  RSI: ae80 RDI: 0005
> RBP: 0082 R08: 7f9ecbc38700 R09: 
> R10: 7f9ecbc38700 R11: 0297 R12: 
> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>
>
>
> // autogenerated by syzkaller (http://github.com/google/syzkaller)
> #define _GNU_SOURCE
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
>
> int kvmcpu;
> struct kvm_run *run;
>
> void* thr(void* arg)
> {
>   int res;
>   res = ioctl(kvmcpu, KVM_RUN, 0);
>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>   res, run->exit_reason, run->internal.suberror);
>   return 0;
> }
>
> void test()
> {
>   int i, kvm, kvmvm;
>   pthread_t th[4];
>
>   kvm = open("/dev/kvm", O_RDWR);
>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_SHARED, kvmcpu, 0);
>   srand(getpid());
>   for (i = 0; i < 4; i++) {
> pthread_create([i], 0, thr, 0);
> usleep(rand() % 1);
>   }
>   for (i = 0; i < 4; i++)
> pthread_join(th[i], 0);
> }
>
> int main()
> {
>   for (;;) {
> int pid = fork();
> if (pid < 0)
>   exit(1);
> if (pid == 0) {
>   test();
>   exit(0);
> }
> int status;
> while (waitpid(pid, , __WALL) != pid) {}
>   }
>   return 0;
> }

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Wanpeng Li]

2017-08-10 1:07 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
>
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530

Please have a try. https://lkml.org/lkml/2017/8/10/27

Regards,
Wanpeng Li

>
>
> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>  panic+0x1e4/0x417 kernel/panic.c:180
>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>  report_bug+0x211/0x2d0 lib/bug.c:183
>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
> RSP: 0018:8800627cf670 EFLAGS: 00010297
> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
> RDX:  RSI:  RDI: 880067d08260
> RBP: 8800627cfa40 R08: 0001 R09: 
> R10: 8800627cfa58 R11:  R12: 0001
> R13:  R14: 88006b934880 R15: 880067d08040
>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>  vfs_ioctl fs/ioctl.c:45 [inline]
>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe
> RIP: 0033:0x44ccc9
> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 0044ccc9
> RDX:  RSI: ae80 RDI: 0005
> RBP: 0082 R08: 7f9ecbc38700 R09: 
> R10: 7f9ecbc38700 R11: 0297 R12: 
> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>
>
>
> // autogenerated by syzkaller (http://github.com/google/syzkaller)
> #define _GNU_SOURCE
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
>
> int kvmcpu;
> struct kvm_run *run;
>
> void* thr(void* arg)
> {
>   int res;
>   res = ioctl(kvmcpu, KVM_RUN, 0);
>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>   res, run->exit_reason, run->internal.suberror);
>   return 0;
> }
>
> void test()
> {
>   int i, kvm, kvmvm;
>   pthread_t th[4];
>
>   kvm = open("/dev/kvm", O_RDWR);
>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_SHARED, kvmcpu, 0);
>   srand(getpid());
>   for (i = 0; i < 4; i++) {
> pthread_create([i], 0, thr, 0);
> usleep(rand() % 1);
>   }
>   for (i = 0; i < 4; i++)
> pthread_join(th[i], 0);
> }
>
> int main()
> {
>   for (;;) {
> int pid = fork();
> if (pid < 0)
>   exit(1);
> if (pid == 0) {
>   test();
>   exit(0);
> }
> int status;
> while (waitpid(pid, , __WALL) != pid) {}
>   }
>   return 0;
> }

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Wanpeng Li]

2017-08-10 1:07 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
>
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530
>
>
> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>  panic+0x1e4/0x417 kernel/panic.c:180
>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>  report_bug+0x211/0x2d0 lib/bug.c:183
>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
> RSP: 0018:8800627cf670 EFLAGS: 00010297
> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
> RDX:  RSI:  RDI: 880067d08260
> RBP: 8800627cfa40 R08: 0001 R09: 
> R10: 8800627cfa58 R11:  R12: 0001
> R13:  R14: 88006b934880 R15: 880067d08040
>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>  vfs_ioctl fs/ioctl.c:45 [inline]
>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe
> RIP: 0033:0x44ccc9
> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 0044ccc9
> RDX:  RSI: ae80 RDI: 0005
> RBP: 0082 R08: 7f9ecbc38700 R09: 
> R10: 7f9ecbc38700 R11: 0297 R12: 
> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>
>
>
> // autogenerated by syzkaller (http://github.com/google/syzkaller)
> #define _GNU_SOURCE
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
>
> int kvmcpu;
> struct kvm_run *run;
>
> void* thr(void* arg)
> {
>   int res;
>   res = ioctl(kvmcpu, KVM_RUN, 0);
>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>   res, run->exit_reason, run->internal.suberror);
>   return 0;
> }
>
> void test()
> {
>   int i, kvm, kvmvm;
>   pthread_t th[4];
>
>   kvm = open("/dev/kvm", O_RDWR);
>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_SHARED, kvmcpu, 0);
>   srand(getpid());
>   for (i = 0; i < 4; i++) {
> pthread_create([i], 0, thr, 0);
> usleep(rand() % 1);
>   }
>   for (i = 0; i < 4; i++)
> pthread_join(th[i], 0);
> }
>
> int main()
> {
>   for (;;) {
> int pid = fork();
> if (pid < 0)
>   exit(1);
> if (pid == 0) {
>   test();
>   exit(0);
> }
> int status;
> while (waitpid(pid, , __WALL) != pid) {}
>   }
>   return 0;
> }

Thanks for the report and I can reproduce, I will take a look.

Regards,
Wanpeng Li

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Wanpeng Li]

2017-08-10 1:07 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
>
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530
>
>
> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>  panic+0x1e4/0x417 kernel/panic.c:180
>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>  report_bug+0x211/0x2d0 lib/bug.c:183
>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
> RSP: 0018:8800627cf670 EFLAGS: 00010297
> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
> RDX:  RSI:  RDI: 880067d08260
> RBP: 8800627cfa40 R08: 0001 R09: 
> R10: 8800627cfa58 R11:  R12: 0001
> R13:  R14: 88006b934880 R15: 880067d08040
>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>  vfs_ioctl fs/ioctl.c:45 [inline]
>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe
> RIP: 0033:0x44ccc9
> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 0044ccc9
> RDX:  RSI: ae80 RDI: 0005
> RBP: 0082 R08: 7f9ecbc38700 R09: 
> R10: 7f9ecbc38700 R11: 0297 R12: 
> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>
>
>
> // autogenerated by syzkaller (http://github.com/google/syzkaller)
> #define _GNU_SOURCE
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
>
> int kvmcpu;
> struct kvm_run *run;
>
> void* thr(void* arg)
> {
>   int res;
>   res = ioctl(kvmcpu, KVM_RUN, 0);
>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>   res, run->exit_reason, run->internal.suberror);
>   return 0;
> }
>
> void test()
> {
>   int i, kvm, kvmvm;
>   pthread_t th[4];
>
>   kvm = open("/dev/kvm", O_RDWR);
>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_SHARED, kvmcpu, 0);
>   srand(getpid());
>   for (i = 0; i < 4; i++) {
> pthread_create([i], 0, thr, 0);
> usleep(rand() % 1);
>   }
>   for (i = 0; i < 4; i++)
> pthread_join(th[i], 0);
> }
>
> int main()
> {
>   for (;;) {
> int pid = fork();
> if (pid < 0)
>   exit(1);
> if (pid == 0) {
>   test();
>   exit(0);
> }
> int status;
> while (waitpid(pid, , __WALL) != pid) {}
>   }
>   return 0;
> }

Thanks for the report and I can reproduce, I will take a look.

Regards,
Wanpeng Li

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

On Wed, Aug 9, 2017 at 10:24 PM, David Hildenbrand  wrote:
> On 09.08.2017 19:07, Dmitry Vyukov wrote:
>> Hello,
>>
>> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
>> This is easily reproducible and reproducer is attached at the bottom.
>> The report is on upstream commit
>> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
>> kvm-intel.unrestricted_guest=0 on the machine, with
>> unrestricted_guest=1 the WARNING does not happen. Output of the
>> program is:
>>
>> ret1=0 exit_reason=17 suberror=1
>> ret2=0 exit_reason=8 suberror=65530
>>
>>
>
> I wonder if the following is possible:
>
> 1) we set vcpu->arch.pio.count != 0
>
> 2) we set vcpu->arch.complete_userspace_io = complete_emulated_pio;
> (in x86_emulate_instruction)
>
> 3) in kvm_arch_vcpu_ioctl_run(), we execute
> vcpu->arch.complete_userspace_io and set it to NULL.
>
> 4. complete_emulated_pio() calls complete_emulated_io(), which fails
> (in some different way) and leaves vcpu->arch.pio.count set.
>
> 5. we now have vcpu->arch.pio.count set but
> vcpu->arch.complete_userspace_io cleared.
>
> Same should not apply to vcpu->mmio_needed, as we clear it before
> calling complete_emulated_io() in complete_emulated_mmio().
>
> AFAIK, we cannot simply clear vcpu->arch.pio.count before calling
> complete_emulated_io().
>
> Maybe something like this could help: (untested, wild guess)
>
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 86242ee..60296fef 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7227,10 +7227,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, 
> struct kvm_run *kvm_run)
> int (*cui)(struct kvm_vcpu *) = 
> vcpu->arch.complete_userspace_io;
> vcpu->arch.complete_userspace_io = NULL;
> r = cui(vcpu);
> +   if (!vcpu->arch.complete_userspace_io) {
> +   /* no new handler -> everything handled */
> +   vcpu->arch.pio.count = 0;
> +   vcpu->mmio_needed = 0;
> +   }
> if (r <= 0)
> goto out;
> -   } else
> -   WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed);
> +   }
>
> if (kvm_run->immediate_exit)
> r = -EINTR;
>
>
> Alternatively, something like this
>
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 86242ee..58be388 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7127,9 +7127,15 @@ static inline int complete_emulated_io(struct kvm_vcpu 
> *vcpu)
>
>  static int complete_emulated_pio(struct kvm_vcpu *vcpu)
>  {
> +   int ret;
> +
> BUG_ON(!vcpu->arch.pio.count);
>
> -   return complete_emulated_io(vcpu);
> +   ret = complete_emulated_io(vcpu);
> +   if (ret && !vcpu->arch.complete_userspace_io)
> +   vcpu->arch.pio.count = 0;
> +
> +   return ret;
>  }
>
>  /*
>
>
> Haven't tried to reproduce on upstream yet (at least on my Fedora 25
> I can't trigger it), so only wild guesses.


It is know to happen back to at least 4.3. Make sure you have
kvm-intel.unrestricted_guest=0, it does not reproduce without it (at
least not with this program).

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

On Wed, Aug 9, 2017 at 10:24 PM, David Hildenbrand  wrote:
> On 09.08.2017 19:07, Dmitry Vyukov wrote:
>> Hello,
>>
>> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
>> This is easily reproducible and reproducer is attached at the bottom.
>> The report is on upstream commit
>> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
>> kvm-intel.unrestricted_guest=0 on the machine, with
>> unrestricted_guest=1 the WARNING does not happen. Output of the
>> program is:
>>
>> ret1=0 exit_reason=17 suberror=1
>> ret2=0 exit_reason=8 suberror=65530
>>
>>
>
> I wonder if the following is possible:
>
> 1) we set vcpu->arch.pio.count != 0
>
> 2) we set vcpu->arch.complete_userspace_io = complete_emulated_pio;
> (in x86_emulate_instruction)
>
> 3) in kvm_arch_vcpu_ioctl_run(), we execute
> vcpu->arch.complete_userspace_io and set it to NULL.
>
> 4. complete_emulated_pio() calls complete_emulated_io(), which fails
> (in some different way) and leaves vcpu->arch.pio.count set.
>
> 5. we now have vcpu->arch.pio.count set but
> vcpu->arch.complete_userspace_io cleared.
>
> Same should not apply to vcpu->mmio_needed, as we clear it before
> calling complete_emulated_io() in complete_emulated_mmio().
>
> AFAIK, we cannot simply clear vcpu->arch.pio.count before calling
> complete_emulated_io().
>
> Maybe something like this could help: (untested, wild guess)
>
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 86242ee..60296fef 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7227,10 +7227,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, 
> struct kvm_run *kvm_run)
> int (*cui)(struct kvm_vcpu *) = 
> vcpu->arch.complete_userspace_io;
> vcpu->arch.complete_userspace_io = NULL;
> r = cui(vcpu);
> +   if (!vcpu->arch.complete_userspace_io) {
> +   /* no new handler -> everything handled */
> +   vcpu->arch.pio.count = 0;
> +   vcpu->mmio_needed = 0;
> +   }
> if (r <= 0)
> goto out;
> -   } else
> -   WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed);
> +   }
>
> if (kvm_run->immediate_exit)
> r = -EINTR;
>
>
> Alternatively, something like this
>
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 86242ee..58be388 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7127,9 +7127,15 @@ static inline int complete_emulated_io(struct kvm_vcpu 
> *vcpu)
>
>  static int complete_emulated_pio(struct kvm_vcpu *vcpu)
>  {
> +   int ret;
> +
> BUG_ON(!vcpu->arch.pio.count);
>
> -   return complete_emulated_io(vcpu);
> +   ret = complete_emulated_io(vcpu);
> +   if (ret && !vcpu->arch.complete_userspace_io)
> +   vcpu->arch.pio.count = 0;
> +
> +   return ret;
>  }
>
>  /*
>
>
> Haven't tried to reproduce on upstream yet (at least on my Fedora 25
> I can't trigger it), so only wild guesses.


It is know to happen back to at least 4.3. Make sure you have
kvm-intel.unrestricted_guest=0, it does not reproduce without it (at
least not with this program).

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[David Hildenbrand]

On 09.08.2017 19:07, Dmitry Vyukov wrote:
> Hello,
> 
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
> 
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530
> 
> 

I wonder if the following is possible:

1) we set vcpu->arch.pio.count != 0

2) we set vcpu->arch.complete_userspace_io = complete_emulated_pio;
(in x86_emulate_instruction)

3) in kvm_arch_vcpu_ioctl_run(), we execute
vcpu->arch.complete_userspace_io and set it to NULL.

4. complete_emulated_pio() calls complete_emulated_io(), which fails
(in some different way) and leaves vcpu->arch.pio.count set.

5. we now have vcpu->arch.pio.count set but
vcpu->arch.complete_userspace_io cleared.

Same should not apply to vcpu->mmio_needed, as we clear it before
calling complete_emulated_io() in complete_emulated_mmio().

AFAIK, we cannot simply clear vcpu->arch.pio.count before calling
complete_emulated_io().

Maybe something like this could help: (untested, wild guess)


diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 86242ee..60296fef 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7227,10 +7227,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, 
struct kvm_run *kvm_run)
int (*cui)(struct kvm_vcpu *) = 
vcpu->arch.complete_userspace_io;
vcpu->arch.complete_userspace_io = NULL;
r = cui(vcpu);
+   if (!vcpu->arch.complete_userspace_io) {
+   /* no new handler -> everything handled */
+   vcpu->arch.pio.count = 0;
+   vcpu->mmio_needed = 0;
+   }
if (r <= 0)
goto out;
-   } else
-   WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed);
+   }
 
if (kvm_run->immediate_exit)
r = -EINTR;


Alternatively, something like this


diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 86242ee..58be388 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7127,9 +7127,15 @@ static inline int complete_emulated_io(struct kvm_vcpu 
*vcpu)
 
 static int complete_emulated_pio(struct kvm_vcpu *vcpu)
 {
+   int ret;
+
BUG_ON(!vcpu->arch.pio.count);
 
-   return complete_emulated_io(vcpu);
+   ret = complete_emulated_io(vcpu);
+   if (ret && !vcpu->arch.complete_userspace_io)
+   vcpu->arch.pio.count = 0;
+
+   return ret;
 }
 
 /*


Haven't tried to reproduce on upstream yet (at least on my Fedora 25
I can't trigger it), so only wild guesses.

-- 

Thanks,

David

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[David Hildenbrand]

On 09.08.2017 19:07, Dmitry Vyukov wrote:
> Hello,
> 
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
> 
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530
> 
> 

I wonder if the following is possible:

1) we set vcpu->arch.pio.count != 0

2) we set vcpu->arch.complete_userspace_io = complete_emulated_pio;
(in x86_emulate_instruction)

3) in kvm_arch_vcpu_ioctl_run(), we execute
vcpu->arch.complete_userspace_io and set it to NULL.

4. complete_emulated_pio() calls complete_emulated_io(), which fails
(in some different way) and leaves vcpu->arch.pio.count set.

5. we now have vcpu->arch.pio.count set but
vcpu->arch.complete_userspace_io cleared.

Same should not apply to vcpu->mmio_needed, as we clear it before
calling complete_emulated_io() in complete_emulated_mmio().

AFAIK, we cannot simply clear vcpu->arch.pio.count before calling
complete_emulated_io().

Maybe something like this could help: (untested, wild guess)


diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 86242ee..60296fef 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7227,10 +7227,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, 
struct kvm_run *kvm_run)
int (*cui)(struct kvm_vcpu *) = 
vcpu->arch.complete_userspace_io;
vcpu->arch.complete_userspace_io = NULL;
r = cui(vcpu);
+   if (!vcpu->arch.complete_userspace_io) {
+   /* no new handler -> everything handled */
+   vcpu->arch.pio.count = 0;
+   vcpu->mmio_needed = 0;
+   }
if (r <= 0)
goto out;
-   } else
-   WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed);
+   }
 
if (kvm_run->immediate_exit)
r = -EINTR;


Alternatively, something like this


diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 86242ee..58be388 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7127,9 +7127,15 @@ static inline int complete_emulated_io(struct kvm_vcpu 
*vcpu)
 
 static int complete_emulated_pio(struct kvm_vcpu *vcpu)
 {
+   int ret;
+
BUG_ON(!vcpu->arch.pio.count);
 
-   return complete_emulated_io(vcpu);
+   ret = complete_emulated_io(vcpu);
+   if (ret && !vcpu->arch.complete_userspace_io)
+   vcpu->arch.pio.count = 0;
+
+   return ret;
 }
 
 /*


Haven't tried to reproduce on upstream yet (at least on my Fedora 25
I can't trigger it), so only wild guesses.

-- 

Thanks,

David

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

On Wed, Aug 9, 2017 at 7:09 PM, Dmitry Vyukov  wrote:
> -non-existent email address
>
> On Wed, Aug 9, 2017 at 7:07 PM, Dmitry Vyukov  wrote:
>> Hello,
>>
>> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
>> This is easily reproducible and reproducer is attached at the bottom.
>> The report is on upstream commit
>> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
>> kvm-intel.unrestricted_guest=0 on the machine, with
>> unrestricted_guest=1 the WARNING does not happen. Output of the
>> program is:
>>
>> ret1=0 exit_reason=17 suberror=1
>> ret2=0 exit_reason=8 suberror=65530
>>
>>
>> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
>> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
>> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>>  panic+0x1e4/0x417 kernel/panic.c:180
>>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>>  report_bug+0x211/0x2d0 lib/bug.c:183
>>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
>> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
>> RSP: 0018:8800627cf670 EFLAGS: 00010297
>> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
>> RDX:  RSI:  RDI: 880067d08260
>> RBP: 8800627cfa40 R08: 0001 R09: 
>> R10: 8800627cfa58 R11:  R12: 0001
>> R13:  R14: 88006b934880 R15: 880067d08040
>>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>>  vfs_ioctl fs/ioctl.c:45 [inline]
>>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>>  SYSC_ioctl fs/ioctl.c:700 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>>  entry_SYSCALL_64_fastpath+0x1f/0xbe
>> RIP: 0033:0x44ccc9
>> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
>> RAX: ffda RBX:  RCX: 0044ccc9
>> RDX:  RSI: ae80 RDI: 0005
>> RBP: 0082 R08: 7f9ecbc38700 R09: 
>> R10: 7f9ecbc38700 R11: 0297 R12: 
>> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>>
>>
>>
>> // autogenerated by syzkaller (http://github.com/google/syzkaller)
>> #define _GNU_SOURCE
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>>
>> int kvmcpu;
>> struct kvm_run *run;
>>
>> void* thr(void* arg)
>> {
>>   int res;
>>   res = ioctl(kvmcpu, KVM_RUN, 0);
>>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>>   res, run->exit_reason, run->internal.suberror);
>>   return 0;
>> }
>>
>> void test()
>> {
>>   int i, kvm, kvmvm;
>>   pthread_t th[4];
>>
>>   kvm = open("/dev/kvm", O_RDWR);
>>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
>> MAP_SHARED, kvmcpu, 0);
>>   srand(getpid());
>>   for (i = 0; i < 4; i++) {
>> pthread_create([i], 0, thr, 0);
>> usleep(rand() % 1);
>>   }
>>   for (i = 0; i < 4; i++)
>> pthread_join(th[i], 0);
>> }
>>
>> int main()
>> {
>>   for (;;) {
>> int pid = fork();
>> if (pid < 0)
>>   exit(1);
>> if (pid == 0) {
>>   test();
>>   exit(0);
>> }
>> int status;
>> while (waitpid(pid, , __WALL) != pid) {}
>>   }
>>   return 0;
>> }



For the record, James suggested the following patch, but it does _not_ help:

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6c97c82814c4..21391c9241bc 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5383,6 +5383,7 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu)
vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_EMULATION;
vcpu->run->internal.ndata = 0;
+   vcpu->arch.pio.count = 0;
r = EMULATE_FAIL;
}
kvm_queue_exception(vcpu, UD_VECTOR);

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

On Wed, Aug 9, 2017 at 7:09 PM, Dmitry Vyukov  wrote:
> -non-existent email address
>
> On Wed, Aug 9, 2017 at 7:07 PM, Dmitry Vyukov  wrote:
>> Hello,
>>
>> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
>> This is easily reproducible and reproducer is attached at the bottom.
>> The report is on upstream commit
>> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
>> kvm-intel.unrestricted_guest=0 on the machine, with
>> unrestricted_guest=1 the WARNING does not happen. Output of the
>> program is:
>>
>> ret1=0 exit_reason=17 suberror=1
>> ret2=0 exit_reason=8 suberror=65530
>>
>>
>> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
>> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
>> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>>  panic+0x1e4/0x417 kernel/panic.c:180
>>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>>  report_bug+0x211/0x2d0 lib/bug.c:183
>>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
>> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
>> RSP: 0018:8800627cf670 EFLAGS: 00010297
>> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
>> RDX:  RSI:  RDI: 880067d08260
>> RBP: 8800627cfa40 R08: 0001 R09: 
>> R10: 8800627cfa58 R11:  R12: 0001
>> R13:  R14: 88006b934880 R15: 880067d08040
>>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>>  vfs_ioctl fs/ioctl.c:45 [inline]
>>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>>  SYSC_ioctl fs/ioctl.c:700 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>>  entry_SYSCALL_64_fastpath+0x1f/0xbe
>> RIP: 0033:0x44ccc9
>> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
>> RAX: ffda RBX:  RCX: 0044ccc9
>> RDX:  RSI: ae80 RDI: 0005
>> RBP: 0082 R08: 7f9ecbc38700 R09: 
>> R10: 7f9ecbc38700 R11: 0297 R12: 
>> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>>
>>
>>
>> // autogenerated by syzkaller (http://github.com/google/syzkaller)
>> #define _GNU_SOURCE
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>> #include 
>>
>> int kvmcpu;
>> struct kvm_run *run;
>>
>> void* thr(void* arg)
>> {
>>   int res;
>>   res = ioctl(kvmcpu, KVM_RUN, 0);
>>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>>   res, run->exit_reason, run->internal.suberror);
>>   return 0;
>> }
>>
>> void test()
>> {
>>   int i, kvm, kvmvm;
>>   pthread_t th[4];
>>
>>   kvm = open("/dev/kvm", O_RDWR);
>>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
>> MAP_SHARED, kvmcpu, 0);
>>   srand(getpid());
>>   for (i = 0; i < 4; i++) {
>> pthread_create([i], 0, thr, 0);
>> usleep(rand() % 1);
>>   }
>>   for (i = 0; i < 4; i++)
>> pthread_join(th[i], 0);
>> }
>>
>> int main()
>> {
>>   for (;;) {
>> int pid = fork();
>> if (pid < 0)
>>   exit(1);
>> if (pid == 0) {
>>   test();
>>   exit(0);
>> }
>> int status;
>> while (waitpid(pid, , __WALL) != pid) {}
>>   }
>>   return 0;
>> }



For the record, James suggested the following patch, but it does _not_ help:

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6c97c82814c4..21391c9241bc 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5383,6 +5383,7 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu)
vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_EMULATION;
vcpu->run->internal.ndata = 0;
+   vcpu->arch.pio.count = 0;
r = EMULATE_FAIL;
}
kvm_queue_exception(vcpu, UD_VECTOR);

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

-non-existent email address

On Wed, Aug 9, 2017 at 7:07 PM, Dmitry Vyukov  wrote:
> Hello,
>
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
>
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530
>
>
> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>  panic+0x1e4/0x417 kernel/panic.c:180
>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>  report_bug+0x211/0x2d0 lib/bug.c:183
>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
> RSP: 0018:8800627cf670 EFLAGS: 00010297
> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
> RDX:  RSI:  RDI: 880067d08260
> RBP: 8800627cfa40 R08: 0001 R09: 
> R10: 8800627cfa58 R11:  R12: 0001
> R13:  R14: 88006b934880 R15: 880067d08040
>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>  vfs_ioctl fs/ioctl.c:45 [inline]
>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe
> RIP: 0033:0x44ccc9
> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 0044ccc9
> RDX:  RSI: ae80 RDI: 0005
> RBP: 0082 R08: 7f9ecbc38700 R09: 
> R10: 7f9ecbc38700 R11: 0297 R12: 
> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>
>
>
> // autogenerated by syzkaller (http://github.com/google/syzkaller)
> #define _GNU_SOURCE
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
>
> int kvmcpu;
> struct kvm_run *run;
>
> void* thr(void* arg)
> {
>   int res;
>   res = ioctl(kvmcpu, KVM_RUN, 0);
>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>   res, run->exit_reason, run->internal.suberror);
>   return 0;
> }
>
> void test()
> {
>   int i, kvm, kvmvm;
>   pthread_t th[4];
>
>   kvm = open("/dev/kvm", O_RDWR);
>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_SHARED, kvmcpu, 0);
>   srand(getpid());
>   for (i = 0; i < 4; i++) {
> pthread_create([i], 0, thr, 0);
> usleep(rand() % 1);
>   }
>   for (i = 0; i < 4; i++)
> pthread_join(th[i], 0);
> }
>
> int main()
> {
>   for (;;) {
> int pid = fork();
> if (pid < 0)
>   exit(1);
> if (pid == 0) {
>   test();
>   exit(0);
> }
> int status;
> while (waitpid(pid, , __WALL) != pid) {}
>   }
>   return 0;
> }

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

-non-existent email address

On Wed, Aug 9, 2017 at 7:07 PM, Dmitry Vyukov  wrote:
> Hello,
>
> syzkaller fuzzer has hit the following WARNING in kvm_arch_vcpu_ioctl_run.
> This is easily reproducible and reproducer is attached at the bottom.
> The report is on upstream commit
> 26c5cebfdb6ca799186f1e56be7d6f2480c5012c. This requires setting
> kvm-intel.unrestricted_guest=0 on the machine, with
> unrestricted_guest=1 the WARNING does not happen. Output of the
> program is:
>
> ret1=0 exit_reason=17 suberror=1
> ret2=0 exit_reason=8 suberror=65530
>
>
> WARNING: CPU: 1 PID: 2850 at arch/x86/kvm/x86.c:7223
> kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7223
> CPU: 1 PID: 2850 Comm: a.out Not tainted 4.13.0-rc3+ #445
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x194/0x257 lib/dump_stack.c:52
>  panic+0x1e4/0x417 kernel/panic.c:180
>  __warn+0x1c4/0x1d9 kernel/panic.c:541
>  report_bug+0x211/0x2d0 lib/bug.c:183
>  fixup_bug+0x40/0x90 arch/x86/kernel/traps.c:190
>  do_trap_no_signal arch/x86/kernel/traps.c:224 [inline]
>  do_trap+0x260/0x390 arch/x86/kernel/traps.c:273
>  do_error_trap+0x120/0x390 arch/x86/kernel/traps.c:310
>  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:323
>  invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:846
> RIP: 0010:kvm_arch_vcpu_ioctl_run+0x213/0x5870 arch/x86/kvm/x86.c:7222
> RSP: 0018:8800627cf670 EFLAGS: 00010297
> RAX: 880066c44480 RBX: 88006b07f000 RCX: 880067d0845c
> RDX:  RSI:  RDI: 880067d08260
> RBP: 8800627cfa40 R08: 0001 R09: 
> R10: 8800627cfa58 R11:  R12: 0001
> R13:  R14: 88006b934880 R15: 880067d08040
>  kvm_vcpu_ioctl+0x64c/0x1010 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2592
>  vfs_ioctl fs/ioctl.c:45 [inline]
>  do_vfs_ioctl+0x1b1/0x1520 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe
> RIP: 0033:0x44ccc9
> RSP: 002b:7f9ecbc37db8 EFLAGS: 0297 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 0044ccc9
> RDX:  RSI: ae80 RDI: 0005
> RBP: 0082 R08: 7f9ecbc38700 R09: 
> R10: 7f9ecbc38700 R11: 0297 R12: 
> R13:  R14: 7f9ecbc389c0 R15: 7f9ecbc38700
>
>
>
> // autogenerated by syzkaller (http://github.com/google/syzkaller)
> #define _GNU_SOURCE
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
> #include 
>
> int kvmcpu;
> struct kvm_run *run;
>
> void* thr(void* arg)
> {
>   int res;
>   res = ioctl(kvmcpu, KVM_RUN, 0);
>   printf("ret1=%d exit_reason=%d suberror=%d\n",
>   res, run->exit_reason, run->internal.suberror);
>   return 0;
> }
>
> void test()
> {
>   int i, kvm, kvmvm;
>   pthread_t th[4];
>
>   kvm = open("/dev/kvm", O_RDWR);
>   kvmvm = ioctl(kvm, KVM_CREATE_VM, 0);
>   kvmcpu = ioctl(kvmvm, KVM_CREATE_VCPU, 0);
>   run = (struct kvm_run*)mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_SHARED, kvmcpu, 0);
>   srand(getpid());
>   for (i = 0; i < 4; i++) {
> pthread_create([i], 0, thr, 0);
> usleep(rand() % 1);
>   }
>   for (i = 0; i < 4; i++)
> pthread_join(th[i], 0);
> }
>
> int main()
> {
>   for (;;) {
> int pid = fork();
> if (pid < 0)
>   exit(1);
> if (pid == 0) {
>   test();
>   exit(0);
> }
> int status;
> while (waitpid(pid, , __WALL) != pid) {}
>   }
>   return 0;
> }

Re: kvm: warning in kvm_load_guest_fpu

[Wanpeng Li]

2017-05-10 9:48 GMT+08:00 Wanpeng Li :
> 2017-05-09 22:04 GMT+08:00 Andrey Konovalov :
>> Hi,
>>
>> I've got the following error report while fuzzing the kernel with syzkaller.
>>
>> On commit 2868b2513aa732a99ea4a0a6bf10dc93c1f3dac2 (4.11+).
>>
>> A reproducer and .config are attached.
>
> If there are beauty codes for testing?

It messes the MXCSR Register reserve bits, I will make a patch.

Regards,
Wanpeng Li

>
>>
>> [ cut here ]
>> WARNING: CPU: 0 PID: 4108 at ./arch/x86/include/asm/fpu/internal.h:169
>> kvm_load_guest_fpu.part.163+0x2a9/0x430
>> Modules linked in:
>> CPU: 0 PID: 4108 Comm: a.out Not tainted 4.11.0+ #331
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> task: 880068b2c200 task.stack: 88006921
>> RIP: 0010:copy_kernel_to_fxregs ./arch/x86/include/asm/fpu/internal.h:169
>> RIP: 0010:__copy_kernel_to_fpregs ./arch/x86/include/asm/fpu/internal.h:459
>> RIP: 0010:kvm_load_guest_fpu.part.163+0x2a9/0x430 arch/x86/kvm/x86.c:7596
>> RSP: 0018:8800692176e8 EFLAGS: 00010297
>> RAX: 880068b2c200 RBX: 11000d242edd RCX: 8800696fc5ec
>> RDX:  RSI:  RDI: 880068b2d4c5
>> RBP: 8800692177b0 R08: 11000d242ebf R09: f8f8
>> R10:  R11: 0002 R12: 8800696f8000
>> R13: 880069217788 R14: dc00 R15: 8800696f8000
>> FS:  7fb239f787c0() GS:88006ca0() knlGS:
>> CS:  0010 DS:  ES:  CR0: 80050033
>> CR2: 2001f000 CR3: 6c55a000 CR4: 26f0
>> Call Trace:
>>  kvm_load_guest_fpu arch/x86/kvm/x86.c:6737
>>  vcpu_enter_guest arch/x86/kvm/x86.c:6842
>>  vcpu_run arch/x86/kvm/x86.c:7030
>>  kvm_arch_vcpu_ioctl_run+0x1f61/0x4860 arch/x86/kvm/x86.c:7191
>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2568
>>  vfs_ioctl fs/ioctl.c:45
>>  do_vfs_ioctl+0x1bf/0x1660 fs/ioctl.c:685
>>  SYSC_ioctl fs/ioctl.c:700
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>>  entry_SYSCALL_64_fastpath+0x1f/0xbe arch/x86/entry/entry_64.S:204
>> RIP: 0033:0x7fb23968bb79
>> RSP: 002b:7ffec57db2d8 EFLAGS: 0202 ORIG_RAX: 0010
>> RAX: ffda RBX: 7ffec57db480 RCX: 7fb23968bb79
>> RDX:  RSI: ae80 RDI: 0005
>> RBP: 00400b40 R08:  R09: 
>> R10:  R11: 0202 R12: 
>> R13: 7ffec57db480 R14:  R15: 
>> Code: 4e 00 65 ff 0d f9 72 f5 7e e9 5c fe ff ff e8 7f e4 4e 00 31 c0
>> 49 0f ae 8c 24 00 0b 00 00 85 c0 0f 84 35 fe ff ff e8 67 e4 4e 00 <0f>
>> ff e9 29 fe ff ff e8 5b e4 4e 00 65 ff 05 c4 72 f5 7e 4d 8d
>> ---[ end trace 7a89c6ce24f92b9b ]---

Re: kvm: warning in kvm_load_guest_fpu

[Wanpeng Li]

2017-05-10 9:48 GMT+08:00 Wanpeng Li :
> 2017-05-09 22:04 GMT+08:00 Andrey Konovalov :
>> Hi,
>>
>> I've got the following error report while fuzzing the kernel with syzkaller.
>>
>> On commit 2868b2513aa732a99ea4a0a6bf10dc93c1f3dac2 (4.11+).
>>
>> A reproducer and .config are attached.
>
> If there are beauty codes for testing?

It messes the MXCSR Register reserve bits, I will make a patch.

Regards,
Wanpeng Li

>
>>
>> [ cut here ]
>> WARNING: CPU: 0 PID: 4108 at ./arch/x86/include/asm/fpu/internal.h:169
>> kvm_load_guest_fpu.part.163+0x2a9/0x430
>> Modules linked in:
>> CPU: 0 PID: 4108 Comm: a.out Not tainted 4.11.0+ #331
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> task: 880068b2c200 task.stack: 88006921
>> RIP: 0010:copy_kernel_to_fxregs ./arch/x86/include/asm/fpu/internal.h:169
>> RIP: 0010:__copy_kernel_to_fpregs ./arch/x86/include/asm/fpu/internal.h:459
>> RIP: 0010:kvm_load_guest_fpu.part.163+0x2a9/0x430 arch/x86/kvm/x86.c:7596
>> RSP: 0018:8800692176e8 EFLAGS: 00010297
>> RAX: 880068b2c200 RBX: 11000d242edd RCX: 8800696fc5ec
>> RDX:  RSI:  RDI: 880068b2d4c5
>> RBP: 8800692177b0 R08: 11000d242ebf R09: f8f8
>> R10:  R11: 0002 R12: 8800696f8000
>> R13: 880069217788 R14: dc00 R15: 8800696f8000
>> FS:  7fb239f787c0() GS:88006ca0() knlGS:
>> CS:  0010 DS:  ES:  CR0: 80050033
>> CR2: 2001f000 CR3: 6c55a000 CR4: 26f0
>> Call Trace:
>>  kvm_load_guest_fpu arch/x86/kvm/x86.c:6737
>>  vcpu_enter_guest arch/x86/kvm/x86.c:6842
>>  vcpu_run arch/x86/kvm/x86.c:7030
>>  kvm_arch_vcpu_ioctl_run+0x1f61/0x4860 arch/x86/kvm/x86.c:7191
>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2568
>>  vfs_ioctl fs/ioctl.c:45
>>  do_vfs_ioctl+0x1bf/0x1660 fs/ioctl.c:685
>>  SYSC_ioctl fs/ioctl.c:700
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>>  entry_SYSCALL_64_fastpath+0x1f/0xbe arch/x86/entry/entry_64.S:204
>> RIP: 0033:0x7fb23968bb79
>> RSP: 002b:7ffec57db2d8 EFLAGS: 0202 ORIG_RAX: 0010
>> RAX: ffda RBX: 7ffec57db480 RCX: 7fb23968bb79
>> RDX:  RSI: ae80 RDI: 0005
>> RBP: 00400b40 R08:  R09: 
>> R10:  R11: 0202 R12: 
>> R13: 7ffec57db480 R14:  R15: 
>> Code: 4e 00 65 ff 0d f9 72 f5 7e e9 5c fe ff ff e8 7f e4 4e 00 31 c0
>> 49 0f ae 8c 24 00 0b 00 00 85 c0 0f 84 35 fe ff ff e8 67 e4 4e 00 <0f>
>> ff e9 29 fe ff ff e8 5b e4 4e 00 65 ff 05 c4 72 f5 7e 4d 8d
>> ---[ end trace 7a89c6ce24f92b9b ]---

Re: kvm: warning in kvm_load_guest_fpu

[Wanpeng Li]

2017-05-09 22:04 GMT+08:00 Andrey Konovalov :
> Hi,
>
> I've got the following error report while fuzzing the kernel with syzkaller.
>
> On commit 2868b2513aa732a99ea4a0a6bf10dc93c1f3dac2 (4.11+).
>
> A reproducer and .config are attached.

If there are beauty codes for testing?

>
> [ cut here ]
> WARNING: CPU: 0 PID: 4108 at ./arch/x86/include/asm/fpu/internal.h:169
> kvm_load_guest_fpu.part.163+0x2a9/0x430
> Modules linked in:
> CPU: 0 PID: 4108 Comm: a.out Not tainted 4.11.0+ #331
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> task: 880068b2c200 task.stack: 88006921
> RIP: 0010:copy_kernel_to_fxregs ./arch/x86/include/asm/fpu/internal.h:169
> RIP: 0010:__copy_kernel_to_fpregs ./arch/x86/include/asm/fpu/internal.h:459
> RIP: 0010:kvm_load_guest_fpu.part.163+0x2a9/0x430 arch/x86/kvm/x86.c:7596
> RSP: 0018:8800692176e8 EFLAGS: 00010297
> RAX: 880068b2c200 RBX: 11000d242edd RCX: 8800696fc5ec
> RDX:  RSI:  RDI: 880068b2d4c5
> RBP: 8800692177b0 R08: 11000d242ebf R09: f8f8
> R10:  R11: 0002 R12: 8800696f8000
> R13: 880069217788 R14: dc00 R15: 8800696f8000
> FS:  7fb239f787c0() GS:88006ca0() knlGS:
> CS:  0010 DS:  ES:  CR0: 80050033
> CR2: 2001f000 CR3: 6c55a000 CR4: 26f0
> Call Trace:
>  kvm_load_guest_fpu arch/x86/kvm/x86.c:6737
>  vcpu_enter_guest arch/x86/kvm/x86.c:6842
>  vcpu_run arch/x86/kvm/x86.c:7030
>  kvm_arch_vcpu_ioctl_run+0x1f61/0x4860 arch/x86/kvm/x86.c:7191
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2568
>  vfs_ioctl fs/ioctl.c:45
>  do_vfs_ioctl+0x1bf/0x1660 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe arch/x86/entry/entry_64.S:204
> RIP: 0033:0x7fb23968bb79
> RSP: 002b:7ffec57db2d8 EFLAGS: 0202 ORIG_RAX: 0010
> RAX: ffda RBX: 7ffec57db480 RCX: 7fb23968bb79
> RDX:  RSI: ae80 RDI: 0005
> RBP: 00400b40 R08:  R09: 
> R10:  R11: 0202 R12: 
> R13: 7ffec57db480 R14:  R15: 
> Code: 4e 00 65 ff 0d f9 72 f5 7e e9 5c fe ff ff e8 7f e4 4e 00 31 c0
> 49 0f ae 8c 24 00 0b 00 00 85 c0 0f 84 35 fe ff ff e8 67 e4 4e 00 <0f>
> ff e9 29 fe ff ff e8 5b e4 4e 00 65 ff 05 c4 72 f5 7e 4d 8d
> ---[ end trace 7a89c6ce24f92b9b ]---

Re: kvm: warning in kvm_load_guest_fpu

[Wanpeng Li]

2017-05-09 22:04 GMT+08:00 Andrey Konovalov :
> Hi,
>
> I've got the following error report while fuzzing the kernel with syzkaller.
>
> On commit 2868b2513aa732a99ea4a0a6bf10dc93c1f3dac2 (4.11+).
>
> A reproducer and .config are attached.

If there are beauty codes for testing?

>
> [ cut here ]
> WARNING: CPU: 0 PID: 4108 at ./arch/x86/include/asm/fpu/internal.h:169
> kvm_load_guest_fpu.part.163+0x2a9/0x430
> Modules linked in:
> CPU: 0 PID: 4108 Comm: a.out Not tainted 4.11.0+ #331
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> task: 880068b2c200 task.stack: 88006921
> RIP: 0010:copy_kernel_to_fxregs ./arch/x86/include/asm/fpu/internal.h:169
> RIP: 0010:__copy_kernel_to_fpregs ./arch/x86/include/asm/fpu/internal.h:459
> RIP: 0010:kvm_load_guest_fpu.part.163+0x2a9/0x430 arch/x86/kvm/x86.c:7596
> RSP: 0018:8800692176e8 EFLAGS: 00010297
> RAX: 880068b2c200 RBX: 11000d242edd RCX: 8800696fc5ec
> RDX:  RSI:  RDI: 880068b2d4c5
> RBP: 8800692177b0 R08: 11000d242ebf R09: f8f8
> R10:  R11: 0002 R12: 8800696f8000
> R13: 880069217788 R14: dc00 R15: 8800696f8000
> FS:  7fb239f787c0() GS:88006ca0() knlGS:
> CS:  0010 DS:  ES:  CR0: 80050033
> CR2: 2001f000 CR3: 6c55a000 CR4: 26f0
> Call Trace:
>  kvm_load_guest_fpu arch/x86/kvm/x86.c:6737
>  vcpu_enter_guest arch/x86/kvm/x86.c:6842
>  vcpu_run arch/x86/kvm/x86.c:7030
>  kvm_arch_vcpu_ioctl_run+0x1f61/0x4860 arch/x86/kvm/x86.c:7191
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2568
>  vfs_ioctl fs/ioctl.c:45
>  do_vfs_ioctl+0x1bf/0x1660 fs/ioctl.c:685
>  SYSC_ioctl fs/ioctl.c:700
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
>  entry_SYSCALL_64_fastpath+0x1f/0xbe arch/x86/entry/entry_64.S:204
> RIP: 0033:0x7fb23968bb79
> RSP: 002b:7ffec57db2d8 EFLAGS: 0202 ORIG_RAX: 0010
> RAX: ffda RBX: 7ffec57db480 RCX: 7fb23968bb79
> RDX:  RSI: ae80 RDI: 0005
> RBP: 00400b40 R08:  R09: 
> R10:  R11: 0202 R12: 
> R13: 7ffec57db480 R14:  R15: 
> Code: 4e 00 65 ff 0d f9 72 f5 7e e9 5c fe ff ff e8 7f e4 4e 00 31 c0
> 49 0f ae 8c 24 00 0b 00 00 85 c0 0f 84 35 fe ff ff e8 67 e4 4e 00 <0f>
> ff e9 29 fe ff ff e8 5b e4 4e 00 65 ff 05 c4 72 f5 7e 4d 8d
> ---[ end trace 7a89c6ce24f92b9b ]---

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Mar 14, 2017 at 4:17 PM, Radim Krčmář  wrote:
> 2017-03-12 12:20+0100, Dmitry Vyukov:
>> On Tue, Jan 17, 2017 at 5:00 PM, Dmitry Vyukov  wrote:
>>> On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:


 On 13/01/2017 12:15, Dmitry Vyukov wrote:
>
> I've commented out the WARNING for now, but I am seeing lots of
> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
>
>
> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
> Read of size 8 by task syz-executor2/16715
> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) 
> index:0x0
> flags: 0x500()
> raw: 0500   
> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
> page dumped because: kasan: bad access detected
> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
> 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  kasan_report_error mm/kasan/report.c:213 [inline]
>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
> [inline]
>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: ffda RBX: 2000 RCX: 00445329
> RDX: 0003 RSI: 00af1000 RDI: 2000
> RBP: 006dfe90 R08:  R09: 
> R10: 0032 R11: 0282 R12: 0070
> R13: 0006 R14:  R15: 20001000
> Memory state around the buggy address:
>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>   ^
>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> ==

 This could be related to the gfn_to_rmap issues.
>>>
>>>
>>> Humm... That's possible. Potentially I am not seeing any more of
>>> spte-related crashes after I applied the following patch:
>>>
>>> --- a/virt/kvm/kvm_main.c
>>> +++ b/virt/kvm/kvm_main.c
>>> @@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
>>> /* Check for overlaps */
>>> r = -EEXIST;
>>> kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
>>> -   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
>>> -   (slot->id == id))
>>> +   if (slot->id == id)
>>> continue;
>>> if (!((base_gfn + npages <= slot->base_gfn) ||
>>>   (base_gfn >= slot->base_gfn + slot->npages)))
>
> I don't understand how this fixes the test: the only memslot that the
> test creates is at memory range 0x0-0x1000, which should not overlap
> with any private memslots.
> There should be just the IDENTITY_PAGETABLE_PRIVATE_MEMSLOT @
> 0xfffbc000ul.
>
> Do you get any ouput with this hunk?
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index a17d78759727..7e1929432232 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -888,6 +888,14 @@ static struct kvm_memslots 

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Mar 14, 2017 at 4:17 PM, Radim Krčmář  wrote:
> 2017-03-12 12:20+0100, Dmitry Vyukov:
>> On Tue, Jan 17, 2017 at 5:00 PM, Dmitry Vyukov  wrote:
>>> On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:


 On 13/01/2017 12:15, Dmitry Vyukov wrote:
>
> I've commented out the WARNING for now, but I am seeing lots of
> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
>
>
> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
> Read of size 8 by task syz-executor2/16715
> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) 
> index:0x0
> flags: 0x500()
> raw: 0500   
> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
> page dumped because: kasan: bad access detected
> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
> 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  kasan_report_error mm/kasan/report.c:213 [inline]
>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
> [inline]
>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: ffda RBX: 2000 RCX: 00445329
> RDX: 0003 RSI: 00af1000 RDI: 2000
> RBP: 006dfe90 R08:  R09: 
> R10: 0032 R11: 0282 R12: 0070
> R13: 0006 R14:  R15: 20001000
> Memory state around the buggy address:
>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>   ^
>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> ==

 This could be related to the gfn_to_rmap issues.
>>>
>>>
>>> Humm... That's possible. Potentially I am not seeing any more of
>>> spte-related crashes after I applied the following patch:
>>>
>>> --- a/virt/kvm/kvm_main.c
>>> +++ b/virt/kvm/kvm_main.c
>>> @@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
>>> /* Check for overlaps */
>>> r = -EEXIST;
>>> kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
>>> -   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
>>> -   (slot->id == id))
>>> +   if (slot->id == id)
>>> continue;
>>> if (!((base_gfn + npages <= slot->base_gfn) ||
>>>   (base_gfn >= slot->base_gfn + slot->npages)))
>
> I don't understand how this fixes the test: the only memslot that the
> test creates is at memory range 0x0-0x1000, which should not overlap
> with any private memslots.
> There should be just the IDENTITY_PAGETABLE_PRIVATE_MEMSLOT @
> 0xfffbc000ul.
>
> Do you get any ouput with this hunk?
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index a17d78759727..7e1929432232 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -888,6 +888,14 @@ static struct kvm_memslots *install_new_memslots(struct 
> kvm *kvm,
> return 

Re: kvm: WARNING In kvm_apic_accept_events

[David Hildenbrand]

On 22.03.2017 16:34, Dmitry Vyukov wrote:
> On Wed, Feb 15, 2017 at 11:10 AM, Paolo Bonzini  wrote:
>>
>>
>> On 15/02/2017 10:26, David Hildenbrand wrote:

 Any progress with this? Should we commit this as is?

>>> Paolo, should I resend as a proper notcopieddiff mail?
>>
>> Yes, please do (CCing sta...@vger.kernel.org too).
>>
>> Paolo
> 
> Has this reached any tree? I don't see this in kvm/master.
> 

Whops, seems like I missed Paolo's mail. Will send out the patch in a
minute.

-- 

Thanks,

David

Re: kvm: WARNING In kvm_apic_accept_events

[David Hildenbrand]

On 22.03.2017 16:34, Dmitry Vyukov wrote:
> On Wed, Feb 15, 2017 at 11:10 AM, Paolo Bonzini  wrote:
>>
>>
>> On 15/02/2017 10:26, David Hildenbrand wrote:

 Any progress with this? Should we commit this as is?

>>> Paolo, should I resend as a proper notcopieddiff mail?
>>
>> Yes, please do (CCing sta...@vger.kernel.org too).
>>
>> Paolo
> 
> Has this reached any tree? I don't see this in kvm/master.
> 

Whops, seems like I missed Paolo's mail. Will send out the patch in a
minute.

-- 

Thanks,

David

Re: kvm: WARNING In kvm_apic_accept_events

[Dmitry Vyukov]

On Wed, Feb 15, 2017 at 11:10 AM, Paolo Bonzini  wrote:
>
>
> On 15/02/2017 10:26, David Hildenbrand wrote:
>>>
>>> Any progress with this? Should we commit this as is?
>>>
>> Paolo, should I resend as a proper notcopieddiff mail?
>
> Yes, please do (CCing sta...@vger.kernel.org too).
>
> Paolo

Has this reached any tree? I don't see this in kvm/master.

Re: kvm: WARNING In kvm_apic_accept_events

[Dmitry Vyukov]

On Wed, Feb 15, 2017 at 11:10 AM, Paolo Bonzini  wrote:
>
>
> On 15/02/2017 10:26, David Hildenbrand wrote:
>>>
>>> Any progress with this? Should we commit this as is?
>>>
>> Paolo, should I resend as a proper notcopieddiff mail?
>
> Yes, please do (CCing sta...@vger.kernel.org too).
>
> Paolo

Has this reached any tree? I don't see this in kvm/master.

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Radim Krčmář]

2017-03-12 12:20+0100, Dmitry Vyukov:
> On Tue, Jan 17, 2017 at 5:00 PM, Dmitry Vyukov  wrote:
>> On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:
>>>
>>>
>>> On 13/01/2017 12:15, Dmitry Vyukov wrote:

 I've commented out the WARNING for now, but I am seeing lots of
 use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:


 BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
 arch/x86/kvm/mmu.c:597 at addr 880068ae2008
 Read of size 8 by task syz-executor2/16715
 page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
 flags: 0x500()
 raw: 0500   
 raw: ea00017ec5a0 ea0001783d48 88006aec5d98
 page dumped because: kasan: bad access detected
 CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
 01/01/2011
 Call Trace:
  __dump_stack lib/dump_stack.c:15 [inline]
  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
  kasan_report_error mm/kasan/report.c:213 [inline]
  kasan_report+0x42d/0x460 mm/kasan/report.c:307
  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
 arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
 [inline]
  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
  unmap_region+0x2f8/0x560 mm/mmap.c:2460
  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
  do_mmap_pgoff include/linux/mm.h:2031 [inline]
  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
  entry_SYSCALL_64_fastpath+0x1f/0xc2
 RIP: 0033:0x445329
 RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
 RAX: ffda RBX: 2000 RCX: 00445329
 RDX: 0003 RSI: 00af1000 RDI: 2000
 RBP: 006dfe90 R08:  R09: 
 R10: 0032 R11: 0282 R12: 0070
 R13: 0006 R14:  R15: 20001000
 Memory state around the buggy address:
  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
   ^
  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
 ==
>>>
>>> This could be related to the gfn_to_rmap issues.
>>
>>
>> Humm... That's possible. Potentially I am not seeing any more of
>> spte-related crashes after I applied the following patch:
>>
>> --- a/virt/kvm/kvm_main.c
>> +++ b/virt/kvm/kvm_main.c
>> @@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
>> /* Check for overlaps */
>> r = -EEXIST;
>> kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
>> -   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
>> -   (slot->id == id))
>> +   if (slot->id == id)
>> continue;
>> if (!((base_gfn + npages <= slot->base_gfn) ||
>>   (base_gfn >= slot->base_gfn + slot->npages)))

I don't understand how this fixes the test: the only memslot that the
test creates is at memory range 0x0-0x1000, which should not overlap
with any private memslots.
There should be just the IDENTITY_PAGETABLE_PRIVATE_MEMSLOT @
0xfffbc000ul.

Do you get any ouput with this hunk?

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index a17d78759727..7e1929432232 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -888,6 +888,14 @@ static struct kvm_memslots *install_new_memslots(struct 
kvm *kvm,
return old_memslots;
 }
 
+void kvm_dump_slot(struct kvm_memory_slot *slot)
+{
+   printk("kvm_memory_slot %p { .id = %u, .base_gfn = %#llx, 

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Radim Krčmář]

2017-03-12 12:20+0100, Dmitry Vyukov:
> On Tue, Jan 17, 2017 at 5:00 PM, Dmitry Vyukov  wrote:
>> On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:
>>>
>>>
>>> On 13/01/2017 12:15, Dmitry Vyukov wrote:

 I've commented out the WARNING for now, but I am seeing lots of
 use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:


 BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
 arch/x86/kvm/mmu.c:597 at addr 880068ae2008
 Read of size 8 by task syz-executor2/16715
 page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
 flags: 0x500()
 raw: 0500   
 raw: ea00017ec5a0 ea0001783d48 88006aec5d98
 page dumped because: kasan: bad access detected
 CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
 01/01/2011
 Call Trace:
  __dump_stack lib/dump_stack.c:15 [inline]
  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
  kasan_report_error mm/kasan/report.c:213 [inline]
  kasan_report+0x42d/0x460 mm/kasan/report.c:307
  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
 arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
 [inline]
  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
  unmap_region+0x2f8/0x560 mm/mmap.c:2460
  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
  do_mmap_pgoff include/linux/mm.h:2031 [inline]
  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
  entry_SYSCALL_64_fastpath+0x1f/0xc2
 RIP: 0033:0x445329
 RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
 RAX: ffda RBX: 2000 RCX: 00445329
 RDX: 0003 RSI: 00af1000 RDI: 2000
 RBP: 006dfe90 R08:  R09: 
 R10: 0032 R11: 0282 R12: 0070
 R13: 0006 R14:  R15: 20001000
 Memory state around the buggy address:
  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
   ^
  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
 ==
>>>
>>> This could be related to the gfn_to_rmap issues.
>>
>>
>> Humm... That's possible. Potentially I am not seeing any more of
>> spte-related crashes after I applied the following patch:
>>
>> --- a/virt/kvm/kvm_main.c
>> +++ b/virt/kvm/kvm_main.c
>> @@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
>> /* Check for overlaps */
>> r = -EEXIST;
>> kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
>> -   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
>> -   (slot->id == id))
>> +   if (slot->id == id)
>> continue;
>> if (!((base_gfn + npages <= slot->base_gfn) ||
>>   (base_gfn >= slot->base_gfn + slot->npages)))

I don't understand how this fixes the test: the only memslot that the
test creates is at memory range 0x0-0x1000, which should not overlap
with any private memslots.
There should be just the IDENTITY_PAGETABLE_PRIVATE_MEMSLOT @
0xfffbc000ul.

Do you get any ouput with this hunk?

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index a17d78759727..7e1929432232 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -888,6 +888,14 @@ static struct kvm_memslots *install_new_memslots(struct 
kvm *kvm,
return old_memslots;
 }
 
+void kvm_dump_slot(struct kvm_memory_slot *slot)
+{
+   printk("kvm_memory_slot %p { .id = %u, .base_gfn = %#llx, .npages = 
%lu, "
+  

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Jan 17, 2017 at 5:00 PM, Dmitry Vyukov  wrote:
> On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:
>>
>>
>> On 13/01/2017 12:15, Dmitry Vyukov wrote:
>>>
>>> I've commented out the WARNING for now, but I am seeing lots of
>>> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
>>>
>>>
>>> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
>>> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
>>> Read of size 8 by task syz-executor2/16715
>>> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
>>> flags: 0x500()
>>> raw: 0500   
>>> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
>>> page dumped because: kasan: bad access detected
>>> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>>  kasan_report_error mm/kasan/report.c:213 [inline]
>>>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>>>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>>>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>>>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>>>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>>>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>>>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>>>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>>>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
>>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>>>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>>>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
>>> [inline]
>>>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>>>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>>>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>>>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>>>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>>>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>>>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>>>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>>>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>>>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>>>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x445329
>>> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
>>> RAX: ffda RBX: 2000 RCX: 00445329
>>> RDX: 0003 RSI: 00af1000 RDI: 2000
>>> RBP: 006dfe90 R08:  R09: 
>>> R10: 0032 R11: 0282 R12: 0070
>>> R13: 0006 R14:  R15: 20001000
>>> Memory state around the buggy address:
>>>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>>   ^
>>>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>> ==
>>
>> This could be related to the gfn_to_rmap issues.
>
>
> Humm... That's possible. Potentially I am not seeing any more of
> spte-related crashes after I applied the following patch:
>
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
> /* Check for overlaps */
> r = -EEXIST;
> kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
> -   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
> -   (slot->id == id))
> +   if (slot->id == id)
> continue;
> if (!((base_gfn + npages <= slot->base_gfn) ||
>   (base_gfn >= slot->base_gfn + slot->npages)))




Friendly ping. Just hit it on
mmotm/86292b33d4b79ee03e2f43ea0381ef85f077c760 (without the above
change):

[ cut here ]
WARNING: CPU: 1 PID: 31060 at arch/x86/kvm/mmu.c:682
mmu_spte_clear_track_bits+0x3a1/0x420 arch/x86/kvm/mmu.c:682
CPU: 1 PID: 31060 Comm: syz-executor0 Not tainted 4.11.0-rc1+ #328
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x1a7/0x26a lib/dump_stack.c:52
 panic+0x1f8/0x40f kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 mmu_spte_clear_track_bits+0x3a1/0x420 arch/x86/kvm/mmu.c:682
 drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1323
 mmu_page_zap_pte+0x223/0x350 arch/x86/kvm/mmu.c:2438
 

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Jan 17, 2017 at 5:00 PM, Dmitry Vyukov  wrote:
> On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:
>>
>>
>> On 13/01/2017 12:15, Dmitry Vyukov wrote:
>>>
>>> I've commented out the WARNING for now, but I am seeing lots of
>>> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
>>>
>>>
>>> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
>>> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
>>> Read of size 8 by task syz-executor2/16715
>>> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
>>> flags: 0x500()
>>> raw: 0500   
>>> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
>>> page dumped because: kasan: bad access detected
>>> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>>  kasan_report_error mm/kasan/report.c:213 [inline]
>>>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>>>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>>>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>>>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>>>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>>>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>>>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>>>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>>>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
>>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>>>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>>>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
>>> [inline]
>>>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>>>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>>>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>>>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>>>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>>>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>>>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>>>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>>>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>>>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>>>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x445329
>>> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
>>> RAX: ffda RBX: 2000 RCX: 00445329
>>> RDX: 0003 RSI: 00af1000 RDI: 2000
>>> RBP: 006dfe90 R08:  R09: 
>>> R10: 0032 R11: 0282 R12: 0070
>>> R13: 0006 R14:  R15: 20001000
>>> Memory state around the buggy address:
>>>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>>   ^
>>>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>> ==
>>
>> This could be related to the gfn_to_rmap issues.
>
>
> Humm... That's possible. Potentially I am not seeing any more of
> spte-related crashes after I applied the following patch:
>
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
> /* Check for overlaps */
> r = -EEXIST;
> kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
> -   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
> -   (slot->id == id))
> +   if (slot->id == id)
> continue;
> if (!((base_gfn + npages <= slot->base_gfn) ||
>   (base_gfn >= slot->base_gfn + slot->npages)))




Friendly ping. Just hit it on
mmotm/86292b33d4b79ee03e2f43ea0381ef85f077c760 (without the above
change):

[ cut here ]
WARNING: CPU: 1 PID: 31060 at arch/x86/kvm/mmu.c:682
mmu_spte_clear_track_bits+0x3a1/0x420 arch/x86/kvm/mmu.c:682
CPU: 1 PID: 31060 Comm: syz-executor0 Not tainted 4.11.0-rc1+ #328
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x1a7/0x26a lib/dump_stack.c:52
 panic+0x1f8/0x40f kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 mmu_spte_clear_track_bits+0x3a1/0x420 arch/x86/kvm/mmu.c:682
 drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1323
 mmu_page_zap_pte+0x223/0x350 arch/x86/kvm/mmu.c:2438
 kvm_mmu_page_unlink_children 

Re: kvm: WARNING in nested_vmx_vmexit

[Radim Krčmář]

2017-03-02 20:28+0800, Wanpeng Li:
> 2017-03-02 2:31 GMT+08:00 Radim Krčmář :
>> 2017-03-01 10:44+0100, Dmitry Vyukov:
>>> On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> The following program triggers WARNING in nested_vmx_vmexit:
> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>
>
> [ cut here ]
> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
> 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>  kvm_vcpu_ioctl+0x232/0x1120 
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x451229
> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 00451229
> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
> RBP:  R08:  R09: 
> R10:  R11: 0246 R12: 
> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>
>
> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570

 I git checkout to this commit for linus tree, however the line number
 11029 doesn't match any warning, and there are several warnings in the
 function nested_vmx_vmexit(), could you point out which one?
>>>
>>> Ah sorry, I have some local diff.
>>> So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
>>> https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt
>>>
>>> The warning is:
>>>
>>> /* trying to cancel vmlaunch/vmresume is a bug */
>>> WARN_ON_ONCE(vmx->nested.nested_run_pending);
>>
>> I can't reproduce this one, but it is there:
>>
>> The warning gets thrown while doing
>>
>> (*(uint32_t*)0x20aecfe8 = (uint32_t)0x1);
>> (*(uint32_t*)0x20aecfec = (uint32_t)0x0);
>> (*(uint32_t*)0x20aecff0 = (uint32_t)0x3a);
>> (*(uint32_t*)0x20aecff4 = (uint32_t)0x0);
>> (*(uint64_t*)0x20aecff8 = (uint64_t)0x0);
>> r[29] = syscall(__NR_ioctl, r[4], 0x4008ae89ul,
>> 0x20aecfe8ul, 0, 0, 0, 0, 0, 0);
>>
>> i.e. KVM_SET_MSR ioctl with
>>
>>   struct kvm_msrs {
>> .nmsrs = 1,
>> .pad = 0,
>> .entries = {
>> {.index = MSR_IA32_FEATURE_CONTROL,
>>  .reserved = 0,
>>  .data = 0}
>> }
>>   }
>>
>> and the relevant setter is
>>
>>   case MSR_IA32_FEATURE_CONTROL:
>> if (!vmx_feature_control_msr_valid(vcpu, data) ||
>> (to_vmx(vcpu)->msr_ia32_feature_control &
>>  FEATURE_CONTROL_LOCKED && !msr_info->host_initiated))
>> return 1;
>> vmx->msr_ia32_feature_control = data;
>> if (msr_info->host_initiated && data == 0)
>> vmx_leave_nested(vcpu);
>> break;
>>
>> The condition is wrong as it is no longer only VMX bits in data, so VMX
>> can get disabled while the data is nonzero.  KVM might not need to force
>> nested VM exit when userspace goes crazy, so the condition would just go
>> away ... need to check.
> 
> If the bits just influence the new VMXON execution instead of
> currently running hypervisor?

If nesting is exited some other way during a reset from userspace.
There is nothing like that, so a disgusting

  vmx->nested.nested_run_pending = 0;
  nested_vmx_vmexit(vcpu, -1, 0, 0);

in vmx_leave_nested() is the safest solution.  The CPU is going to be
reset hence there should be nothing pending.

Re: kvm: WARNING in nested_vmx_vmexit

[Radim Krčmář]

2017-03-02 20:28+0800, Wanpeng Li:
> 2017-03-02 2:31 GMT+08:00 Radim Krčmář :
>> 2017-03-01 10:44+0100, Dmitry Vyukov:
>>> On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> The following program triggers WARNING in nested_vmx_vmexit:
> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>
>
> [ cut here ]
> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
> 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>  kvm_vcpu_ioctl+0x232/0x1120 
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x451229
> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 00451229
> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
> RBP:  R08:  R09: 
> R10:  R11: 0246 R12: 
> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>
>
> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570

 I git checkout to this commit for linus tree, however the line number
 11029 doesn't match any warning, and there are several warnings in the
 function nested_vmx_vmexit(), could you point out which one?
>>>
>>> Ah sorry, I have some local diff.
>>> So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
>>> https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt
>>>
>>> The warning is:
>>>
>>> /* trying to cancel vmlaunch/vmresume is a bug */
>>> WARN_ON_ONCE(vmx->nested.nested_run_pending);
>>
>> I can't reproduce this one, but it is there:
>>
>> The warning gets thrown while doing
>>
>> (*(uint32_t*)0x20aecfe8 = (uint32_t)0x1);
>> (*(uint32_t*)0x20aecfec = (uint32_t)0x0);
>> (*(uint32_t*)0x20aecff0 = (uint32_t)0x3a);
>> (*(uint32_t*)0x20aecff4 = (uint32_t)0x0);
>> (*(uint64_t*)0x20aecff8 = (uint64_t)0x0);
>> r[29] = syscall(__NR_ioctl, r[4], 0x4008ae89ul,
>> 0x20aecfe8ul, 0, 0, 0, 0, 0, 0);
>>
>> i.e. KVM_SET_MSR ioctl with
>>
>>   struct kvm_msrs {
>> .nmsrs = 1,
>> .pad = 0,
>> .entries = {
>> {.index = MSR_IA32_FEATURE_CONTROL,
>>  .reserved = 0,
>>  .data = 0}
>> }
>>   }
>>
>> and the relevant setter is
>>
>>   case MSR_IA32_FEATURE_CONTROL:
>> if (!vmx_feature_control_msr_valid(vcpu, data) ||
>> (to_vmx(vcpu)->msr_ia32_feature_control &
>>  FEATURE_CONTROL_LOCKED && !msr_info->host_initiated))
>> return 1;
>> vmx->msr_ia32_feature_control = data;
>> if (msr_info->host_initiated && data == 0)
>> vmx_leave_nested(vcpu);
>> break;
>>
>> The condition is wrong as it is no longer only VMX bits in data, so VMX
>> can get disabled while the data is nonzero.  KVM might not need to force
>> nested VM exit when userspace goes crazy, so the condition would just go
>> away ... need to check.
> 
> If the bits just influence the new VMXON execution instead of
> currently running hypervisor?

If nesting is exited some other way during a reset from userspace.
There is nothing like that, so a disgusting

  vmx->nested.nested_run_pending = 0;
  nested_vmx_vmexit(vcpu, -1, 0, 0);

in vmx_leave_nested() is the safest solution.  The CPU is going to be
reset hence there should be nothing pending.

Re: kvm: WARNING in nested_vmx_vmexit

[Wanpeng Li]

2017-03-02 2:31 GMT+08:00 Radim Krčmář :
> 2017-03-01 10:44+0100, Dmitry Vyukov:
>> On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
>>> 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
 Hello,

 The following program triggers WARNING in nested_vmx_vmexit:
 https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt


 [ cut here ]
 WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
 nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
 CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
 01/01/2011
 Call Trace:
  __dump_stack lib/dump_stack.c:15 [inline]
  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
  panic+0x1fb/0x412 kernel/panic.c:179
  __warn+0x1c4/0x1e0 kernel/panic.c:540
  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
  __msr_io arch/x86/kvm/x86.c:2577 [inline]
  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
  vfs_ioctl fs/ioctl.c:43 [inline]
  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
  SYSC_ioctl fs/ioctl.c:698 [inline]
  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
  entry_SYSCALL_64_fastpath+0x1f/0xc2
 RIP: 0033:0x451229
 RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
 RAX: ffda RBX:  RCX: 00451229
 RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
 RBP:  R08:  R09: 
 R10:  R11: 0246 R12: 
 R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700


 On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
>>>
>>> I git checkout to this commit for linus tree, however the line number
>>> 11029 doesn't match any warning, and there are several warnings in the
>>> function nested_vmx_vmexit(), could you point out which one?
>>
>> Ah sorry, I have some local diff.
>> So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
>> https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt
>>
>> The warning is:
>>
>> /* trying to cancel vmlaunch/vmresume is a bug */
>> WARN_ON_ONCE(vmx->nested.nested_run_pending);
>
> I can't reproduce this one, but it is there:
>
> The warning gets thrown while doing
>
> (*(uint32_t*)0x20aecfe8 = (uint32_t)0x1);
> (*(uint32_t*)0x20aecfec = (uint32_t)0x0);
> (*(uint32_t*)0x20aecff0 = (uint32_t)0x3a);
> (*(uint32_t*)0x20aecff4 = (uint32_t)0x0);
> (*(uint64_t*)0x20aecff8 = (uint64_t)0x0);
> r[29] = syscall(__NR_ioctl, r[4], 0x4008ae89ul,
> 0x20aecfe8ul, 0, 0, 0, 0, 0, 0);
>
> i.e. KVM_SET_MSR ioctl with
>
>   struct kvm_msrs {
> .nmsrs = 1,
> .pad = 0,
> .entries = {
> {.index = MSR_IA32_FEATURE_CONTROL,
>  .reserved = 0,
>  .data = 0}
> }
>   }
>
> and the relevant setter is
>
>   case MSR_IA32_FEATURE_CONTROL:
> if (!vmx_feature_control_msr_valid(vcpu, data) ||
> (to_vmx(vcpu)->msr_ia32_feature_control &
>  FEATURE_CONTROL_LOCKED && !msr_info->host_initiated))
> return 1;
> vmx->msr_ia32_feature_control = data;
> if (msr_info->host_initiated && data == 0)
> vmx_leave_nested(vcpu);
> break;
>
> The condition is wrong as it is no longer only VMX bits in data, so VMX
> can get disabled while the data is nonzero.  KVM might not need to force
> nested VM exit when userspace goes crazy, so the condition would just go
> away ... need to check.

If the bits just influence the new VMXON execution instead of
currently running hypervisor?

Regards,
Wanpeng Li

Re: kvm: WARNING in nested_vmx_vmexit

[Wanpeng Li]

2017-03-02 2:31 GMT+08:00 Radim Krčmář :
> 2017-03-01 10:44+0100, Dmitry Vyukov:
>> On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
>>> 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
 Hello,

 The following program triggers WARNING in nested_vmx_vmexit:
 https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt


 [ cut here ]
 WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
 nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
 CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
 01/01/2011
 Call Trace:
  __dump_stack lib/dump_stack.c:15 [inline]
  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
  panic+0x1fb/0x412 kernel/panic.c:179
  __warn+0x1c4/0x1e0 kernel/panic.c:540
  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
  __msr_io arch/x86/kvm/x86.c:2577 [inline]
  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
  vfs_ioctl fs/ioctl.c:43 [inline]
  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
  SYSC_ioctl fs/ioctl.c:698 [inline]
  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
  entry_SYSCALL_64_fastpath+0x1f/0xc2
 RIP: 0033:0x451229
 RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
 RAX: ffda RBX:  RCX: 00451229
 RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
 RBP:  R08:  R09: 
 R10:  R11: 0246 R12: 
 R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700


 On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
>>>
>>> I git checkout to this commit for linus tree, however the line number
>>> 11029 doesn't match any warning, and there are several warnings in the
>>> function nested_vmx_vmexit(), could you point out which one?
>>
>> Ah sorry, I have some local diff.
>> So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
>> https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt
>>
>> The warning is:
>>
>> /* trying to cancel vmlaunch/vmresume is a bug */
>> WARN_ON_ONCE(vmx->nested.nested_run_pending);
>
> I can't reproduce this one, but it is there:
>
> The warning gets thrown while doing
>
> (*(uint32_t*)0x20aecfe8 = (uint32_t)0x1);
> (*(uint32_t*)0x20aecfec = (uint32_t)0x0);
> (*(uint32_t*)0x20aecff0 = (uint32_t)0x3a);
> (*(uint32_t*)0x20aecff4 = (uint32_t)0x0);
> (*(uint64_t*)0x20aecff8 = (uint64_t)0x0);
> r[29] = syscall(__NR_ioctl, r[4], 0x4008ae89ul,
> 0x20aecfe8ul, 0, 0, 0, 0, 0, 0);
>
> i.e. KVM_SET_MSR ioctl with
>
>   struct kvm_msrs {
> .nmsrs = 1,
> .pad = 0,
> .entries = {
> {.index = MSR_IA32_FEATURE_CONTROL,
>  .reserved = 0,
>  .data = 0}
> }
>   }
>
> and the relevant setter is
>
>   case MSR_IA32_FEATURE_CONTROL:
> if (!vmx_feature_control_msr_valid(vcpu, data) ||
> (to_vmx(vcpu)->msr_ia32_feature_control &
>  FEATURE_CONTROL_LOCKED && !msr_info->host_initiated))
> return 1;
> vmx->msr_ia32_feature_control = data;
> if (msr_info->host_initiated && data == 0)
> vmx_leave_nested(vcpu);
> break;
>
> The condition is wrong as it is no longer only VMX bits in data, so VMX
> can get disabled while the data is nonzero.  KVM might not need to force
> nested VM exit when userspace goes crazy, so the condition would just go
> away ... need to check.

If the bits just influence the new VMXON execution instead of
currently running hypervisor?

Regards,
Wanpeng Li

Re: kvm: WARNING in nested_vmx_merge_msr_bitmap

[Radim Krčmář]

2017-02-28 13:07-0800, Jim Mattson:
> I believe this happens when the VMCS12 MSR bitmap address is valid,
> but no device is configured to respond to the bus request. I agree
> that the warning should be removed. However, in this case, the VMCS12
> MSR bitmap should read as all 1s. The same is true everywhere that
> nested_get_page returns NULL. Reads should return all 1s and writes
> should be ignored.

Yes, we have already checked that the address is within physical limits
and page-aligned.  After that, any errors seem like undefined behavior.

Going to do a bare-metal check and send a patch, thanks.

> On Tue, Feb 28, 2017 at 11:06 AM, Dmitry Vyukov  wrote:
>>
>> [ cut here ]
>> WARNING: CPU: 0 PID: 7832 at arch/x86/kvm/vmx.c:9709
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>  nested_vmx_merge_msr_bitmap arch/x86/kvm/vmx.c:9709 [inline]
>>  nested_get_vmcs12_pages+0xfb6/0x15c0 arch/x86/kvm/vmx.c:9640
>>  enter_vmx_non_root_mode arch/x86/kvm/vmx.c:10471 [inline]
>>  nested_vmx_run+0x6186/0xaab0 arch/x86/kvm/vmx.c:10561
>>
>> If I am reading this correctly, failure to map msr bitmap is trivially
>> trigger-able by guest (specify an invalid address?) and thus should
>> not be a warning. The case is already handled by disabling
>> acceleration. So can we remove the warning? or replace with pr_err?

Re: kvm: WARNING in nested_vmx_merge_msr_bitmap

[Radim Krčmář]

2017-02-28 13:07-0800, Jim Mattson:
> I believe this happens when the VMCS12 MSR bitmap address is valid,
> but no device is configured to respond to the bus request. I agree
> that the warning should be removed. However, in this case, the VMCS12
> MSR bitmap should read as all 1s. The same is true everywhere that
> nested_get_page returns NULL. Reads should return all 1s and writes
> should be ignored.

Yes, we have already checked that the address is within physical limits
and page-aligned.  After that, any errors seem like undefined behavior.

Going to do a bare-metal check and send a patch, thanks.

> On Tue, Feb 28, 2017 at 11:06 AM, Dmitry Vyukov  wrote:
>>
>> [ cut here ]
>> WARNING: CPU: 0 PID: 7832 at arch/x86/kvm/vmx.c:9709
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>  nested_vmx_merge_msr_bitmap arch/x86/kvm/vmx.c:9709 [inline]
>>  nested_get_vmcs12_pages+0xfb6/0x15c0 arch/x86/kvm/vmx.c:9640
>>  enter_vmx_non_root_mode arch/x86/kvm/vmx.c:10471 [inline]
>>  nested_vmx_run+0x6186/0xaab0 arch/x86/kvm/vmx.c:10561
>>
>> If I am reading this correctly, failure to map msr bitmap is trivially
>> trigger-able by guest (specify an invalid address?) and thus should
>> not be a warning. The case is already handled by disabling
>> acceleration. So can we remove the warning? or replace with pr_err?

Re: kvm: WARNING in nested_vmx_vmexit

[Radim Krčmář]

2017-03-01 10:44+0100, Dmitry Vyukov:
> On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
>> 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
>>> Hello,
>>>
>>> The following program triggers WARNING in nested_vmx_vmexit:
>>> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>>>
>>>
>>> [ cut here ]
>>> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
>>> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>>>  panic+0x1fb/0x412 kernel/panic.c:179
>>>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>>>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>>>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>>>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>>>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>>>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>>>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>>>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x451229
>>> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
>>> RAX: ffda RBX:  RCX: 00451229
>>> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
>>> RBP:  R08:  R09: 
>>> R10:  R11: 0246 R12: 
>>> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>>>
>>>
>>> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
>>
>> I git checkout to this commit for linus tree, however the line number
>> 11029 doesn't match any warning, and there are several warnings in the
>> function nested_vmx_vmexit(), could you point out which one?
> 
> Ah sorry, I have some local diff.
> So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
> https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt
> 
> The warning is:
> 
> /* trying to cancel vmlaunch/vmresume is a bug */
> WARN_ON_ONCE(vmx->nested.nested_run_pending);

I can't reproduce this one, but it is there:

The warning gets thrown while doing

(*(uint32_t*)0x20aecfe8 = (uint32_t)0x1);
(*(uint32_t*)0x20aecfec = (uint32_t)0x0);
(*(uint32_t*)0x20aecff0 = (uint32_t)0x3a);
(*(uint32_t*)0x20aecff4 = (uint32_t)0x0);
(*(uint64_t*)0x20aecff8 = (uint64_t)0x0);
r[29] = syscall(__NR_ioctl, r[4], 0x4008ae89ul,
0x20aecfe8ul, 0, 0, 0, 0, 0, 0);

i.e. KVM_SET_MSR ioctl with

  struct kvm_msrs {
.nmsrs = 1,
.pad = 0,
.entries = {
{.index = MSR_IA32_FEATURE_CONTROL,
 .reserved = 0,
 .data = 0}
}
  }

and the relevant setter is

  case MSR_IA32_FEATURE_CONTROL:
if (!vmx_feature_control_msr_valid(vcpu, data) ||
(to_vmx(vcpu)->msr_ia32_feature_control &
 FEATURE_CONTROL_LOCKED && !msr_info->host_initiated))
return 1;
vmx->msr_ia32_feature_control = data;
if (msr_info->host_initiated && data == 0)
vmx_leave_nested(vcpu);
break;

The condition is wrong as it is no longer only VMX bits in data, so VMX
can get disabled while the data is nonzero.  KVM might not need to force
nested VM exit when userspace goes crazy, so the condition would just go
away ... need to check.

Re: kvm: WARNING in nested_vmx_vmexit

[Radim Krčmář]

2017-03-01 10:44+0100, Dmitry Vyukov:
> On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
>> 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
>>> Hello,
>>>
>>> The following program triggers WARNING in nested_vmx_vmexit:
>>> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>>>
>>>
>>> [ cut here ]
>>> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
>>> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>>>  panic+0x1fb/0x412 kernel/panic.c:179
>>>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>>>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>>>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>>>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>>>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>>>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>>>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>>>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x451229
>>> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
>>> RAX: ffda RBX:  RCX: 00451229
>>> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
>>> RBP:  R08:  R09: 
>>> R10:  R11: 0246 R12: 
>>> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>>>
>>>
>>> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
>>
>> I git checkout to this commit for linus tree, however the line number
>> 11029 doesn't match any warning, and there are several warnings in the
>> function nested_vmx_vmexit(), could you point out which one?
> 
> Ah sorry, I have some local diff.
> So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
> https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt
> 
> The warning is:
> 
> /* trying to cancel vmlaunch/vmresume is a bug */
> WARN_ON_ONCE(vmx->nested.nested_run_pending);

I can't reproduce this one, but it is there:

The warning gets thrown while doing

(*(uint32_t*)0x20aecfe8 = (uint32_t)0x1);
(*(uint32_t*)0x20aecfec = (uint32_t)0x0);
(*(uint32_t*)0x20aecff0 = (uint32_t)0x3a);
(*(uint32_t*)0x20aecff4 = (uint32_t)0x0);
(*(uint64_t*)0x20aecff8 = (uint64_t)0x0);
r[29] = syscall(__NR_ioctl, r[4], 0x4008ae89ul,
0x20aecfe8ul, 0, 0, 0, 0, 0, 0);

i.e. KVM_SET_MSR ioctl with

  struct kvm_msrs {
.nmsrs = 1,
.pad = 0,
.entries = {
{.index = MSR_IA32_FEATURE_CONTROL,
 .reserved = 0,
 .data = 0}
}
  }

and the relevant setter is

  case MSR_IA32_FEATURE_CONTROL:
if (!vmx_feature_control_msr_valid(vcpu, data) ||
(to_vmx(vcpu)->msr_ia32_feature_control &
 FEATURE_CONTROL_LOCKED && !msr_info->host_initiated))
return 1;
vmx->msr_ia32_feature_control = data;
if (msr_info->host_initiated && data == 0)
vmx_leave_nested(vcpu);
break;

The condition is wrong as it is no longer only VMX bits in data, so VMX
can get disabled while the data is nonzero.  KVM might not need to force
nested VM exit when userspace goes crazy, so the condition would just go
away ... need to check.

Re: kvm: WARNING in nested_vmx_vmexit

[Radim Krčmář]

2017-02-28 13:48+0100, Dmitry Vyukov:
> On Tue, Feb 28, 2017 at 1:15 PM, Dmitry Vyukov  wrote:
>> Hello,
>>
>> The following program triggers WARNING in nested_vmx_vmexit:
>> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>>
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
>> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>>  panic+0x1fb/0x412 kernel/panic.c:179
>>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x451229
>> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
>> RAX: ffda RBX:  RCX: 00451229
>> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
>> RBP:  R08:  R09: 
>> R10:  R11: 0246 R12: 
>> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>>
>>
>> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
> 
> 
> The bug that I tried to localize is a different one:
> 
> WARNING: CPU: 1 PID: 4106 at mm/filemap.c:259
> __delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
> CPU: 1 PID: 4106 Comm: syz-executor Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  __delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
>  delete_from_page_cache+0x242/0x720 mm/filemap.c:282
>  truncate_complete_page mm/truncate.c:156 [inline]
>  truncate_inode_page+0x2ce/0x510 mm/truncate.c:195
>  shmem_undo_range+0x90c/0x2720 mm/shmem.c:828
>  shmem_truncate_range+0x27/0xa0 mm/shmem.c:956
>  shmem_evict_inode+0x35f/0xca0 mm/shmem.c:1047
>  evict+0x46e/0x980 fs/inode.c:553
>  iput_final fs/inode.c:1515 [inline]
>  iput+0x589/0xb20 fs/inode.c:1542
>  dentry_unlink_inode+0x43b/0x600 fs/dcache.c:343
>  __dentry_kill+0x34d/0x740 fs/dcache.c:538
>  dentry_kill fs/dcache.c:579 [inline]
>  dput.part.27+0x5ce/0x7c0 fs/dcache.c:791
>  dput+0x1f/0x30 fs/dcache.c:753
>  __fput+0x527/0x7f0 fs/file_table.c:226
>  fput+0x15/0x20 fs/file_table.c:244
>  task_work_run+0x18a/0x260 kernel/task_work.c:116
>  tracehook_notify_resume include/linux/tracehook.h:191 [inline]
>  exit_to_usermode_loop+0x23b/0x2a0 arch/x86/entry/common.c:160
>  prepare_exit_to_usermode arch/x86/entry/common.c:190 [inline]
>  syscall_return_slowpath+0x4d3/0x570 arch/x86/entry/common.c:259
>  entry_SYSCALL_64_fastpath+0xc0/0xc2
> RIP: 0033:0x4458d9
> RSP: 002b:7fb393062b58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: 2000 RBX: 00708000 RCX: 004458d9
> RDX: 0003 RSI: 00af7000 RDI: 2000
> RBP: 2f20 R08:  R09: 
> R10: 4032 R11: 0282 R12: 006e0fe0
> R13: 2000 R14: 00af7000 R15: 0003
> 
> 
> But it only reproduces when I run the following syzkaller program
> using syz-execprog utility:
> 
> mmap(&(0x7f00/0xaef000)=nil, (0xaef000), 0x3, 0x31,
> 0x, 0x0)
> r0 = openat$kvm(0xff9c,
> &(0x7f005000-0x9)="2f6465762f6b766d00", 0x0, 0x0)
> r1 = ioctl$KVM_CREATE_VM(r0, 0xae01, 0x0)
> r2 = ioctl$KVM_CREATE_VCPU(r1, 0xae41, 0x0)
> getpid()
> mmap(&(0x7faef000/0x1000)=nil, (0x1000), 0x3, 0x32, 0x, 
> 0x0)
> syz_kvm_setup_cpu$x86(r1, r2, &(0x7f274000/0x18000)=nil,
> &(0x7fadf000)=[@text64={0x40,
> 

Re: kvm: WARNING in nested_vmx_vmexit

[Radim Krčmář]

2017-02-28 13:48+0100, Dmitry Vyukov:
> On Tue, Feb 28, 2017 at 1:15 PM, Dmitry Vyukov  wrote:
>> Hello,
>>
>> The following program triggers WARNING in nested_vmx_vmexit:
>> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>>
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
>> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>>  panic+0x1fb/0x412 kernel/panic.c:179
>>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x451229
>> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
>> RAX: ffda RBX:  RCX: 00451229
>> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
>> RBP:  R08:  R09: 
>> R10:  R11: 0246 R12: 
>> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>>
>>
>> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
> 
> 
> The bug that I tried to localize is a different one:
> 
> WARNING: CPU: 1 PID: 4106 at mm/filemap.c:259
> __delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
> CPU: 1 PID: 4106 Comm: syz-executor Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  __delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
>  delete_from_page_cache+0x242/0x720 mm/filemap.c:282
>  truncate_complete_page mm/truncate.c:156 [inline]
>  truncate_inode_page+0x2ce/0x510 mm/truncate.c:195
>  shmem_undo_range+0x90c/0x2720 mm/shmem.c:828
>  shmem_truncate_range+0x27/0xa0 mm/shmem.c:956
>  shmem_evict_inode+0x35f/0xca0 mm/shmem.c:1047
>  evict+0x46e/0x980 fs/inode.c:553
>  iput_final fs/inode.c:1515 [inline]
>  iput+0x589/0xb20 fs/inode.c:1542
>  dentry_unlink_inode+0x43b/0x600 fs/dcache.c:343
>  __dentry_kill+0x34d/0x740 fs/dcache.c:538
>  dentry_kill fs/dcache.c:579 [inline]
>  dput.part.27+0x5ce/0x7c0 fs/dcache.c:791
>  dput+0x1f/0x30 fs/dcache.c:753
>  __fput+0x527/0x7f0 fs/file_table.c:226
>  fput+0x15/0x20 fs/file_table.c:244
>  task_work_run+0x18a/0x260 kernel/task_work.c:116
>  tracehook_notify_resume include/linux/tracehook.h:191 [inline]
>  exit_to_usermode_loop+0x23b/0x2a0 arch/x86/entry/common.c:160
>  prepare_exit_to_usermode arch/x86/entry/common.c:190 [inline]
>  syscall_return_slowpath+0x4d3/0x570 arch/x86/entry/common.c:259
>  entry_SYSCALL_64_fastpath+0xc0/0xc2
> RIP: 0033:0x4458d9
> RSP: 002b:7fb393062b58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: 2000 RBX: 00708000 RCX: 004458d9
> RDX: 0003 RSI: 00af7000 RDI: 2000
> RBP: 2f20 R08:  R09: 
> R10: 4032 R11: 0282 R12: 006e0fe0
> R13: 2000 R14: 00af7000 R15: 0003
> 
> 
> But it only reproduces when I run the following syzkaller program
> using syz-execprog utility:
> 
> mmap(&(0x7f00/0xaef000)=nil, (0xaef000), 0x3, 0x31,
> 0x, 0x0)
> r0 = openat$kvm(0xff9c,
> &(0x7f005000-0x9)="2f6465762f6b766d00", 0x0, 0x0)
> r1 = ioctl$KVM_CREATE_VM(r0, 0xae01, 0x0)
> r2 = ioctl$KVM_CREATE_VCPU(r1, 0xae41, 0x0)
> getpid()
> mmap(&(0x7faef000/0x1000)=nil, (0x1000), 0x3, 0x32, 0x, 
> 0x0)
> syz_kvm_setup_cpu$x86(r1, r2, &(0x7f274000/0x18000)=nil,
> &(0x7fadf000)=[@text64={0x40,
> &(0x7f99d000-0x64)="b6c3f8e788595d2a1ba31779d22e2453ab6fe204d8cb17bc3c4ab8e3e0483b9931418b5c1612cb68cb1f08acd253883205213823efd610026d3b892f9ecf43c837882ddb41cb3a22a62644cc9081d865b5c7d6d371bfbc1b7da5ab28911fcb5667d0e8b0ca",
> 0x65}], 0x1, 

Re: kvm: WARNING in nested_vmx_vmexit

[Dmitry Vyukov]

On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
> 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
>> Hello,
>>
>> The following program triggers WARNING in nested_vmx_vmexit:
>> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>>
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
>> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>>  panic+0x1fb/0x412 kernel/panic.c:179
>>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x451229
>> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
>> RAX: ffda RBX:  RCX: 00451229
>> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
>> RBP:  R08:  R09: 
>> R10:  R11: 0246 R12: 
>> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>>
>>
>> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
>
> I git checkout to this commit for linus tree, however the line number
> 11029 doesn't match any warning, and there are several warnings in the
> function nested_vmx_vmexit(), could you point out which one?

Ah sorry, I have some local diff.
So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt

The warning is:

/* trying to cancel vmlaunch/vmresume is a bug */
WARN_ON_ONCE(vmx->nested.nested_run_pending);

Re: kvm: WARNING in nested_vmx_vmexit

[Dmitry Vyukov]

On Wed, Mar 1, 2017 at 7:13 AM, Wanpeng Li  wrote:
> 2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
>> Hello,
>>
>> The following program triggers WARNING in nested_vmx_vmexit:
>> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>>
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
>> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>>  panic+0x1fb/0x412 kernel/panic.c:179
>>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x451229
>> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
>> RAX: ffda RBX:  RCX: 00451229
>> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
>> RBP:  R08:  R09: 
>> R10:  R11: 0246 R12: 
>> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>>
>>
>> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570
>
> I git checkout to this commit for linus tree, however the line number
> 11029 doesn't match any warning, and there are several warnings in the
> function nested_vmx_vmexit(), could you point out which one?

Ah sorry, I have some local diff.
So now I am on 86292b33d4b79ee03e2f43ea0381ef85f077c760 and here is my diff:
https://gist.githubusercontent.com/dvyukov/6bb88c6e3584c3fd989df01386efcb74/raw/1ae08a3a682ad9467de5eb7fb96d457db514f9d2/gistfile1.txt

The warning is:

/* trying to cancel vmlaunch/vmresume is a bug */
WARN_ON_ONCE(vmx->nested.nested_run_pending);

Re: kvm: WARNING in nested_vmx_vmexit

[Wanpeng Li]

2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> The following program triggers WARNING in nested_vmx_vmexit:
> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>
>
> [ cut here ]
> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x451229
> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 00451229
> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
> RBP:  R08:  R09: 
> R10:  R11: 0246 R12: 
> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>
>
> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570

I git checkout to this commit for linus tree, however the line number
11029 doesn't match any warning, and there are several warnings in the
function nested_vmx_vmexit(), could you point out which one?

Regards,
Wanpeng Li

Re: kvm: WARNING in nested_vmx_vmexit

[Wanpeng Li]

2017-02-28 20:15 GMT+08:00 Dmitry Vyukov :
> Hello,
>
> The following program triggers WARNING in nested_vmx_vmexit:
> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>
>
> [ cut here ]
> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x451229
> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 00451229
> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
> RBP:  R08:  R09: 
> R10:  R11: 0246 R12: 
> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>
>
> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570

I git checkout to this commit for linus tree, however the line number
11029 doesn't match any warning, and there are several warnings in the
function nested_vmx_vmexit(), could you point out which one?

Regards,
Wanpeng Li

Re: kvm: WARNING in nested_vmx_merge_msr_bitmap

[Jim Mattson]

I believe this happens when the VMCS12 MSR bitmap address is valid,
but no device is configured to respond to the bus request. I agree
that the warning should be removed. However, in this case, the VMCS12
MSR bitmap should read as all 1s. The same is true everywhere that
nested_get_page returns NULL. Reads should return all 1s and writes
should be ignored.

On Tue, Feb 28, 2017 at 11:06 AM, Dmitry Vyukov  wrote:
> Hello,
>
> I've got the following WARNING while running syzkaller fuzzer:
>
> [ cut here ]
> WARNING: CPU: 0 PID: 7832 at arch/x86/kvm/vmx.c:9709
> nested_vmx_merge_msr_bitmap arch/x86/kvm/vmx.c:9709 [inline]
> WARNING: CPU: 0 PID: 7832 at arch/x86/kvm/vmx.c:9709
> nested_get_vmcs12_pages+0xfb6/0x15c0 arch/x86/kvm/vmx.c:9640
> Kernel panic - not syncing: panic_on_warn set ...
> CPU: 0 PID: 7832 Comm: syz-executor1 Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_merge_msr_bitmap arch/x86/kvm/vmx.c:9709 [inline]
>  nested_get_vmcs12_pages+0xfb6/0x15c0 arch/x86/kvm/vmx.c:9640
>  enter_vmx_non_root_mode arch/x86/kvm/vmx.c:10471 [inline]
>  nested_vmx_run+0x6186/0xaab0 arch/x86/kvm/vmx.c:10561
>  handle_vmlaunch+0x1a/0x20 arch/x86/kvm/vmx.c:7312
>  vmx_handle_exit+0xfc0/0x3f00 arch/x86/kvm/vmx.c:8526
>  vcpu_enter_guest arch/x86/kvm/x86.c:6982 [inline]
>  vcpu_run arch/x86/kvm/x86.c:7044 [inline]
>  kvm_arch_vcpu_ioctl_run+0x1418/0x4840 arch/x86/kvm/x86.c:7205
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2570
> sctp: [Deprecated]: syz-executor2 (pid 7865) Use of int in max_burst
> socket option.
> Use struct sctp_assoc_value instead
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> 9pnet_virtio: no channels available for device ./bus
> RIP: 0033:0x4458d9
> RSP: 002b:7f248b1abb58 EFLAGS: 0286 ORIG_RAX: 0010
> RAX: ffda RBX: 001b RCX: 004458d9
> RDX:  RSI: ae80 RDI: 001b
> RBP: 006dfcd0 R08:  R09: 
> R10:  R11: 0286 R12: 007080a8
> R13: 001a R14: 001b R15: 201b3000
>
>
> If I am reading this correctly, failure to map msr bitmap is trivially
> trigger-able by guest (specify an invalid address?) and thus should
> not be a warning. The case is already handled by disabling
> acceleration. So can we remove the warning? or replace with pr_err?

Re: kvm: WARNING in nested_vmx_merge_msr_bitmap

[Jim Mattson]

I believe this happens when the VMCS12 MSR bitmap address is valid,
but no device is configured to respond to the bus request. I agree
that the warning should be removed. However, in this case, the VMCS12
MSR bitmap should read as all 1s. The same is true everywhere that
nested_get_page returns NULL. Reads should return all 1s and writes
should be ignored.

On Tue, Feb 28, 2017 at 11:06 AM, Dmitry Vyukov  wrote:
> Hello,
>
> I've got the following WARNING while running syzkaller fuzzer:
>
> [ cut here ]
> WARNING: CPU: 0 PID: 7832 at arch/x86/kvm/vmx.c:9709
> nested_vmx_merge_msr_bitmap arch/x86/kvm/vmx.c:9709 [inline]
> WARNING: CPU: 0 PID: 7832 at arch/x86/kvm/vmx.c:9709
> nested_get_vmcs12_pages+0xfb6/0x15c0 arch/x86/kvm/vmx.c:9640
> Kernel panic - not syncing: panic_on_warn set ...
> CPU: 0 PID: 7832 Comm: syz-executor1 Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_merge_msr_bitmap arch/x86/kvm/vmx.c:9709 [inline]
>  nested_get_vmcs12_pages+0xfb6/0x15c0 arch/x86/kvm/vmx.c:9640
>  enter_vmx_non_root_mode arch/x86/kvm/vmx.c:10471 [inline]
>  nested_vmx_run+0x6186/0xaab0 arch/x86/kvm/vmx.c:10561
>  handle_vmlaunch+0x1a/0x20 arch/x86/kvm/vmx.c:7312
>  vmx_handle_exit+0xfc0/0x3f00 arch/x86/kvm/vmx.c:8526
>  vcpu_enter_guest arch/x86/kvm/x86.c:6982 [inline]
>  vcpu_run arch/x86/kvm/x86.c:7044 [inline]
>  kvm_arch_vcpu_ioctl_run+0x1418/0x4840 arch/x86/kvm/x86.c:7205
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2570
> sctp: [Deprecated]: syz-executor2 (pid 7865) Use of int in max_burst
> socket option.
> Use struct sctp_assoc_value instead
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> 9pnet_virtio: no channels available for device ./bus
> RIP: 0033:0x4458d9
> RSP: 002b:7f248b1abb58 EFLAGS: 0286 ORIG_RAX: 0010
> RAX: ffda RBX: 001b RCX: 004458d9
> RDX:  RSI: ae80 RDI: 001b
> RBP: 006dfcd0 R08:  R09: 
> R10:  R11: 0286 R12: 007080a8
> R13: 001a R14: 001b R15: 201b3000
>
>
> If I am reading this correctly, failure to map msr bitmap is trivially
> trigger-able by guest (specify an invalid address?) and thus should
> not be a warning. The case is already handled by disabling
> acceleration. So can we remove the warning? or replace with pr_err?

Re: kvm: WARNING in nested_vmx_vmexit

[Dmitry Vyukov]

On Tue, Feb 28, 2017 at 1:15 PM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program triggers WARNING in nested_vmx_vmexit:
> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>
>
> [ cut here ]
> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x451229
> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 00451229
> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
> RBP:  R08:  R09: 
> R10:  R11: 0246 R12: 
> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>
>
> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570


The bug that I tried to localize is a different one:

WARNING: CPU: 1 PID: 4106 at mm/filemap.c:259
__delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
CPU: 1 PID: 4106 Comm: syz-executor Not tainted 4.10.0+ #229
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 panic+0x1fb/0x412 kernel/panic.c:179
 __warn+0x1c4/0x1e0 kernel/panic.c:540
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
 __delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
 delete_from_page_cache+0x242/0x720 mm/filemap.c:282
 truncate_complete_page mm/truncate.c:156 [inline]
 truncate_inode_page+0x2ce/0x510 mm/truncate.c:195
 shmem_undo_range+0x90c/0x2720 mm/shmem.c:828
 shmem_truncate_range+0x27/0xa0 mm/shmem.c:956
 shmem_evict_inode+0x35f/0xca0 mm/shmem.c:1047
 evict+0x46e/0x980 fs/inode.c:553
 iput_final fs/inode.c:1515 [inline]
 iput+0x589/0xb20 fs/inode.c:1542
 dentry_unlink_inode+0x43b/0x600 fs/dcache.c:343
 __dentry_kill+0x34d/0x740 fs/dcache.c:538
 dentry_kill fs/dcache.c:579 [inline]
 dput.part.27+0x5ce/0x7c0 fs/dcache.c:791
 dput+0x1f/0x30 fs/dcache.c:753
 __fput+0x527/0x7f0 fs/file_table.c:226
 fput+0x15/0x20 fs/file_table.c:244
 task_work_run+0x18a/0x260 kernel/task_work.c:116
 tracehook_notify_resume include/linux/tracehook.h:191 [inline]
 exit_to_usermode_loop+0x23b/0x2a0 arch/x86/entry/common.c:160
 prepare_exit_to_usermode arch/x86/entry/common.c:190 [inline]
 syscall_return_slowpath+0x4d3/0x570 arch/x86/entry/common.c:259
 entry_SYSCALL_64_fastpath+0xc0/0xc2
RIP: 0033:0x4458d9
RSP: 002b:7fb393062b58 EFLAGS: 0282 ORIG_RAX: 0009
RAX: 2000 RBX: 00708000 RCX: 004458d9
RDX: 0003 RSI: 00af7000 RDI: 2000
RBP: 2f20 R08:  R09: 
R10: 4032 R11: 0282 R12: 006e0fe0
R13: 2000 R14: 00af7000 R15: 0003


But it only reproduces when I run the following syzkaller program
using syz-execprog utility:

mmap(&(0x7f00/0xaef000)=nil, (0xaef000), 0x3, 0x31,
0x, 0x0)
r0 = openat$kvm(0xff9c,
&(0x7f005000-0x9)="2f6465762f6b766d00", 0x0, 0x0)
r1 = ioctl$KVM_CREATE_VM(r0, 0xae01, 0x0)
r2 = ioctl$KVM_CREATE_VCPU(r1, 0xae41, 0x0)
getpid()
mmap(&(0x7faef000/0x1000)=nil, (0x1000), 0x3, 0x32, 0x, 0x0)
syz_kvm_setup_cpu$x86(r1, r2, &(0x7f274000/0x18000)=nil,
&(0x7fadf000)=[@text64={0x40,
&(0x7f99d000-0x64)="b6c3f8e788595d2a1ba31779d22e2453ab6fe204d8cb17bc3c4ab8e3e0483b9931418b5c1612cb68cb1f08acd253883205213823efd610026d3b892f9ecf43c837882ddb41cb3a22a62644cc9081d865b5c7d6d371bfbc1b7da5ab28911fcb5667d0e8b0ca",
0x65}], 0x1, 0x42, &(0x7fae2000-0x10)=[@vmwrite={0x8, 0x0, 0x1ff,
0x0, 0x4, 0x0, 0x1, 0x0, 0x6}], 0x1)
getpid()
ioctl$KVM_RUN(r2, 0xae80)
mmap(&(0x7f00/0xaf7000)=nil, (0xaf7000), 0x3,

Re: kvm: WARNING in nested_vmx_vmexit

[Dmitry Vyukov]

On Tue, Feb 28, 2017 at 1:15 PM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program triggers WARNING in nested_vmx_vmexit:
> https://gist.githubusercontent.com/dvyukov/16b946d7dc703bb07b9b933f12fb8a6e/raw/dac60506feb8dd9dd22828c486e46ee8a5e30f13/gistfile1.txt
>
>
> [ cut here ]
> WARNING: CPU: 1 PID: 27742 at arch/x86/kvm/vmx.c:11029
> nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
> CPU: 1 PID: 27742 Comm: a.out Not tainted 4.10.0+ #229
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
>  panic+0x1fb/0x412 kernel/panic.c:179
>  __warn+0x1c4/0x1e0 kernel/panic.c:540
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
>  nested_vmx_vmexit+0x5c35/0x74d0 arch/x86/kvm/vmx.c:11029
>  vmx_leave_nested arch/x86/kvm/vmx.c:11136 [inline]
>  vmx_set_msr+0x1565/0x1910 arch/x86/kvm/vmx.c:3324
>  kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1099
>  do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1128
>  __msr_io arch/x86/kvm/x86.c:2577 [inline]
>  msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2614
>  kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3497
>  kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x451229
> RSP: 002b:7fc1e7ebec98 EFLAGS: 0246 ORIG_RAX: 0010
> RAX: ffda RBX:  RCX: 00451229
> RDX: 20aecfe8 RSI: 4008ae89 RDI: 0008
> RBP:  R08:  R09: 
> R10:  R11: 0246 R12: 
> R13:  R14: 7fc1e7ebf9c0 R15: 7fc1e7ebf700
>
>
> On commit e5d56efc97f8240d0b5d66c03949382b6d7e5570


The bug that I tried to localize is a different one:

WARNING: CPU: 1 PID: 4106 at mm/filemap.c:259
__delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
CPU: 1 PID: 4106 Comm: syz-executor Not tainted 4.10.0+ #229
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:15 [inline]
 dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
 panic+0x1fb/0x412 kernel/panic.c:179
 __warn+0x1c4/0x1e0 kernel/panic.c:540
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:583
 __delete_from_page_cache+0x1066/0x1390 mm/filemap.c:259
 delete_from_page_cache+0x242/0x720 mm/filemap.c:282
 truncate_complete_page mm/truncate.c:156 [inline]
 truncate_inode_page+0x2ce/0x510 mm/truncate.c:195
 shmem_undo_range+0x90c/0x2720 mm/shmem.c:828
 shmem_truncate_range+0x27/0xa0 mm/shmem.c:956
 shmem_evict_inode+0x35f/0xca0 mm/shmem.c:1047
 evict+0x46e/0x980 fs/inode.c:553
 iput_final fs/inode.c:1515 [inline]
 iput+0x589/0xb20 fs/inode.c:1542
 dentry_unlink_inode+0x43b/0x600 fs/dcache.c:343
 __dentry_kill+0x34d/0x740 fs/dcache.c:538
 dentry_kill fs/dcache.c:579 [inline]
 dput.part.27+0x5ce/0x7c0 fs/dcache.c:791
 dput+0x1f/0x30 fs/dcache.c:753
 __fput+0x527/0x7f0 fs/file_table.c:226
 fput+0x15/0x20 fs/file_table.c:244
 task_work_run+0x18a/0x260 kernel/task_work.c:116
 tracehook_notify_resume include/linux/tracehook.h:191 [inline]
 exit_to_usermode_loop+0x23b/0x2a0 arch/x86/entry/common.c:160
 prepare_exit_to_usermode arch/x86/entry/common.c:190 [inline]
 syscall_return_slowpath+0x4d3/0x570 arch/x86/entry/common.c:259
 entry_SYSCALL_64_fastpath+0xc0/0xc2
RIP: 0033:0x4458d9
RSP: 002b:7fb393062b58 EFLAGS: 0282 ORIG_RAX: 0009
RAX: 2000 RBX: 00708000 RCX: 004458d9
RDX: 0003 RSI: 00af7000 RDI: 2000
RBP: 2f20 R08:  R09: 
R10: 4032 R11: 0282 R12: 006e0fe0
R13: 2000 R14: 00af7000 R15: 0003


But it only reproduces when I run the following syzkaller program
using syz-execprog utility:

mmap(&(0x7f00/0xaef000)=nil, (0xaef000), 0x3, 0x31,
0x, 0x0)
r0 = openat$kvm(0xff9c,
&(0x7f005000-0x9)="2f6465762f6b766d00", 0x0, 0x0)
r1 = ioctl$KVM_CREATE_VM(r0, 0xae01, 0x0)
r2 = ioctl$KVM_CREATE_VCPU(r1, 0xae41, 0x0)
getpid()
mmap(&(0x7faef000/0x1000)=nil, (0x1000), 0x3, 0x32, 0x, 0x0)
syz_kvm_setup_cpu$x86(r1, r2, &(0x7f274000/0x18000)=nil,
&(0x7fadf000)=[@text64={0x40,
&(0x7f99d000-0x64)="b6c3f8e788595d2a1ba31779d22e2453ab6fe204d8cb17bc3c4ab8e3e0483b9931418b5c1612cb68cb1f08acd253883205213823efd610026d3b892f9ecf43c837882ddb41cb3a22a62644cc9081d865b5c7d6d371bfbc1b7da5ab28911fcb5667d0e8b0ca",
0x65}], 0x1, 0x42, &(0x7fae2000-0x10)=[@vmwrite={0x8, 0x0, 0x1ff,
0x0, 0x4, 0x0, 0x1, 0x0, 0x6}], 0x1)
getpid()
ioctl$KVM_RUN(r2, 0xae80)
mmap(&(0x7f00/0xaf7000)=nil, (0xaf7000), 0x3,
0x4032, 

Re: kvm: WARNING In kvm_apic_accept_events

[Paolo Bonzini]

On 15/02/2017 10:26, David Hildenbrand wrote:
>>
>> Any progress with this? Should we commit this as is?
>>
> Paolo, should I resend as a proper notcopieddiff mail?

Yes, please do (CCing sta...@vger.kernel.org too).

Paolo

Re: kvm: WARNING In kvm_apic_accept_events

[Paolo Bonzini]

On 15/02/2017 10:26, David Hildenbrand wrote:
>>
>> Any progress with this? Should we commit this as is?
>>
> Paolo, should I resend as a proper notcopieddiff mail?

Yes, please do (CCing sta...@vger.kernel.org too).

Paolo

Re: kvm: WARNING In kvm_apic_accept_events

[David Hildenbrand]

>>> From: David Hildenbrand 
>>> Date: Mon, 23 Jan 2017 16:42:23 +0100
>>> Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM
>>>
>>> If we already entered/are about to enter SMM, don't allow
>>> switching to INIT/SIPI_RECEIVED, otherwise the next call to
>>> kvm_apic_accept_events() will report a warning.
>>>
>>> Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management 
>>> mode")
>>> Reported-by: Dmitry Vyukov 
>>> Signed-off-by: David Hildenbrand 
>>> ---
>>>  arch/x86/kvm/x86.c | 6 ++
>>>  1 file changed, 6 insertions(+)
>>>
>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>> index 57d8a85..06ffc91 100644
>>> --- a/arch/x86/kvm/x86.c
>>> +++ b/arch/x86/kvm/x86.c
>>> @@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
>>> *vcpu,
>>>   mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
>>>   return -EINVAL;
>>>
>>> + /* INITs are latched while in SMM */
>>> + if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
>>> + (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
>>> +  mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
>>> + return -EINVAL;
>>> +
>>>   if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
>>>   vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
>>>   set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
>>>
>>
>> Looks good, thanks for analyzing it.  I'll try making a minimal testcase...
> 
> 
> Any progress with this? Should we commit this as is?
> 

Paolo, should I resend as a proper notcopieddiff mail?

-- 
Thanks,

David

Re: kvm: WARNING In kvm_apic_accept_events

[David Hildenbrand]

>>> From: David Hildenbrand 
>>> Date: Mon, 23 Jan 2017 16:42:23 +0100
>>> Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM
>>>
>>> If we already entered/are about to enter SMM, don't allow
>>> switching to INIT/SIPI_RECEIVED, otherwise the next call to
>>> kvm_apic_accept_events() will report a warning.
>>>
>>> Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management 
>>> mode")
>>> Reported-by: Dmitry Vyukov 
>>> Signed-off-by: David Hildenbrand 
>>> ---
>>>  arch/x86/kvm/x86.c | 6 ++
>>>  1 file changed, 6 insertions(+)
>>>
>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>> index 57d8a85..06ffc91 100644
>>> --- a/arch/x86/kvm/x86.c
>>> +++ b/arch/x86/kvm/x86.c
>>> @@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
>>> *vcpu,
>>>   mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
>>>   return -EINVAL;
>>>
>>> + /* INITs are latched while in SMM */
>>> + if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
>>> + (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
>>> +  mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
>>> + return -EINVAL;
>>> +
>>>   if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
>>>   vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
>>>   set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
>>>
>>
>> Looks good, thanks for analyzing it.  I'll try making a minimal testcase...
> 
> 
> Any progress with this? Should we commit this as is?
> 

Paolo, should I resend as a proper notcopieddiff mail?

-- 
Thanks,

David

Re: kvm: WARNING In kvm_apic_accept_events

[Dmitry Vyukov]

On Mon, Jan 23, 2017 at 5:29 PM, Paolo Bonzini  wrote:
>
> On 23/01/2017 17:03, David Hildenbrand wrote:
>> Am 22.01.2017 um 09:52 schrieb Dmitry Vyukov:
>>> On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
 Hello,

 The following program triggers WARNING in kvm_apic_accept_events:
 https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt

 On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).

 [ cut here ]
 WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
 kvm_apic_accept_events+0x3f6/0x460
 Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
 tainted 4.9.0-rc4+ #41
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
 01/01/2011
  88006be77978 81c2f79b  
  8322d6a0 81127426 88006be779c0 8123ffef
  811acc46 091a 8322d6a0 091a
 Call Trace:
  [< inline >] __dump_stack lib/dump_stack.c:15
  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
  [] __warn+0x19f/0x1e0 kernel/panic.c:550
  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
  [] kvm_apic_accept_events+0x3f6/0x460
 arch/x86/kvm/lapic.c:2330
  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
 arch/x86/kvm/x86.c:6984
  [] kvm_vcpu_ioctl+0x61e/0xdd0
 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
  [< inline >] vfs_ioctl fs/ioctl.c:43
  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
  [< inline >] SYSC_ioctl fs/ioctl.c:694
  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
 arch/x86/entry/entry_64.S:209
 ---[ end trace b2415ddd1b26dce6 ]---
>>>
>>>
>>> ping
>>>
>>
>> Our VCPU is in SMM mode but is switched into KVM_MP_STATE_INIT_RECEIVED.
>>
>> The relevant warning is (arch/x86/kvm/lapic.c:kvm_apic_accept_events())
>>
>> if (is_smm(vcpu)) {
>> WARN_ON_ONCE(vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED);
>> 
>>
>> Something like that might help (untested):
>>
>>
>> From: David Hildenbrand 
>> Date: Mon, 23 Jan 2017 16:42:23 +0100
>> Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM
>>
>> If we already entered/are about to enter SMM, don't allow
>> switching to INIT/SIPI_RECEIVED, otherwise the next call to
>> kvm_apic_accept_events() will report a warning.
>>
>> Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management mode")
>> Reported-by: Dmitry Vyukov 
>> Signed-off-by: David Hildenbrand 
>> ---
>>  arch/x86/kvm/x86.c | 6 ++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 57d8a85..06ffc91 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
>> *vcpu,
>>   mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
>>   return -EINVAL;
>>
>> + /* INITs are latched while in SMM */
>> + if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
>> + (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
>> +  mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
>> + return -EINVAL;
>> +
>>   if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
>>   vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
>>   set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
>>
>
> Looks good, thanks for analyzing it.  I'll try making a minimal testcase...


Any progress with this? Should we commit this as is?

Re: kvm: WARNING In kvm_apic_accept_events

[Dmitry Vyukov]

On Mon, Jan 23, 2017 at 5:29 PM, Paolo Bonzini  wrote:
>
> On 23/01/2017 17:03, David Hildenbrand wrote:
>> Am 22.01.2017 um 09:52 schrieb Dmitry Vyukov:
>>> On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
 Hello,

 The following program triggers WARNING in kvm_apic_accept_events:
 https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt

 On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).

 [ cut here ]
 WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
 kvm_apic_accept_events+0x3f6/0x460
 Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
 tainted 4.9.0-rc4+ #41
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
 01/01/2011
  88006be77978 81c2f79b  
  8322d6a0 81127426 88006be779c0 8123ffef
  811acc46 091a 8322d6a0 091a
 Call Trace:
  [< inline >] __dump_stack lib/dump_stack.c:15
  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
  [] __warn+0x19f/0x1e0 kernel/panic.c:550
  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
  [] kvm_apic_accept_events+0x3f6/0x460
 arch/x86/kvm/lapic.c:2330
  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
 arch/x86/kvm/x86.c:6984
  [] kvm_vcpu_ioctl+0x61e/0xdd0
 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
  [< inline >] vfs_ioctl fs/ioctl.c:43
  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
  [< inline >] SYSC_ioctl fs/ioctl.c:694
  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
 arch/x86/entry/entry_64.S:209
 ---[ end trace b2415ddd1b26dce6 ]---
>>>
>>>
>>> ping
>>>
>>
>> Our VCPU is in SMM mode but is switched into KVM_MP_STATE_INIT_RECEIVED.
>>
>> The relevant warning is (arch/x86/kvm/lapic.c:kvm_apic_accept_events())
>>
>> if (is_smm(vcpu)) {
>> WARN_ON_ONCE(vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED);
>> 
>>
>> Something like that might help (untested):
>>
>>
>> From: David Hildenbrand 
>> Date: Mon, 23 Jan 2017 16:42:23 +0100
>> Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM
>>
>> If we already entered/are about to enter SMM, don't allow
>> switching to INIT/SIPI_RECEIVED, otherwise the next call to
>> kvm_apic_accept_events() will report a warning.
>>
>> Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management mode")
>> Reported-by: Dmitry Vyukov 
>> Signed-off-by: David Hildenbrand 
>> ---
>>  arch/x86/kvm/x86.c | 6 ++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 57d8a85..06ffc91 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
>> *vcpu,
>>   mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
>>   return -EINVAL;
>>
>> + /* INITs are latched while in SMM */
>> + if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
>> + (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
>> +  mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
>> + return -EINVAL;
>> +
>>   if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
>>   vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
>>   set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
>>
>
> Looks good, thanks for analyzing it.  I'll try making a minimal testcase...


Any progress with this? Should we commit this as is?

Re: kvm: WARNING In kvm_apic_accept_events

[Paolo Bonzini]

On 23/01/2017 17:03, David Hildenbrand wrote:
> Am 22.01.2017 um 09:52 schrieb Dmitry Vyukov:
>> On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
>>> Hello,
>>>
>>> The following program triggers WARNING in kvm_apic_accept_events:
>>> https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt
>>>
>>> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>>>
>>> [ cut here ]
>>> WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
>>> kvm_apic_accept_events+0x3f6/0x460
>>> Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
>>> tainted 4.9.0-rc4+ #41
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  88006be77978 81c2f79b  
>>>  8322d6a0 81127426 88006be779c0 8123ffef
>>>  811acc46 091a 8322d6a0 091a
>>> Call Trace:
>>>  [< inline >] __dump_stack lib/dump_stack.c:15
>>>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>>>  [] __warn+0x19f/0x1e0 kernel/panic.c:550
>>>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>>>  [] kvm_apic_accept_events+0x3f6/0x460
>>> arch/x86/kvm/lapic.c:2330
>>>  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
>>>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
>>>  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
>>> arch/x86/kvm/x86.c:6984
>>>  [] kvm_vcpu_ioctl+0x61e/0xdd0
>>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>>>  [< inline >] vfs_ioctl fs/ioctl.c:43
>>>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>>>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>>>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>>>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> arch/x86/entry/entry_64.S:209
>>> ---[ end trace b2415ddd1b26dce6 ]---
>>
>>
>> ping
>>
> 
> Our VCPU is in SMM mode but is switched into KVM_MP_STATE_INIT_RECEIVED.
> 
> The relevant warning is (arch/x86/kvm/lapic.c:kvm_apic_accept_events())
> 
> if (is_smm(vcpu)) {
> WARN_ON_ONCE(vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED);
> 
> 
> Something like that might help (untested):
> 
> 
> From: David Hildenbrand 
> Date: Mon, 23 Jan 2017 16:42:23 +0100
> Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM
> 
> If we already entered/are about to enter SMM, don't allow
> switching to INIT/SIPI_RECEIVED, otherwise the next call to
> kvm_apic_accept_events() will report a warning.
> 
> Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management mode")
> Reported-by: Dmitry Vyukov 
> Signed-off-by: David Hildenbrand 
> ---
>  arch/x86/kvm/x86.c | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 57d8a85..06ffc91 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
> *vcpu,
>   mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
>   return -EINVAL;
>  
> + /* INITs are latched while in SMM */
> + if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
> + (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
> +  mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
> + return -EINVAL;
> +
>   if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
>   vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
>   set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
> 

Looks good, thanks for analyzing it.  I'll try making a minimal testcase...

Paolo

Re: kvm: WARNING In kvm_apic_accept_events

[Paolo Bonzini]

On 23/01/2017 17:03, David Hildenbrand wrote:
> Am 22.01.2017 um 09:52 schrieb Dmitry Vyukov:
>> On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
>>> Hello,
>>>
>>> The following program triggers WARNING in kvm_apic_accept_events:
>>> https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt
>>>
>>> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>>>
>>> [ cut here ]
>>> WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
>>> kvm_apic_accept_events+0x3f6/0x460
>>> Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
>>> tainted 4.9.0-rc4+ #41
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  88006be77978 81c2f79b  
>>>  8322d6a0 81127426 88006be779c0 8123ffef
>>>  811acc46 091a 8322d6a0 091a
>>> Call Trace:
>>>  [< inline >] __dump_stack lib/dump_stack.c:15
>>>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>>>  [] __warn+0x19f/0x1e0 kernel/panic.c:550
>>>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>>>  [] kvm_apic_accept_events+0x3f6/0x460
>>> arch/x86/kvm/lapic.c:2330
>>>  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
>>>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
>>>  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
>>> arch/x86/kvm/x86.c:6984
>>>  [] kvm_vcpu_ioctl+0x61e/0xdd0
>>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>>>  [< inline >] vfs_ioctl fs/ioctl.c:43
>>>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>>>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>>>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>>>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> arch/x86/entry/entry_64.S:209
>>> ---[ end trace b2415ddd1b26dce6 ]---
>>
>>
>> ping
>>
> 
> Our VCPU is in SMM mode but is switched into KVM_MP_STATE_INIT_RECEIVED.
> 
> The relevant warning is (arch/x86/kvm/lapic.c:kvm_apic_accept_events())
> 
> if (is_smm(vcpu)) {
> WARN_ON_ONCE(vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED);
> 
> 
> Something like that might help (untested):
> 
> 
> From: David Hildenbrand 
> Date: Mon, 23 Jan 2017 16:42:23 +0100
> Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM
> 
> If we already entered/are about to enter SMM, don't allow
> switching to INIT/SIPI_RECEIVED, otherwise the next call to
> kvm_apic_accept_events() will report a warning.
> 
> Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management mode")
> Reported-by: Dmitry Vyukov 
> Signed-off-by: David Hildenbrand 
> ---
>  arch/x86/kvm/x86.c | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 57d8a85..06ffc91 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
> *vcpu,
>   mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
>   return -EINVAL;
>  
> + /* INITs are latched while in SMM */
> + if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
> + (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
> +  mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
> + return -EINVAL;
> +
>   if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
>   vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
>   set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
> 

Looks good, thanks for analyzing it.  I'll try making a minimal testcase...

Paolo

Re: kvm: WARNING In kvm_apic_accept_events

[David Hildenbrand]

Am 22.01.2017 um 09:52 schrieb Dmitry Vyukov:
> On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
>> Hello,
>>
>> The following program triggers WARNING in kvm_apic_accept_events:
>> https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt
>>
>> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
>> kvm_apic_accept_events+0x3f6/0x460
>> Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
>> tainted 4.9.0-rc4+ #41
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>  88006be77978 81c2f79b  
>>  8322d6a0 81127426 88006be779c0 8123ffef
>>  811acc46 091a 8322d6a0 091a
>> Call Trace:
>>  [< inline >] __dump_stack lib/dump_stack.c:15
>>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>>  [] __warn+0x19f/0x1e0 kernel/panic.c:550
>>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>>  [] kvm_apic_accept_events+0x3f6/0x460
>> arch/x86/kvm/lapic.c:2330
>>  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
>>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
>>  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
>> arch/x86/kvm/x86.c:6984
>>  [] kvm_vcpu_ioctl+0x61e/0xdd0
>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>>  [< inline >] vfs_ioctl fs/ioctl.c:43
>>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
>> arch/x86/entry/entry_64.S:209
>> ---[ end trace b2415ddd1b26dce6 ]---
> 
> 
> ping
> 

Our VCPU is in SMM mode but is switched into KVM_MP_STATE_INIT_RECEIVED.

The relevant warning is (arch/x86/kvm/lapic.c:kvm_apic_accept_events())

if (is_smm(vcpu)) {
WARN_ON_ONCE(vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED);


Something like that might help (untested):


From: David Hildenbrand 
Date: Mon, 23 Jan 2017 16:42:23 +0100
Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM

If we already entered/are about to enter SMM, don't allow
switching to INIT/SIPI_RECEIVED, otherwise the next call to
kvm_apic_accept_events() will report a warning.

Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management mode")
Reported-by: Dmitry Vyukov 
Signed-off-by: David Hildenbrand 
---
 arch/x86/kvm/x86.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 57d8a85..06ffc91 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
*vcpu,
mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
return -EINVAL;
 
+   /* INITs are latched while in SMM */
+   if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
+   (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
+mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
+   return -EINVAL;
+
if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
-- 
2.9.3



-- 

David

Re: kvm: WARNING In kvm_apic_accept_events

[David Hildenbrand]

Am 22.01.2017 um 09:52 schrieb Dmitry Vyukov:
> On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
>> Hello,
>>
>> The following program triggers WARNING in kvm_apic_accept_events:
>> https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt
>>
>> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
>> kvm_apic_accept_events+0x3f6/0x460
>> Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
>> tainted 4.9.0-rc4+ #41
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>  88006be77978 81c2f79b  
>>  8322d6a0 81127426 88006be779c0 8123ffef
>>  811acc46 091a 8322d6a0 091a
>> Call Trace:
>>  [< inline >] __dump_stack lib/dump_stack.c:15
>>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>>  [] __warn+0x19f/0x1e0 kernel/panic.c:550
>>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>>  [] kvm_apic_accept_events+0x3f6/0x460
>> arch/x86/kvm/lapic.c:2330
>>  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
>>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
>>  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
>> arch/x86/kvm/x86.c:6984
>>  [] kvm_vcpu_ioctl+0x61e/0xdd0
>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>>  [< inline >] vfs_ioctl fs/ioctl.c:43
>>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
>> arch/x86/entry/entry_64.S:209
>> ---[ end trace b2415ddd1b26dce6 ]---
> 
> 
> ping
> 

Our VCPU is in SMM mode but is switched into KVM_MP_STATE_INIT_RECEIVED.

The relevant warning is (arch/x86/kvm/lapic.c:kvm_apic_accept_events())

if (is_smm(vcpu)) {
WARN_ON_ONCE(vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED);


Something like that might help (untested):


From: David Hildenbrand 
Date: Mon, 23 Jan 2017 16:42:23 +0100
Subject: [PATCH] KVM: x86: fix illegal MP_STATE when in/entering SMM

If we already entered/are about to enter SMM, don't allow
switching to INIT/SIPI_RECEIVED, otherwise the next call to
kvm_apic_accept_events() will report a warning.

Fixes: cd7764fe9f73 ("KVM: x86: latch INITs while in system management mode")
Reported-by: Dmitry Vyukov 
Signed-off-by: David Hildenbrand 
---
 arch/x86/kvm/x86.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 57d8a85..06ffc91 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7247,6 +7247,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu 
*vcpu,
mp_state->mp_state != KVM_MP_STATE_RUNNABLE)
return -EINVAL;
 
+   /* INITs are latched while in SMM */
+   if ((is_smm(vcpu) || vcpu->arch.smi_pending) &&
+   (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED ||
+mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
+   return -EINVAL;
+
if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
set_bit(KVM_APIC_SIPI, >arch.apic->pending_events);
-- 
2.9.3



-- 

David

Re: kvm: WARNING In kvm_apic_accept_events

[Dmitry Vyukov]

On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program triggers WARNING in kvm_apic_accept_events:
> https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt
>
> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>
> [ cut here ]
> WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
> kvm_apic_accept_events+0x3f6/0x460
> Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
> tainted 4.9.0-rc4+ #41
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  88006be77978 81c2f79b  
>  8322d6a0 81127426 88006be779c0 8123ffef
>  811acc46 091a 8322d6a0 091a
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>  [] __warn+0x19f/0x1e0 kernel/panic.c:550
>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>  [] kvm_apic_accept_events+0x3f6/0x460
> arch/x86/kvm/lapic.c:2330
>  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
>  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
> arch/x86/kvm/x86.c:6984
>  [] kvm_vcpu_ioctl+0x61e/0xdd0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
> arch/x86/entry/entry_64.S:209
> ---[ end trace b2415ddd1b26dce6 ]---


ping

Re: kvm: WARNING In kvm_apic_accept_events

[Dmitry Vyukov]

On Sat, Nov 12, 2016 at 2:56 AM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program triggers WARNING in kvm_apic_accept_events:
> https://gist.githubusercontent.com/dvyukov/95b845a2e637485568ea1ef181a72370/raw/d90717dd67128b21715c5e794568a1600f613d97/gistfile1.txt
>
> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>
> [ cut here ]
> WARNING: CPU: 1 PID: 23523 at arch/x86/kvm/lapic.c:2330
> kvm_apic_accept_events+0x3f6/0x460
> Modules linked in:[   55.632648] CPU: 1 PID: 2812 Comm: a.out Not
> tainted 4.9.0-rc4+ #41
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  88006be77978 81c2f79b  
>  8322d6a0 81127426 88006be779c0 8123ffef
>  811acc46 091a 8322d6a0 091a
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>  [] __warn+0x19f/0x1e0 kernel/panic.c:550
>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>  [] kvm_apic_accept_events+0x3f6/0x460
> arch/x86/kvm/lapic.c:2330
>  [< inline >] vcpu_block arch/x86/kvm/x86.c:6793
>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6828
>  [] kvm_arch_vcpu_ioctl_run+0x1580/0x5a90
> arch/x86/kvm/x86.c:6984
>  [] kvm_vcpu_ioctl+0x61e/0xdd0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
> arch/x86/entry/entry_64.S:209
> ---[ end trace b2415ddd1b26dce6 ]---


ping

Re: kvm: WARNING in kvm_load_guest_fpu

[Dmitry Vyukov]

On Sat, Nov 12, 2016 at 4:57 AM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program triggers WARNING in kvm_load_guest_fpu:
> https://gist.githubusercontent.com/dvyukov/5bc076073b48772e22b5f33acbe2b743/raw/1000df869f0d58a5c6e637268453c711280b255d/gistfile1.txt
>
> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>
> WARNING: CPU: 3 PID: 23924 at
> ./arch/x86/include/asm/fpu/internal.h:164
> kvm_load_guest_fpu.part.163+0xf1/0x340
> Kernel panic - not syncing: panic_on_warn set ...
>
> CPU: 3 PID: 23924 Comm: syz-executor Not tainted 4.9.0-rc4+ #44
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  88006ce67920 81c2f79b 83271fc0 88006ce679f8
>  8321c300 81096631 88006ce679e8 81548463
>  41b58ab3 837cd505 815482ac 815490e9
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>  [] panic+0x1b7/0x3a3 kernel/panic.c:179
>  [] __warn+0x1c4/0x1e0 kernel/panic.c:542
>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>  [< inline >] copy_kernel_to_fxregs
> arch/x86/include/asm/fpu/internal.h:164
>  [< inline >] __copy_kernel_to_fpregs
> arch/x86/include/asm/fpu/internal.h:454
>  [] kvm_load_guest_fpu.part.163+0xf1/0x340
> arch/x86/kvm/x86.c:7383
>  [< inline >] kvm_load_guest_fpu arch/x86/kvm/x86.c:6456
>  [< inline >] vcpu_enter_guest arch/x86/kvm/x86.c:6660
>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6826
>  [] kvm_arch_vcpu_ioctl_run+0x3b13/0x5a90
> arch/x86/kvm/x86.c:6984
>  [] kvm_vcpu_ioctl+0x61e/0xdd0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
> Dumping ftrace buffer:
>(ftrace buffer empty)
> Kernel Offset: disabled
> reboot: cpu_has_vmx: ecx=80a02021 1

ping
this may be an information leak

Re: kvm: WARNING in kvm_load_guest_fpu

[Dmitry Vyukov]

On Sat, Nov 12, 2016 at 4:57 AM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program triggers WARNING in kvm_load_guest_fpu:
> https://gist.githubusercontent.com/dvyukov/5bc076073b48772e22b5f33acbe2b743/raw/1000df869f0d58a5c6e637268453c711280b255d/gistfile1.txt
>
> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>
> WARNING: CPU: 3 PID: 23924 at
> ./arch/x86/include/asm/fpu/internal.h:164
> kvm_load_guest_fpu.part.163+0xf1/0x340
> Kernel panic - not syncing: panic_on_warn set ...
>
> CPU: 3 PID: 23924 Comm: syz-executor Not tainted 4.9.0-rc4+ #44
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  88006ce67920 81c2f79b 83271fc0 88006ce679f8
>  8321c300 81096631 88006ce679e8 81548463
>  41b58ab3 837cd505 815482ac 815490e9
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>  [] panic+0x1b7/0x3a3 kernel/panic.c:179
>  [] __warn+0x1c4/0x1e0 kernel/panic.c:542
>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>  [< inline >] copy_kernel_to_fxregs
> arch/x86/include/asm/fpu/internal.h:164
>  [< inline >] __copy_kernel_to_fpregs
> arch/x86/include/asm/fpu/internal.h:454
>  [] kvm_load_guest_fpu.part.163+0xf1/0x340
> arch/x86/kvm/x86.c:7383
>  [< inline >] kvm_load_guest_fpu arch/x86/kvm/x86.c:6456
>  [< inline >] vcpu_enter_guest arch/x86/kvm/x86.c:6660
>  [< inline >] vcpu_run arch/x86/kvm/x86.c:6826
>  [] kvm_arch_vcpu_ioctl_run+0x3b13/0x5a90
> arch/x86/kvm/x86.c:6984
>  [] kvm_vcpu_ioctl+0x61e/0xdd0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
> Dumping ftrace buffer:
>(ftrace buffer empty)
> Kernel Offset: disabled
> reboot: cpu_has_vmx: ecx=80a02021 1

ping
this may be an information leak

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:
>
>
> On 13/01/2017 12:15, Dmitry Vyukov wrote:
>>
>> I've commented out the WARNING for now, but I am seeing lots of
>> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
>>
>>
>> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
>> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
>> Read of size 8 by task syz-executor2/16715
>> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
>> flags: 0x500()
>> raw: 0500   
>> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
>> page dumped because: kasan: bad access detected
>> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>  kasan_report_error mm/kasan/report.c:213 [inline]
>>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
>> [inline]
>>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x445329
>> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
>> RAX: ffda RBX: 2000 RCX: 00445329
>> RDX: 0003 RSI: 00af1000 RDI: 2000
>> RBP: 006dfe90 R08:  R09: 
>> R10: 0032 R11: 0282 R12: 0070
>> R13: 0006 R14:  R15: 20001000
>> Memory state around the buggy address:
>>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>   ^
>>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> ==
>
> This could be related to the gfn_to_rmap issues.


Humm... That's possible. Potentially I am not seeing any more of
spte-related crashes after I applied the following patch:

--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
/* Check for overlaps */
r = -EEXIST;
kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
-   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
-   (slot->id == id))
+   if (slot->id == id)
continue;
if (!((base_gfn + npages <= slot->base_gfn) ||
  (base_gfn >= slot->base_gfn + slot->npages)))

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Jan 17, 2017 at 4:20 PM, Paolo Bonzini  wrote:
>
>
> On 13/01/2017 12:15, Dmitry Vyukov wrote:
>>
>> I've commented out the WARNING for now, but I am seeing lots of
>> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
>>
>>
>> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
>> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
>> Read of size 8 by task syz-executor2/16715
>> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
>> flags: 0x500()
>> raw: 0500   
>> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
>> page dumped because: kasan: bad access detected
>> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>  kasan_report_error mm/kasan/report.c:213 [inline]
>>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 
>> [inline]
>>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x445329
>> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
>> RAX: ffda RBX: 2000 RCX: 00445329
>> RDX: 0003 RSI: 00af1000 RDI: 2000
>> RBP: 006dfe90 R08:  R09: 
>> R10: 0032 R11: 0282 R12: 0070
>> R13: 0006 R14:  R15: 20001000
>> Memory state around the buggy address:
>>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>   ^
>>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> ==
>
> This could be related to the gfn_to_rmap issues.


Humm... That's possible. Potentially I am not seeing any more of
spte-related crashes after I applied the following patch:

--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -968,8 +968,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
/* Check for overlaps */
r = -EEXIST;
kvm_for_each_memslot(slot, __kvm_memslots(kvm, as_id)) {
-   if ((slot->id >= KVM_USER_MEM_SLOTS) ||
-   (slot->id == id))
+   if (slot->id == id)
continue;
if (!((base_gfn + npages <= slot->base_gfn) ||
  (base_gfn >= slot->base_gfn + slot->npages)))

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Paolo Bonzini]

On 13/01/2017 12:15, Dmitry Vyukov wrote:
> 
> I've commented out the WARNING for now, but I am seeing lots of
> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
> 
> 
> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
> Read of size 8 by task syz-executor2/16715
> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
> flags: 0x500()
> raw: 0500   
> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
> page dumped because: kasan: bad access detected
> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  kasan_report_error mm/kasan/report.c:213 [inline]
>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 [inline]
>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: ffda RBX: 2000 RCX: 00445329
> RDX: 0003 RSI: 00af1000 RDI: 2000
> RBP: 006dfe90 R08:  R09: 
> R10: 0032 R11: 0282 R12: 0070
> R13: 0006 R14:  R15: 20001000
> Memory state around the buggy address:
>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>   ^
>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> ==

This could be related to the gfn_to_rmap issues.

Paolo

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Paolo Bonzini]

On 13/01/2017 12:15, Dmitry Vyukov wrote:
> 
> I've commented out the WARNING for now, but I am seeing lots of
> use-after-free's and rcu stalls involving mmu_spte_clear_track_bits:
> 
> 
> BUG: KASAN: use-after-free in mmu_spte_clear_track_bits+0x186/0x190
> arch/x86/kvm/mmu.c:597 at addr 880068ae2008
> Read of size 8 by task syz-executor2/16715
> page:ea00016e6170 count:0 mapcount:0 mapping:  (null) index:0x0
> flags: 0x500()
> raw: 0500   
> raw: ea00017ec5a0 ea0001783d48 88006aec5d98
> page dumped because: kasan: bad access detected
> CPU: 2 PID: 16715 Comm: syz-executor2 Not tainted 4.10.0-rc3+ #163
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  kasan_report_error mm/kasan/report.c:213 [inline]
>  kasan_report+0x42d/0x460 mm/kasan/report.c:307
>  __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:333
>  mmu_spte_clear_track_bits+0x186/0x190 arch/x86/kvm/mmu.c:597
>  drop_spte+0x24/0x280 arch/x86/kvm/mmu.c:1182
>  kvm_zap_rmapp+0x119/0x260 arch/x86/kvm/mmu.c:1401
>  kvm_unmap_rmapp+0x1d/0x30 arch/x86/kvm/mmu.c:1412
>  kvm_handle_hva_range+0x54a/0x7d0 arch/x86/kvm/mmu.c:1565
>  kvm_unmap_hva_range+0x2e/0x40 arch/x86/kvm/mmu.c:1591
>  kvm_mmu_notifier_invalidate_range_start+0xae/0x140
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:360
>  __mmu_notifier_invalidate_range_start+0x1f8/0x300 mm/mmu_notifier.c:199
>  mmu_notifier_invalidate_range_start include/linux/mmu_notifier.h:282 [inline]
>  unmap_vmas+0x14b/0x1b0 mm/memory.c:1368
>  unmap_region+0x2f8/0x560 mm/mmap.c:2460
>  do_munmap+0x7b8/0xfa0 mm/mmap.c:2657
>  mmap_region+0x68f/0x18e0 mm/mmap.c:1612
>  do_mmap+0x6a2/0xd40 mm/mmap.c:1450
>  do_mmap_pgoff include/linux/mm.h:2031 [inline]
>  vm_mmap_pgoff+0x1a9/0x200 mm/util.c:305
>  SYSC_mmap_pgoff mm/mmap.c:1500 [inline]
>  SyS_mmap_pgoff+0x22c/0x5d0 mm/mmap.c:1458
>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:95 [inline]
>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:86
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7fb33933cb58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: ffda RBX: 2000 RCX: 00445329
> RDX: 0003 RSI: 00af1000 RDI: 2000
> RBP: 006dfe90 R08:  R09: 
> R10: 0032 R11: 0282 R12: 0070
> R13: 0006 R14:  R15: 20001000
> Memory state around the buggy address:
>  880068ae1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  880068ae1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 880068ae2000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>   ^
>  880068ae2080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>  880068ae2100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> ==

This could be related to the gfn_to_rmap issues.

Paolo

Re: kvm: WARNING in x86_emulate_insn

[Dmitry Vyukov]

On Tue, Jan 17, 2017 at 12:34 PM, Dmitry Vyukov  wrote:
> On Fri, Jan 13, 2017 at 6:47 PM, Radim Krčmář  wrote:
>> 2017-01-12 14:55+0100, Dmitry Vyukov:
>>> Hello,
>>>
>>> I've got the following WARNING in x86_emulate_insn while running
>>> syzkaller fuzzer:
>>>
>>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/emulate.c:5558
>>> x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>>> Modules linked in:
>>> CPU: 2 PID: 18646 Comm: syz-executor Not tainted 4.10.0-rc3+ #155
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>>  x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>>>  x86_emulate_instruction+0x403/0x1cc0 arch/x86/kvm/x86.c:5618
>>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x445329
>>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>>> RAX: ffda RBX: 0018 RCX: 00445329
>>> RDX:  RSI: ae80 RDI: 0018
>>> RBP: 006deb40 R08:  R09: 
>>> R10:  R11: 0286 R12: 00700150
>>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>>> ---[ end trace 6b54f749506b620c ]---
>>> [ cut here ]
>>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/x86.c:366
>>> exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>>> Modules linked in:
>>> CPU: 2 PID: 18646 Comm: syz-executor Tainted: GW   4.10.0-rc3+ 
>>> #155
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>>  exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>>>  x86_emulate_instruction+0x1356/0x1cc0 arch/x86/kvm/x86.c:5664
>>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x445329
>>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>>> RAX: ffda RBX: 0018 RCX: 00445329
>>> RDX:  RSI: ae80 RDI: 0018
>>> RBP: 006deb40 R08:  R09: 
>>> R10:  R11: 0286 R12: 00700150
>>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>>> ---[ end trace 6b54f749506b620d ]---
>>>
>>> On commit ba836a6f5ab1243ff5e08a941a2d1de8b31244e1.
>>>
>>> Unfortunately I can't reproduce it with a C program.
>>> It reproduces with the following syzkaller program within a minute, though:
>>> https://gist.githubusercontent.com/dvyukov/d09118fb9d986a9385487d80a1b50680/raw/884c68d22c3a80778ae596a6c5daf7467ea41b68/gistfile1.txt
>>> It can be executed following these instructions:
>>> https://github.com/google/syzkaller/wiki/How-to-execute-syzkaller-programs
>>> I run syz-execprog as:
>>> ./syz-execprog -repeat=0 -procs=8 -sandbox=none gistfile1.txt
>>>
>>> Note that syz_kvm_setup_cpu is a pseudo syscall that setups vcpu into
>>> a complex state:
>>> https://github.com/google/syzkaller/blob/master/executor/common_kvm_amd64.h#L271
>>>
>>> My bet would be on some race where VM memory is overwritten
>>> concurrently, and it affects either guest execution or
>>> emulate_instruction in a bad way...
>>
>> Yeah, all functions that return X86EMUL_PROPAGATE_FAULT seem to set
>> exception.vector to something sane.  The only easy way to get a bad value 
>> there
>> is when x86_emulate_instruction() clears it to -1U, 

Re: kvm: WARNING in x86_emulate_insn

[Dmitry Vyukov]

On Tue, Jan 17, 2017 at 12:34 PM, Dmitry Vyukov  wrote:
> On Fri, Jan 13, 2017 at 6:47 PM, Radim Krčmář  wrote:
>> 2017-01-12 14:55+0100, Dmitry Vyukov:
>>> Hello,
>>>
>>> I've got the following WARNING in x86_emulate_insn while running
>>> syzkaller fuzzer:
>>>
>>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/emulate.c:5558
>>> x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>>> Modules linked in:
>>> CPU: 2 PID: 18646 Comm: syz-executor Not tainted 4.10.0-rc3+ #155
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>>  x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>>>  x86_emulate_instruction+0x403/0x1cc0 arch/x86/kvm/x86.c:5618
>>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x445329
>>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>>> RAX: ffda RBX: 0018 RCX: 00445329
>>> RDX:  RSI: ae80 RDI: 0018
>>> RBP: 006deb40 R08:  R09: 
>>> R10:  R11: 0286 R12: 00700150
>>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>>> ---[ end trace 6b54f749506b620c ]---
>>> [ cut here ]
>>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/x86.c:366
>>> exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>>> Modules linked in:
>>> CPU: 2 PID: 18646 Comm: syz-executor Tainted: GW   4.10.0-rc3+ 
>>> #155
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> Call Trace:
>>>  __dump_stack lib/dump_stack.c:15 [inline]
>>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>>  exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>>>  x86_emulate_instruction+0x1356/0x1cc0 arch/x86/kvm/x86.c:5664
>>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> RIP: 0033:0x445329
>>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>>> RAX: ffda RBX: 0018 RCX: 00445329
>>> RDX:  RSI: ae80 RDI: 0018
>>> RBP: 006deb40 R08:  R09: 
>>> R10:  R11: 0286 R12: 00700150
>>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>>> ---[ end trace 6b54f749506b620d ]---
>>>
>>> On commit ba836a6f5ab1243ff5e08a941a2d1de8b31244e1.
>>>
>>> Unfortunately I can't reproduce it with a C program.
>>> It reproduces with the following syzkaller program within a minute, though:
>>> https://gist.githubusercontent.com/dvyukov/d09118fb9d986a9385487d80a1b50680/raw/884c68d22c3a80778ae596a6c5daf7467ea41b68/gistfile1.txt
>>> It can be executed following these instructions:
>>> https://github.com/google/syzkaller/wiki/How-to-execute-syzkaller-programs
>>> I run syz-execprog as:
>>> ./syz-execprog -repeat=0 -procs=8 -sandbox=none gistfile1.txt
>>>
>>> Note that syz_kvm_setup_cpu is a pseudo syscall that setups vcpu into
>>> a complex state:
>>> https://github.com/google/syzkaller/blob/master/executor/common_kvm_amd64.h#L271
>>>
>>> My bet would be on some race where VM memory is overwritten
>>> concurrently, and it affects either guest execution or
>>> emulate_instruction in a bad way...
>>
>> Yeah, all functions that return X86EMUL_PROPAGATE_FAULT seem to set
>> exception.vector to something sane.  The only easy way to get a bad value 
>> there
>> is when x86_emulate_instruction() clears it to -1U, but I don't see how a 
>> race
>> would 

Re: kvm: WARNING in x86_emulate_insn

[Dmitry Vyukov]

On Fri, Jan 13, 2017 at 6:47 PM, Radim Krčmář  wrote:
> 2017-01-12 14:55+0100, Dmitry Vyukov:
>> Hello,
>>
>> I've got the following WARNING in x86_emulate_insn while running
>> syzkaller fuzzer:
>>
>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/emulate.c:5558
>> x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>> Modules linked in:
>> CPU: 2 PID: 18646 Comm: syz-executor Not tainted 4.10.0-rc3+ #155
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>  x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>>  x86_emulate_instruction+0x403/0x1cc0 arch/x86/kvm/x86.c:5618
>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x445329
>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>> RAX: ffda RBX: 0018 RCX: 00445329
>> RDX:  RSI: ae80 RDI: 0018
>> RBP: 006deb40 R08:  R09: 
>> R10:  R11: 0286 R12: 00700150
>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>> ---[ end trace 6b54f749506b620c ]---
>> [ cut here ]
>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/x86.c:366
>> exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>> Modules linked in:
>> CPU: 2 PID: 18646 Comm: syz-executor Tainted: GW   4.10.0-rc3+ 
>> #155
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>  exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>>  x86_emulate_instruction+0x1356/0x1cc0 arch/x86/kvm/x86.c:5664
>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x445329
>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>> RAX: ffda RBX: 0018 RCX: 00445329
>> RDX:  RSI: ae80 RDI: 0018
>> RBP: 006deb40 R08:  R09: 
>> R10:  R11: 0286 R12: 00700150
>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>> ---[ end trace 6b54f749506b620d ]---
>>
>> On commit ba836a6f5ab1243ff5e08a941a2d1de8b31244e1.
>>
>> Unfortunately I can't reproduce it with a C program.
>> It reproduces with the following syzkaller program within a minute, though:
>> https://gist.githubusercontent.com/dvyukov/d09118fb9d986a9385487d80a1b50680/raw/884c68d22c3a80778ae596a6c5daf7467ea41b68/gistfile1.txt
>> It can be executed following these instructions:
>> https://github.com/google/syzkaller/wiki/How-to-execute-syzkaller-programs
>> I run syz-execprog as:
>> ./syz-execprog -repeat=0 -procs=8 -sandbox=none gistfile1.txt
>>
>> Note that syz_kvm_setup_cpu is a pseudo syscall that setups vcpu into
>> a complex state:
>> https://github.com/google/syzkaller/blob/master/executor/common_kvm_amd64.h#L271
>>
>> My bet would be on some race where VM memory is overwritten
>> concurrently, and it affects either guest execution or
>> emulate_instruction in a bad way...
>
> Yeah, all functions that return X86EMUL_PROPAGATE_FAULT seem to set
> exception.vector to something sane.  The only easy way to get a bad value 
> there
> is when x86_emulate_instruction() clears it to -1U, but I don't see how a race
> would play out.
>
> Anyway, I can't reproduce on bare metal [got another warning, see below].
> Will try after rebuilding a guest kernel.

Re: kvm: WARNING in x86_emulate_insn

[Dmitry Vyukov]

On Fri, Jan 13, 2017 at 6:47 PM, Radim Krčmář  wrote:
> 2017-01-12 14:55+0100, Dmitry Vyukov:
>> Hello,
>>
>> I've got the following WARNING in x86_emulate_insn while running
>> syzkaller fuzzer:
>>
>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/emulate.c:5558
>> x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>> Modules linked in:
>> CPU: 2 PID: 18646 Comm: syz-executor Not tainted 4.10.0-rc3+ #155
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>  x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>>  x86_emulate_instruction+0x403/0x1cc0 arch/x86/kvm/x86.c:5618
>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x445329
>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>> RAX: ffda RBX: 0018 RCX: 00445329
>> RDX:  RSI: ae80 RDI: 0018
>> RBP: 006deb40 R08:  R09: 
>> R10:  R11: 0286 R12: 00700150
>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>> ---[ end trace 6b54f749506b620c ]---
>> [ cut here ]
>> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/x86.c:366
>> exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>> Modules linked in:
>> CPU: 2 PID: 18646 Comm: syz-executor Tainted: GW   4.10.0-rc3+ 
>> #155
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:15 [inline]
>>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>>  __warn+0x19f/0x1e0 kernel/panic.c:547
>>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>>  exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>>  x86_emulate_instruction+0x1356/0x1cc0 arch/x86/kvm/x86.c:5664
>>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>>  vfs_ioctl fs/ioctl.c:43 [inline]
>>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>>  SYSC_ioctl fs/ioctl.c:698 [inline]
>>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>>  entry_SYSCALL_64_fastpath+0x1f/0xc2
>> RIP: 0033:0x445329
>> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
>> RAX: ffda RBX: 0018 RCX: 00445329
>> RDX:  RSI: ae80 RDI: 0018
>> RBP: 006deb40 R08:  R09: 
>> R10:  R11: 0286 R12: 00700150
>> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
>> ---[ end trace 6b54f749506b620d ]---
>>
>> On commit ba836a6f5ab1243ff5e08a941a2d1de8b31244e1.
>>
>> Unfortunately I can't reproduce it with a C program.
>> It reproduces with the following syzkaller program within a minute, though:
>> https://gist.githubusercontent.com/dvyukov/d09118fb9d986a9385487d80a1b50680/raw/884c68d22c3a80778ae596a6c5daf7467ea41b68/gistfile1.txt
>> It can be executed following these instructions:
>> https://github.com/google/syzkaller/wiki/How-to-execute-syzkaller-programs
>> I run syz-execprog as:
>> ./syz-execprog -repeat=0 -procs=8 -sandbox=none gistfile1.txt
>>
>> Note that syz_kvm_setup_cpu is a pseudo syscall that setups vcpu into
>> a complex state:
>> https://github.com/google/syzkaller/blob/master/executor/common_kvm_amd64.h#L271
>>
>> My bet would be on some race where VM memory is overwritten
>> concurrently, and it affects either guest execution or
>> emulate_instruction in a bad way...
>
> Yeah, all functions that return X86EMUL_PROPAGATE_FAULT seem to set
> exception.vector to something sane.  The only easy way to get a bad value 
> there
> is when x86_emulate_instruction() clears it to -1U, but I don't see how a race
> would play out.
>
> Anyway, I can't reproduce on bare metal [got another warning, see below].
> Will try after rebuilding a guest kernel.


I've dumped exception 

Re: kvm: WARNING in x86_emulate_insn

[Radim Krčmář]

2017-01-12 14:55+0100, Dmitry Vyukov:
> Hello,
> 
> I've got the following WARNING in x86_emulate_insn while running
> syzkaller fuzzer:
> 
> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/emulate.c:5558
> x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
> Modules linked in:
> CPU: 2 PID: 18646 Comm: syz-executor Not tainted 4.10.0-rc3+ #155
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  __warn+0x19f/0x1e0 kernel/panic.c:547
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>  x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>  x86_emulate_instruction+0x403/0x1cc0 arch/x86/kvm/x86.c:5618
>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
> RAX: ffda RBX: 0018 RCX: 00445329
> RDX:  RSI: ae80 RDI: 0018
> RBP: 006deb40 R08:  R09: 
> R10:  R11: 0286 R12: 00700150
> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
> ---[ end trace 6b54f749506b620c ]---
> [ cut here ]
> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/x86.c:366
> exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
> Modules linked in:
> CPU: 2 PID: 18646 Comm: syz-executor Tainted: GW   4.10.0-rc3+ 
> #155
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  __warn+0x19f/0x1e0 kernel/panic.c:547
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>  exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>  x86_emulate_instruction+0x1356/0x1cc0 arch/x86/kvm/x86.c:5664
>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
> RAX: ffda RBX: 0018 RCX: 00445329
> RDX:  RSI: ae80 RDI: 0018
> RBP: 006deb40 R08:  R09: 
> R10:  R11: 0286 R12: 00700150
> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
> ---[ end trace 6b54f749506b620d ]---
> 
> On commit ba836a6f5ab1243ff5e08a941a2d1de8b31244e1.
> 
> Unfortunately I can't reproduce it with a C program.
> It reproduces with the following syzkaller program within a minute, though:
> https://gist.githubusercontent.com/dvyukov/d09118fb9d986a9385487d80a1b50680/raw/884c68d22c3a80778ae596a6c5daf7467ea41b68/gistfile1.txt
> It can be executed following these instructions:
> https://github.com/google/syzkaller/wiki/How-to-execute-syzkaller-programs
> I run syz-execprog as:
> ./syz-execprog -repeat=0 -procs=8 -sandbox=none gistfile1.txt
> 
> Note that syz_kvm_setup_cpu is a pseudo syscall that setups vcpu into
> a complex state:
> https://github.com/google/syzkaller/blob/master/executor/common_kvm_amd64.h#L271
> 
> My bet would be on some race where VM memory is overwritten
> concurrently, and it affects either guest execution or
> emulate_instruction in a bad way...

Yeah, all functions that return X86EMUL_PROPAGATE_FAULT seem to set
exception.vector to something sane.  The only easy way to get a bad value there
is when x86_emulate_instruction() clears it to -1U, but I don't see how a race
would play out.

Anyway, I can't reproduce on bare metal [got another warning, see below].
Will try after rebuilding a guest kernel.

Thanks.


The best result was this warning after 300k executions:

[ cut here ]
WARNING: CPU: 7 PID: 20187 at lib/debugobjects.c:263 

Re: kvm: WARNING in x86_emulate_insn

[Radim Krčmář]

2017-01-12 14:55+0100, Dmitry Vyukov:
> Hello,
> 
> I've got the following WARNING in x86_emulate_insn while running
> syzkaller fuzzer:
> 
> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/emulate.c:5558
> x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
> Modules linked in:
> CPU: 2 PID: 18646 Comm: syz-executor Not tainted 4.10.0-rc3+ #155
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  __warn+0x19f/0x1e0 kernel/panic.c:547
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>  x86_emulate_insn+0x16a5/0x4090 arch/x86/kvm/emulate.c:5572
>  x86_emulate_instruction+0x403/0x1cc0 arch/x86/kvm/x86.c:5618
>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
> RAX: ffda RBX: 0018 RCX: 00445329
> RDX:  RSI: ae80 RDI: 0018
> RBP: 006deb40 R08:  R09: 
> R10:  R11: 0286 R12: 00700150
> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
> ---[ end trace 6b54f749506b620c ]---
> [ cut here ]
> WARNING: CPU: 2 PID: 18646 at arch/x86/kvm/x86.c:366
> exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
> Modules linked in:
> CPU: 2 PID: 18646 Comm: syz-executor Tainted: GW   4.10.0-rc3+ 
> #155
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:15 [inline]
>  dump_stack+0x292/0x3a2 lib/dump_stack.c:51
>  __warn+0x19f/0x1e0 kernel/panic.c:547
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:582
>  exception_type+0x73/0x80 arch/x86/kvm/x86.c:366
>  x86_emulate_instruction+0x1356/0x1cc0 arch/x86/kvm/x86.c:5664
>  emulate_instruction arch/x86/include/asm/kvm_host.h:1127 [inline]
>  handle_exception+0x594/0xfd0 arch/x86/kvm/vmx.c:5762
>  vmx_handle_exit+0x2b7/0x38b0 arch/x86/kvm/vmx.c:8625
>  vcpu_enter_guest arch/x86/kvm/x86.c:6888 [inline]
>  vcpu_run arch/x86/kvm/x86.c:6947 [inline]
>  kvm_arch_vcpu_ioctl_run+0xf3d/0x4660 arch/x86/kvm/x86.c:7105
>  kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2569
>  vfs_ioctl fs/ioctl.c:43 [inline]
>  do_vfs_ioctl+0x1bf/0x1780 fs/ioctl.c:683
>  SYSC_ioctl fs/ioctl.c:698 [inline]
>  SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689
>  entry_SYSCALL_64_fastpath+0x1f/0xc2
> RIP: 0033:0x445329
> RSP: 002b:7f9e6e22fb58 EFLAGS: 0286 ORIG_RAX: 0010
> RAX: ffda RBX: 0018 RCX: 00445329
> RDX:  RSI: ae80 RDI: 0018
> RBP: 006deb40 R08:  R09: 
> R10:  R11: 0286 R12: 00700150
> R13:  R14: 7f9e6e2309c0 R15: 7f9e6e230700
> ---[ end trace 6b54f749506b620d ]---
> 
> On commit ba836a6f5ab1243ff5e08a941a2d1de8b31244e1.
> 
> Unfortunately I can't reproduce it with a C program.
> It reproduces with the following syzkaller program within a minute, though:
> https://gist.githubusercontent.com/dvyukov/d09118fb9d986a9385487d80a1b50680/raw/884c68d22c3a80778ae596a6c5daf7467ea41b68/gistfile1.txt
> It can be executed following these instructions:
> https://github.com/google/syzkaller/wiki/How-to-execute-syzkaller-programs
> I run syz-execprog as:
> ./syz-execprog -repeat=0 -procs=8 -sandbox=none gistfile1.txt
> 
> Note that syz_kvm_setup_cpu is a pseudo syscall that setups vcpu into
> a complex state:
> https://github.com/google/syzkaller/blob/master/executor/common_kvm_amd64.h#L271
> 
> My bet would be on some race where VM memory is overwritten
> concurrently, and it affects either guest execution or
> emulate_instruction in a bad way...

Yeah, all functions that return X86EMUL_PROPAGATE_FAULT seem to set
exception.vector to something sane.  The only easy way to get a bad value there
is when x86_emulate_instruction() clears it to -1U, but I don't see how a race
would play out.

Anyway, I can't reproduce on bare metal [got another warning, see below].
Will try after rebuilding a guest kernel.

Thanks.


The best result was this warning after 300k executions:

[ cut here ]
WARNING: CPU: 7 PID: 20187 at lib/debugobjects.c:263 

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Dec 13, 2016 at 8:50 PM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program:
> https://gist.githubusercontent.com/dvyukov/23d8bd622fd526d7701ac2057bbbc9c2/raw/aacd20451e6f460232f5e1da262b653fb3155613/gistfile1.txt
>
> leads to WARNING in mmu_spte_clear_track_bits and later to splash of
> BUG: Bad page state in process a.out  pfn:619b5
>
> On commit e7aa8c2eb11ba69b1b69099c3c7bd6be3087b0ba (Dec 12).
>
> [ cut here ]
> WARNING: CPU: 0 PID: 6907 at mmu_spte_clear_track_bits+0x326/0x3a0
> arch/x86/kvm/mmu.c:614
> Modules linked in:
> CPU: 0 PID: 6907 Comm: a.out Not tainted 4.9.0+ #85
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
> [< inline >] __dump_stack lib/dump_stack.c:15
> [] dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
> [] __warn+0x1a4/0x1e0 kernel/panic.c:550
> [] warn_slowpath_null+0x31/0x40 kernel/panic.c:585
> [] mmu_spte_clear_track_bits+0x326/0x3a0
> arch/x86/kvm/mmu.c:614
> [] drop_spte+0x29/0x220 arch/x86/kvm/mmu.c:1182
> [] mmu_page_zap_pte+0x209/0x300 arch/x86/kvm/mmu.c:2306
> [< inline >] kvm_mmu_page_unlink_children arch/x86/kvm/mmu.c:2328
> [] kvm_mmu_prepare_zap_page+0x1cd/0x1240
> arch/x86/kvm/mmu.c:2372
> [< inline >] kvm_zap_obsolete_pages arch/x86/kvm/mmu.c:4915
> [] kvm_mmu_invalidate_zap_all_pages+0x4af/0x6f0
> arch/x86/kvm/mmu.c:4956
> [] kvm_arch_flush_shadow_all+0x1a/0x20
> arch/x86/kvm/x86.c:8177
> [] kvm_mmu_notifier_release+0x76/0xb0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:467
> [] __mmu_notifier_release+0x1fe/0x6c0 
> mm/mmu_notifier.c:74
> [< inline >] mmu_notifier_release ./include/linux/mmu_notifier.h:235
> [] exit_mmap+0x3d1/0x4a0 mm/mmap.c:2918
> [< inline >] __mmput kernel/fork.c:868
> [] mmput+0x1fd/0x690 kernel/fork.c:890
> [< inline >] exit_mm kernel/exit.c:521
> [] do_exit+0x9e7/0x2930 kernel/exit.c:826
> [] do_group_exit+0x14e/0x420 kernel/exit.c:943
> [< inline >] SYSC_exit_group kernel/exit.c:954
> [] SyS_exit_group+0x22/0x30 kernel/exit.c:952
> [] entry_SYSCALL_64_fastpath+0x23/0xc6
> arch/x86/entry/entry_64.S:203
> RIP: 0033:0x43f4d9
> RSP: 002b:7ffc7e83f548 EFLAGS: 0246 ORIG_RAX: 00e7
> RAX: ffda RBX: 006d6660 RCX: 0043f4d9
> RDX:  RSI:  RDI: 
> RBP: 0001 R08: 003c R09: 00e7
> R10: ffd0 R11: 0246 R12: 
> R13:  R14: 7fe58e3869c0 R15: 7fe58e386700
> ---[ end trace 37ef4e3d7e4c81a9 ]---
>
>
> BUG: Bad page state in process a.out  pfn:61fb5
> page:ea000187ed40 count:0 mapcount:0 mapping:  (null) index:0x0
> flags: 0x5fffc14(referenced|dirty)
> raw: 05fffc14   
> raw:  ea000187ed60  
> page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag set
> bad because of flags: 0x14(referenced|dirty)
> Modules linked in:
> CPU: 2 PID: 7169 Comm: a.out Tainted: GW   4.9.0+ #85
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
> [< inline >] __dump_stack lib/dump_stack.c:15
> [] dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
> [] bad_page+0x29c/0x320 mm/page_alloc.c:550
> [] check_new_page_bad+0x203/0x2f0 mm/page_alloc.c:1682
> [< inline >] check_new_page mm/page_alloc.c:1694
> [< inline >] check_new_pages mm/page_alloc.c:1731
> [] buffered_rmqueue+0x1770/0x2900 mm/page_alloc.c:2668
> [] get_page_from_freelist+0x213/0x1180
> mm/page_alloc.c:2985
> [] __alloc_pages_nodemask+0x3b2/0xc90 
> mm/page_alloc.c:3801
> [< inline >] __alloc_pages ./include/linux/gfp.h:433
> [< inline >] __alloc_pages_node ./include/linux/gfp.h:446
> [] alloc_pages_vma+0x723/0xa30 mm/mempolicy.c:2012
> [] do_huge_pmd_anonymous_page+0x35f/0x1b10
> mm/huge_memory.c:704
> [< inline >] create_huge_pmd mm/memory.c:3476
> [< inline >] __handle_mm_fault mm/memory.c:3626
> [] handle_mm_fault+0x1975/0x2b90 mm/memory.c:3687
> [] __do_page_fault+0x4fb/0xb60 arch/x86/mm/fault.c:1396
> [] trace_do_page_fault+0x159/0x810
> arch/x86/mm/fault.c:1489
> [] do_async_page_fault+0x77/0xd0 
> arch/x86/kernel/kvm.c:264
> 

Re: kvm: WARNING in mmu_spte_clear_track_bits

[Dmitry Vyukov]

On Tue, Dec 13, 2016 at 8:50 PM, Dmitry Vyukov  wrote:
> Hello,
>
> The following program:
> https://gist.githubusercontent.com/dvyukov/23d8bd622fd526d7701ac2057bbbc9c2/raw/aacd20451e6f460232f5e1da262b653fb3155613/gistfile1.txt
>
> leads to WARNING in mmu_spte_clear_track_bits and later to splash of
> BUG: Bad page state in process a.out  pfn:619b5
>
> On commit e7aa8c2eb11ba69b1b69099c3c7bd6be3087b0ba (Dec 12).
>
> [ cut here ]
> WARNING: CPU: 0 PID: 6907 at mmu_spte_clear_track_bits+0x326/0x3a0
> arch/x86/kvm/mmu.c:614
> Modules linked in:
> CPU: 0 PID: 6907 Comm: a.out Not tainted 4.9.0+ #85
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
> [< inline >] __dump_stack lib/dump_stack.c:15
> [] dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
> [] __warn+0x1a4/0x1e0 kernel/panic.c:550
> [] warn_slowpath_null+0x31/0x40 kernel/panic.c:585
> [] mmu_spte_clear_track_bits+0x326/0x3a0
> arch/x86/kvm/mmu.c:614
> [] drop_spte+0x29/0x220 arch/x86/kvm/mmu.c:1182
> [] mmu_page_zap_pte+0x209/0x300 arch/x86/kvm/mmu.c:2306
> [< inline >] kvm_mmu_page_unlink_children arch/x86/kvm/mmu.c:2328
> [] kvm_mmu_prepare_zap_page+0x1cd/0x1240
> arch/x86/kvm/mmu.c:2372
> [< inline >] kvm_zap_obsolete_pages arch/x86/kvm/mmu.c:4915
> [] kvm_mmu_invalidate_zap_all_pages+0x4af/0x6f0
> arch/x86/kvm/mmu.c:4956
> [] kvm_arch_flush_shadow_all+0x1a/0x20
> arch/x86/kvm/x86.c:8177
> [] kvm_mmu_notifier_release+0x76/0xb0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:467
> [] __mmu_notifier_release+0x1fe/0x6c0 
> mm/mmu_notifier.c:74
> [< inline >] mmu_notifier_release ./include/linux/mmu_notifier.h:235
> [] exit_mmap+0x3d1/0x4a0 mm/mmap.c:2918
> [< inline >] __mmput kernel/fork.c:868
> [] mmput+0x1fd/0x690 kernel/fork.c:890
> [< inline >] exit_mm kernel/exit.c:521
> [] do_exit+0x9e7/0x2930 kernel/exit.c:826
> [] do_group_exit+0x14e/0x420 kernel/exit.c:943
> [< inline >] SYSC_exit_group kernel/exit.c:954
> [] SyS_exit_group+0x22/0x30 kernel/exit.c:952
> [] entry_SYSCALL_64_fastpath+0x23/0xc6
> arch/x86/entry/entry_64.S:203
> RIP: 0033:0x43f4d9
> RSP: 002b:7ffc7e83f548 EFLAGS: 0246 ORIG_RAX: 00e7
> RAX: ffda RBX: 006d6660 RCX: 0043f4d9
> RDX:  RSI:  RDI: 
> RBP: 0001 R08: 003c R09: 00e7
> R10: ffd0 R11: 0246 R12: 
> R13:  R14: 7fe58e3869c0 R15: 7fe58e386700
> ---[ end trace 37ef4e3d7e4c81a9 ]---
>
>
> BUG: Bad page state in process a.out  pfn:61fb5
> page:ea000187ed40 count:0 mapcount:0 mapping:  (null) index:0x0
> flags: 0x5fffc14(referenced|dirty)
> raw: 05fffc14   
> raw:  ea000187ed60  
> page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag set
> bad because of flags: 0x14(referenced|dirty)
> Modules linked in:
> CPU: 2 PID: 7169 Comm: a.out Tainted: GW   4.9.0+ #85
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
> [< inline >] __dump_stack lib/dump_stack.c:15
> [] dump_stack+0x2ee/0x3ef lib/dump_stack.c:51
> [] bad_page+0x29c/0x320 mm/page_alloc.c:550
> [] check_new_page_bad+0x203/0x2f0 mm/page_alloc.c:1682
> [< inline >] check_new_page mm/page_alloc.c:1694
> [< inline >] check_new_pages mm/page_alloc.c:1731
> [] buffered_rmqueue+0x1770/0x2900 mm/page_alloc.c:2668
> [] get_page_from_freelist+0x213/0x1180
> mm/page_alloc.c:2985
> [] __alloc_pages_nodemask+0x3b2/0xc90 
> mm/page_alloc.c:3801
> [< inline >] __alloc_pages ./include/linux/gfp.h:433
> [< inline >] __alloc_pages_node ./include/linux/gfp.h:446
> [] alloc_pages_vma+0x723/0xa30 mm/mempolicy.c:2012
> [] do_huge_pmd_anonymous_page+0x35f/0x1b10
> mm/huge_memory.c:704
> [< inline >] create_huge_pmd mm/memory.c:3476
> [< inline >] __handle_mm_fault mm/memory.c:3626
> [] handle_mm_fault+0x1975/0x2b90 mm/memory.c:3687
> [] __do_page_fault+0x4fb/0xb60 arch/x86/mm/fault.c:1396
> [] trace_do_page_fault+0x159/0x810
> arch/x86/mm/fault.c:1489
> [] do_async_page_fault+0x77/0xd0 
> arch/x86/kernel/kvm.c:264
> [

Re: kvm: WARNING in em_jmp_far

[Paolo Bonzini]

On 15/11/2016 06:39, Nadav Amit wrote:
> 
>> On Nov 14, 2016, at 9:30 PM, Dmitry Vyukov  wrote:
>>
>> On Tue, Nov 15, 2016 at 6:24 AM, Nadav Amit  wrote:
>>>
 On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:

 Hello,

 The following program triggers WARNING in em_jmp_far:
 https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt


 WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
 em_jmp_far+0x4a7/0x530
>>>
>>> I don’t know how to “read” the test, but it seems that this warning
>>> can be triggered if CS base/limit cause a #GP exception when EIP
>>> is loaded.
>>>
>>> I think it safe to remove this warning (which I introduced) as well as
>>> the redundant “return rc” that follows it. The code should handle the
>>> emulation correctly regardless of the warning.
>>
>> There was also a similar WARNING in em_ret_far:
>> https://groups.google.com/forum/#!msg/syzkaller/o5ZftARBhrs/r1ivQ-HtBgAJ
>>
>> Please mail a fix and add a test.
> 
> I am sorry, but I don’t think my current employer allows me to contribute
> to KVM in such a manner.

No problem, we can take care of it.

Paolo

Re: kvm: WARNING in em_jmp_far

[Paolo Bonzini]

On 15/11/2016 06:39, Nadav Amit wrote:
> 
>> On Nov 14, 2016, at 9:30 PM, Dmitry Vyukov  wrote:
>>
>> On Tue, Nov 15, 2016 at 6:24 AM, Nadav Amit  wrote:
>>>
 On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:

 Hello,

 The following program triggers WARNING in em_jmp_far:
 https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt


 WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
 em_jmp_far+0x4a7/0x530
>>>
>>> I don’t know how to “read” the test, but it seems that this warning
>>> can be triggered if CS base/limit cause a #GP exception when EIP
>>> is loaded.
>>>
>>> I think it safe to remove this warning (which I introduced) as well as
>>> the redundant “return rc” that follows it. The code should handle the
>>> emulation correctly regardless of the warning.
>>
>> There was also a similar WARNING in em_ret_far:
>> https://groups.google.com/forum/#!msg/syzkaller/o5ZftARBhrs/r1ivQ-HtBgAJ
>>
>> Please mail a fix and add a test.
> 
> I am sorry, but I don’t think my current employer allows me to contribute
> to KVM in such a manner.

No problem, we can take care of it.

Paolo

Re: kvm: WARNING in em_jmp_far

[Nadav Amit]

> On Nov 14, 2016, at 9:30 PM, Dmitry Vyukov  wrote:
> 
> On Tue, Nov 15, 2016 at 6:24 AM, Nadav Amit  wrote:
>> 
>>> On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:
>>> 
>>> Hello,
>>> 
>>> The following program triggers WARNING in em_jmp_far:
>>> https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt
>>> 
>>> 
>>> WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
>>> em_jmp_far+0x4a7/0x530
>> 
>> I don’t know how to “read” the test, but it seems that this warning
>> can be triggered if CS base/limit cause a #GP exception when EIP
>> is loaded.
>> 
>> I think it safe to remove this warning (which I introduced) as well as
>> the redundant “return rc” that follows it. The code should handle the
>> emulation correctly regardless of the warning.
> 
> There was also a similar WARNING in em_ret_far:
> https://groups.google.com/forum/#!msg/syzkaller/o5ZftARBhrs/r1ivQ-HtBgAJ
> 
> Please mail a fix and add a test.

I am sorry, but I don’t think my current employer allows me to contribute
to KVM in such a manner.

Regards,
Nadav

Re: kvm: WARNING in em_jmp_far

[Nadav Amit]

> On Nov 14, 2016, at 9:30 PM, Dmitry Vyukov  wrote:
> 
> On Tue, Nov 15, 2016 at 6:24 AM, Nadav Amit  wrote:
>> 
>>> On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:
>>> 
>>> Hello,
>>> 
>>> The following program triggers WARNING in em_jmp_far:
>>> https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt
>>> 
>>> 
>>> WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
>>> em_jmp_far+0x4a7/0x530
>> 
>> I don’t know how to “read” the test, but it seems that this warning
>> can be triggered if CS base/limit cause a #GP exception when EIP
>> is loaded.
>> 
>> I think it safe to remove this warning (which I introduced) as well as
>> the redundant “return rc” that follows it. The code should handle the
>> emulation correctly regardless of the warning.
> 
> There was also a similar WARNING in em_ret_far:
> https://groups.google.com/forum/#!msg/syzkaller/o5ZftARBhrs/r1ivQ-HtBgAJ
> 
> Please mail a fix and add a test.

I am sorry, but I don’t think my current employer allows me to contribute
to KVM in such a manner.

Regards,
Nadav

Re: kvm: WARNING in em_jmp_far

[Nadav Amit]

 
> On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:
> 
> Hello,
> 
> The following program triggers WARNING in em_jmp_far:
> https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt
> 
> 
> WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
> em_jmp_far+0x4a7/0x530

I don’t know how to “read” the test, but it seems that this warning
can be triggered if CS base/limit cause a #GP exception when EIP
is loaded.

I think it safe to remove this warning (which I introduced) as well as
the redundant “return rc” that follows it. The code should handle the
emulation correctly regardless of the warning.

Regards,
Nadav

Re: kvm: WARNING in em_jmp_far

[Nadav Amit]

 
> On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:
> 
> Hello,
> 
> The following program triggers WARNING in em_jmp_far:
> https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt
> 
> 
> WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
> em_jmp_far+0x4a7/0x530

I don’t know how to “read” the test, but it seems that this warning
can be triggered if CS base/limit cause a #GP exception when EIP
is loaded.

I think it safe to remove this warning (which I introduced) as well as
the redundant “return rc” that follows it. The code should handle the
emulation correctly regardless of the warning.

Regards,
Nadav

Re: kvm: WARNING in em_jmp_far

[Dmitry Vyukov]

On Tue, Nov 15, 2016 at 6:24 AM, Nadav Amit  wrote:
>
>> On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:
>>
>> Hello,
>>
>> The following program triggers WARNING in em_jmp_far:
>> https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt
>>
>>
>> WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
>> em_jmp_far+0x4a7/0x530
>
> I don’t know how to “read” the test, but it seems that this warning
> can be triggered if CS base/limit cause a #GP exception when EIP
> is loaded.
>
> I think it safe to remove this warning (which I introduced) as well as
> the redundant “return rc” that follows it. The code should handle the
> emulation correctly regardless of the warning.

There was also a similar WARNING in em_ret_far:
https://groups.google.com/forum/#!msg/syzkaller/o5ZftARBhrs/r1ivQ-HtBgAJ

Please mail a fix and add a test.

Thanks

Re: kvm: WARNING in em_jmp_far

[Dmitry Vyukov]

On Tue, Nov 15, 2016 at 6:24 AM, Nadav Amit  wrote:
>
>> On Nov 14, 2016, at 9:06 PM, Dmitry Vyukov  wrote:
>>
>> Hello,
>>
>> The following program triggers WARNING in em_jmp_far:
>> https://gist.githubusercontent.com/dvyukov/16bfd3d68fa7d5461101ef74e07796e4/raw/e6d663980681f2c5838ff6cd361cede7d3204838/gistfile1.txt
>>
>>
>> WARNING: CPU: 1 PID: 15748 at arch/x86/kvm/emulate.c:2128 
>> em_jmp_far+0x4a7/0x530
>
> I don’t know how to “read” the test, but it seems that this warning
> can be triggered if CS base/limit cause a #GP exception when EIP
> is loaded.
>
> I think it safe to remove this warning (which I introduced) as well as
> the redundant “return rc” that follows it. The code should handle the
> emulation correctly regardless of the warning.

There was also a similar WARNING in em_ret_far:
https://groups.google.com/forum/#!msg/syzkaller/o5ZftARBhrs/r1ivQ-HtBgAJ

Please mail a fix and add a test.

Thanks

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Paolo Bonzini]

On 14/11/2016 15:09, Dmitry Vyukov wrote:
> 
> I made dump_vmcs() no-op locally. You should see the warning
> regardless, but probably it is difficult to notice. Or maybe tons of
> output affect timings so that the warning does not happen.

I let it run for several minutes, and only got 50 splats out of several
thousand iterations.  Reproducing is not going to help much anyway---if
we can get to the root cause, writing a reliable reproducer would be
easier anyway.

Paolo

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Paolo Bonzini]

On 14/11/2016 15:09, Dmitry Vyukov wrote:
> 
> I made dump_vmcs() no-op locally. You should see the warning
> regardless, but probably it is difficult to notice. Or maybe tons of
> output affect timings so that the warning does not happen.

I let it run for several minutes, and only got 50 splats out of several
thousand iterations.  Reproducing is not going to help much anyway---if
we can get to the root cause, writing a reliable reproducer would be
easier anyway.

Paolo

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

On Mon, Nov 14, 2016 at 3:03 PM, Paolo Bonzini  wrote:
>
>
> On 14/11/2016 15:00, Dmitry Vyukov wrote:
>> Hello,
>>
>> The following program triggers WARNING in kvm_arch_vcpu_ioctl_run in
>> run in parallel loop:
>> https://gist.githubusercontent.com/dvyukov/08091eee6f38548ff9e6905c5e0eaaee/raw/8cae95f2e88eeb98c4ddc06d227670c1e248222f/gistfile1.txt
>>
>> On commit e234832afb623fe5c7d1d5703d6619494d8d703f (Nov 13)
>>
>> WARNING: CPU: 3 PID: 11141 at arch/x86/kvm/x86.c:6984
>> kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
>> CPU: 3 PID: 11141 Comm: syz-executor Not tainted 4.9.0-rc5+ #24
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>  880039cef698 834c3959 0003 11000739de66
>>  ed000739de5e 41b58ab3 89576450 834c366b
>>  41b58ab3 894d2830 81590020 810d90f0
>> Call Trace:
>>  [< inline >] __dump_stack lib/dump_stack.c:15
>>  [] dump_stack+0x2ee/0x3f5 lib/dump_stack.c:51
>>  [] panic+0x200/0x425 kernel/panic.c:179
>>  [] __warn+0x1c9/0x1e0 kernel/panic.c:542
>>  [] warn_slowpath_null+0x31/0x40 kernel/panic.c:585
>>  [] kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
>> arch/x86/kvm/x86.c:6984
>>  [] kvm_vcpu_ioctl+0x678/0x11c0
>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>>  [< inline >] vfs_ioctl fs/ioctl.c:43
>>  [] do_vfs_ioctl+0x1c4/0x1630 fs/ioctl.c:679
>>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>>  [] SyS_ioctl+0x94/0xc0 fs/ioctl.c:685
>>  [] entry_SYSCALL_64_fastpath+0x23/0xc6
>
> What I got for now is this dmesg splat:


I made dump_vmcs() no-op locally. You should see the warning
regardless, but probably it is difficult to notice. Or maybe tons of
output affect timings so that the warning does not happen.




> [31705.171034] *** Guest State ***
> [31705.171039] CR0: actual=0x0030, shadow=0x6010, 
> gh_mask=fff7
> [31705.171040] CR4: actual=0x2050, shadow=0x, 
> gh_mask=f871
> [31705.171040] CR3 = 0xfffbc000
> [31705.171041] RSP = 0x0007  RIP = 0x0010
> [31705.171042] RFLAGS=0x0ad0 DR7 = 0x0400
> [31705.171043] Sysenter RSP= CS:RIP=:
> [31705.171044] CS:   sel=0x, attr=0x0009b, limit=0x, 
> base=0x
> [31705.171045] DS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171046] SS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171046] ES:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171047] FS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171048] GS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171049] GDTR:   limit=0x, 
> base=0x
> [31705.171050] LDTR: sel=0x, attr=0x00082, limit=0x, 
> base=0x
> [31705.171050] IDTR:   limit=0x, 
> base=0x
> [31705.171051] TR:   sel=0x, attr=0x0008b, limit=0x, 
> base=0x
> [31705.171052] EFER = 0x  PAT = 0x0007040600070406
> [31705.171053] DebugCtl = 0x  DebugExceptions = 
> 0x
> [31705.171054] Interruptibility =   ActivityState = 
> [31705.171054] *** Host State ***
> [31705.171055] RIP = 0xc0be5138  RSP = 0x8a4bc8967cd0
> [31705.171056] CS=0010 SS=0018 DS= ES= FS= GS= TR=0040
> [31705.171057] FSBase=7fe434945700 GSBase=8a4bde24 
> TRBase=8a4bde256c80
> [31705.171058] GDTBase=8a4bde249000 IDTBase=ff57b000
> [31705.171058] CR0=80050033 CR3=0001934aa000 CR4=001426e0
> [31705.171059] Sysenter RSP= CS:RIP=0010:86803e10
> [31705.171060] EFER = 0x0d01  PAT = 0x0407010600070106
> [31705.171061] *** Control State ***
> [31705.171061] PinBased=003f CPUBased=b6a1edfe SecondaryExec=00eb
> [31705.171062] EntryControls=d1ff ExitControls=002fefff
> [31705.171063] ExceptionBitmap=00060042 PFECmask= PFECmatch=
> [31705.171064] VMEntry: intr_info=80ff errcode= ilen=
> [31705.171065] VMExit: intr_info= errcode= ilen=0002
> [31705.171065] reason=8021 qualification=
> [31705.171066] IDTVectoring: info= errcode=
> [31705.171067] TSC Offset = 0xc75eb7825681
> [31705.171067] TPR Threshold = 0x00
> [31705.171068] EPT pointer = 0x00019361301e
> [31705.171069] Virtual processor ID = 0x0001
>
> --
> You received this message because you are subscribed to the Google Groups 
> "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to syzkaller+unsubscr...@googlegroups.com.
> For more 

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Dmitry Vyukov]

On Mon, Nov 14, 2016 at 3:03 PM, Paolo Bonzini  wrote:
>
>
> On 14/11/2016 15:00, Dmitry Vyukov wrote:
>> Hello,
>>
>> The following program triggers WARNING in kvm_arch_vcpu_ioctl_run in
>> run in parallel loop:
>> https://gist.githubusercontent.com/dvyukov/08091eee6f38548ff9e6905c5e0eaaee/raw/8cae95f2e88eeb98c4ddc06d227670c1e248222f/gistfile1.txt
>>
>> On commit e234832afb623fe5c7d1d5703d6619494d8d703f (Nov 13)
>>
>> WARNING: CPU: 3 PID: 11141 at arch/x86/kvm/x86.c:6984
>> kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
>> CPU: 3 PID: 11141 Comm: syz-executor Not tainted 4.9.0-rc5+ #24
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>  880039cef698 834c3959 0003 11000739de66
>>  ed000739de5e 41b58ab3 89576450 834c366b
>>  41b58ab3 894d2830 81590020 810d90f0
>> Call Trace:
>>  [< inline >] __dump_stack lib/dump_stack.c:15
>>  [] dump_stack+0x2ee/0x3f5 lib/dump_stack.c:51
>>  [] panic+0x200/0x425 kernel/panic.c:179
>>  [] __warn+0x1c9/0x1e0 kernel/panic.c:542
>>  [] warn_slowpath_null+0x31/0x40 kernel/panic.c:585
>>  [] kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
>> arch/x86/kvm/x86.c:6984
>>  [] kvm_vcpu_ioctl+0x678/0x11c0
>> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>>  [< inline >] vfs_ioctl fs/ioctl.c:43
>>  [] do_vfs_ioctl+0x1c4/0x1630 fs/ioctl.c:679
>>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>>  [] SyS_ioctl+0x94/0xc0 fs/ioctl.c:685
>>  [] entry_SYSCALL_64_fastpath+0x23/0xc6
>
> What I got for now is this dmesg splat:


I made dump_vmcs() no-op locally. You should see the warning
regardless, but probably it is difficult to notice. Or maybe tons of
output affect timings so that the warning does not happen.




> [31705.171034] *** Guest State ***
> [31705.171039] CR0: actual=0x0030, shadow=0x6010, 
> gh_mask=fff7
> [31705.171040] CR4: actual=0x2050, shadow=0x, 
> gh_mask=f871
> [31705.171040] CR3 = 0xfffbc000
> [31705.171041] RSP = 0x0007  RIP = 0x0010
> [31705.171042] RFLAGS=0x0ad0 DR7 = 0x0400
> [31705.171043] Sysenter RSP= CS:RIP=:
> [31705.171044] CS:   sel=0x, attr=0x0009b, limit=0x, 
> base=0x
> [31705.171045] DS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171046] SS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171046] ES:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171047] FS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171048] GS:   sel=0x, attr=0x00093, limit=0x, 
> base=0x
> [31705.171049] GDTR:   limit=0x, 
> base=0x
> [31705.171050] LDTR: sel=0x, attr=0x00082, limit=0x, 
> base=0x
> [31705.171050] IDTR:   limit=0x, 
> base=0x
> [31705.171051] TR:   sel=0x, attr=0x0008b, limit=0x, 
> base=0x
> [31705.171052] EFER = 0x  PAT = 0x0007040600070406
> [31705.171053] DebugCtl = 0x  DebugExceptions = 
> 0x
> [31705.171054] Interruptibility =   ActivityState = 
> [31705.171054] *** Host State ***
> [31705.171055] RIP = 0xc0be5138  RSP = 0x8a4bc8967cd0
> [31705.171056] CS=0010 SS=0018 DS= ES= FS= GS= TR=0040
> [31705.171057] FSBase=7fe434945700 GSBase=8a4bde24 
> TRBase=8a4bde256c80
> [31705.171058] GDTBase=8a4bde249000 IDTBase=ff57b000
> [31705.171058] CR0=80050033 CR3=0001934aa000 CR4=001426e0
> [31705.171059] Sysenter RSP= CS:RIP=0010:86803e10
> [31705.171060] EFER = 0x0d01  PAT = 0x0407010600070106
> [31705.171061] *** Control State ***
> [31705.171061] PinBased=003f CPUBased=b6a1edfe SecondaryExec=00eb
> [31705.171062] EntryControls=d1ff ExitControls=002fefff
> [31705.171063] ExceptionBitmap=00060042 PFECmask= PFECmatch=
> [31705.171064] VMEntry: intr_info=80ff errcode= ilen=
> [31705.171065] VMExit: intr_info= errcode= ilen=0002
> [31705.171065] reason=8021 qualification=
> [31705.171066] IDTVectoring: info= errcode=
> [31705.171067] TSC Offset = 0xc75eb7825681
> [31705.171067] TPR Threshold = 0x00
> [31705.171068] EPT pointer = 0x00019361301e
> [31705.171069] Virtual processor ID = 0x0001
>
> --
> You received this message because you are subscribed to the Google Groups 
> "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to syzkaller+unsubscr...@googlegroups.com.
> For more options, visit 

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Paolo Bonzini]

On 14/11/2016 15:00, Dmitry Vyukov wrote:
> Hello,
> 
> The following program triggers WARNING in kvm_arch_vcpu_ioctl_run in
> run in parallel loop:
> https://gist.githubusercontent.com/dvyukov/08091eee6f38548ff9e6905c5e0eaaee/raw/8cae95f2e88eeb98c4ddc06d227670c1e248222f/gistfile1.txt
> 
> On commit e234832afb623fe5c7d1d5703d6619494d8d703f (Nov 13)
> 
> WARNING: CPU: 3 PID: 11141 at arch/x86/kvm/x86.c:6984
> kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
> CPU: 3 PID: 11141 Comm: syz-executor Not tainted 4.9.0-rc5+ #24
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  880039cef698 834c3959 0003 11000739de66
>  ed000739de5e 41b58ab3 89576450 834c366b
>  41b58ab3 894d2830 81590020 810d90f0
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0x2ee/0x3f5 lib/dump_stack.c:51
>  [] panic+0x200/0x425 kernel/panic.c:179
>  [] __warn+0x1c9/0x1e0 kernel/panic.c:542
>  [] warn_slowpath_null+0x31/0x40 kernel/panic.c:585
>  [] kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
> arch/x86/kvm/x86.c:6984
>  [] kvm_vcpu_ioctl+0x678/0x11c0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x1c4/0x1630 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x94/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x23/0xc6

What I got for now is this dmesg splat:

[31705.171034] *** Guest State ***
[31705.171039] CR0: actual=0x0030, shadow=0x6010, 
gh_mask=fff7
[31705.171040] CR4: actual=0x2050, shadow=0x, 
gh_mask=f871
[31705.171040] CR3 = 0xfffbc000
[31705.171041] RSP = 0x0007  RIP = 0x0010
[31705.171042] RFLAGS=0x0ad0 DR7 = 0x0400
[31705.171043] Sysenter RSP= CS:RIP=:
[31705.171044] CS:   sel=0x, attr=0x0009b, limit=0x, 
base=0x
[31705.171045] DS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171046] SS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171046] ES:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171047] FS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171048] GS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171049] GDTR:   limit=0x, 
base=0x
[31705.171050] LDTR: sel=0x, attr=0x00082, limit=0x, 
base=0x
[31705.171050] IDTR:   limit=0x, 
base=0x
[31705.171051] TR:   sel=0x, attr=0x0008b, limit=0x, 
base=0x
[31705.171052] EFER = 0x  PAT = 0x0007040600070406
[31705.171053] DebugCtl = 0x  DebugExceptions = 
0x
[31705.171054] Interruptibility =   ActivityState = 
[31705.171054] *** Host State ***
[31705.171055] RIP = 0xc0be5138  RSP = 0x8a4bc8967cd0
[31705.171056] CS=0010 SS=0018 DS= ES= FS= GS= TR=0040
[31705.171057] FSBase=7fe434945700 GSBase=8a4bde24 
TRBase=8a4bde256c80
[31705.171058] GDTBase=8a4bde249000 IDTBase=ff57b000
[31705.171058] CR0=80050033 CR3=0001934aa000 CR4=001426e0
[31705.171059] Sysenter RSP= CS:RIP=0010:86803e10
[31705.171060] EFER = 0x0d01  PAT = 0x0407010600070106
[31705.171061] *** Control State ***
[31705.171061] PinBased=003f CPUBased=b6a1edfe SecondaryExec=00eb
[31705.171062] EntryControls=d1ff ExitControls=002fefff
[31705.171063] ExceptionBitmap=00060042 PFECmask= PFECmatch=
[31705.171064] VMEntry: intr_info=80ff errcode= ilen=
[31705.171065] VMExit: intr_info= errcode= ilen=0002
[31705.171065] reason=8021 qualification=
[31705.171066] IDTVectoring: info= errcode=
[31705.171067] TSC Offset = 0xc75eb7825681
[31705.171067] TPR Threshold = 0x00
[31705.171068] EPT pointer = 0x00019361301e
[31705.171069] Virtual processor ID = 0x0001

Re: kvm: WARNING in kvm_arch_vcpu_ioctl_run

[Paolo Bonzini]

On 14/11/2016 15:00, Dmitry Vyukov wrote:
> Hello,
> 
> The following program triggers WARNING in kvm_arch_vcpu_ioctl_run in
> run in parallel loop:
> https://gist.githubusercontent.com/dvyukov/08091eee6f38548ff9e6905c5e0eaaee/raw/8cae95f2e88eeb98c4ddc06d227670c1e248222f/gistfile1.txt
> 
> On commit e234832afb623fe5c7d1d5703d6619494d8d703f (Nov 13)
> 
> WARNING: CPU: 3 PID: 11141 at arch/x86/kvm/x86.c:6984
> kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
> CPU: 3 PID: 11141 Comm: syz-executor Not tainted 4.9.0-rc5+ #24
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  880039cef698 834c3959 0003 11000739de66
>  ed000739de5e 41b58ab3 89576450 834c366b
>  41b58ab3 894d2830 81590020 810d90f0
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0x2ee/0x3f5 lib/dump_stack.c:51
>  [] panic+0x200/0x425 kernel/panic.c:179
>  [] __warn+0x1c9/0x1e0 kernel/panic.c:542
>  [] warn_slowpath_null+0x31/0x40 kernel/panic.c:585
>  [] kvm_arch_vcpu_ioctl_run+0x1f0/0x4f50
> arch/x86/kvm/x86.c:6984
>  [] kvm_vcpu_ioctl+0x678/0x11c0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x1c4/0x1630 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x94/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x23/0xc6

What I got for now is this dmesg splat:

[31705.171034] *** Guest State ***
[31705.171039] CR0: actual=0x0030, shadow=0x6010, 
gh_mask=fff7
[31705.171040] CR4: actual=0x2050, shadow=0x, 
gh_mask=f871
[31705.171040] CR3 = 0xfffbc000
[31705.171041] RSP = 0x0007  RIP = 0x0010
[31705.171042] RFLAGS=0x0ad0 DR7 = 0x0400
[31705.171043] Sysenter RSP= CS:RIP=:
[31705.171044] CS:   sel=0x, attr=0x0009b, limit=0x, 
base=0x
[31705.171045] DS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171046] SS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171046] ES:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171047] FS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171048] GS:   sel=0x, attr=0x00093, limit=0x, 
base=0x
[31705.171049] GDTR:   limit=0x, 
base=0x
[31705.171050] LDTR: sel=0x, attr=0x00082, limit=0x, 
base=0x
[31705.171050] IDTR:   limit=0x, 
base=0x
[31705.171051] TR:   sel=0x, attr=0x0008b, limit=0x, 
base=0x
[31705.171052] EFER = 0x  PAT = 0x0007040600070406
[31705.171053] DebugCtl = 0x  DebugExceptions = 
0x
[31705.171054] Interruptibility =   ActivityState = 
[31705.171054] *** Host State ***
[31705.171055] RIP = 0xc0be5138  RSP = 0x8a4bc8967cd0
[31705.171056] CS=0010 SS=0018 DS= ES= FS= GS= TR=0040
[31705.171057] FSBase=7fe434945700 GSBase=8a4bde24 
TRBase=8a4bde256c80
[31705.171058] GDTBase=8a4bde249000 IDTBase=ff57b000
[31705.171058] CR0=80050033 CR3=0001934aa000 CR4=001426e0
[31705.171059] Sysenter RSP= CS:RIP=0010:86803e10
[31705.171060] EFER = 0x0d01  PAT = 0x0407010600070106
[31705.171061] *** Control State ***
[31705.171061] PinBased=003f CPUBased=b6a1edfe SecondaryExec=00eb
[31705.171062] EntryControls=d1ff ExitControls=002fefff
[31705.171063] ExceptionBitmap=00060042 PFECmask= PFECmatch=
[31705.171064] VMEntry: intr_info=80ff errcode= ilen=
[31705.171065] VMExit: intr_info= errcode= ilen=0002
[31705.171065] reason=8021 qualification=
[31705.171066] IDTVectoring: info= errcode=
[31705.171067] TSC Offset = 0xc75eb7825681
[31705.171067] TPR Threshold = 0x00
[31705.171068] EPT pointer = 0x00019361301e
[31705.171069] Virtual processor ID = 0x0001

Re: kvm: WARNING in em_ret_far

[Paolo Bonzini]

Reproduced.

Paolo

On 12/11/2016 22:31, Dmitry Vyukov wrote:
> Hello,
> 
> The following program triggers WARNING in em_ret_far:
> https://gist.githubusercontent.com/dvyukov/12bc06a5f638d91953f94bcbe49bcc89/raw/6d863470808a695c085620664b7134864b2eb2a9/gistfile1.txt
> 
> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
> 
> WARNING: CPU: 2 PID: 3668 at arch/x86/kvm/emulate.c:2217 
> em_ret_far+0x428/0x480
> Kernel panic - not syncing: panic_on_warn set ...
> 
> CPU: 2 PID: 3668 Comm: syz-executor Not tainted 4.9.0-rc4+ #49
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  88006b32f618 81c2e46b 83270ec0 88006b32f6f0
>  83225600 8110a218 88006b32f6e0 81546463
>  41b58ab3 837cda55 815462ac 815470e9
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>  [] panic+0x1b7/0x3a3 kernel/panic.c:179
>  [] __warn+0x1c4/0x1e0 kernel/panic.c:542
>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>  [] em_ret_far+0x428/0x480 arch/x86/kvm/emulate.c:2217
>  [] em_ret_far_imm+0x17/0x70 arch/x86/kvm/emulate.c:2227
>  [] x86_emulate_insn+0x87a/0x3730 
> arch/x86/kvm/emulate.c:5294
>  [] x86_emulate_instruction+0x520/0x1ba0
> arch/x86/kvm/x86.c:5545
>  [< inline >] emulate_instruction arch/x86/include/asm/kvm_host.h:1116
>  [< inline >] complete_emulated_io arch/x86/kvm/x86.c:6870
>  [] complete_emulated_mmio+0x4e9/0x710 
> arch/x86/kvm/x86.c:6934
>  [] kvm_arch_vcpu_ioctl_run+0x3b7a/0x5a90
> arch/x86/kvm/x86.c:6978
>  [] kvm_vcpu_ioctl+0x61e/0xdd0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
> Dumping ftrace buffer:
>(ftrace buffer empty)
> Kernel Offset: disabled
> reboot: cpu_has_vmx: ecx=80a02021 1
> 

Re: kvm: WARNING in em_ret_far

[Paolo Bonzini]

Reproduced.

Paolo

On 12/11/2016 22:31, Dmitry Vyukov wrote:
> Hello,
> 
> The following program triggers WARNING in em_ret_far:
> https://gist.githubusercontent.com/dvyukov/12bc06a5f638d91953f94bcbe49bcc89/raw/6d863470808a695c085620664b7134864b2eb2a9/gistfile1.txt
> 
> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
> 
> WARNING: CPU: 2 PID: 3668 at arch/x86/kvm/emulate.c:2217 
> em_ret_far+0x428/0x480
> Kernel panic - not syncing: panic_on_warn set ...
> 
> CPU: 2 PID: 3668 Comm: syz-executor Not tainted 4.9.0-rc4+ #49
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  88006b32f618 81c2e46b 83270ec0 88006b32f6f0
>  83225600 8110a218 88006b32f6e0 81546463
>  41b58ab3 837cda55 815462ac 815470e9
> Call Trace:
>  [< inline >] __dump_stack lib/dump_stack.c:15
>  [] dump_stack+0xb3/0x118 lib/dump_stack.c:51
>  [] panic+0x1b7/0x3a3 kernel/panic.c:179
>  [] __warn+0x1c4/0x1e0 kernel/panic.c:542
>  [] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>  [] em_ret_far+0x428/0x480 arch/x86/kvm/emulate.c:2217
>  [] em_ret_far_imm+0x17/0x70 arch/x86/kvm/emulate.c:2227
>  [] x86_emulate_insn+0x87a/0x3730 
> arch/x86/kvm/emulate.c:5294
>  [] x86_emulate_instruction+0x520/0x1ba0
> arch/x86/kvm/x86.c:5545
>  [< inline >] emulate_instruction arch/x86/include/asm/kvm_host.h:1116
>  [< inline >] complete_emulated_io arch/x86/kvm/x86.c:6870
>  [] complete_emulated_mmio+0x4e9/0x710 
> arch/x86/kvm/x86.c:6934
>  [] kvm_arch_vcpu_ioctl_run+0x3b7a/0x5a90
> arch/x86/kvm/x86.c:6978
>  [] kvm_vcpu_ioctl+0x61e/0xdd0
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2557
>  [< inline >] vfs_ioctl fs/ioctl.c:43
>  [] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
>  [< inline >] SYSC_ioctl fs/ioctl.c:694
>  [] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
>  [] entry_SYSCALL_64_fastpath+0x1f/0xc2
> Dumping ftrace buffer:
>(ftrace buffer empty)
> Kernel Offset: disabled
> reboot: cpu_has_vmx: ecx=80a02021 1
> 

Re: KVM: WARNING: at .. kvm_release_pfn_clean

[Xiao Guangrong]

On 10/05/2012 04:33 PM, Jiri Slaby wrote:
> Hi,
> 
> I'm getting this warning while running qemu-kvm 1.2.rc1 on the top of
> 3.6.0-next-20121001:
> WARNING: at virt/kvm/kvm_main.c:1325 kvm_release_pfn_clean+0x70/0x80()
> Hardware name: To Be Filled By O.E.M.
> Modules linked in: nls_cp437 vfat fat dvb_usb_dib0700 dib0090 dib7000p
> dib7000m dib0070 dib8000 dib3000mc dibx000_common microcode
> Pid: 11848, comm: qemu-kvm Tainted: GW
> 3.6.0-next-20121001_64+ #70
> Call Trace:
>  [] warn_slowpath_common+0x7f/0xc0
>  [] warn_slowpath_null+0x1a/0x20
>  [] kvm_release_pfn_clean+0x70/0x80
>  [] paging64_page_fault+0x686/0x830
>  [] ? paging64_gva_to_gpa+0x35/0x80
>  [] kvm_mmu_page_fault+0x31/0x100
>  [] handle_exception+0x231/0x3d0
>  [] vmx_handle_exit+0xcc/0x780
>  [] kvm_arch_vcpu_ioctl_run+0x444/0xf60
>  [] ? kvm_arch_vcpu_load+0x57/0x1b0
>  [] kvm_vcpu_ioctl+0x472/0x610
>  [] ? fsnotify+0x24b/0x340
>  [] do_vfs_ioctl+0x99/0x580
>  [] ? pick_next_task_fair+0x13b/0x1a0
>  [] ? __enqueue_entity+0x78/0x80
>  [] ? kvm_on_user_return+0x64/0x70
>  [] sys_ioctl+0x50/0x90
>  [] system_call_fastpath+0x16/0x1b
> 
> I.e. is_error_pfn(pfn) is true.

Jiri, thanks for your report!

This is a known bug which can be fixed by this patch:
https://lkml.org/lkml/2012/9/21/46

But this patch need be improved, i will post the new version after
full test.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: KVM: WARNING: at .. kvm_release_pfn_clean

[Xiao Guangrong]

On 10/05/2012 04:33 PM, Jiri Slaby wrote:
 Hi,
 
 I'm getting this warning while running qemu-kvm 1.2.rc1 on the top of
 3.6.0-next-20121001:
 WARNING: at virt/kvm/kvm_main.c:1325 kvm_release_pfn_clean+0x70/0x80()
 Hardware name: To Be Filled By O.E.M.
 Modules linked in: nls_cp437 vfat fat dvb_usb_dib0700 dib0090 dib7000p
 dib7000m dib0070 dib8000 dib3000mc dibx000_common microcode
 Pid: 11848, comm: qemu-kvm Tainted: GW
 3.6.0-next-20121001_64+ #70
 Call Trace:
  [8107f19f] warn_slowpath_common+0x7f/0xc0
  [8107f1fa] warn_slowpath_null+0x1a/0x20
  [81004cd0] kvm_release_pfn_clean+0x70/0x80
  [81027ac6] paging64_page_fault+0x686/0x830
  [81024745] ? paging64_gva_to_gpa+0x35/0x80
  [81021121] kvm_mmu_page_fault+0x31/0x100
  [81040e81] handle_exception+0x231/0x3d0
  [81041fbc] vmx_handle_exit+0xcc/0x780
  [8101be44] kvm_arch_vcpu_ioctl_run+0x444/0xf60
  [81017117] ? kvm_arch_vcpu_load+0x57/0x1b0
  [81005302] kvm_vcpu_ioctl+0x472/0x610
  [811aa36b] ? fsnotify+0x24b/0x340
  [8117d609] do_vfs_ioctl+0x99/0x580
  [810b620b] ? pick_next_task_fair+0x13b/0x1a0
  [810b4f68] ? __enqueue_entity+0x78/0x80
  [81010114] ? kvm_on_user_return+0x64/0x70
  [8117db40] sys_ioctl+0x50/0x90
  [816a46e2] system_call_fastpath+0x16/0x1b
 
 I.e. is_error_pfn(pfn) is true.

Jiri, thanks for your report!

This is a known bug which can be fixed by this patch:
https://lkml.org/lkml/2012/9/21/46

But this patch need be improved, i will post the new version after
full test.

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: kvm warning

[Satyam Sharma]

On Mon, 13 Aug 2007, Luck, Tony wrote:

> > Solution (1) above sounds preferable, unless there are mysterious reasons
> > why ia64 wants to avoid Kconfig.preempt (adding Tony Luck to Cc:).
> 
> Send me a patch and unless it causes more problems than it solves, I'll
> put it in.


[PATCH] ia64: Include Kconfig.preempt

arch/ia64/Kconfig failed to include kernel/Kconfig.preempt that meant it
did not support PREEMPT_VOLUNTARY and PREEMPT_BKL (inadvertently).
This was recently noticed when the newly-added PREEMPT_NOTIFIERS in
Kconfig.preempt that was "select"ed from drivers/kvm/Kconfig (therefore)
started giving bogus warnings ('select' used by config symbol 'KVM' refers
to undefined symbol 'PREEMPT_NOTIFIERS') on ia64 builds.

So let's remove the open-coded definition of CONFIG_PREEMPT in
arch/ia64/Kconfig and replace it with just including Kconfig.preempt
instead, like the other archs do.

Signed-off-by: Satyam Sharma <[EMAIL PROTECTED]>

---

 arch/ia64/Kconfig |   12 +---
 1 files changed, 1 insertions(+), 11 deletions(-)

diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index 21aa4fc..8c39913 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -327,17 +327,7 @@ config FORCE_CPEI_RETARGET
This option it useful to enable this feature on older BIOS's as well.
You can also enable this by using boot command line option force_cpei=1.
 
-config PREEMPT
-   bool "Preemptible Kernel"
-help
-  This option reduces the latency of the kernel when reacting to
-  real-time or interactive events by allowing a low priority process to
-  be preempted even if it is in kernel mode executing a system call.
-  This allows applications to run more reliably even when the system is
-  under load.
-
-  Say Y here if you are building a kernel for a desktop, embedded
-  or real-time system.  Say N if you are unsure.
+source "kernel/Kconfig.preempt"
 
 source "mm/Kconfig"
 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: kvm warning

[Luck, Tony]

> Solution (1) above sounds preferable, unless there are mysterious reasons
> why ia64 wants to avoid Kconfig.preempt (adding Tony Luck to Cc:).

Send me a patch and unless it causes more problems than it solves, I'll
put it in.

-Tony
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Satyam Sharma]

On 8/9/07, Avi Kivity <[EMAIL PROTECTED]> wrote:
> Andrew Morton wrote:
> > On Thu, 09 Aug 2007 01:48:07 +0300
> > Avi Kivity <[EMAIL PROTECTED]> wrote:
> >
> >> Ingo Molnar wrote:
> >>
> >>> * Andrew Morton <[EMAIL PROTECTED]> wrote:
> >>>
>  ia64 allmodconfig says
> 
>  drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM'
>  refers to undefined symbol 'PREEMPT_NOTIFIERS'
> 
> >>> hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other
> >>> arches? Due to that ia64 also misses out on voluntary preempt and on
> >>> preempt-bkl.
> >>>
> >> Even more hm, how does ia64 manage to enable kvm?  It 'depends on X86'
> >> at this moment.
> >
> > beats me.  CONFIG_KVM doesn't get set.  But it seems that kconfig wants
> > to do error-checking on that item anyway.
>
> I could do a 'select PREEMPT_NOTIFIERS if X86' but it seems silly.  I'd
> call it a Kconfig bug.

I remember this issue well [1] ... it's a Kconfig deficiency indeed. Basically
kconfig options for generic stuff cannot use "select" for symbols that are
defined only for specific archs (here ia64 synthetically created this situation
by failing to source "kernel/Kconfig.preempt" in its arch/ia64/Kconfig).
Leads to no real build problems, but causes these bogus warnings to
be printed on other archs (that don't have that symbol visible to them).

Possible solutions could be to (1) source "kernel/Kconfig.preempt" from
arch/ia64/Kconfig, or (2) Use "default y if KVM" for PREEMPT_NOTIFIERS
(but this would be kludgy, horrible and tasteless). I'm not sure if
"select PREEMPT_NOTIFIERS if X86" would cut it to get rid of the bogus
warnings,
though.

Solution (1) above sounds preferable, unless there are mysterious reasons
why ia64 wants to avoid Kconfig.preempt (adding Tony Luck to Cc:).


Satyam

[1] Last time this happened was with something called ATARI_KBD_CORE.
I submitted a patch that unfortunately degenerated into a flamewar :-) The
bogus warnings remained (causing more traffic) for a month or so, before
an alternate solution got finally merged.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Satyam Sharma]

On 8/9/07, Avi Kivity [EMAIL PROTECTED] wrote:
 Andrew Morton wrote:
  On Thu, 09 Aug 2007 01:48:07 +0300
  Avi Kivity [EMAIL PROTECTED] wrote:
 
  Ingo Molnar wrote:
 
  * Andrew Morton [EMAIL PROTECTED] wrote:
 
  ia64 allmodconfig says
 
  drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM'
  refers to undefined symbol 'PREEMPT_NOTIFIERS'
 
  hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other
  arches? Due to that ia64 also misses out on voluntary preempt and on
  preempt-bkl.
 
  Even more hm, how does ia64 manage to enable kvm?  It 'depends on X86'
  at this moment.
 
  beats me.  CONFIG_KVM doesn't get set.  But it seems that kconfig wants
  to do error-checking on that item anyway.

 I could do a 'select PREEMPT_NOTIFIERS if X86' but it seems silly.  I'd
 call it a Kconfig bug.

I remember this issue well [1] ... it's a Kconfig deficiency indeed. Basically
kconfig options for generic stuff cannot use select for symbols that are
defined only for specific archs (here ia64 synthetically created this situation
by failing to source kernel/Kconfig.preempt in its arch/ia64/Kconfig).
Leads to no real build problems, but causes these bogus warnings to
be printed on other archs (that don't have that symbol visible to them).

Possible solutions could be to (1) source kernel/Kconfig.preempt from
arch/ia64/Kconfig, or (2) Use default y if KVM for PREEMPT_NOTIFIERS
(but this would be kludgy, horrible and tasteless). I'm not sure if
select PREEMPT_NOTIFIERS if X86 would cut it to get rid of the bogus
warnings,
though.

Solution (1) above sounds preferable, unless there are mysterious reasons
why ia64 wants to avoid Kconfig.preempt (adding Tony Luck to Cc:).


Satyam

[1] Last time this happened was with something called ATARI_KBD_CORE.
I submitted a patch that unfortunately degenerated into a flamewar :-) The
bogus warnings remained (causing more traffic) for a month or so, before
an alternate solution got finally merged.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: kvm warning

[Luck, Tony]

 Solution (1) above sounds preferable, unless there are mysterious reasons
 why ia64 wants to avoid Kconfig.preempt (adding Tony Luck to Cc:).

Send me a patch and unless it causes more problems than it solves, I'll
put it in.

-Tony
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: kvm warning

[Satyam Sharma]

On Mon, 13 Aug 2007, Luck, Tony wrote:

  Solution (1) above sounds preferable, unless there are mysterious reasons
  why ia64 wants to avoid Kconfig.preempt (adding Tony Luck to Cc:).
 
 Send me a patch and unless it causes more problems than it solves, I'll
 put it in.


[PATCH] ia64: Include Kconfig.preempt

arch/ia64/Kconfig failed to include kernel/Kconfig.preempt that meant it
did not support PREEMPT_VOLUNTARY and PREEMPT_BKL (inadvertently).
This was recently noticed when the newly-added PREEMPT_NOTIFIERS in
Kconfig.preempt that was selected from drivers/kvm/Kconfig (therefore)
started giving bogus warnings ('select' used by config symbol 'KVM' refers
to undefined symbol 'PREEMPT_NOTIFIERS') on ia64 builds.

So let's remove the open-coded definition of CONFIG_PREEMPT in
arch/ia64/Kconfig and replace it with just including Kconfig.preempt
instead, like the other archs do.

Signed-off-by: Satyam Sharma [EMAIL PROTECTED]

---

 arch/ia64/Kconfig |   12 +---
 1 files changed, 1 insertions(+), 11 deletions(-)

diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index 21aa4fc..8c39913 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -327,17 +327,7 @@ config FORCE_CPEI_RETARGET
This option it useful to enable this feature on older BIOS's as well.
You can also enable this by using boot command line option force_cpei=1.
 
-config PREEMPT
-   bool Preemptible Kernel
-help
-  This option reduces the latency of the kernel when reacting to
-  real-time or interactive events by allowing a low priority process to
-  be preempted even if it is in kernel mode executing a system call.
-  This allows applications to run more reliably even when the system is
-  under load.
-
-  Say Y here if you are building a kernel for a desktop, embedded
-  or real-time system.  Say N if you are unsure.
+source kernel/Kconfig.preempt
 
 source mm/Kconfig
 
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Avi Kivity]

Andrew Morton wrote:

On Thu, 09 Aug 2007 01:48:07 +0300
Avi Kivity <[EMAIL PROTECTED]> wrote:

  

Ingo Molnar wrote:


* Andrew Morton <[EMAIL PROTECTED]> wrote:

  
  

ia64 allmodconfig says

drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM' 
refers to undefined symbol 'PREEMPT_NOTIFIERS'


hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other 
arches? Due to that ia64 also misses out on voluntary preempt and on 
preempt-bkl.


  
  
Even more hm, how does ia64 manage to enable kvm?  It 'depends on X86' 
at this moment.





beats me.  CONFIG_KVM doesn't get set.  But it seems that kconfig wants
to do error-checking on that item anyway.


  


I could do a 'select PREEMPT_NOTIFIERS if X86' but it seems silly.  I'd 
call it a Kconfig bug.




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Andrew Morton]

On Thu, 09 Aug 2007 01:48:07 +0300
Avi Kivity <[EMAIL PROTECTED]> wrote:

> Ingo Molnar wrote:
> > * Andrew Morton <[EMAIL PROTECTED]> wrote:
> >
> >   
> >> ia64 allmodconfig says
> >>
> >> drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM' 
> >> refers to undefined symbol 'PREEMPT_NOTIFIERS'
> >> 
> >
> > hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other 
> > arches? Due to that ia64 also misses out on voluntary preempt and on 
> > preempt-bkl.
> >
> >   
> 
> Even more hm, how does ia64 manage to enable kvm?  It 'depends on X86' 
> at this moment.
> 

beats me.  CONFIG_KVM doesn't get set.  But it seems that kconfig wants
to do error-checking on that item anyway.



btw, testing of Kconfig can be done for any architecture without
installation of a toolchain for that architecture.  Set $ARCH and run
mrproper then use menuconfig/oldconfig/allmodconfig/allconfig as usual.

Judging by the number of Kconfig problem I see, this is a big secret ;)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Avi Kivity]

Ingo Molnar wrote:

* Andrew Morton <[EMAIL PROTECTED]> wrote:

  

ia64 allmodconfig says

drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM' 
refers to undefined symbol 'PREEMPT_NOTIFIERS'



hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other 
arches? Due to that ia64 also misses out on voluntary preempt and on 
preempt-bkl.


  


Even more hm, how does ia64 manage to enable kvm?  It 'depends on X86' 
at this moment.



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Ingo Molnar]

* Andrew Morton <[EMAIL PROTECTED]> wrote:

> ia64 allmodconfig says
> 
> drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM' 
> refers to undefined symbol 'PREEMPT_NOTIFIERS'

hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other 
arches? Due to that ia64 also misses out on voluntary preempt and on 
preempt-bkl.

Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Ingo Molnar]

* Andrew Morton [EMAIL PROTECTED] wrote:

 ia64 allmodconfig says
 
 drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM' 
 refers to undefined symbol 'PREEMPT_NOTIFIERS'

hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other 
arches? Due to that ia64 also misses out on voluntary preempt and on 
preempt-bkl.

Ingo
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: kvm warning

[Avi Kivity]

Ingo Molnar wrote:

* Andrew Morton [EMAIL PROTECTED] wrote:

  

ia64 allmodconfig says

drivers/kvm/Kconfig:14:warning: 'select' used by config symbol 'KVM' 
refers to undefined symbol 'PREEMPT_NOTIFIERS'



hm, why doesnt ia64 pick up kernel/Kconfig.preempt, like all the other 
arches? Due to that ia64 also misses out on voluntary preempt and on 
preempt-bkl.


  


Even more hm, how does ia64 manage to enable kvm?  It 'depends on X86' 
at this moment.



-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/