Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 27.03.20 12:08, Eugenio Pérez wrote: > Hi Christian. > > Sorry for the late response. Could we try this one over > eccb852f1fe6bede630e2e4f1a121a81e34354ab, and see if you still > can reproduce the bug? To much time has passed and too many things have changed on that system. I have trouble

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 14.02.20 13:26, Eugenio Pérez wrote: > On Fri, 2020-02-14 at 13:22 +0100, Christian Borntraeger wrote: >> >> On 14.02.20 13:17, Eugenio Pérez wrote: >>> Can you try the inlined patch over 52c36ce7f334 ("vhost: use batched >>> version by default")? My intention is to check >>> if >>> "strange

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 14.02.20 13:17, Eugenio Pérez wrote: > Can you try the inlined patch over 52c36ce7f334 ("vhost: use batched version > by default")? My intention is to check if > "strange VHOST_SET_VRING_BASE" line appears. In previous tests, it appears > very fast, but maybe it takes some time for > it to

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

repro On 14.02.20 08:43, Christian Borntraeger wrote: > > > On 14.02.20 08:40, Eugenio Perez Martin wrote: >> Hi. >> >> Were the vhost and vhost_net modules loaded with dyndbg='+plt'? I miss >> all the others regular debug traces on that one. > > I did > > echo -n 'file

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 14.02.20 08:40, Eugenio Perez Martin wrote: > Hi. > > Were the vhost and vhost_net modules loaded with dyndbg='+plt'? I miss > all the others regular debug traces on that one. I did echo -n 'file drivers/vhost/vhost.c +plt' > control and echo -n 'file drivers/vhost/net.c +plt' > control

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

I did ping -c 20 -f ... ; reboot twice The ping after the first reboot showed ...E this was on the host console [ 55.951885] CPU: 34 PID: 1908 Comm: CPU 0/KVM Not tainted 5.5.0+ #21 [ 55.951891] Hardware name: IBM 3906 M04 704 (LPAR) [ 55.951892] Call Trace: [ 55.951902]

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 13.02.20 17:29, Eugenio Pérez wrote: > Can we try with this traces? Does not apply on eccb852f1fe6bede630e2e4f1a121a81e34354ab, can you double check? > > From b793b4106085ab1970bdedb340e49f37843ed585 Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= > Date: Thu, 13 Feb

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 13.02.20 11:47, Eugenio Pérez wrote: > > Can we try tracing last_avail_idx with the attached patch? Can you enable > also line and thread id (dyndbg='+plt')? [ 326.584446] [2127] 1648: VHOST_SET_VRING_BASE [vq=36fdfcb7][vq->last_avail_idx=0][vq->avail_idx=0] [ 326.584457]

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 12.02.20 17:34, Eugenio Pérez wrote: > On Tue, 2020-02-11 at 14:13 +0100, Christian Borntraeger wrote: >> >> On 11.02.20 14:04, Eugenio Pérez wrote: >>> On Mon, 2020-02-10 at 12:01 +0100, Christian Borntraeger wrote: On 10.02.20 10:47, Eugenio Perez Martin wrote: > Hi Christian.

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Tue, Feb 11, 2020 at 02:04:54PM +0100, Eugenio Pérez wrote: > On Mon, 2020-02-10 at 12:01 +0100, Christian Borntraeger wrote: > > > > On 10.02.20 10:47, Eugenio Perez Martin wrote: > > > Hi Christian. > > > > > > I'm not able to reproduce the failure with > > >

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 11.02.20 14:04, Eugenio Pérez wrote: > On Mon, 2020-02-10 at 12:01 +0100, Christian Borntraeger wrote: >> >> On 10.02.20 10:47, Eugenio Perez Martin wrote: >>> Hi Christian. >>> >>> I'm not able to reproduce the failure with >>> eccb852f1fe6bede630e2e4f1a121a81e34354ab commit. Could you add

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 11.02.20 10:56, Christian Borntraeger wrote: > > > On 11.02.20 10:33, Eugenio Pérez wrote: >> On Mon, 2020-02-10 at 12:01 +0100, Christian Borntraeger wrote: >>> >>> On 10.02.20 10:47, Eugenio Perez Martin wrote: Hi Christian. I'm not able to reproduce the failure with

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 11.02.20 10:33, Eugenio Pérez wrote: > On Mon, 2020-02-10 at 12:01 +0100, Christian Borntraeger wrote: >> >> On 10.02.20 10:47, Eugenio Perez Martin wrote: >>> Hi Christian. >>> >>> I'm not able to reproduce the failure with >>> eccb852f1fe6bede630e2e4f1a121a81e34354ab commit. Could you add

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 10.02.20 10:47, Eugenio Perez Martin wrote: > Hi Christian. > > I'm not able to reproduce the failure with > eccb852f1fe6bede630e2e4f1a121a81e34354ab commit. Could you add more data? > Your configuration (libvirt or qemu line), and host's dmesg output if any? > > Thanks! If it was not

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 10.02.20 10:40, Eugenio Perez Martin wrote: > Hi Christian. > > I'm not able to reproduce the failure with > eccb852f1fe6bede630e2e4f1a121a81e34354ab commit. Could you add more data? > Your configuration (libvirt or qemu line), and host's dmesg output if any? I do the following in the

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Fri, Feb 07, 2020 at 09:53:53AM +0100, Cornelia Huck wrote: > On Fri, 7 Feb 2020 09:13:14 +0100 > Christian Borntraeger wrote: > > > On 07.02.20 08:58, Michael S. Tsirkin wrote: > > > On Fri, Feb 07, 2020 at 08:47:14AM +0100, Christian Borntraeger wrote: > > >> Also adding Cornelia. > > >>

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Fri, 7 Feb 2020 09:13:14 +0100 Christian Borntraeger wrote: > On 07.02.20 08:58, Michael S. Tsirkin wrote: > > On Fri, Feb 07, 2020 at 08:47:14AM +0100, Christian Borntraeger wrote: > >> Also adding Cornelia. > >> > >> > >> On 06.02.20 23:17, Michael S. Tsirkin wrote: > >>> On Thu, Feb

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 07.02.20 08:58, Michael S. Tsirkin wrote: > On Fri, Feb 07, 2020 at 08:47:14AM +0100, Christian Borntraeger wrote: >> Also adding Cornelia. >> >> >> On 06.02.20 23:17, Michael S. Tsirkin wrote: >>> On Thu, Feb 06, 2020 at 04:12:21PM +0100, Christian Borntraeger wrote: On

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Fri, Feb 07, 2020 at 08:47:14AM +0100, Christian Borntraeger wrote: > Also adding Cornelia. > > > On 06.02.20 23:17, Michael S. Tsirkin wrote: > > On Thu, Feb 06, 2020 at 04:12:21PM +0100, Christian Borntraeger wrote: > >> > >> > >> On 06.02.20 15:22, epere...@redhat.com wrote: > >>> Hi

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

Also adding Cornelia. On 06.02.20 23:17, Michael S. Tsirkin wrote: > On Thu, Feb 06, 2020 at 04:12:21PM +0100, Christian Borntraeger wrote: >> >> >> On 06.02.20 15:22, epere...@redhat.com wrote: >>> Hi Christian. >>> >>> Could you try this patch on top of ("38ced0208491 vhost: use batched >>>

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Thu, Feb 06, 2020 at 04:12:21PM +0100, Christian Borntraeger wrote: > > > On 06.02.20 15:22, epere...@redhat.com wrote: > > Hi Christian. > > > > Could you try this patch on top of ("38ced0208491 vhost: use batched > > version by default")? > > > > It will not solve your first random crash

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Thu, Feb 06, 2020 at 03:22:39PM +0100, epere...@redhat.com wrote: > Hi Christian. > > Could you try this patch on top of ("38ced0208491 vhost: use batched version > by default")? > > It will not solve your first random crash but it should help with the lost of > network connectivity. > >

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 06.02.20 15:22, epere...@redhat.com wrote: > Hi Christian. > > Could you try this patch on top of ("38ced0208491 vhost: use batched version > by default")? > > It will not solve your first random crash but it should help with the lost of > network connectivity. > > Please let me know

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 20.01.20 07:27, Michael S. Tsirkin wrote: > On Tue, Jan 07, 2020 at 01:16:50PM +0100, Christian Borntraeger wrote: >> On 07.01.20 12:55, Michael S. Tsirkin wrote: >> >>> >>> I pushed batched-v3 - same head but bisect should work now. >>> >> >> With >> commit

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Tue, Jan 07, 2020 at 01:16:50PM +0100, Christian Borntraeger wrote: > On 07.01.20 12:55, Michael S. Tsirkin wrote: > > > > > I pushed batched-v3 - same head but bisect should work now. > > > > With > commit 38ced0208491103b50f1056f0d1c8f28e2e13d08 (HEAD) > Author: Michael S. Tsirkin >

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 07.01.20 12:55, Michael S. Tsirkin wrote: > > I pushed batched-v3 - same head but bisect should work now. > With commit 38ced0208491103b50f1056f0d1c8f28e2e13d08 (HEAD) Author: Michael S. Tsirkin AuthorDate: Wed Dec 11 12:19:26 2019 -0500 Commit: Michael S. Tsirkin CommitDate: Tue

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Tue, Jan 07, 2020 at 12:34:50PM +0100, Christian Borntraeger wrote: > > > On 07.01.20 10:39, Michael S. Tsirkin wrote: > > On Tue, Jan 07, 2020 at 09:59:16AM +0100, Christian Borntraeger wrote: > >> > >> > >> On 06.01.20 11:50, Michael S. Tsirkin wrote: > >>> On Wed, Dec 18, 2019 at

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Tue, Jan 07, 2020 at 12:34:50PM +0100, Christian Borntraeger wrote: > > > On 07.01.20 10:39, Michael S. Tsirkin wrote: > > On Tue, Jan 07, 2020 at 09:59:16AM +0100, Christian Borntraeger wrote: > >> > >> > >> On 06.01.20 11:50, Michael S. Tsirkin wrote: > >>> On Wed, Dec 18, 2019 at

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 07.01.20 10:39, Michael S. Tsirkin wrote: > On Tue, Jan 07, 2020 at 09:59:16AM +0100, Christian Borntraeger wrote: >> >> >> On 06.01.20 11:50, Michael S. Tsirkin wrote: >>> On Wed, Dec 18, 2019 at 04:59:02PM +0100, Christian Borntraeger wrote: On 18.12.19 16:10, Michael S. Tsirkin wrote:

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 06.01.20 11:50, Michael S. Tsirkin wrote: > On Wed, Dec 18, 2019 at 04:59:02PM +0100, Christian Borntraeger wrote: >> On 18.12.19 16:10, Michael S. Tsirkin wrote: >>> On Wed, Dec 18, 2019 at 03:43:43PM +0100, Christian Borntraeger wrote: Michael, with commit

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Tue, Jan 07, 2020 at 09:59:16AM +0100, Christian Borntraeger wrote: > > > On 06.01.20 11:50, Michael S. Tsirkin wrote: > > On Wed, Dec 18, 2019 at 04:59:02PM +0100, Christian Borntraeger wrote: > >> On 18.12.19 16:10, Michael S. Tsirkin wrote: > >>> On Wed, Dec 18, 2019 at 03:43:43PM +0100,

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Wed, Dec 18, 2019 at 04:59:02PM +0100, Christian Borntraeger wrote: > On 18.12.19 16:10, Michael S. Tsirkin wrote: > > On Wed, Dec 18, 2019 at 03:43:43PM +0100, Christian Borntraeger wrote: > >> Michael, > >> > >> with > >> commit db7286b100b503ef80612884453bed53d74c9a16 > >>

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On 18.12.19 16:10, Michael S. Tsirkin wrote: > On Wed, Dec 18, 2019 at 03:43:43PM +0100, Christian Borntraeger wrote: >> Michael, >> >> with >> commit db7286b100b503ef80612884453bed53d74c9a16 >> (refs/bisect/skip-db7286b100b503ef80612884453bed53d74c9a16) >> vhost: use batched version by

vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

Michael, with commit db7286b100b503ef80612884453bed53d74c9a16 (refs/bisect/skip-db7286b100b503ef80612884453bed53d74c9a16) vhost: use batched version by default plus commit 6bd262d5eafcdf8cdfae491e2e748e4e434dcda6 (HEAD, refs/bisect/bad) Revert "vhost/net: add an option to test new code"

Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Wed, Dec 18, 2019 at 03:43:43PM +0100, Christian Borntraeger wrote: > Michael, > > with > commit db7286b100b503ef80612884453bed53d74c9a16 > (refs/bisect/skip-db7286b100b503ef80612884453bed53d74c9a16) > vhost: use batched version by default > plus > commit