Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-09-09 Thread Hans van Kranenburg
On 06/09/2018 18:23, Hans van Kranenburg wrote: > > Anyway, I think the future proof solution here is to have clear > documentation about how to configure related settings, instead of trying > to find values that suit all users and that are not ridiculously high. I just assisted a user in #xen

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-09-06 Thread Hans van Kranenburg
On 02/28/2018 08:54 AM, Valentin Vidic wrote: > On Tue, Feb 27, 2018 at 08:22:50PM +0100, Valentin Vidic wrote: >> Since I can't reproduce it easily anymore I suspect something was >> fixed in the meanwhile. My original report was for 4.9.30-2+deb9u2 >> and since then there seems to be a number

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-27 Thread Valentin Vidic
On Tue, Feb 27, 2018 at 08:22:50PM +0100, Valentin Vidic wrote: > Since I can't reproduce it easily anymore I suspect something was > fixed in the meanwhile. My original report was for 4.9.30-2+deb9u2 > and since then there seems to be a number of fixes that could be > related to this: Just

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-27 Thread Christian Schwamborn
I much appreciate the effort you all did and like the idea to ship the xen-diag tool and maybe a hint somewhere about the issues that occurred and the possible solution by raising max_nr_frames. On 27.02.2018 17:05, Hans van Kranenburg wrote: ad 1. Christian, Valentin, can you give more

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-27 Thread Christian Schwamborn
I much appreciate the effort you all did and like the idea to ship the xen-diag tool and maybe a hint somewhere about the issues that occurred and the possible solution by raising max_nr_frames. On 27.02.2018 17:05, Hans van Kranenburg wrote: ad 1. Christian, Valentin, can you give more

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-27 Thread Valentin Vidic
On Tue, Feb 27, 2018 at 05:05:06PM +0100, Hans van Kranenburg wrote: > ad 1. Christian, Valentin, can you give more specific info that can help > someone else to set up a test environment to trigger > 32 values. I can't touch the original VM that had this issue and tried to reproduce on another

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-27 Thread Hans van Kranenburg
On 02/27/2018 05:05 PM, Hans van Kranenburg wrote: > [...] > > ...I doubt if it's useful (priority wise) to keep spending a lot of time > on this, since the work is really time consuming. It is, but it's also an interesting problem. Idle just started domU starts at nr_frames=6 or 7 in all

Bug#880554: [Pkg-xen-devel] Bug#880554: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-27 Thread Hans van Kranenburg
On 02/27/2018 12:40 AM, Hans van Kranenburg wrote: > [...] > > But, the main thing I wanted to test is if the change would result in a > much lower total amount of grants, which is not the case. So, * I couldn't reproduce a number > 32 * The proposed fix doesn't help. There's two scenarios

Bug#880554: [Pkg-xen-devel] Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-26 Thread Hans van Kranenburg
On 02/26/2018 07:35 PM, Hans van Kranenburg wrote: > On 02/26/2018 03:52 PM, Ian Jackson wrote: >> Christian Schwamborn writes ("Re: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64"): >>> I can try, but the only system I can really test thi

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-26 Thread Hans van Kranenburg
On 02/26/2018 03:52 PM, Ian Jackson wrote: > Christian Schwamborn writes ("Re: Bug#880554: xen domu freezes with kernel > linux-image-4.9.0-4-amd64"): >> I can try, but the only system I can really test this is a productive >> system, as this 'reliable' shows

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-26 Thread Ian Jackson
Christian Schwamborn writes ("Re: Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64"): > I can try, but the only system I can really test this is a productive > system, as this 'reliable' shows this issue (and I don't want to crash > it on purpose on a regu

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-26 Thread Christian Schwamborn
Hi Hans, I can try, but the only system I can really test this is a productive system, as this 'reliable' shows this issue (and I don't want to crash it on purpose on a regular basis). Since I set gnttab_max_frame to a higher value it runs smooth. If you're confident this will work I can try

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-02-23 Thread Hans van Kranenburg
Hi Valentin, Christian, Finally getting back to you about the max grant frames issue. We discussed this with upstream Xen developers, and a different fix was proposed. I would really appreciate if you could test it and confirm it also solves the issue. Testing does not involve recompiling the

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-15 Thread Valentin Vidic
On Mon, Jan 15, 2018 at 11:12:03AM +0100, Christian Schwamborn wrote: > Is there a easy way to get/monitor the used 'grants' frames? As I understand > it, the xen-diag tool you mentioned doesn't compile in xen 4.8? Here is a status from another host: domid=0: nr_frames=4, max_nr_frames=256

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-15 Thread Valentin Vidic
On Mon, Jan 15, 2018 at 11:12:03AM +0100, Christian Schwamborn wrote: > Is there a easy way to get/monitor the used 'grants' frames? As I understand > it, the xen-diag tool you mentioned doesn't compile in xen 4.8? I just gave it another try and after modifying xen-diag.c a bit to work with 4.8

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-15 Thread Christian Schwamborn
Hi Hans and Valentin, first of all: Thanks for your help and explanations, that is very helpfull. I was on vacation last week and couldn't answer right away. On 07.01.2018 19:36, Hans van Kranenburg wrote: If this is something users are going to run into while not doing more unusual things

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-12 Thread Hans van Kranenburg
On 01/12/2018 12:43 PM, Valentin Vidic wrote: > On Fri, Jan 12, 2018 at 01:34:10AM +0100, Hans van Kranenburg wrote: >> Is the 59 your lots-o-vcpu-monster? > > Yes, that is the one with a larger vcpu count. Check. >> I just finished with the initial preparation of a Xen 4.10 package for >>

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-12 Thread Valentin Vidic
On Fri, Jan 12, 2018 at 01:34:10AM +0100, Hans van Kranenburg wrote: > Is the 59 your lots-o-vcpu-monster? Yes, that is the one with a larger vcpu count. > I just finished with the initial preparation of a Xen 4.10 package for > unstable and have it running in my test environment. Unrelated to

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-11 Thread Hans van Kranenburg
Hi, On 08/01/2018 13:38, Valentin Vidic wrote: > On Sun, Jan 07, 2018 at 07:36:40PM +0100, Hans van Kranenburg wrote: >> Recently a tool was added to "dump guest grant table info". You could >> see if it compiles on the 4.8 source and see if it works? Would be >> interesting to get some idea

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-08 Thread Valentin Vidic
On Sun, Jan 07, 2018 at 07:36:40PM +0100, Hans van Kranenburg wrote: > Recently a tool was added to "dump guest grant table info". You could > see if it compiles on the 4.8 source and see if it works? Would be > interesting to get some idea about how high or low these numbers are in > different

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-07 Thread Hans van Kranenburg
On 01/07/2018 10:05 AM, Valentin Vidic wrote: > On Sat, Jan 06, 2018 at 11:17:00PM +0100, Hans van Kranenburg wrote: >> I agree that the upstream default, 32 is quite low. This is indeed a >> configuration issue. I myself ran into this years ago with a growing >> number of domUs and network

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-07 Thread Valentin Vidic
On Sat, Jan 06, 2018 at 11:17:00PM +0100, Hans van Kranenburg wrote: > I agree that the upstream default, 32 is quite low. This is indeed a > configuration issue. I myself ran into this years ago with a growing > number of domUs and network interfaces in use. We have been using >

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-06 Thread Hans van Kranenburg
Hi Christian and everyone else, Ack on reassign to Xen. On 01/06/2018 04:11 PM, Yves-Alexis Perez wrote: > control: reassign -1 xen-hypervisor-4.8-amd64 > > On Sat, 2018-01-06 at 15:23 +0100, Valentin Vidic wrote: >> On Sat, Jan 06, 2018 at 03:08:26PM +0100, Yves-Alexis Perez wrote: >>>

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-06 Thread Yves-Alexis Perez
control: reassign -1 xen-hypervisor-4.8-amd64 On Sat, 2018-01-06 at 15:23 +0100, Valentin Vidic wrote: > On Sat, Jan 06, 2018 at 03:08:26PM +0100, Yves-Alexis Perez wrote: > > According to that link, the fix seems to be configuration rather than > > code. > > Does this mean this bug against the

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-06 Thread Valentin Vidic
On Sat, Jan 06, 2018 at 03:08:26PM +0100, Yves-Alexis Perez wrote: > According to that link, the fix seems to be configuration rather than code. > Does this mean this bug against the kernel should be closed? Yes, the problem seems to be in the Xen hypervisor and not the Linux kernel itself. The

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2018-01-06 Thread Yves-Alexis Perez
On Fri, 2017-11-17 at 07:39 +0100, Valentin Vidic wrote: > Hi, > > The problem seems to be caused by the new multi-queue xen blk driver > and I was advised by the Xen devs to increase the gnttab_max_frames=256 > parameter for the hypervisor. This has solved the blocking issue > for me and it has

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2017-11-16 Thread Valentin Vidic
Hi, The problem seems to be caused by the new multi-queue xen blk driver and I was advised by the Xen devs to increase the gnttab_max_frames=256 parameter for the hypervisor. This has solved the blocking issue for me and it has been running without problems for a few months now. I/O to LUNs

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2017-11-14 Thread Martin von Wittich
We're having the same problem here. For some reason, only 2 domUs are affected (the dom0 has a total of 22 domUs, 14 of those are running Debian stretch, and 13 of those are running Linux 4.9.51-1). The `xl console` output of the first domU (according to our monitoring it hangs since

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2017-11-13 Thread Christian Schwamborn
Update: First of all: Forget my observation about the 'system boot time'. I mixed up something, the dom0 boot time was increased, but this happened probably due to the not (well/propper) handled lvm thin activation during system boot. One last thing I pulled from domu with the original

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2017-11-02 Thread Christian Schwamborn
Update: Sadly the my productive system froze in the early afternoon today again with the older kernel as well (4.9.30-2+deb9u5). so that wasn't a temp workaround. Paradoxically nothing showed up on the xl console (within a screen) at dom0. No errors, nothing, the vm just stopped responding.

Bug#880554: xen domu freezes with kernel linux-image-4.9.0-4-amd64

2017-11-02 Thread Christian Schwamborn
Package: linux-image-4.9.0-4-amd64 Version: 4.9.51-1 Severity: critical As I can tell right now, the domu system simply freezes. The logs simply end at some point until the new reboot stuff comes up. Sometimes it's still possible to log on to the system, but nothing really works. It is like