[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-08-07 Thread Peter Waller
That's my understanding too, except in one of the scenarios I observed 100% SYS CPU for long stretches even when there was a significant amount (~50GB) of the device unused. However, if it was a soft lockup it was for 8 hours, during which the machine was totally unresponsive to HTTP requests,

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-08-05 Thread Peter Waller
Repost of what I sent to the mailing list just now: My current interpretation of this problem is that it is some pathological condition caused by not rebalancing and being nearly out of space for allocating more metadata and hence it is rarely being seen by anyone else (because most users are

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-08-02 Thread Peter Waller
The production machine hasn't had a lockup since moving to 3.15.7-031507-generic (it's been up for 4 days) even though we could reproduce the lockup on a new machine with that kernel using a snapshot of the old volume. Another twist is that on the productino machine I'm now reliably seeing No

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-08-01 Thread Peter Waller
btrfs was created with `mkfs.btrfs /dev/mapper/vg-lv`. It isn't a hard requirement except that it's a pain to migrate since that requires downtime to move the files. Something I'd rather not do unless absolutely necessary. The machine freezes are inconvenient but represent a few minutes downtime

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-08-01 Thread Peter Waller
The filesystem may have been originally created on an older version of BTRFS from Ubuntu Saucy, which I suppose may not have detected the SSD? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1349711

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-08-01 Thread Peter Waller
smb: Yeah, the system the filesystem was created on was PV, the device name was xvd*. Now it's on HVM with xvd*. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1349711 Title: Machine lockup in

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-31 Thread Peter Waller
I've got a way to rapidly reproduce the error now. I can do it reliably with a turnaround time of 5-10 minutes. I've reproduced the crash on the new Kernel, so it has now been observed on both 3.13.0-32-generic and 3.15.7-031507-generic. I'll try 3.16 next. I've also discovered this new stack

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-31 Thread Peter Waller
Now reproduced on 3.16. I'm out of things to try for now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1349711 Title: Machine lockup in btrfs-transaction To manage notifications about this bug go

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-31 Thread Peter Waller
This gist contains a stack trace every 10 seconds taken with `echo l /proc/sysrq-trigger` whilst the machine was spinning in the kernel but still responsive. https://gist.github.com/pwaller/c7dd0f4807459acedcdf The machine remained responsive for 5-10 minutes before becoming totally

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-31 Thread Peter Waller
** Tags added: kernel-bug-exists-upstream ** Changed in: linux (Ubuntu) Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1349711 Title: Machine lockup in

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-31 Thread Peter Waller
Hm, I'm not sure I can give a thorough description since I don't understand enough about the exact workload myself. It is a fairly arbitrary workload generated by our users. In the end, it boils down to creating, reading and writing many (~20,000) sqlite files of size 16kb - 12GB across many

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-31 Thread Peter Waller
(otherwise unloaded test machines) On a dual core machine, 100% system CPU usage with zero writes is seen on one core for 5-10 minutes, spending time in BTRFS threads. On a single thread machine 100% system CPU is used and I haven't yet been able to cause it to hang entirely. I do observe almost

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-30 Thread Peter Waller
I found an additional stack trace from a previous machine lockup. [1093202.136107] INFO: task kworker/u30:1:31455 blocked for more than 120 seconds. [1093202.141596] Tainted: GF3.13.0-30-generic #54-Ubuntu [1093202.146201] echo 0 /proc/sys/kernel/hung_task_timeout_secs

[Bug 1349711] [NEW] Machine lockup in btrfs-transaction

2014-07-29 Thread Peter Waller
Public bug reported: This has happened twice now. I'm on an AWS EC2 m3.large instance with the official Ubuntu AMI ami- 776d9700. # cat /proc/version_signature Ubuntu 3.13.0-32.57-generic 3.13.11.4 After running for many days, the machine locked up with the below messages appearing on the

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-29 Thread Peter Waller
I've also started a thread on linux-btrfs: http://thread.gmane.org/gmane.comp.file-systems.btrfs/37224 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1349711 Title: Machine lockup in

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-29 Thread Peter Waller
@brad-figg, apologies I missed your response. Is there a way to generate the output without automatically uploading it? I would like to review it first. I tried `apport-cli --save` but that doesn't do anything unless there are any crash files that I can tell. -- You received this bug

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-29 Thread Peter Waller
One thing I am unsure of is that the bug did not manifest for at least 12 days running originally. So I'm not sure it is going to be possible to reliably decide that it is fixed by moving to a particular kernel. What is the standard here? -- You received this bug notification because you are a

[Bug 1349711] Re: Machine lockup in btrfs-transaction

2014-07-29 Thread Peter Waller
The crashes became more frequent. The approximate time was 12 days running, then ~2 days running, then 6 hours, then 1 hour. I since moved to 3.15.7-031507-generic. One thing I have observed is that (EXT4 filesystem) /var/log/nginx/access.log contained ~2KB of NULL characters in place of any

[Bug 1074564] Re: Upstart logfiles should be readable by adm group

2014-07-08 Thread Peter Waller
Some admins are typing sudo bash because it's inconvenient to have to sudo to look at each log file. This is pretty annoying, what does it take to get this fixed? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1301015] Re: Networking does not restart

2014-07-08 Thread Peter Waller
I've added the following file: /etc/network/interfaces.d$ cat lo1.cfg auto lo1 iface lo1 inet loopback address 127.0.1.1 netmask 255.0.0.0 How do I get it to take effect if not `restart networking`? I've tried `ifup lo1` but it just says cannot find device lo1. I have a

[Bug 296122] Re: XNEST crashes in X_PolyFillRectangle with error config/hal: NewInputDeviceRequest failed

2014-03-24 Thread Peter Waller
I'm also seeing the same with Xnest on 13.10. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/296122 Title: XNEST crashes in X_PolyFillRectangle with error config/hal: NewInputDeviceRequest failed

[Bug 886427] Re: oidentd doesn't have a status option in the init script

2014-02-19 Thread Peter Waller
Any chance of this fix finding its way into Ubuntu? It's causing some configuration managers to think that oidentd always needs to be restarted because `status` always returns a non-zero exit status. -- You received this bug notification because you are a member of Ubuntu Bugs, which is

[Bug 1264674] Re: nginx segfault when adding add_header in configuration

2014-02-12 Thread Peter Waller
This bug says fix released but it isn't actually updating on my machine. How is this? nginx won't currently start - when is this going to be fixed?! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1094319] Re: p11-kit: couldn't load module: /usr/lib/i386-linux-gnu/pkcs11/gnome-keyring-pkcs11.so: /usr/lib/i386-linux-gnu/pkcs11/gnome-keyring-pkcs11.so: cannot open shared object file: No such

2013-12-11 Thread Peter Waller
Any chance of an update for precise? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1094319 Title: p11-kit: couldn't load module: /usr/lib/i386-linux-gnu/pkcs11/gnome- keyring-pkcs11.so:

[Bug 1250208] [NEW] `stop salt-master` causes upstart to hang

2013-11-11 Thread Peter Waller
Public bug reported: On the Saucy (13.10) 64-bit image provided by Amazon web services, `stop salt-master` causes upstart to hang. This makes it difficult to even un-install salt-master. I've mentioned this problem upstream* in an issue that shows related behaviour. However, that upstream issue

[Bug 1228686] Re: X crashes on logout or user switching

2013-09-22 Thread Peter Waller
After X crashes, if I try to restart it, frequently the screen locks up entirely. I've discovered more information in the Xorg.1.log (attached). Here is an extract: [ 40345.063] (II) NVIDIA(0): NVIDIA GPU GeForce GTX 560 Ti (GF114) at PCI:1:0:0 (GPU-0) [ 40345.063] (--) NVIDIA(0): Memory:

[Bug 1228686] [NEW] X crashes on logout or user switching

2013-09-21 Thread Peter Waller
Public bug reported: If I try to logout or switch to another user, I am frequently presented with a VT with a blinking cursor. It seems to be about a 50% chance that it will work and do the right thing, and a 50% chance that X dies. I'm on 13.10 saucy daily, 64 bit. dmesg says the following,

[Bug 312462] Re: document_new_from_data() arg1 must be without null bytes

2013-09-15 Thread Peter Waller
Oh, duh, `gir1.2-poppler-0.18` on ubuntu. Was blind to it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/312462 Title: document_new_from_data() arg1 must be without null bytes To manage

[Bug 312462] Re: document_new_from_data() arg1 must be without null bytes

2013-09-15 Thread Peter Waller
BenjaminBerg, this wasn't obvious. I'm glad to hear there is something replacing the bindings and it's not just that it is totally dead. However, if I try and use it: from gi.repository import Poppler ERROR:root:Could not find any typelib for Poppler I can't see any obvious packages that I

[Bug 312462] Re: document_new_from_data() arg1 must be without null bytes

2013-09-12 Thread Peter Waller
Ping. I really want to use poppler-python and stuff like this makes me cringe. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/312462 Title: document_new_from_data() arg1 must be without null bytes

[Bug 1062118] [NEW] Missing some geometry header files (TGeo*)

2012-10-05 Thread Peter Waller
Public bug reported: The files in the `geombuilder` directory seem to be missing from `libroot-geom-dev` package. They are found here, and should be part of the standard root installation, so far as I can tell. http://root.cern.ch/viewvc/trunk/geom/geombuilder/inc/ ** Affects: root-system