Re: memory leak in =3.11.6

2013-10-30 Thread Duncan
Kai Krakow posted on Wed, 30 Oct 2013 00:30:26 +0100 as excerpted: For me it started late 3.9 if I remember right. But I'm pretty sure it startet with 3.10 which I mostly upgraded to for using skinny extents. While skinny extents has helped subjective performance a little bit I since

Re: memory leak in =3.11.6

2013-10-29 Thread Duncan
Jérôme Poulin posted on Mon, 28 Oct 2013 23:24:10 -0400 as excerpted: On Mon, Oct 28, 2013 at 6:34 PM, Kai Krakow hurikhan77+bt...@gmail.com wrote: If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low

Re: memory leak in =3.11.6

2013-10-29 Thread Jérôme Poulin
On Tue, Oct 29, 2013 at 12:42 PM, Duncan 1i5t5.dun...@cox.net wrote: So I'd call the qgroups feature broken at this time I agree. I made a page about Quotas on the wiki so to document my findings as documentation is really sparse. If you consult the section Known issues, you will find those: *

Re: memory leak in =3.11.6

2013-10-29 Thread Kai Krakow
Jérôme Poulin jeromepou...@gmail.com schrieb: If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low at 2 GB while after a few minutes uptime cache is about 5 GB. I probably have the same problem over

Re: memory leak in =3.11.6

2013-10-29 Thread Kai Krakow
Tomasz Chmielewski t...@virtall.com schrieb: I probably have the same problem over here, after about 2 weeks of random read/write it seems my memory and swap get almost full and even after killing all process and getting in single user mode, memory won't free up. Would you happen to have

Re: memory leak in =3.11.6

2013-10-29 Thread Wang Shilong
On 10/30/2013 06:46 AM, Jérôme Poulin wrote: On Tue, Oct 29, 2013 at 12:42 PM, Duncan 1i5t5.dun...@cox.net wrote: So I'd call the qgroups feature broken at this time I agree. I made a page about Quotas on the wiki so to document my findings as documentation is really sparse. If you consult the

Re: memory leak in =3.11.6

2013-10-28 Thread Jérôme Poulin
On Mon, Oct 28, 2013 at 6:34 PM, Kai Krakow hurikhan77+bt...@gmail.com wrote: If I close all my desktop programs, my system stays at +12 GB RAM usage while after a fresh boot it has 12+ GB free (of 16 GB). Cache stays low at 2 GB while after a few minutes uptime cache is about 5 GB. I

Re: memory leak in =3.11.6

2013-10-28 Thread Tomasz Chmielewski
I probably have the same problem over here, after about 2 weeks of random read/write it seems my memory and swap get almost full and even after killing all process and getting in single user mode, memory won't free up. Would you happen to have quota enabled too? Kernel is 3.11.4 I

Re: Memory leak?

2011-07-12 Thread Stephane Chazelas
2011-07-11 10:01:21 +0100, Stephane Chazelas: [...] Same without dmcrypt. So to sum up, BUG() reached in btrfs-fixup thread when doing an - rsync (though I also got (back when on ubuntu and 2.6.38) at least one occurrence using bsdtar | bsdtar) - of a large amount of data (with a large

Re: Memory leak?

2011-07-11 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-11 05:01:21 -0400: 2011-07-10 19:37:28 +0100, Stephane Chazelas: 2011-07-10 08:44:34 -0400, Chris Mason: [...] Great, we're on the right track. Does it trigger with mount -o compress instead of mount -o compress_force? [...]

Re: Memory leak?

2011-07-11 Thread Stephane Chazelas
2011-07-11 11:00:19 -0400, Chris Mason: Excerpts from Stephane Chazelas's message of 2011-07-11 05:01:21 -0400: 2011-07-10 19:37:28 +0100, Stephane Chazelas: 2011-07-10 08:44:34 -0400, Chris Mason: [...] Great, we're on the right track. Does it trigger with mount -o compress

Re: Memory leak?

2011-07-11 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-11 11:35:56 -0400: 2011-07-11 11:00:19 -0400, Chris Mason: Excerpts from Stephane Chazelas's message of 2011-07-11 05:01:21 -0400: 2011-07-10 19:37:28 +0100, Stephane Chazelas: 2011-07-10 08:44:34 -0400, Chris Mason: [...]

Re: Memory leak?

2011-07-11 Thread Stephane Chazelas
2011-07-11 12:25:51 -0400, Chris Mason: [...] Also, when I resume the rsync (so it doesn't transfer the already transfered files), it does BUG() again. Ok, could you please send along the exact rsync command you were running? [...] I did earlier, but here it is again: rsync --archive

Re: Memory leak?

2011-07-10 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-09 16:36:50 -0400: 2011-07-09 13:25:00 -0600, cwillu: On Sat, Jul 9, 2011 at 11:09 AM, Stephane Chazelas stephane_chaze...@yahoo.fr wrote: 2011-07-08 11:06:08 -0400, Chris Mason: [...] I would do two things.  First, I'd turn off

Re: Memory leak?

2011-07-10 Thread Stephane Chazelas
2011-07-10 08:44:34 -0400, Chris Mason: [...] Great, we're on the right track. Does it trigger with mount -o compress instead of mount -o compress_force? [...] It does trigger. I get that same invalid opcode. BTW, I tried with CONFIG_SLUB and slub_debug and no more useful information than

Re: Memory leak?

2011-07-09 Thread Stephane Chazelas
2011-07-08 12:17:54 -0400, Chris Mason: [...] How easily can you recompile your kernel with more debugging flags? That should help narrow it down. I'm looking for CONFIG_SLAB_DEBUG (or slub) and CONFIG_DEBUG_PAGEALLOC [...] I tried that (with CONFIG_DEBUG_SLAB_LEAK as well) but no difference

Re: Memory leak?

2011-07-09 Thread cwillu
On Sat, Jul 9, 2011 at 11:09 AM, Stephane Chazelas stephane_chaze...@yahoo.fr wrote: 2011-07-08 11:06:08 -0400, Chris Mason: [...] I would do two things.  First, I'd turn off compress_force.  There's no explicit reason for this, it just seems like the mostly likely place for a bug. [...] I

Re: Memory leak?

2011-07-09 Thread Stephane Chazelas
2011-07-09 13:25:00 -0600, cwillu: On Sat, Jul 9, 2011 at 11:09 AM, Stephane Chazelas stephane_chaze...@yahoo.fr wrote: 2011-07-08 11:06:08 -0400, Chris Mason: [...] I would do two things.  First, I'd turn off compress_force.  There's no explicit reason for this, it just seems like the

Re: Memory leak?

2011-07-09 Thread Stephane Chazelas
2011-07-09 08:09:55 +0100, Stephane Chazelas: 2011-07-08 16:12:28 -0400, Chris Mason: [...] I'm running a dstat -df at the same time and I'm seeing substantive amount of disk writes on the disks that hold the source FS (and I'm rsyncing from read-only snapshot subvolumes in case

Re: Memory leak?

2011-07-08 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-08 08:44:29 -0400: 2011-07-06 09:11:11 +0100, Stephane Chazelas: 2011-07-03 13:38:57 -0600, cwillu: On Sun, Jul 3, 2011 at 1:09 PM, Stephane Chazelas stephane_chaze...@yahoo.fr wrote: [...] Now, on a few occasions (actually, most

Re: Memory leak?

2011-07-08 Thread Stephane Chazelas
2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger problem. Does the btrfs-fixup-0 oops come before or

Re: Memory leak?

2011-07-08 Thread Stephane Chazelas
2011-07-08 16:41:23 +0100, Stephane Chazelas: 2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger

Re: Memory leak?

2011-07-08 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-08 11:41:23 -0400: 2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly)

Re: Memory leak?

2011-07-08 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-08 12:11:03 -0400: 2011-07-08 16:41:23 +0100, Stephane Chazelas: 2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to

Re: Memory leak?

2011-07-08 Thread Stephane Chazelas
2011-07-08 12:17:54 -0400, Chris Mason: [...] Jun 5 00:58:10 BUG: Bad page state in process rsync pfn:1bfdf Jun 5 00:58:10 page:ea61f8c8 count:0 mapcount:0 mapping: (null) index:0x2300 Jun 5 00:58:10 page flags: 0x110(dirty) Jun 5 00:58:10 Pid:

Re: Memory leak?

2011-07-08 Thread Stephane Chazelas
2011-07-08 12:15:08 -0400, Chris Mason: [...] You described this workload as rsync, is there anything else running? [...] Nope. Nothing else. And at least initially, that was onto an empty drive so basic copy. rsync --archive --xattrs --hard-links --numeric-ids --sparse --acls Cheers,

Re: Memory leak?

2011-07-08 Thread Stephane Chazelas
2011-07-08 12:15:08 -0400, Chris Mason: [...] I'd definitely try without -o compress_force. [...] Just started that over the night. I'm running a dstat -df at the same time and I'm seeing substantive amount of disk writes on the disks that hold the source FS (and I'm rsyncing from read-only

Re: Memory leak?

2011-07-08 Thread Chris Mason
Excerpts from Stephane Chazelas's message of 2011-07-08 16:04:12 -0400: 2011-07-08 12:15:08 -0400, Chris Mason: [...] I'd definitely try without -o compress_force. [...] Just started that over the night. I'm running a dstat -df at the same time and I'm seeing substantive amount of disk

Re: Memory leak?

2011-07-07 Thread Stephane Chazelas
2011-07-06 09:11:11 +0100, Stephane Chazelas: [...] extent_map delayed_node btrfs_inode_cache btrfs_free_space_cache (in bytes) [...] 01:00 267192640 668595744 23216460003418048 01:10 267192640 668595744 23216460003418048 01:20 267192640 668595744 23216460003418048

Re: Memory leak?

2011-07-07 Thread Li Zefan
Stephane Chazelas wrote: 2011-07-06 09:11:11 +0100, Stephane Chazelas: [...] extent_map delayed_node btrfs_inode_cache btrfs_free_space_cache (in bytes) [...] 01:00 267192640 668595744 23216460003418048 01:10 267192640 668595744 23216460003418048 01:20 267192640 668595744

Re: Memory leak?

2011-07-07 Thread Stephane Chazelas
2011-07-07 16:20:20 +0800, Li Zefan: [...] btrfs_inode_cache is a slab cache for in memory inodes, which is of struct btrfs_inode. [...] Thanks Li. If that's a cache, the system should be able to reuse the space there when it's low on memory, wouldn't it? What would be the conditions where

Re: Memory leak?

2011-07-06 Thread Stephane Chazelas
2011-07-03 13:38:57 -0600, cwillu: On Sun, Jul 3, 2011 at 1:09 PM, Stephane Chazelas stephane_chaze...@yahoo.fr wrote: [...] Now, on a few occasions (actually, most of the time), when I rsynced the data (about 2.5TB) onto the external drive, the system would crash after some time with Out