Re: exclusive subvolume space missing

Tomasz Pala posted on Sat, 02 Dec 2017 01:53:39 +0100 as excerpted: > # btrfs fi usage / > Overall: > Device size: 128.00GiB > Device allocated:117.19GiB > Device unallocated: 10.81GiB > Device missing: 0.00B > Used:

Re: exclusive subvolume space missing

On 2017年12月02日 10:21, Tomasz Pala wrote: > On Sat, Dec 02, 2017 at 09:47:19 +0800, Qu Wenruo wrote: > >>> Actually I should rephrase the problem: >>> >>> "snapshot has taken 8 GB of space despite nothing has altered source >>> subvolume" > > Actually, after: > > # btrfs balance start -v

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 09:47:19 +0800, Qu Wenruo wrote: >> Actually I should rephrase the problem: >> >> "snapshot has taken 8 GB of space despite nothing has altered source >> subvolume" Actually, after: # btrfs balance start -v -dconvert=raid1 / ctrl-c on block group 35G/113G # btrfs

Re: exclusive subvolume space missing

On 2017年12月02日 09:43, Tomasz Pala wrote: > On Sat, Dec 02, 2017 at 09:05:50 +0800, Qu Wenruo wrote: > >>> qgroupid rfer excl >>> >>> 0/26012.25GiB 3.22GiB from 170712 - first snapshot >>> 0/31217.54GiB 4.56GiB from

Re: exclusive subvolume space missing

On 2017年12月02日 09:23, Tomasz Pala wrote: > On Sat, Dec 02, 2017 at 08:27:56 +0800, Qu Wenruo wrote: > >> I assume there is program eating up the space. >> Not btrfs itself. > > Very doubtful. I've encountered ext3 "eating" problem once, that couldn't be > find by lsof on 3.4.75 kernel, but the

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 09:05:50 +0800, Qu Wenruo wrote: >> qgroupid rfer excl >> >> 0/26012.25GiB 3.22GiB from 170712 - first snapshot >> 0/31217.54GiB 4.56GiB from 170811 >> 0/36625.59GiB 2.44GiB

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 08:27:56 +0800, Qu Wenruo wrote: > I assume there is program eating up the space. > Not btrfs itself. Very doubtful. I've encountered ext3 "eating" problem once, that couldn't be find by lsof on 3.4.75 kernel, but the space was returning after killing Xorg. The system I'm

Re: exclusive subvolume space missing

>>> Now, the weird part for me is exclusive data count: >>> >>> # btrfs sub sh ./snapshot-171125 >>> [...] >>> Subvolume ID: 388 >>> # btrfs fi du -s ./snapshot-171125 >>> Total Exclusive Set shared Filename >>> 21.50GiB63.35MiB20.77GiB snapshot-171125 >>>

Re: exclusive subvolume space missing

On Fri, Dec 01, 2017 at 21:36:14 +, Hugo Mills wrote: >The thing I'd first go looking for here is some rogue process > writing lots of data. I've had something like this happen to me > before, a few times. First, I'd look for large files with "du -ms /* | > sort -n", then work down into

Re: exclusive subvolume space missing

On 2017年12月02日 00:15, Tomasz Pala wrote: > Hello, > > I got a problem with btrfs running out of space (not THE > Internet-wide, well known issues with interpretation). > > The problem is: something eats the space while not running anything that > justifies this. There were 18 GB free space

Re: [PATCH 5/5] btrfs: Greatly simplify btrfs_read_dev_super

On 12/01/2017 05:19 PM, Nikolay Borisov wrote: Currently this function executes the inner loop at most 1 due to the i = 0; i < 1 condition. Furthermore, the btrfs_super_generation(super) > transid code in the if condition is never executed due to latest always set to NULL hence the first part

Re: btrfs-transacti hammering the system

Well, it's at zero now... # btrfs fi df /export/ Data, single: total=30.45TiB, used=30.25TiB System, DUP: total=32.00MiB, used=3.62MiB Metadata, DUP: total=66.50GiB, used=65.16GiB GlobalReserve, single: total=512.00MiB, used=0.00B On 01/12/17 16:47, Duncan wrote: Hans van Kranenburg posted on

Re: btrfs-transacti hammering the system

Hans van Kranenburg posted on Fri, 01 Dec 2017 18:06:23 +0100 as excerpted: > On 12/01/2017 05:31 PM, Matt McKinnon wrote: >> Sorry, I missed your in-line reply: >> >> >>> 2) How big is this filesystem? What does your `btrfs fi df >>> /mountpoint` say? >>> >> >> # btrfs fi df /export/ >> Data,

Re: exclusive subvolume space missing

On Fri, Dec 01, 2017 at 05:15:55PM +0100, Tomasz Pala wrote: > Hello, > > I got a problem with btrfs running out of space (not THE > Internet-wide, well known issues with interpretation). > > The problem is: something eats the space while not running anything that > justifies this. There were 18

Re: exclusive subvolume space missing

Tomasz Pala posted on Fri, 01 Dec 2017 17:15:55 +0100 as excerpted: > Hello, > > I got a problem with btrfs running out of space (not THE > Internet-wide, well known issues with interpretation). > > The problem is: something eats the space while not running anything that > justifies this. There

Re: btrfs-transacti hammering the system

On Fri, Dec 1, 2017 at 12:07 PM, Matt McKinnon wrote: > Right. The file system is 48T, with 17T available, so we're not quite > pushing it yet. > > So far so good on the space_cache=v2 mount. I'm surprised this isn't on the > gotcha page in the wiki; it may end up making a

Re: btrfs-progs - failed btrfs replace on RAID1 seems to have left things in a wrong state

Patrik Lundquist posted on Fri, 01 Dec 2017 10:29:43 +0100 as excerpted: > On 1 December 2017 at 08:18, Duncan <1i5t5.dun...@cox.net> wrote: >> >> When udev sees a device it triggers >> a btrfs device scan, which lets btrfs know which devices belong to which >> individual btrfs. But once it

Re: btrfs-transacti hammering the system

Right. The file system is 48T, with 17T available, so we're not quite pushing it yet. So far so good on the space_cache=v2 mount. I'm surprised this isn't on the gotcha page in the wiki; it may end up making a world of difference to the users here Thanks again, Matt On 01/12/17 13:24,

Re: btrfs-transacti hammering the system

On 12/01/2017 06:57 PM, Holger Hoffstätte wrote: > On 12/01/17 18:34, Matt McKinnon wrote: >> Thanks, I'll give space_cache=v2 a shot. > > Yes, very much recommended. > >> My mount options are: rw,relatime,space_cache,autodefrag,subvolid=5,subvol=/ > > Turn autodefrag off and use noatime

Re: btrfs-transacti hammering the system

On 2017-12-01 12:13, Andrei Borzenkov wrote: 01.12.2017 20:06, Hans van Kranenburg пишет: Additional tips (forgot to ask for your /proc/mounts before): * Use the noatime mount option, so that only accessing files does not lead to changes in metadata, Is not 'lazytime" default today? It gives

Re: btrfs-transacti hammering the system

On 12/01/17 18:34, Matt McKinnon wrote: > Thanks, I'll give space_cache=v2 a shot. Yes, very much recommended. > My mount options are: rw,relatime,space_cache,autodefrag,subvolid=5,subvol=/ Turn autodefrag off and use noatime instead of relatime. Your filesystem also seems very full, that's

Re: btrfs-transacti hammering the system

Thanks, I'll give space_cache=v2 a shot. My mount options are: rw,relatime,space_cache,autodefrag,subvolid=5,subvol=/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: btrfs-transacti hammering the system

01.12.2017 20:06, Hans van Kranenburg пишет: > > Additional tips (forgot to ask for your /proc/mounts before): > * Use the noatime mount option, so that only accessing files does not > lead to changes in metadata, Is not 'lazytime" default today? It gives you correct atime + no extra metadata

Re: btrfs-transacti hammering the system

On 12/01/2017 05:31 PM, Matt McKinnon wrote: > Sorry, I missed your in-line reply: > > >> 1) The one right above, btrfs_write_out_cache, is the write-out of the >> free space cache v1. Do you see this for multiple seconds going on, and >> does it match the time when it's writing X MB/s to disk?

exclusive subvolume space missing

Hello, I got a problem with btrfs running out of space (not THE Internet-wide, well known issues with interpretation). The problem is: something eats the space while not running anything that justifies this. There were 18 GB free space available, suddenly it dropped to 8 GB and then to 63 MB

Re: btrfs-transacti hammering the system

Sorry, I missed your in-line reply: 1) The one right above, btrfs_write_out_cache, is the write-out of the free space cache v1. Do you see this for multiple seconds going on, and does it match the time when it's writing X MB/s to disk? It seems to only last until the next watch update. []

Re: btrfs-transacti hammering the system

These seem to come up most often: [] transaction_kthread+0x133/0x1c0 [btrfs] [] kthread+0x109/0x140 [] ret_from_fork+0x25/0x30 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: btrfs-transacti hammering the system

On 12/01/2017 04:24 PM, Matt McKinnon wrote: > Thanks for this.  Here's what I get: Ok, and which one is displaying most of the time? > [...] > > [] io_schedule+0x16/0x40 > [] get_request+0x23e/0x720 > [] blk_queue_bio+0xc1/0x3a0 > [] generic_make_request+0xf8/0x2a0 > [] submit_bio+0x75/0x150 >

Re: btrfs-transacti hammering the system

Thanks for this. Here's what I get: [] transaction_kthread+0x133/0x1c0 [btrfs] [] kthread+0x109/0x140 [] ret_from_fork+0x25/0x30 ... [] io_schedule+0x16/0x40 [] get_request+0x23e/0x720 [] blk_queue_bio+0xc1/0x3a0 [] generic_make_request+0xf8/0x2a0 [] submit_bio+0x75/0x150 []

Re: btrfs-transacti hammering the system

On 12/01/2017 03:25 PM, Matt McKinnon wrote: > > Is there any way to figure out what exactly btrfs-transacti is chugging > on?  I have a few file systems that seem to get wedged for days on end > with this process pegged around 100%.  I've stopped all snapshots, made > sure no quotas were

btrfs-transacti hammering the system

Hi All, Is there any way to figure out what exactly btrfs-transacti is chugging on? I have a few file systems that seem to get wedged for days on end with this process pegged around 100%. I've stopped all snapshots, made sure no quotas were enabled, turned on autodefrag in the mount

RE: btrfs-progs - failed btrfs replace on RAID1 seems to have left things in a wrong state

Duncan, Thank you for your thorough response to my problem. I am now wiser in my understanding of how btrfs works in RAID1 thanks to your words. Last night I worked with someone in the IRC channel and we essentially came to the exact same conclusion. I used wipefs -a on the errant drive. Rebooted

Re: [PATCHSET v2] cgroup, writeback, btrfs: make sure btrfs issues metadata IOs from the root cgroup

On Wed 29-11-17 13:38:26, Chris Mason wrote: > On 11/29/2017 12:05 PM, Tejun Heo wrote: > >On Wed, Nov 29, 2017 at 09:03:30AM -0800, Tejun Heo wrote: > >>Hello, > >> > >>On Wed, Nov 29, 2017 at 05:56:08PM +0100, Jan Kara wrote: > >>>What has happened with this patch set? > >> > >>No idea. cc'ing

Re: btrfs-progs - failed btrfs replace on RAID1 seems to have left things in a wrong state

On 1 December 2017 at 08:18, Duncan <1i5t5.dun...@cox.net> wrote: > > When udev sees a device it triggers > a btrfs device scan, which lets btrfs know which devices belong to which > individual btrfs. But once it associates a device with a particular > btrfs, there's nothing to unassociate it --

[PATCH 5/5] btrfs: Greatly simplify btrfs_read_dev_super

Currently this function executes the inner loop at most 1 due to the i = 0; i < 1 condition. Furthermore, the btrfs_super_generation(super) > transid code in the if condition is never executed due to latest always set to NULL hence the first part of the condition always triggering. The gist of

[PATCH 1/5] btrfs: Remove dead code

trans was statically assigned to NULL and this never changed over the course of btrfs_get_extent. So remove any code which checks whether trans != NULL and just hardcode the fact trans is always NULL. This fixes CID#112806 Signed-off-by: Nikolay Borisov --- fs/btrfs/inode.c |

[PATCH 3/5] btrfs: Fix possible off-by-one in btrfs_search_path_in_tree

The name char array passed to btrfs_search_path_in_tree is of size BTRFS_INO_LOOKUP_PATH_MAX (4080). So the actual accessible char indexes are in the range of [0, 4079]. Currently the code uses the define but this represents an off-by-one. Signed-off-by: Nikolay Borisov ---

[PATCH 4/5] btrfs: Remove redundant NULL check

Before returning hole_em in btrfs_get_fiemap_extent we check if it's different than null. However, by the time this null check is triggered we already know hole_em is not null because it means it points to the em we found and it has already been dereferenced. Signed-off-by: Nikolay Borisov

[PATCH 2/5] btrfs: Remove dead code

'clear' is always set to 0 (BTRFS_FEATURE_COMPAT_SAFE_CLEAR, BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR and BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR are all defined to 0). So remove the code that logically can never execute. Signed-off-by: Nikolay Borisov --- fs/btrfs/sysfs.c | 2 -- 1 file

[PATCH 0/5] Misc cleanups

Here's a bunch of stuff that coverity found, this survived a full xfstest run. Nikolay Borisov (5): btrfs: Remove dead code btrfs: Remove dead code btrfs: Fix possible off-by-one in btrfs_search_path_in_tree btrfs: Remove redundant NULL check btrfs: Greatly simplify