Re: exclusive subvolume space missing

Tomasz Pala posted on Fri, 15 Dec 2017 09:22:14 +0100 as excerpted: > I wonder how this one db-library behaves: > > $ find . -name \*.sqlite | xargs ls -gGhS | head -n1 > -rw-r--r-- 1 15M 2017-12-08 12:14 > ./.mozilla/firefox/vni9ojqi.default/extension-data/ublock0.sqlite > > $ ~/fiemap

Re: exclusive subvolume space missing

On Tue, Dec 12, 2017 at 08:50:15 +0800, Qu Wenruo wrote: > Even without snapshot, things can easily go crazy. > > This will write 128M file (max btrfs file extent size) and write it to disk. > # xfs_io -f -c "pwrite 0 128M" -c "sync" /mnt/btrfs/file > > Then, overwrite the 1~128M range. > #

Re: exclusive subvolume space missing

On 2017年12月11日 19:40, Tomasz Pala wrote: > On Mon, Dec 11, 2017 at 07:44:46 +0800, Qu Wenruo wrote: > >>> I could debug something before I'll clean this up, is there anything you >>> want to me to check/know about the files? >> >> fiemap result along with btrfs dump-tree -t2 result. > > fiemap

Re: exclusive subvolume space missing

On Mon, Dec 11, 2017 at 07:44:46 +0800, Qu Wenruo wrote: >> I could debug something before I'll clean this up, is there anything you >> want to me to check/know about the files? > > fiemap result along with btrfs dump-tree -t2 result. fiemap attached, but dump-tree requires unmounted fs,

Re: exclusive subvolume space missing

On 2017年12月11日 07:44, Qu Wenruo wrote: > > > On 2017年12月10日 19:27, Tomasz Pala wrote: >> On Mon, Dec 04, 2017 at 08:34:28 +0800, Qu Wenruo wrote: >> 1. is there any switch resulting in 'defrag only exclusive data'? >>> >>> IIRC, no. >> >> I have found a directory - pam_abl databases,

Re: exclusive subvolume space missing

On 2017年12月10日 19:27, Tomasz Pala wrote: > On Mon, Dec 04, 2017 at 08:34:28 +0800, Qu Wenruo wrote: > >>> 1. is there any switch resulting in 'defrag only exclusive data'? >> >> IIRC, no. > > I have found a directory - pam_abl databases, which occupy 10 MB (yes, > TEN MEGAbytes) and released

Re: exclusive subvolume space missing

On Sun, Dec 10, 2017 at 12:27:38 +0100, Tomasz Pala wrote: > I have found a directory - pam_abl databases, which occupy 10 MB (yes, > TEN MEGAbytes) and released ...8.7 GB (almost NINE GIGAbytes) after # df Filesystem Size Used Avail Use% Mounted on /dev/sda264G 61G

Re: exclusive subvolume space missing

On Mon, Dec 04, 2017 at 08:34:28 +0800, Qu Wenruo wrote: >> 1. is there any switch resulting in 'defrag only exclusive data'? > > IIRC, no. I have found a directory - pam_abl databases, which occupy 10 MB (yes, TEN MEGAbytes) and released ...8.7 GB (almost NINE GIGAbytes) after defrag. After

Re: exclusive subvolume space missing

On Sun, Dec 03, 2017 at 01:45:45 +, Duncan wrote: > OTOH, it's also quite possible that people chose btrfs at least partly > for other reasons, say the "storage pool" qualities, and would rather Well, to name some: 1. filesystem-level backups via snapshot/send/receive - much cleaner and

How exclusive in parent qgroup is computed? (was: Re: exclusive subvolume space missing)

02.12.2017 03:27, Qu Wenruo пишет: > > That's the difference between how sub show and quota works. > > For quota, it's per-root owner check. > Means even a file extent is shared between different inodes, if all > inodes are inside the same subvolume, it's counted as exclusive. > And if any of

Re: exclusive subvolume space missing

On Sun, Dec 3, 2017 at 3:47 AM, Adam Borowski wrote: > I'd say that the only good use for nocow is "I wish I have placed this file > on a non-btrfs, but it'd be too much hassle to repartition". > > If you snapshot nocow at all, you get the worst of both worlds. I think it's

Re: exclusive subvolume space missing

On Fri, Dec 1, 2017 at 5:53 PM, Tomasz Pala wrote: > # btrfs fi usage / > Overall: > Device size: 128.00GiB > Device allocated:117.19GiB > Device unallocated: 10.81GiB > Device missing: 0.00B > Used:

Re: exclusive subvolume space missing

On 2017年12月02日 17:33, Tomasz Pala wrote: > OK, I seriously need to address that, as during the night I lost > 3 GB again: > > On Sat, Dec 02, 2017 at 10:35:12 +0800, Qu Wenruo wrote: > >>> # btrfs fi sh / >>> Label: none uuid: 17a3de25-6e26-4b0b-9665-ac267f6f6c4a >>> Total devices 2

Re: exclusive subvolume space missing

On Sun, Dec 03, 2017 at 01:45:45AM +, Duncan wrote: > Tomasz Pala posted on Sat, 02 Dec 2017 18:18:19 +0100 as excerpted: > >> I got ~500 small files (100-500 kB) updated partially in regular > >> intervals: > >> > >> # du -Lc **/*.rrd | tail -n1 > >> 105Mtotal > > FWIW, I've no idea

Re: exclusive subvolume space missing

Tomasz Pala posted on Sat, 02 Dec 2017 18:18:19 +0100 as excerpted: > On Sat, Dec 02, 2017 at 17:28:12 +0100, Tomasz Pala wrote: > >>> Suppose you start with a 100 MiB file (I'm adjusting the sizes down from >> [...] >>> Now make various small changes to the file, say under 16 KiB each. These

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 17:28:12 +0100, Tomasz Pala wrote: >> Suppose you start with a 100 MiB file (I'm adjusting the sizes down from > [...] >> Now make various small changes to the file, say under 16 KiB each. These >> will each be COWed elsewhere as one might expect. by default 16 KiB at >>

Re: exclusive subvolume space missing

On Fri, 01 Dec 2017 18:57:08 -0800, Duncan wrote: > OK, is this supposed to be raid1 or single data, because the above shows > metadata as all raid1, while some data is single tho most is raid1, and > while old mkfs used to create unused single chunks on raid1 that had to > be removed manually

Re: exclusive subvolume space missing

OK, I seriously need to address that, as during the night I lost 3 GB again: On Sat, Dec 02, 2017 at 10:35:12 +0800, Qu Wenruo wrote: >> # btrfs fi sh / >> Label: none uuid: 17a3de25-6e26-4b0b-9665-ac267f6f6c4a >> Total devices 2 FS bytes used 44.10GiB Total devices 2 FS

Re: exclusive subvolume space missing

Tomasz Pala posted on Sat, 02 Dec 2017 01:53:39 +0100 as excerpted: > # btrfs fi usage / > Overall: > Device size: 128.00GiB > Device allocated:117.19GiB > Device unallocated: 10.81GiB > Device missing: 0.00B > Used:

Re: exclusive subvolume space missing

On 2017年12月02日 10:21, Tomasz Pala wrote: > On Sat, Dec 02, 2017 at 09:47:19 +0800, Qu Wenruo wrote: > >>> Actually I should rephrase the problem: >>> >>> "snapshot has taken 8 GB of space despite nothing has altered source >>> subvolume" > > Actually, after: > > # btrfs balance start -v

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 09:47:19 +0800, Qu Wenruo wrote: >> Actually I should rephrase the problem: >> >> "snapshot has taken 8 GB of space despite nothing has altered source >> subvolume" Actually, after: # btrfs balance start -v -dconvert=raid1 / ctrl-c on block group 35G/113G # btrfs

Re: exclusive subvolume space missing

On 2017年12月02日 09:43, Tomasz Pala wrote: > On Sat, Dec 02, 2017 at 09:05:50 +0800, Qu Wenruo wrote: > >>> qgroupid rfer excl >>> >>> 0/26012.25GiB 3.22GiB from 170712 - first snapshot >>> 0/31217.54GiB 4.56GiB from

Re: exclusive subvolume space missing

On 2017年12月02日 09:23, Tomasz Pala wrote: > On Sat, Dec 02, 2017 at 08:27:56 +0800, Qu Wenruo wrote: > >> I assume there is program eating up the space. >> Not btrfs itself. > > Very doubtful. I've encountered ext3 "eating" problem once, that couldn't be > find by lsof on 3.4.75 kernel, but the

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 09:05:50 +0800, Qu Wenruo wrote: >> qgroupid rfer excl >> >> 0/26012.25GiB 3.22GiB from 170712 - first snapshot >> 0/31217.54GiB 4.56GiB from 170811 >> 0/36625.59GiB 2.44GiB

Re: exclusive subvolume space missing

On Sat, Dec 02, 2017 at 08:27:56 +0800, Qu Wenruo wrote: > I assume there is program eating up the space. > Not btrfs itself. Very doubtful. I've encountered ext3 "eating" problem once, that couldn't be find by lsof on 3.4.75 kernel, but the space was returning after killing Xorg. The system I'm

Re: exclusive subvolume space missing

>>> Now, the weird part for me is exclusive data count: >>> >>> # btrfs sub sh ./snapshot-171125 >>> [...] >>> Subvolume ID: 388 >>> # btrfs fi du -s ./snapshot-171125 >>> Total Exclusive Set shared Filename >>> 21.50GiB63.35MiB20.77GiB snapshot-171125 >>>

Re: exclusive subvolume space missing

On Fri, Dec 01, 2017 at 21:36:14 +, Hugo Mills wrote: >The thing I'd first go looking for here is some rogue process > writing lots of data. I've had something like this happen to me > before, a few times. First, I'd look for large files with "du -ms /* | > sort -n", then work down into

Re: exclusive subvolume space missing

On 2017年12月02日 00:15, Tomasz Pala wrote: > Hello, > > I got a problem with btrfs running out of space (not THE > Internet-wide, well known issues with interpretation). > > The problem is: something eats the space while not running anything that > justifies this. There were 18 GB free space

Re: exclusive subvolume space missing

On Fri, Dec 01, 2017 at 05:15:55PM +0100, Tomasz Pala wrote: > Hello, > > I got a problem with btrfs running out of space (not THE > Internet-wide, well known issues with interpretation). > > The problem is: something eats the space while not running anything that > justifies this. There were 18

Re: exclusive subvolume space missing

Tomasz Pala posted on Fri, 01 Dec 2017 17:15:55 +0100 as excerpted: > Hello, > > I got a problem with btrfs running out of space (not THE > Internet-wide, well known issues with interpretation). > > The problem is: something eats the space while not running anything that > justifies this. There

exclusive subvolume space missing

Hello, I got a problem with btrfs running out of space (not THE Internet-wide, well known issues with interpretation). The problem is: something eats the space while not running anything that justifies this. There were 18 GB free space available, suddenly it dropped to 8 GB and then to 63 MB