Re: Understanding "btrfs filesystem usage"

2018-10-29 Thread Hugo Mills
egin:vcard > fn:Remi Gauvin > n:Gauvin;Remi > org:Georgian Infotech > adr:;;3-51 Sykes St. N.;Meaford;ON;N4L 1X3;Canada > email;internet:r...@georgianit.com > tel;work:226-256-1545 > version:2.1 > end:vcard > -- Hugo Mills | Great oxymorons of the world, no. 8: hugo@... carfax.org.uk | The Latest In Proven Technology http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Urgent: Need BTRFS-Expert

2018-10-17 Thread Hugo Mills
ur problem here on this mailing list, you'll get *most* of the experts looking at it, rather than just the one, and you'll generally get a much better (and easier to use) service. Hugo. -- Hugo Mills | The early bird gets the worm, but the second mouse hugo@... carfax.org.uk | gets t

Re: Interpreting `btrfs filesystem show'

2018-10-15 Thread Hugo Mills
On Mon, Oct 15, 2018 at 05:40:40PM +0300, Anton Shepelev wrote: > Hugo Mills to Anton Shepelev: > > >>While trying to resolve free space problems, and found > >>that I cannot interpret the output of: > >> > >>> btrfs filesystem show > >> >

Re: Interpreting `btrfs filesystem show'

2018-10-15 Thread Hugo Mills
On Mon, Oct 15, 2018 at 02:26:41PM +, Hugo Mills wrote: > On Mon, Oct 15, 2018 at 05:24:08PM +0300, Anton Shepelev wrote: > > Hello, all > > > > While trying to resolve free space problems, and found that > > I cannot interpret the output of: > > > >

Re: Interpreting `btrfs filesystem show'

2018-10-15 Thread Hugo Mills
uot; on the FS is the total amount of actual data and metadata in that allocation. You will also need to look at the output of "btrfs fi df" to see the breakdown of the 37.82 GiB into data, metadata and currently unused. See https://btrfs.wiki.kernel.org/index.php/FAQ#Understanding_free

Re: Which device is missing ?

2018-10-08 Thread Hugo Mills
On Mon, Oct 08, 2018 at 11:01:35PM +0200, Pierre Couderc wrote: > On 10/08/2018 06:14 PM, Hugo Mills wrote: > >On Mon, Oct 08, 2018 at 04:10:55PM +0000, Hugo Mills wrote: > >>On Mon, Oct 08, 2018 at 03:49:53PM +0200, Pierre Couderc wrote: > >>>I ma trying to make a &

Re: Which device is missing ?

2018-10-08 Thread Hugo Mills
On Mon, Oct 08, 2018 at 04:10:55PM +, Hugo Mills wrote: > On Mon, Oct 08, 2018 at 03:49:53PM +0200, Pierre Couderc wrote: > > I ma trying to make a "RAID1" with /dev/sda2 ans /dev/sdb (or similar). > > > > But I have stranges status or errors  about

Re: Which device is missing ?

2018-10-08 Thread Hugo Mills
g a device, it will probably need to be mounted in degraded mode (-o degraded), and that on kernels earlier than (IIRC) 4.14, this can only be done *once* without the FS becoming more or less permanently read-only. On recent kernels, it _should_ be OK. *WARNING ENDS* Hugo. [snip] -- H

Re: btrfs receive incremental stream on another uuid

2018-09-18 Thread Hugo Mills
trfs/msg44089.html Hugo. -- Hugo Mills | Great films about cricket: Monster's No-Ball hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: DRDY errors are not consistent with scrub results

2018-08-29 Thread Hugo Mills
quite a bit of time! Hah. Only a day? It's up to 2 days now. The devices get bigger. The interfaces don't get faster at the same rate. Back in the late '90s, it was only an hour or so to run a badblocks pass on a big disk... Hugo. -- Hugo Mills | Nostalgia isn't

Re: BTRFS and databases

2018-08-01 Thread Hugo Mills
the old and > reliable ext4 for those applications? > > > Kind regards, > MegaBrutal -- Hugo Mills | In theory, theory and practice are the same. In hugo@... carfax.org.uk | practice, they're different. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: btrfs filesystem corruptions with 4.18. git kernels

2018-07-20 Thread Hugo Mills
ly in btrfs. It could be a bug in some other kernel component where it does some pointer arithmetic wrong, or uses some uninitialised data as a pointer. My money's is on bad RAM, though (by a small margin). Hugo. -- Hugo Mills | Stick them with the pointy end. hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | Jon Snow signature.asc Description: Digital signature

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-20 Thread Hugo Mills
itely got into /dev/sdac territory. I don't recall what RAID level they were using. I think it was either RAID-1 or -10. That's the largest I can recall seeing mention of, though. Hugo. -- Hugo Mills | Have found Lost City of Atlantis. High Priest is hugo@... carfax.org.uk | winn

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-18 Thread Hugo Mills
ends up working most of the time with equivalent > sized devices).) Well, it's an *accurate* observation. It's just not a particularly *useful* one. :) Hugo. -- Hugo Mills | I gave up smoking, drinking and sex once. It was the hugo@... carfax.org.uk | scariest 20 minutes of my life. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: [PATCH 0/4] 3- and 4- copy RAID1

2018-07-15 Thread Hugo Mills
. The upper-case version tends to make the letters and numbers merge into each other. With lower-case c, s, p, the taller digits (or M) stand out: 1c 1cMs2p 2c3s8p (OK, just kidding about this one) Hugo. -- Hugo Mills | The English language has the mot juste for every hugo@...

Re: unsolvable technical issues?

2018-06-25 Thread Hugo Mills
erhaps even > media as well. I started doing this a couple of years ago, but it turned out to be impossible to keep even vaguely accurate or up to date, without going round and bugging the developers individually on a per-release basis. I don't think it's going to happen. Hugo. -- Hugo Mills

Re: [PATCH] btrfs: Add more details while checking tree block

2018-06-22 Thread Hugo Mills
;>ret = -EIO; > >>goto err; > >>} > >> @@ -628,8 +628,8 @@ static int btree_readpage_end_io_hook(struct > >> btrfs_io_bio *io_bio, > >>} > >>found_level = btrfs_header_level(eb); > >>if (found_lev

Re: About more loose parameter sequence requirement

2018-06-18 Thread Hugo Mills
oose order inside > > subcomand while still can distinguish global option, will it be accepted > > (if it's quality is acceptable) ? > > I think it's not worth updating the parser just to support an IMHO > narrow usecase. -- Hugo Mills | Turning, pages turning in t

Re: status page

2018-04-25 Thread Hugo Mills
;stable" in a way that's actually measurable to get any useful answer, and even then, see my previous comment about ETAs. There have been example patches in the last few months on the subject of closing the write hole, so there's clear ongoing work on that particular item, but again, see the

Re: Recovery from full metadata with all device space consumed?

2018-04-19 Thread Hugo Mills
On Thu, Apr 19, 2018 at 04:12:39PM -0700, Drew Bloechl wrote: > On Thu, Apr 19, 2018 at 10:43:57PM +0000, Hugo Mills wrote: > >Given that both data and metadata levels here require paired > > chunks, try adding _two_ temporary devices so that it can allocate a > > new bl

Re: Recovery from full metadata with all device space consumed?

2018-04-19 Thread Hugo Mills
other steps that can be tried to rescue a > filesystem in this state? I still have it mounted in the same state, and > I'm willing to try other things or extract debugging info. > > Question 2: Is there something I could have done to prevent this from > happening in

Re: [wiki] Please clarify how to check whether barriers are properly implemented in hardware

2018-04-02 Thread Hugo Mills
Hugo. > https://btrfs.wiki.kernel.org/index.php/FAQ#How_does_this_happen.3F -- Hugo Mills | "Damn and blast British Telecom!" said Dirk, hugo@... carfax.org.uk | the words coming easily from force of habit. http://carfax.org.uk/ |

Re: Out of space and incorrect size reported

2018-03-21 Thread Hugo Mills
m and Metadata overhead)? Most likely, you need to ugrade your kernel to get past the known bug (fixed in about 4.6 or so, if I recall correctly), and then mount with -o clear_cache to force the free space cache to be rebuilt. Hugo. -- Hugo Mills | Q: What goes, "Pieces of

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

2018-03-19 Thread Hugo Mills
On Mon, Mar 19, 2018 at 02:02:23PM +0100, David Sterba wrote: > On Mon, Mar 19, 2018 at 08:20:10AM +0000, Hugo Mills wrote: > > On Mon, Mar 19, 2018 at 05:16:42PM +0900, Misono, Tomohiro wrote: > > > Currently, the top-level subvolume lacks the UUID. As a result, both > >

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

2018-03-19 Thread Hugo Mills
int ret; > > ret = btrfs_copy_root(trans, root, root->node, , objectid); > @@ -325,6 +326,8 @@ static int create_tree(struct btrfs_trans_handle *trans, > btrfs_set_root_bytenr(_item, tmp->start); > btrfs_set_root_level(_item, btrfs_header_level(tmp)); > btrf

Re: [PATCH] Improve error stats message

2018-03-07 Thread Hugo Mills
On Wed, Mar 07, 2018 at 08:02:51PM +0100, Diego wrote: > El miércoles, 7 de marzo de 2018 19:24:53 (CET) Hugo Mills escribió: > >On multi-device filesystems, the two are not necessarily the same. > > Ouch. FWIW, I was moved to do this because I saw this conversation on >

Re: [PATCH] Improve error stats message

2018-03-07 Thread Hugo Mills
gt; -rcu_str_deref(dev->name), > + btrfs_err_rl(dev->fs_info, > + "errors: write %u, read %u, flush %u, corrupt %u, generation > %u", > btrfs_dev_stat_read(dev, BTRFS_DEV_STAT_WRITE_ERRS), >

Re: btrfs send/receive in reverse possible?

2018-02-16 Thread Hugo Mills
sg44089.html Hugo. -- Hugo Mills | I can't foretell the future, I just work there. hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 |The Doctor signature.asc Description: Digital signature

Re: Metadata / Data on Heterogeneous Media

2018-02-15 Thread Hugo Mills
rk which should cover a wide range of use-cases: https://www.spinics.net/lists/linux-btrfs/msg33916.html I never got round to implementing it, though -- I ran into issues over storing the properties/metadata needed to configure it. Hugo. -- Hugo Mills | Dullest spy film ever: The

Re: [PATCH 00/26] btrfs-progs: introduce libbtrfsutil, "btrfs-progs as a library"

2018-01-26 Thread Hugo Mills
il/python/error.c > create mode 100644 libbtrfsutil/python/filesystem.c > create mode 100644 libbtrfsutil/python/module.c > create mode 100644 libbtrfsutil/python/qgroup.c > create mode 100755 libbtrfsutil/python/setup.py > create mode 100644 libbtrfsutil/python/subvolum

Re: bad key ordering - repairable?

2018-01-22 Thread Hugo Mills
repairable, it would be nice if this kind of > corruption could be repaired in the future, even if losing a few > files. Or if the corruptions could be avoided in the first place. Given that the current tools crash, the answer's a definite no. However, if you can get a developer interested

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Hugo Mills
instead of device id. > >>>>>> > >>>>>>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I > >>>>>>> do the following: > >>>>>>> convert inode struct to btrfs_inode struct (use btrfsI

Re: Recommendations for balancing as part of regular maintenance?

2018-01-08 Thread Hugo Mills
being allocated when there's theoretically loads of space available to use (but which it may not be practical to use, due to the fragmented free space). After 4.13 (or 4.14), the "ssd" mount option has been fixed, and it no longer has the bad long-term effects that we've seen before, but it won't deal with the existing fragmented free space without a data balance. If you're running an older kernel, it's definitely recommended to mount all filesystems with "nossd" to avoid these issues. Hugo. -- Hugo Mills | As long as you're getting different error messages, hugo@... carfax.org.uk | you're making progress. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: [PATCH 1/2] btrfs-progs: Fix progs_extra build dependencies

2017-12-23 Thread Hugo Mills
ct-dump-super.o: No such file or directory > Makefile:389: recipe for target 'btrfs-show-super' failed > > Signed-off-by: Hans van Kranenburg <h...@knorrie.org> Hans and I worked this one out between us on IRC. Not sure if you need this, but here it is: Signed-off-by:

Re: broken btrfs filesystem

2017-12-12 Thread Hugo Mills
On Tue, Dec 12, 2017 at 04:18:09PM +, Neal Becker wrote: > Is it possible to check while it is mounted? Certainly not while mounted read-write. While mounted read-only -- I'm not certain. Possibly. Hugo. > On Tue, Dec 12, 2017 at 9:52 AM Hugo Mills <h...@carfax.org.

Re: broken btrfs filesystem

2017-12-12 Thread Hugo Mills
mesg.log is here: > https://nbecker.fedorapeople.org/dmesg.txt > > mount | grep btrfs > /dev/sda3 on / type btrfs > (rw,relatime,seclabel,ssd,space_cache,subvolid=257,subvol=/root) > /dev/sda3 on /home type btrfs > (rw,relatime,seclabel,ssd,space_cache,subvolid=318,subvol=/hom

Re: Odd behaviour of replace -- unknown resulting state

2017-12-09 Thread Hugo Mills
On Sat, Dec 09, 2017 at 05:43:48PM +, Hugo Mills wrote: >This is on 4.10, so there may have been fixes made to this since > then. If so, apologies for the noise. > >I had a filesystem on 6 devices with a badly failing drive in it > (/dev/sdi). I replaced the drive

Odd behaviour of replace -- unknown resulting state

2017-12-09 Thread Hugo Mills
of any issues I might have if I leave it in this state for a while? Likewise, any issues I might have from a reboot? (Probably into 4.14) Hugo. (*) as an aside, it was reporting over 300% complete when it finally completed. Not sure if that's been fixed since 4.10, either. -- Hugo Mills

Re: [RFC] Improve subvolume usability for a normal user

2017-12-07 Thread Hugo Mills
increase the creation time, and btrfs is > >currently optimized for fast creation time. > > > >Hmm... What about creating a "temporary" snapshot if not root, then > >walking the tree to check perms and deleting it without ever showing it > >to userspace if the pe

Re: exclusive subvolume space missing

2017-12-01 Thread Hugo Mills
same amount differs in total: 28.75-21.50=7.25 GiB. > And the same happens with other snapshots, much more exclusive data > shown in qgroup than actually found in files. So if not files, where > is that space wasted? Metadata? Personally, I'd trust qgroups' output about as far as I coul

Re: btrfs-image hash collision option, super slow

2017-11-11 Thread Hugo Mills
that the resulting value isn't mappable to an ASCII pre-image, or that the search just gives up before finding one. Hugo. -- Hugo Mills | Yes, this is an example of something that becomes hugo@... carfax.org.uk | less explosive as a

Re: Problem with file system

2017-11-08 Thread Hugo Mills
since a long time, I can't > even think how far back, maybe even before 3.0. Whereas raid56 got it > in 4.12. Yes, I'm pretty sure it's been like that ever since I've been using btrfs (somewhere around the early neolithic). Hugo. -- Hugo Mills | Turning, pages turning i

Re: Seeking Help on Corruption Issues

2017-10-04 Thread Hugo Mills
On Tue, Oct 03, 2017 at 03:49:25PM -0700, Stephen Nesbitt wrote: > > On 10/3/2017 2:11 PM, Hugo Mills wrote: > >Hi, Stephen, > > > >On Tue, Oct 03, 2017 at 08:52:04PM +, Stephen Nesbitt wrote: > >>Here it i. There are a couple of out-of-order entries be

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Hugo Mills
uld be fixable by btrfs check (I think). However, even fixing that, it's not ordered, because 118 is then before 117, which could be another bitflip ("9" -> "4" in the 7th digit), but two bad bits that close to each other seems unlikely to me. Hugo. -- Hugo Mills

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Hugo Mills
o go through the effort of a bare metal > reinstall. > > In the process of researching this I did uncover a bad DIMM. Am I > correct that the problems I'm seeing are likely linked to the > resulting memory errors. > > Thx in advance, > > -steve > -- Hugo Mills | Quidquid latine dictum sit, altum videtur hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Lost about 3TB

2017-10-03 Thread Hugo Mills
r the whole FS. I'm not 100% sure about what defrag will do in this case, and there are some people round here who have investigated the behaviour of partially-overwritten extents in more detail than I have. Hugo. > Fred. > > > - Mail original - > De: "Hugo Mills - h..

Re: Lost about 3TB

2017-10-03 Thread Hugo Mills
: > file data blocks allocated: 183716661284864 ?? what's this ?? > referenced 30095956975616 = 27.3 TB !! > > > > I also used the verbose option of https://github.com/knorrie/btrfs-heatmap/ > to sum up the total size of all DATA EXTENT and found 32TB. > > I did scrub, balance up to -dusage=90 (and also dusage=0) and ended up with > 32TB used. > No snasphots nor subvolumes nor TB hidden under the mount point after > unmounting the BTRFS volume > > > What did I do wrong or am I missing ? > > Thanks in advance. > Frederic Larive. > -- Hugo Mills | Beware geeks bearing GIFs hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: [PATCH v2] btrfs-progs: subvol: change subvol set-default to also accept subvol path

2017-10-02 Thread Hugo Mills
gt; + fullpath = realpath(path, NULL); > > + if (!fullpath) { > > + error("cannot find real path for '%s': %s", > > + path, strerror(errno)); > > + return 1; > > + } > > + > > + ret = get_subvol_info(fullpath, ); > > + free(fullpath); > > + > > + if (ret) > > + return 1; > > + > > + objectid = ri.root_id; > > + } else { > > + /* subvol id and path to the filesystem are specified */ > > + subvolid = argv[optind]; > > + path = argv[optind + 1]; > > + objectid = arg_strtou64(subvolid); > > + } > > > > fd = btrfs_open_dir(path, , 1); > > if (fd < 0) -- Hugo Mills | Great oxymorons of the world, no. 4: hugo@... carfax.org.uk | Future Perfect http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

2017-09-25 Thread Hugo Mills
On Mon, Sep 25, 2017 at 04:04:03PM +0800, Qu Wenruo wrote: > > > On 2017年09月25日 15:52, Hugo Mills wrote: > >On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: > >> > >> > >>On 2017年09月25日 15:42, Marat Khalili wrote: > >>>On 25/09/17 1

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

2017-09-25 Thread Hugo Mills
o be a real mess). It's the fact that a successful run of the command produces noise on stdout, which most commands don't. Hugo. > Thanks, > Qu > > >If you change the message a lot of scripts will have to be > >changed, at least make it worth it. > > > &g

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

2017-09-25 Thread Hugo Mills
st make it worth it. Seconded. Make sure the return code reflects the result, and drop the printed message (or keep it if there's a --verbose flag, maybe). Hugo. -- Hugo Mills | If you see something, say nothing and drink to hugo@... carfax.org.uk | forget http://carfax.org

Re: Does btrfs use crc32 for error correction?

2017-09-19 Thread Hugo Mills
create image, create text file, flip bit, try > read and btrfs show IO-error) > > Thanks! -- Hugo Mills | Dullest spy film ever: The Eastbourne Ultimatum hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | T

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-15 Thread Hugo Mills
On Fri, Sep 15, 2017 at 08:04:35AM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 12:18 AM, Hugo Mills wrote: > >As far as I know, both of these are basically known issues, with no > > good solution, other than not using O_DIRECT. Certainly the first > > issue is one I

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-14 Thread Hugo Mills
> > ssize_t r = pwrite(fd, buffer, FILESIZE, 0); > assert(r == FILESIZE); > > pid_t child; > > child = fork(); > assert(child >= 0); > if (child == 0) > write_thread(); > fprintf(stderr, "w

Re: snapshots of encrypted directories?

2017-09-14 Thread Hugo Mills
eally_ want a system where the encrypted contents of a subvolume can be decrypted by simply snapshotting it? Hugo. -- Hugo Mills | Great films about cricket: Umpire of the Rising Sun hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: generic name for volume and subvolume root?

2017-09-09 Thread Hugo Mills
me" or "subvolume" at all. Yes, it's a filesystem. (Although that does occasionally cause confusion between "the conceptual filesystem implemented by btrfs.ko" and "the concrete filesystem stored on /dev/sda1", but it's generally far less confusing t

Re: generic name for volume and subvolume root?

2017-09-09 Thread Hugo Mills
f you can snapshot it, it's a subvolume. Some subvolumes are also snapshots. (And all snapshots are subvolumes). The subvolume with ID 5 (or ID 0, which is an alias) is the "top level subvolume", and has the unique property that it can't be renamed, deleted or replaced, where all

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Hugo Mills
On Fri, Sep 08, 2017 at 05:12:11PM +0100, Tomasz Kłoczko wrote: > On 8 September 2017 at 16:38, Hugo Mills <h...@carfax.org.uk> wrote: > [..] > >> sometimes I'm really thinking about start rewrite btrfs-progs to make > >> btrfs basic tools syntax as similar as

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Hugo Mills
; Maybe someone already started doing this? The main complaint that can be directed at the btrfs command is that its output is rarely machine-processable. It would therefore make sense to have a "--table" or "--structured" mode for output, which would be more trivially parsab

Re: Is autodefrag recommended? -- re-duplication???

2017-09-05 Thread Hugo Mills
KB of additional space (+metadata) will be allocated each night > without autodefrag. With autodefrag will it be perhaps 4KB+128KB or > something much worse? I'm going for 132 KiB (4+128). Of course, if there's two 4 KiB writes close together, then there's less overhead, as they'll sha

Re: Is autodefrag recommended?

2017-09-04 Thread Hugo Mills
increasing the amount of data written. Hugo. -- Hugo Mills | The future isn't what it used to be. hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: joining to contribute

2017-09-01 Thread Hugo Mills
documentation I'd also point out the Data Structures and Trees pages linked there. Some of the information is a bit out of date, or represents a prototype of what it's describing. The source code is canonical -- use the documentation as a guide to help you see where in the source to look.

Re: BTRFS critical (device sda2): corrupt leaf, bad key order: block=293438636032, root=1, slot=11

2017-08-31 Thread Hugo Mills
--- > Eric Wolf > (201) 316-6098 > 19w...@gmail.com > > > On Thu, Aug 31, 2017 at 2:59 PM, Hugo Mills <h...@carfax.org.uk> wrote: > >(Please don't top-post; edited for conversation flow) > > > > On Thu, Aug 31, 2017 at 02:44:39PM -0400, Eric Wolf w

Re: BTRFS critical (device sda2): corrupt leaf, bad key order: block=293438636032, root=1, slot=11

2017-08-31 Thread Hugo Mills
(Please don't top-post; edited for conversation flow) On Thu, Aug 31, 2017 at 02:44:39PM -0400, Eric Wolf wrote: > On Thu, Aug 31, 2017 at 2:33 PM, Hugo Mills <h...@carfax.org.uk> wrote: > > On Thu, Aug 31, 2017 at 01:53:58PM -0400, Eric Wolf wrote: > >> I'm having

Re: BTRFS critical (device sda2): corrupt leaf, bad key order: block=293438636032, root=1, slot=11

2017-08-31 Thread Hugo Mills
might also be marginal power regulation (blown capacitor somewhere) or a slightly broken CPU. Can you show us the output of "btrfs-debug-tree -b 293438636032 /dev/sda2"? Hugo. -- Hugo Mills | "You got very nice eyes, Dee

Re: deleted subvols don't go away?

2017-08-28 Thread Hugo Mills
This could also explain the more recent than expected generation values. Hugo. -- Hugo Mills | "Big data" doesn't just mean increasing the font hugo@... carfax.org.uk | size. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Hugo Mills
d Available Use% Mounted on - -117220284 95271880 18611032 84% /home/hrm/foo hrm@amelia:~ $ sudo mkdir foo/bar hrm@amelia:~ $ df -T foo/bar Filesystem Type 1K-blocks Used Available Use% Mounted on - -117220284 95271852 18611060 84% /home/hrm/fo

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Hugo Mills
mount options, plus the original source for the mount (which will be a block device for most filesystem mounts, a path for bind mounts, or something FS-specific for network filesystems). Hugo. -- Hugo Mills | And what rough beast, its hour come round at last / hugo@... carfax.o

Re: Btrfs data recovery

2017-08-13 Thread Hugo Mills
st is discoverable). That said, recovering this is going to be somewhere between very hard and miraculous. Hugo. -- Hugo Mills | But somewhere along the line, it seems hugo@... carfax.org.uk | That pimp became cool, and punk mainstream. http

Re: btrfs issue with mariadb incremental backup

2017-08-12 Thread Hugo Mills
rsync mismaches > mean the destination snapshots are bad. Here's what I would do: I don't see how that can happen. Snapshots are atomic -- they're either there or not there. It's not a matter even of copying the metadata part of the subvol. It's literally just adding a pointer to point a

Re: RedHat 7.4 Release Notes: "Btrfs has been deprecated" - wut?

2017-08-12 Thread Hugo Mills
ause the machine was interrupted during write (in which case you can fix it), or is it because the hard disk has had someone write data to it directly, and the data is now toast (in which case you shouldn't fix the I/O error)? Basically, nodatacow bypasses the very mechanisms that are meant to

Re: [PATCH v5 2/5] lib: Add zstd modules

2017-08-10 Thread Hugo Mills
the mount option part of this cycle. In the long run, it'd be great to see most of the btrfs-specific mount options get deprecated and ultimately removed entirely, in favour of attributes/properties, where feasible. Hugo. -- Hugo Mills | Klytus! Are your men on the righ

Re: [PATCH v3] Btrfs: fix out of bounds array access while reading extent buffer

2017-08-09 Thread Hugo Mills
end_i) > return 1; > > @@ -5502,12 +5512,6 @@ int map_private_extent_buffer(struct extent_buffer > *eb, unsigned long start, > *map_start = ((u64)i << PAGE_SHIFT) - start_offset; > } > > - if (start + min_len > eb->len) { > -

Re: Crashed filesystem, nothing helps

2017-08-02 Thread Hugo Mills
ely, it's definitely worth trying to mount with the -o usebackuproot option. (And -o usebackuproot,ro as well). The transid verify failure is a small difference in generations, and it's likely that the older metadata is still there. If that's the case, the mount with usebac

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2017-08-01 Thread Hugo Mills
On Tue, Aug 01, 2017 at 10:56:39AM -0600, Liu Bo wrote: > On Tue, Aug 01, 2017 at 05:28:57PM +0000, Hugo Mills wrote: > >Hi, > > > >Great to see something addressing the write hole at last. > > > > On Tue, Aug 01, 2017 at 10:14:23AM -0600, Liu Bo wrote:

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2017-08-01 Thread Hugo Mills
ransaction.c |2 + > fs/btrfs/volumes.c | 56 +- > fs/btrfs/volumes.h |7 +- > include/uapi/linux/btrfs.h |3 + > include/uapi/linux/btrfs_tree.h |4 + > 10 files changed, 1487 insertions(+), 176 deletions(-) > -- Hugo

Re: Massive loss of disk space

2017-08-01 Thread Hugo Mills
d: 5073330888704 > referenced 5052040339456 > > > > pwm@europium:~$ sudo btrfs scrub status /mnt/snap_04/ > scrub status for c46df8fa-03db-4b32-8beb-5521d9931a31 > scrub started at Mon Jul 31 21:26:50 2017 and finished after > 06:53:47 > total bytes scrubb

Re: Btrfs + compression = slow performance and high cpu usage

2017-07-28 Thread Hugo Mills
en turning on autodefrag will make things worse (but you may be heading for _much_ worse performance in the future as the FS becomes more fragmented -- depending on your write patterns and use case). Hugo. -- Hugo Mills | Great oxymorons of the world, no. 6: hugo@... carfax.or

Re: Raid0 rescue

2017-07-27 Thread Hugo Mills
2, have=269659807399918462 > total bytes 5000989728768 > bytes used 3400345264128 > > > > On Thu, Jul 27, 2017 at 11:10 AM, Hugo Mills <h...@carfax.org.uk> wrote: > > On Thu, Jul 27, 2017 at 10:49:37AM -0400, Alan Brand wrote: > >> I know I am screwed but ho

Re: Raid0 rescue

2017-07-27 Thread Hugo Mills
ta configuration on this FS? Also RAID-0? or RAID-1? > I can't run a normal recovery as only half of each file is there. Welcome to RAID-0... Hugo. -- Hugo Mills | We don't just borrow words; on occasion, English has hugo@... carfax.org.uk | pursued other langua

Re: btrfs raid assurance

2017-07-26 Thread Hugo Mills
On Wed, Jul 26, 2017 at 08:36:54AM -0400, Austin S. Hemmelgarn wrote: > On 2017-07-26 08:27, Hugo Mills wrote: > >On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote: > >>On 2017-07-25 17:45, Hugo Mills wrote: > >>>On Tue, Jul 25, 2017 at 1

Re: btrfs raid assurance

2017-07-26 Thread Hugo Mills
On Wed, Jul 26, 2017 at 12:27:20PM +, Hugo Mills wrote: > On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote: > > On 2017-07-25 17:45, Hugo Mills wrote: > > >On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote: > > >>

Re: btrfs raid assurance

2017-07-26 Thread Hugo Mills
On Wed, Jul 26, 2017 at 08:12:19AM -0400, Austin S. Hemmelgarn wrote: > On 2017-07-25 17:45, Hugo Mills wrote: > >On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote: > >> > >> > >>Hugo Mills wrote: > >>> > >>>>>You can see

Re: btrfs raid assurance

2017-07-25 Thread Hugo Mills
On Tue, Jul 25, 2017 at 11:29:13PM +0200, waxhead wrote: > > > Hugo Mills wrote: > > > >>>You can see about the disk usage in different scenarios with the > >>>online tool at: > >>> > >>>http://carfax.org.uk/btrfs-usage/ > &

Re: btrfs raid assurance

2017-07-25 Thread Hugo Mills
may be a bit wobbly. :) > 2017-07-25 10:51 GMT-03:00 Hugo Mills <h...@carfax.org.uk>: > > On Tue, Jul 25, 2017 at 01:46:56PM +, Hugo Mills wrote: > >> On Tue, Jul 25, 2017 at 09:55:37AM -0300, Hérikz Nawarro wrote: > >> > Hello everyone, > >> > >

Re: btrfs raid assurance

2017-07-25 Thread Hugo Mills
On Tue, Jul 25, 2017 at 01:46:56PM +, Hugo Mills wrote: > On Tue, Jul 25, 2017 at 09:55:37AM -0300, Hérikz Nawarro wrote: > > Hello everyone, > > > > I'm migrating to btrfs and i would like to know, in a btrfs filesystem > > with 4 disks (multiple sizes) with -d

Re: btrfs raid assurance

2017-07-25 Thread Hugo Mills
ll be OK). Hugo. -- Hugo Mills | One of these days, I'll catch that man without a hugo@... carfax.org.uk | quotation, and he'll look undressed. http://carfax.org.uk/ | PGP: E2AB1DE4 | Leto Atreides, Dune signature.asc Description: Digital signature

Re: Best Practice: Add new device to RAID1 pool

2017-07-24 Thread Hugo Mills
On Mon, Jul 24, 2017 at 02:55:00PM -0600, Chris Murphy wrote: > On Mon, Jul 24, 2017 at 2:42 PM, Hugo Mills <h...@carfax.org.uk> wrote: > > > > >In my experience, it's pretty consistent at about a minute per 1 > > GiB for data on rotational drives on RAID-1.

Re: Best Practice: Add new device to RAID1 pool

2017-07-24 Thread Hugo Mills
evels: 2 > [chris@f26s ~]$ > > > I don't think the number of snapshots you have for Docker containers > is the problem. There's this thread (admittedly on SSD) which suggests > decent performance is possible with thousands of containers per day > (100,000 - 200,000 per day but I don't think that's per file system, > I'm actually not sure how many file systems are involved). > > https://www.spinics.net/lists/linux-btrfs/msg67308.html > > > -- Hugo Mills | Two things came out of Berkeley in the 1960s: LSD hugo@... carfax.org.uk | and Unix. This is not a coincidence. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: btrfs device ready purpose

2017-07-22 Thread Hugo Mills
a bit, some years ago. I don't recall now why it was going to be useful, though. I think you have a good use-case for such a new ioctl (or extension to the SCAN_DEV ioctl) now, though. Hugo. -- Hugo Mills | UNIX: Italian pen maker hugo@... carfax.org.uk | http://

Re: interrupt btrfs filesystem defragment -r /

2017-07-08 Thread Hugo Mills
On Sat, Jul 08, 2017 at 01:34:44PM +0200, David Arendt wrote: > Hi, > > Is it safe to interrupt a btrfs filesystem defrag -r / by using ctrl-c > or should it be avoided ? Yes, it's safe. Hugo. -- Hugo Mills | Klytus, I'm bored. What plaything can you o

Re: Exactly what is wrong with RAID5/6

2017-06-20 Thread Hugo Mills
y number of devices (>=4), not just an even number. Obviously, if you have a 6-device array and lose one, you'll need to deal with the loss of redundancy -- either add a new device and rebalance, or replace the missing device with a new one, or (space permitting) rebalance with existing devices

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

2017-06-20 Thread Hugo Mills
On Tue, Jun 20, 2017 at 08:26:48AM -0700, Marc MERLIN wrote: > On Tue, Jun 20, 2017 at 03:23:54PM +0000, Hugo Mills wrote: > > On Tue, Jun 20, 2017 at 07:39:16AM -0700, Marc MERLIN wrote: > > > My filesystem got remounted read only, and yet after a lengthy > > > btrfs c

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

2017-06-20 Thread Hugo Mills
parent transid verify failed on > 1932065177600 wanted 37959 found 3634 > [863279.979033] BTRFS error (device dm-1): parent transid verify failed on > 1932065177600 wanted 37959 found 3634 > [863280.009362] BTRFS error (device dm-1): parent transid verify failed on > 1932065177600

Re: does using different uid/gid/forceuid/... mount options for different subvolumes work / does fuse.bindfs play nice with btrfs?

2017-06-20 Thread Hugo Mills
r this in btrfs (or any other POSIX filesystem), and there are no plans to implement such a feature. > If there is no magical btrfs way, does fuse.bindfs play nice with > btrfs or should I be worried? We haven't had any complaints about it that I'm aware of. Hugo. -- Hugo Mills

Re: Filesystem won't mount (open_ctree failed) or repair (BUG_ON)

2017-06-09 Thread Hugo Mills
id6) doesn't report any errors, trying different > kernels makes no difference, 4.10.17, 4.11.4 and 4.12.0-rc4 all give the same > errors. Everything above was > done with btrfs-progs 4.11. > > Any hints on how I can fix the errors in the filesystem? I don't mind loosing &g

Re: btrfs native encryption

2017-06-09 Thread Hugo Mills
come was). So, there's some interest in a fairly minimal implementation, but progress doesn't seem to be particularly fast. Hugo. -- Hugo Mills | "Are you the man who rules the Universe?" "Well, I hugo@... carfax.org.uk | try not to." http://carfax.org.uk/

Re: BTRFS converted from EXT4 becomes read-only after reboot

2017-05-23 Thread Hugo Mills
t disk and done a fresh btrfs install on it, because I > had to get some work done :) > > Marc -- Hugo Mills | Essex: a branch of philothophy. hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

2017-05-19 Thread Hugo Mills
On Fri, May 19, 2017 at 06:25:22PM -0700, Marc MERLIN wrote: > On Sat, May 20, 2017 at 12:57:09AM +0000, Hugo Mills wrote: > >I think from the POV of removing these BUG_ONs, it doesn't matter > > which FS causes them. "All" you need to know is where the error > >

Re: 4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

2017-05-19 Thread Hugo Mills
On Fri, May 19, 2017 at 05:47:48PM -0700, Marc MERLIN wrote: > On Sat, May 20, 2017 at 12:37:47AM +0000, Hugo Mills wrote: > > > Can I make another plea for just removing all those BUG/BUG_ON? > > > They really have no place in production code, there is no excuse for a >

  1   2   3   4   5   6   7   8   9   10   >