Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2017-08-01 Thread Hugo Mills
| 82 +++ > fs/btrfs/transaction.c |2 + > fs/btrfs/volumes.c | 56 +- > fs/btrfs/volumes.h |7 +- > include/uapi/linux/btrfs.h |3 + > include/uapi/linux/btrfs_tree.h |4 + > 10 files changed, 1487 insertio

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2017-08-01 Thread Hugo Mills
On Tue, Aug 01, 2017 at 10:56:39AM -0600, Liu Bo wrote: > On Tue, Aug 01, 2017 at 05:28:57PM +0000, Hugo Mills wrote: > >Hi, > > > >Great to see something addressing the write hole at last. > > > > On Tue, Aug 01, 2017 at 10:14:23AM -0600, Liu Bo wrote:

Re: Crashed filesystem, nothing helps

2017-08-02 Thread Hugo Mills
. More productively, it's definitely worth trying to mount with the -o usebackuproot option. (And -o usebackuproot,ro as well). The transid verify failure is a small difference in generations, and it's likely that the older metadata is still there. If that

Re: [PATCH v3] Btrfs: fix out of bounds array access while reading extent buffer

2017-08-09 Thread Hugo Mills
turn 1; > > @@ -5502,12 +5512,6 @@ int map_private_extent_buffer(struct extent_buffer > *eb, unsigned long start, > *map_start = ((u64)i << PAGE_SHIFT) - start_offset; > } > > - if (start + min_len > eb->len) { > - WARN

Re: [PATCH v5 2/5] lib: Add zstd modules

2017-08-10 Thread Hugo Mills
trfs properties this time round, and cut out the mount option part of this cycle. In the long run, it'd be great to see most of the btrfs-specific mount options get deprecated and ultimately removed entirely, in favour of attributes/properties, where feasible. Hugo. -- Hugo Mi

Re: RedHat 7.4 Release Notes: "Btrfs has been deprecated" - wut?

2017-08-12 Thread Hugo Mills
when the data and csum don't match, you need to know the _reason_ they don't match -- is it because the machine was interrupted during write (in which case you can fix it), or is it because the hard disk has had someone write data to it directly, and the data is now toast (in which case y

Re: btrfs issue with mariadb incremental backup

2017-08-12 Thread Hugo Mills
ches > mean the destination snapshots are bad. Here's what I would do: I don't see how that can happen. Snapshots are atomic -- they're either there or not there. It's not a matter even of copying the metadata part of the subvol. It's literally just adding a pointer to

Re: Btrfs data recovery

2017-08-13 Thread Hugo Mills
dn't be hard to find that if the rest is discoverable). That said, recovering this is going to be somewhere between very hard and miraculous. Hugo. -- Hugo Mills | But somewhere along the line, it seems hugo@... carfax.org.uk | That pim

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Hugo Mills
lied mount options, plus the original source for the mount (which will be a block device for most filesystem mounts, a path for bind mounts, or something FS-specific for network filesystems). Hugo. -- Hugo Mills | And what rough beast, its hour come round at last / hugo@... car

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Hugo Mills
Type 1K-blocks Used Available Use% Mounted on - -117220284 95271880 18611032 84% /home/hrm/foo hrm@amelia:~ $ sudo mkdir foo/bar hrm@amelia:~ $ df -T foo/bar Filesystem Type 1K-blocks Used Available Use% Mounted on - -117220284 95271852 1

Re: deleted subvols don't go away?

2017-08-28 Thread Hugo Mills
as deleting a file. This could also explain the more recent than expected generation values. Hugo. -- Hugo Mills | "Big data" doesn't just mean increasing the font hugo@... carfax.org.uk | size. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: BTRFS critical (device sda2): corrupt leaf, bad key order: block=293438636032, root=1, slot=11

2017-08-31 Thread Hugo Mills
but might also be marginal power regulation (blown capacitor somewhere) or a slightly broken CPU. Can you show us the output of "btrfs-debug-tree -b 293438636032 /dev/sda2"? Hugo. -- Hugo Mills | "You got very nice eyes

Re: BTRFS critical (device sda2): corrupt leaf, bad key order: block=293438636032, root=1, slot=11

2017-08-31 Thread Hugo Mills
(Please don't top-post; edited for conversation flow) On Thu, Aug 31, 2017 at 02:44:39PM -0400, Eric Wolf wrote: > On Thu, Aug 31, 2017 at 2:33 PM, Hugo Mills wrote: > > On Thu, Aug 31, 2017 at 01:53:58PM -0400, Eric Wolf wrote: > >> I'm having issues with a

Re: BTRFS critical (device sda2): corrupt leaf, bad key order: block=293438636032, root=1, slot=11

2017-08-31 Thread Hugo Mills
onvention, rather than a technical thing... Hugo. > --- > Eric Wolf > (201) 316-6098 > 19w...@gmail.com > > > On Thu, Aug 31, 2017 at 2:59 PM, Hugo Mills wrote: > >(Please don't top-post; edited for conversation flow) > > > > On Thu, Aug 31, 2017

Re: joining to contribute

2017-09-01 Thread Hugo Mills
d, more generally: https://btrfs.wiki.kernel.org/index.php/Main_Page#Developer_documentation I'd also point out the Data Structures and Trees pages linked there. Some of the information is a bit out of date, or represents a prototype of what it's describing. The source code is canon

Re: Is autodefrag recommended?

2017-09-04 Thread Hugo Mills
autodefrag, because it's increasing the amount of data written. Hugo. -- Hugo Mills | The future isn't what it used to be. hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Is autodefrag recommended? -- re-duplication???

2017-09-05 Thread Hugo Mills
gt; 8KB of additional space (+metadata) will be allocated each night > without autodefrag. With autodefrag will it be perhaps 4KB+128KB or > something much worse? I'm going for 132 KiB (4+128). Of course, if there's two 4 KiB writes close together, then there's less over

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Hugo Mills
lready started doing this? The main complaint that can be directed at the btrfs command is that its output is rarely machine-processable. It would therefore make sense to have a "--table" or "--structured" mode for output, which would be more trivially parsable by sh

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Hugo Mills
On Fri, Sep 08, 2017 at 05:12:11PM +0100, Tomasz Kłoczko wrote: > On 8 September 2017 at 16:38, Hugo Mills wrote: > [..] > >> sometimes I'm really thinking about start rewrite btrfs-progs to make > >> btrfs basic tools syntax as similar as it is only possible t

Re: generic name for volume and subvolume root?

2017-09-09 Thread Hugo Mills
can snapshot it, it's a subvolume. Some subvolumes are also snapshots. (And all snapshots are subvolumes). The subvolume with ID 5 (or ID 0, which is an alias) is the "top level subvolume", and has the unique property that it can't be renamed, deleted or replaced, where a

Re: generic name for volume and subvolume root?

2017-09-09 Thread Hugo Mills
the name "volume" or "subvolume" at all. Yes, it's a filesystem. (Although that does occasionally cause confusion between "the conceptual filesystem implemented by btrfs.ko" and "the concrete filesystem stored on /dev/sda1", but it's

Re: snapshots of encrypted directories?

2017-09-14 Thread Hugo Mills
Would you _really_ want a system where the encrypted contents of a subvolume can be decrypted by simply snapshotting it? Hugo. -- Hugo Mills | Great films about cricket: Umpire of the Rising Sun hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-14 Thread Hugo Mills
> assert(fd>=0); > > ssize_t r = pwrite(fd, buffer, FILESIZE, 0); > assert(r == FILESIZE); > > pid_t child; > > child = fork(); > assert(child >= 0); > if (child == 0) > write_thread(); >

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-15 Thread Hugo Mills
On Fri, Sep 15, 2017 at 08:04:35AM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 12:18 AM, Hugo Mills wrote: > >As far as I know, both of these are basically known issues, with no > > good solution, other than not using O_DIRECT. Certainly the first > > issue is one I

Re: Does btrfs use crc32 for error correction?

2017-09-19 Thread Hugo Mills
ck that (i create image, create text file, flip bit, try > read and btrfs show IO-error) > > Thanks! -- Hugo Mills | Dullest spy film ever: The Eastbourne Ultimatum hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 |

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

2017-09-25 Thread Hugo Mills
changed, at least make it worth it. Seconded. Make sure the return code reflects the result, and drop the printed message (or keep it if there's a --verbose flag, maybe). Hugo. -- Hugo Mills | If you see something, say nothing and drink to hugo@... carfax.org.uk | forg

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

2017-09-25 Thread Hugo Mills
#x27;s problematic (although those used to be a real mess). It's the fact that a successful run of the command produces noise on stdout, which most commands don't. Hugo. > Thanks, > Qu > > >If you change the message a lot of scripts will have to be > &g

Re: [PATCH] btrfs-progs: subvolume: outputs message only when operation succeeds

2017-09-25 Thread Hugo Mills
On Mon, Sep 25, 2017 at 04:04:03PM +0800, Qu Wenruo wrote: > > > On 2017年09月25日 15:52, Hugo Mills wrote: > >On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: > >> > >> > >>On 2017年09月25日 15:42, Marat Khalili wrote: > >>>On 25/09/17 1

Re: [PATCH v2] btrfs-progs: subvol: change subvol set-default to also accept subvol path

2017-10-02 Thread Hugo Mills
llpath = realpath(path, NULL); > > + if (!fullpath) { > > + error("cannot find real path for '%s': %s", > > + path, strerror(errno)); > > + return 1; > > + } > > + > > + ret = get_subvol_info(fullpath, &ri); > > + free(fullpath); > > + > > + if (ret) > > + return 1; > > + > > + objectid = ri.root_id; > > + } else { > > + /* subvol id and path to the filesystem are specified */ > > + subvolid = argv[optind]; > > + path = argv[optind + 1]; > > + objectid = arg_strtou64(subvolid); > > + } > > > > fd = btrfs_open_dir(path, &dirstream, 1); > > if (fd < 0) -- Hugo Mills | Great oxymorons of the world, no. 4: hugo@... carfax.org.uk | Future Perfect http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Lost about 3TB

2017-10-03 Thread Hugo Mills
the meaning of "file data blocks allocated") > Code: > file data blocks allocated: 183716661284864 ?? what's this ?? > referenced 30095956975616 = 27.3 TB !! > > > > I also used the verbose option of https://github.com/knorrie/btrfs-heatmap/ > to sum up the total size of all DATA EXTENT and found 32TB. > > I did scrub, balance up to -dusage=90 (and also dusage=0) and ended up with > 32TB used. > No snasphots nor subvolumes nor TB hidden under the mount point after > unmounting the BTRFS volume > > > What did I do wrong or am I missing ? > > Thanks in advance. > Frederic Larive. > -- Hugo Mills | Beware geeks bearing GIFs hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Lost about 3TB

2017-10-03 Thread Hugo Mills
efore you try it for the whole FS. I'm not 100% sure about what defrag will do in this case, and there are some people round here who have investigated the behaviour of partially-overwritten extents in more detail than I have. Hugo. > Fred. > > > - Mail original - >

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Hugo Mills
n't want to go through the effort of a bare metal > reinstall. > > In the process of researching this I did uncover a bad DIMM. Am I > correct that the problems I'm seeing are likely linked to the > resulting memory errors. > > Thx in advance, > > -steve > -- Hugo Mills | Quidquid latine dictum sit, altum videtur hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Hugo Mills
ex digit, which is a single-bit flip, and should be fixable by btrfs check (I think). However, even fixing that, it's not ordered, because 118 is then before 117, which could be another bitflip ("9" -> "4" in the 7th digit), but two bad bits that close to each other seems un

Re: Seeking Help on Corruption Issues

2017-10-04 Thread Hugo Mills
On Tue, Oct 03, 2017 at 03:49:25PM -0700, Stephen Nesbitt wrote: > > On 10/3/2017 2:11 PM, Hugo Mills wrote: > >Hi, Stephen, > > > >On Tue, Oct 03, 2017 at 08:52:04PM +, Stephen Nesbitt wrote: > >>Here it i. There are a couple of out-of-order entries be

Re: Problem with file system

2017-11-08 Thread Hugo Mills
nce a long time, I can't > even think how far back, maybe even before 3.0. Whereas raid56 got it > in 4.12. Yes, I'm pretty sure it's been like that ever since I've been using btrfs (somewhere around the early neolithic). Hugo. -- Hugo Mills | Turning, pa

Re: btrfs-image hash collision option, super slow

2017-11-11 Thread Hugo Mills
of the orginal filenames hashed to the same value. If that happens, one of the hashes is incremented by a small integer in a predictable way before storage. So it may be that the resulting value isn't mappable to an ASCII pre-image, or that the

Re: btrfs dedup - available or experimental? Or yet to be?

2015-03-23 Thread Hugo Mills
any experiences? duperemove is reported as working. > Is there (or will there be,) a bad penalty of fragmentation? With duperemove, it operates on an extent scale, not at the level of blocks, so the fragmentation isn't so bad. Hugo. -- Hugo Mills | ©1973 Unclear Research Ltd

Re: btrfs balance fails with no space errors (despite having plenty)

2015-03-24 Thread Hugo Mills
h, this smells like the known bug that's been on josef's hitlist for months now. He keeps getting side-tracked by data-corruption bugs, so it's not been fixed yet, unfortunately. The only known solution to date is, as you've found, remaking the FS from scratch and restoring from

Re: How to avoid certain file being defragmented?

2015-04-01 Thread Hugo Mills
If you need some kind of raw(ish) block interface, using a file and a loopback device may be more useful. Hugo. -- Hugo Mills | I get nervous when I see words like 'mayhaps' in a hugo@... carfax.org.uk | novel, because I fear that just round the corner

Re: Is this normal? Should I use scrub?

2015-04-01 Thread Hugo Mills
7;t read or that seems > corrupted. I will now run checksums against my last good backup. > > Should I run a scrub as well? Yes. The output you've had so far will be just the pieces that the FS has tried to read, and where, as a result, it's been able to detect the out-of-date data

Re: Is this normal? Should I use scrub?

2015-04-02 Thread Hugo Mills
On Thu, Apr 02, 2015 at 09:58:39AM +, Andy Smith wrote: > Hi Hugo, > > Thanks for your help. Makes a change from you answering my questions. :) > On Wed, Apr 01, 2015 at 03:42:02PM +, Hugo Mills wrote: > > On Wed, Apr 01, 2015 at 03:11:14PM +, Andy Smith wrote:

Re: Btrfs hangs 3.19-10

2015-04-02 Thread Hugo Mills
_commit_transaction+0x375/0xa40 [btrfs] > [ 240.101021] [] ? wait_woken+0x90/0x90 > [ 240.101037] [] transaction_kthread+0x1dd/0x250 [btrfs] > [ 240.101052] [] ? > btrfs_cleanup_transaction+0x550/0x550 [btrfs] > [ 240.101057] [] kthread+0xc9/0xe0 > [ 240.101061] [] ? kthread_create_on

Re: Btrfs hangs 3.19-10

2015-04-04 Thread Hugo Mills
mkfs.btrfs that would make a filesystem > mountable by kernel 3.2.65? If so I'll file a Debian/Jessie bug report > requesting that a specific mention be added to the man page. Yes, there are. It's probably -O^extref, but if you can show the dmesg output from the 3.2 kernel on the fai

Re: Btrfs hangs 3.19-10

2015-04-05 Thread Hugo Mills
On Sun, Apr 05, 2015 at 03:30:47AM +, Russell Coker wrote: > On Sun, 5 Apr 2015 03:16:21 AM Duncan wrote: > > Hugo Mills posted on Sat, 04 Apr 2015 13:00:47 + as excerpted: > > > On Sat, Apr 04, 2015 at 12:55:08PM +, Russell Coker wrote: > > >> As

Re: Btrfs hangs 3.19-10

2015-04-06 Thread Hugo Mills
On Mon, Apr 06, 2015 at 10:40:03AM +0300, Pavel Volkov wrote: > On Sunday, April 5, 2015 1:04:17 PM MSK, Hugo Mills wrote: > > That's these, I think: > > > >#define BTRFS_FEATURE_INCOMPAT_BIG_METADATA (1ULL << 5) > >#define BTRFS_FEATURE_INCOMPAT_EXTEN

Re: btrfs subvolume & default subvolume

2015-04-07 Thread Hugo Mills
subvol=home 0 0 And finally, add this to your kernel command line in the bootloader to specify how to mount root: rootflags=subvol=root Hugo. -- Hugo Mills | The trouble with you, Ibid, is you think you know hugo@... carfax.org.uk | everything. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: btrfs subvolume & default subvolume

2015-04-07 Thread Hugo Mills
On Tue, Apr 07, 2015 at 01:08:59PM +0200, arnaud gaboury wrote: > On Tue, Apr 7, 2015 at 1:03 PM, Hugo Mills wrote: > > On Tue, Apr 07, 2015 at 12:58:28PM +0200, arnaud gaboury wrote: [snip] > >> After more reading, it seems to me creating a top root subvolume is > >

Re: btrfs subvolume & default subvolume

2015-04-07 Thread Hugo Mills
On Tue, Apr 07, 2015 at 06:31:36PM +0200, arnaud gaboury wrote: > On Tue, Apr 7, 2015 at 4:20 PM, Hugo Mills wrote: > > On Tue, Apr 07, 2015 at 01:08:59PM +0200, arnaud gaboury wrote: > >> On Tue, Apr 7, 2015 at 1:03 PM, Hugo Mills wrote: > >> > On Tue, Apr 07, 20

Re: Are attachments allowed?

2015-04-07 Thread Hugo Mills
-info.html#taboo [2] http://www.tux.org/lkml/#s3 -- Hugo Mills | Alert status mauve ocelot: Slight chance of hugo@... carfax.org.uk | brimstone. Be prepared to make a nice cup of tea. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Re: Porting BTRFS to user space

2015-04-08 Thread Hugo Mills
getting BTRFS running in userspace, so there > probably isn't much BTRFS specific literature out there. > > I would, however suggest looking at the FUSE drivers for ext4 and ZFS, > as those are both ported from kernel space, and should give some good > examples of where to start. > >    -- Hugo Mills | "I lost my leg in 1942. Some bastard stole it in a hugo@... carfax.org.uk | pub in Pimlico." http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: incremental full file backups to smaller mediums possible?

2015-04-09 Thread Hugo Mills
ny ideas whether and how this could be done / assisted by btrfs? btrfs sub find-new might be more helpful to you here. That will give you the list of changed files; then just feed that list to your existing bin-packing algorithm for working out what g

Re: btrfs balance {,meta}data to raid5 not working?

2015-04-11 Thread Hugo Mills
gt; I tried today to balance two drive btrfs raid1 to two drive btrfs raid5 > > -- Hugo Mills | The enemy have elected for Death by Powerpoint. hugo@... carfax.org.uk | That's what they shall get. http://carfax.org.uk/ | PGP: E2AB1DE4 |

Re: Degraded volume silently fails to mount

2015-04-13 Thread Hugo Mills
uname -a > Linux ambrosia.homeslice 3.19.3-200.fc21.x86_64 #1 SMP Thu Mar 26 > 21:39:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux > # btrfs --version > Btrfs v3.18.1 > > > Any ideas? -- Hugo Mills | What's a Nazgûl like you doing in a place like this? hugo@...

Re: how many chunk trees and extent trees present

2015-04-17 Thread Hugo Mills
). Note that if all you want is the count of those blocks (rather than the block numbers themselves), then it's already been done with qgroups, and you don't need to write any btrfs code at all. What exactly are you going to be doing with this information? Hugo. -- Hugo M

Re: The FAQ on fsync/O_SYNC

2015-04-19 Thread Hugo Mills
pshotting and reflink copies. However, after the snapshot, writes to either copy will result in that copy being CoWed. (Specifically, writes to an extent of a +C file with more than one reference to the extent will result in a CoW operation, until there is only one reference, and then the writes

Re: The FAQ on fsync/O_SYNC

2015-04-19 Thread Hugo Mills
On Sun, Apr 19, 2015 at 07:50:32PM +0200, Martin Steigerwald wrote: > Am Sonntag, 19. April 2015, 15:18:51 schrieb Hugo Mills: > > On Sun, Apr 19, 2015 at 05:10:30PM +0200, Martin Steigerwald wrote: > > > Am Sonntag, 19. April 2015, 22:31:02 schrieb Craig Ringer: > > >

Re: The FAQ on fsync/O_SYNC

2015-04-19 Thread Hugo Mills
On Sun, Apr 19, 2015 at 08:41:39PM +0200, Martin Steigerwald wrote: > Am Sonntag, 19. April 2015, 18:18:24 schrieb Hugo Mills: > > On Sun, Apr 19, 2015 at 07:50:32PM +0200, Martin Steigerwald wrote: > > > Am Sonntag, 19. April 2015, 15:18:51 schrieb Hugo Mills: > > > &

Re: unable to mount a btrfs file system kernel 3.19.2

2015-04-20 Thread Hugo Mills
> http://sprunge.us/jPMG > > Is it a known bug? Or does someone have an idea about how I could fix > this issue? See the first issue here: https://btrfs.wiki.kernel.org/index.php/Gotchas Hugo. -- Hugo Mills | Do not meddle in the affairs of system hug

Re: btrfs send to file differences

2015-04-20 Thread Hugo Mills
ion if I want to recover incremental recover i.e out1.img + 1Mb ?? > > Here with my steps I can give only out2.img which is of 6Mb or out1.img > which is of 5Mb. > So for incremetal backup, it consumes total of 11Mb. > I want a way to reduce it to total of 6Mb but there should a way

Re: Help with btrfs filesystem problems

2015-04-22 Thread Hugo Mills
and see how that behaves with this filesystem. Another thing to do would be to get hold of a recent (3.19) set of userspace tools, and run btrfs check --readonly on the filesystem (unmounted), and report back what that says. Hugo. -- Hugo Mills |

Re: rollback of a snapshot is just read-only

2015-04-24 Thread Hugo Mills
ad-only (so the FS will complain). What you should probably be considering is not using set-default, but instead using mount options (subvol=/path) to select the subvolume you want to mount. If you want to make a read-only snapshot into a read-write one, you can simply snapshot it a

Re: [PATCH] btrfs-progs: receive explicit parent support

2015-04-26 Thread Hugo Mills
ath = optarg; > + break; > case 'v': > g_verbose++; > break; > @@ -1028,6 +1044,8 @@ const char * const cmd_receive_usage[] = { > " in the data stream. Wit

Re: raid0 needs mounting alternate device every boot? open_ctree failed (3.19.5)

2015-04-26 Thread Hugo Mills
t btrfs-progs from git installed, then the udev hooks probably aren't there, and you'll need to configure it yourself (or arrange to have btrfs dev scan run after the cryptsetup open operation). Hugo. -- Hugo Mills | You can't expect a boy to be vicious until he&#

Re: btrfs freeze/thaw when using with LVM2

2015-04-27 Thread Hugo Mills
ive filesystem corruption, because the kernel can't distinguish between the original block device and the snapshot, and may try using the wrong one (or possibly both). Hugo. -- Hugo Mills | Someone's been throwing dead sheep down my Fun Well hu

Re: btrfs freeze/thaw when using with LVM2

2015-04-27 Thread Hugo Mills
On Mon, Apr 27, 2015 at 02:41:41PM +, sri wrote: > Hugo Mills carfax.org.uk> writes: > > > > > On Mon, Apr 27, 2015 at 12:05:28PM +, sri wrote: > > > Hi, > > > > > > I would like to know if btrfs file system is created on LVM2 logical

Re: btrfs freeze/thaw when using with LVM2

2015-04-27 Thread Hugo Mills
On Mon, Apr 27, 2015 at 10:34:42PM +0500, Roman Mamedov wrote: > On Mon, 27 Apr 2015 14:55:49 + > Hugo Mills wrote: > > > > This is strange considering that I wanted a consistent snapshot of > > > entire btrfs filesystem at volume level. > > > > >

Re: Filesystem with Errors, Unusual behavior

2015-04-28 Thread Hugo Mills
ba > Total devices 8 FS bytes used 9.18TiB > devid1 size 3.64TiB used 3.15TiB path /dev/sdg1 > devid2 size 3.64TiB used 3.15TiB path /dev/sdi1 > devid3 size 3.64TiB used 3.15TiB path /dev/sde1 > devid4 size 3.64TiB used 3.15TiB path /dev/sdf1 > devid 6 siz

Re: raid 5 and different device size

2015-05-11 Thread Hugo Mills
gt; checked for read operations? Parity is spread evenly across all devices, so a read of more than a trivial quantity of data would almost certainly spin up all devices anyway. Hugo. -- Hugo Mills | A gentleman doesn't do damage unless he's paid for hugo@... carfax.org.uk | it. http://carfax.org.uk/ | PGP: E2AB1DE4 |Juri Papay signature.asc Description: Digital signature

Re: Removing a device from a metadata raid1 - data raid0

2015-05-22 Thread Hugo Mills
That at least will work, if be a little risky. IMO, we should definitely support conversion to DUP, and prevent the automatic "upgrade" to RAID-1, but I think we don't at the moment. Hugo. -- Hugo Mills | Two things came out of Berkeley in the 1960s: L

Re: btrfs on disk stability

2015-05-26 Thread Hugo Mills
ee both copies as the same filesystem (because they have the same UUID), and it will get very confused about which device(s) it's meant to be writing to. Hugo. -- Hugo Mills | Q: What goes, "Pieces of seven! Pieces of seven!"? hugo@... carfax.org.uk | A: A parroty error. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: btrfs on disk stability

2015-05-26 Thread Hugo Mills
On Tue, May 26, 2015 at 09:29:25AM +, sri wrote: > Hugo Mills carfax.org.uk> writes: > > > > > On Tue, May 26, 2015 at 08:45:06AM +, sri wrote: > > > Hi, > > > > > > According to btrfs wiki page, under "Stability status" it is w

Re: intermittent -ENOSPC errors on btrfs filesystem with 170G free

2015-05-26 Thread Hugo Mills
lib]# mv abrt abrt2 > mv: cannot move ‘abrt’ to ‘abrt2’: No space left on device > [root@foobox lib]# > > Any tests anyone wants to run on this before I wipe and reinstall > the box? No tests needed. Just run a filtered balance on it to clean up unused chunks: # btrfs balance s

Re: Metadata size limitations of btrfs

2015-05-28 Thread Hugo Mills
e usage=5 or limit=3 suggestion I made above will have any benefit for you. Hugo. > btrfs-progs v4.0 > franzbroetchen# btrfs fi df / > Data, single: total=190.00GiB, used=176.47GiB > System, DUP: total=32.00MiB, used=48.00KiB > Metadata, DUP: total=8.00GiB, used=6.08GiB &g

Re: "@" prefix in subvolume paths

2015-06-10 Thread Hugo Mills
op level 5 path var/lib/named > ID 267 gen 3513372 top level 5 path var/lib/pgsql > ID 268 gen 3517714 top level 5 path var/log > ID 269 gen 3513372 top level 5 path var/opt > ID 270 gen 3517714 top level 5 path var/spool > ID 271 gen 3517706 top level 5 path var/tmp > ID 2

Re: [PATCH] Btrfs: use received_uuid of parent during send

2015-06-11 Thread Hugo Mills
should be sending all three of the parent UUID, the parent's Received UUID (if it exists), and the parent's Parent UUID. That would have to go in the FARv2 update, though. However, since this patch doesn't rule out the above happening at some future date, and I think it'll do the jo

Re: [PATCH] Btrfs: use received_uuid of parent during send

2015-06-11 Thread Hugo Mills
On Thu, Jun 11, 2015 at 01:16:26PM -0400, Josef Bacik wrote: > On 06/11/2015 01:09 PM, Hugo Mills wrote: > >On Thu, Jun 04, 2015 at 05:17:25PM -0400, Josef Bacik wrote: > >>Neil Horman pointed out a problem where if he did something like this > >> > >>receive

Re: CoW with webserver databases: innodb_file_per_table and dedicated tables for blobs?

2015-06-15 Thread Hugo Mills
cause those files to fragment, which causes lots of seeks and short reads, which degrades performance. > Is this likely to give me ok-ish performance? What other > possibilities are there? I would recommend benchmarking over time with your workloads, and seeing how your performance degrad

Re: CoW with webserver databases: innodb_file_per_table and dedicated tables for blobs?

2015-06-16 Thread Hugo Mills
-hoc manner that I think you're unlikely to be achieving the D part of ACID. Hugo. > Ingvar > > > > Am 15.06.15 um 11:57 schrieb Hugo Mills: > >On Mon, Jun 15, 2015 at 11:34:35AM +0200, Ingvar Bogdahn wrote: > >>Hello there, > >> > >>I&#x

Re: RAID10 Balancing Request for Comments and Advices

2015-06-16 Thread Hugo Mills
re there best > practices when using smartctl for periodically testing (long test, > short test) btrfs RAID devices? I can't answer that one, I'm afraid. Hugo. -- Hugo Mills | Welcome to Rivendell, Mr Anderson... hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 |Machinae Supremacy, Hybrid signature.asc Description: Digital signature

Re: Automatic balance after mkfs?

2015-06-16 Thread Hugo Mills
On Tue, Jun 16, 2015 at 01:13:40PM +, Holger Hoffstätte wrote: > > Forking from the other thread.. > > On Tue, 16 Jun 2015 12:25:45 +, Hugo Mills wrote: > > >Yes. It's an artefact of the way that mkfs works. If you run a > > balance on those chunks,

Re: Replacing a drive from a RAID 1 array

2015-06-16 Thread Hugo Mills
it, but that's going to end up with horrible performance during the migration. In my big disk array at home, I have two 4-slot enclosures, and I leave one of them empty specifically for this reason. It's a less attractive proposition with only 4 slots in total, though.

Re: BTRFS: read error corrected: ino 1 off 226840576 (dev /dev/mapper/dshelf1 sector 459432)

2015-06-17 Thread Hugo Mills
's my reading of the situation. Note that the 3.19 kernel is the earliest I would expect this to be able to happen, as it's the first kernel that actually had the full set of parity RAID repair code in it. > I'll do a scrub later, for now I have to wait 20 hours for the raid rebuil

Re: BTRFS: read error corrected: ino 1 off 226840576 (dev /dev/mapper/dshelf1 sector 459432)

2015-06-17 Thread Hugo Mills
On Wed, Jun 17, 2015 at 12:58:35PM +0200, Sander wrote: > Hugo Mills wrote (ao): > > On Wed, Jun 17, 2015 at 12:16:54AM -0700, Marc MERLIN wrote: > > > I had a few power offs due to a faulty power supply, and my mdadm raid5 > > > got into fail mode after 2 drives

Re: RAID10 Balancing Request for Comments and Advices

2015-06-17 Thread Hugo Mills
ad of manually iterating through several values of usage= until you get a useful response, you can use limit= to stop after successful block group relocations. Hugo. -- Hugo Mills | Alert status mauve ocelot: Slight chance of hugo@... carfax.org.uk | brimstone. Be prepared to make a nice cup of tea. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: Send/Receive Use Case

2015-06-27 Thread Hugo Mills
If a file is created, modified and then deleted in-between two > snapshots is it ignored by send/receive or does send/receive > "re-enacts” the journal exactly ? It'll be ignored. The FS doesn't keep track of how it reached a particular state -- only what that

Re: Btrfs - distribute files equally across multiple devices

2015-07-06 Thread Hugo Mills
ith > balanced disk usage + filesystem intelligence. > > Is there something like that already in btrfs or could this be something > the btrfs-devs would consider? > > > [2] Still can read/write multiple files from/to different disks, so less > performance only for "

Re: size 2.73TiB used 240.97GiB after balance

2015-07-06 Thread Hugo Mills
> get a new drive before, but this might take a couple of days (in > which sdc could further degrade). > What is your recommendation? Physically remove the device from the array, mount with -o degraded, optionally add the new device, and run a balance. Hugo. -- Hugo Mills

Re: kernel crash on btrfs device delete missing

2015-07-07 Thread Hugo Mills
eed a developer for. You've already reported it here, but sticking a copy of what you've discovered so far into bugzilla.kernel.org may help it not to get lost. Hugo. -- Hugo Mills | "I don't like the look of it, I tell you." hugo@... carfax.org.uk |

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

2015-07-09 Thread Hugo Mills
f it > ever got in. Are you thinking of the read-only flag? That's not the same thing as the various UUID properties (e.g. parent) which can be used to detemine if a subvolume was made using a snapshot. Hugo. -- Hugo Mills | Someone's been throwing dead sheep down my

Re: Concurrent write access

2015-07-09 Thread Hugo Mills
rtualised 9p, and not be able to host a root filesystem. I haven't tried this one, because Samba and I get on like a house on fire(*). Hugo. (*) Screaming, shouting, people running away, emergency services. -- Hugo Mills | Alert status mauve ocelot: Slight chance of hugo@... c

Re: Send/Receive Use Case

2015-07-10 Thread Hugo Mills
on the underlying raw storage. Hugo. >or do they need to be of the same replication scheme too ? > > > > > On Jun 27, 2015, at 10:34 AM, Vincent Olivier wrote: > > > > ok i’ll go home and rethink my life then ;) > > > >> On Jun 27, 2015, at 10:21 AM, Hu

Re: counting fragments takes more time than defragmenting

2015-07-14 Thread Hugo Mills
lock groups. The same would apply to RAID-10, -5 and -6. (Note, I haven't verified this, but it makes sense based on what I know of the internal data structures). Hugo. -- Hugo Mills | Go not to the elves for counsel, for they will say hugo@... carfax.org.uk | both no and yes. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: counting fragments takes more time than defragmenting

2015-07-14 Thread Hugo Mills
On Tue, Jul 14, 2015 at 09:09:00PM +0200, Patrik Lundquist wrote: > On 14 July 2015 at 20:41, Hugo Mills wrote: > > On Tue, Jul 14, 2015 at 01:57:07PM +0200, Patrik Lundquist wrote: > >> On 24 June 2015 at 12:46, Duncan <1i5t5.dun...@cox.net> wrote: > >> > &g

Re: BTRFS raid6 unmountable after a couple of days of usage.

2015-07-15 Thread Hugo Mills
d of that superblock, and can then identify the location of the System chunks to read the full chunk tree. Once it's got the chunk tree, it can do virtual->physical lookups, and the root tree and log tree locations make sense. I don't know whether btrfs-image works any differently

Re: I'd like a -r flag on btrfs subvolume delete

2015-07-16 Thread Hugo Mills
them from those parts of my filesystems which have a > high churn rate (= more unique extents, occupying a lot of disk) and > yet aren't as important (I need to retain fewer of them) is very > useful. -- Hugo Mills | Hail and greetings. We are a flat-pack invasion hugo@... carfax.org.uk | force from Planet Ikea. We come in pieces. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

2015-07-27 Thread Hugo Mills
--- > Version: GnuPG v2.0.22 (MingW32) > > iQEcBAEBAgAGBQJVtq9AAAoJEE+uni74c4qN5eYIAJAGznsi3RD1tchbSLwhMXJk > bJJ4ORB9taLXHykSfYTsHIaUoVpcVR6tT/I1jz5070DY3mKkQ16a8nwtSxPba4Lv > QiS8YRegFiHMYzZbH1T7Tnm6R9g/RZsaU7GS3JhP9HUYG7hIWGRRuoiOjYn/hoLw > uMXuIFOkPKGYDgyAhDIp3KDYlBjMHT6Oun7CcpvTjXiOnzJFFp3MSt3b6mm

Re: Data single *and* raid?

2015-08-01 Thread Hugo Mills
do this more than once. Hugo. > Why is there Data single and Raid? > Why is Metadata RAID1 and Raid5? > > A scrub is currently running and showed no errors yet. -- Hugo Mills | You've read the project plan. Forget that. We're h

Re: ext4 convert bugs, wiki warning?

2015-08-01 Thread Hugo Mills
ession and it's breaking file systems, > sometimes irreparably? You mean a warning like the very first sentence, in bold, that's already on the wiki page? https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 Hugo. -- Hugo Mills | You've read the proj

Re: Data single *and* raid?

2015-08-01 Thread Hugo Mills
Finally, that's btrfs-progs 4.1.1, the one that's blacklisted due to a > > buggy mkfs.btrfs. If he created the filesystem with that mkfs.btrfs... > > maybe that explains the funky results, as well. > > Good catch. That really ought to be filed as a bug with that distro

Re: Data single *and* raid?

2015-08-02 Thread Hugo Mills
w it will take another balance in the > end because it looks like this is effectively a 2 device raid5, with > the 3rd drive full of single only chunks (which might be empty?). -- Hugo Mills | I'll take your bet, but make it ten thousand francs. hugo@... carfax.org.uk | I&#x

<    1   2   3   4   5   6   7   8   9   10   >