Re: [PATCH] Btrfs: all super blocks of the replaced disk must be scratched

2014-05-20 Thread Anand Jain
On 21/03/14 02:21, David Sterba wrote: On Mon, Mar 17, 2014 at 07:58:06PM +0800, Anand Jain wrote: In a normal scenario when sys-admin replaces a disk, the expeted is btrfs will release the disk completely. However the below test case gives a wrong impression that replaced disk is still is

[PATCH 1/4 RESEND V3] Btrfs: all super blocks of the replaced disk must be scratched

2014-05-20 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com In a normal scenario when sys-admin replaces a disk, the expectation is that btrfs will release the disk completely. However the below test case gives a wrong impression that replaced disk is still is in use. $ btrfs rep start /dev/sde /dev/sdg4 /btrfs $

[PATCH 1/2 v2] btrfs: label should not contain return char

2014-05-20 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com generally if you use echo test /sys/fs/btrfs/fsid/label it would introduce return char at the end and it can not be part of the label. The correct command is echo -n test /sys/fs/btrfs/fsid/label This patch will check for this user error

[PATCH 2/2 v2] btrfs: usage error should not be logged into system log

2014-05-20 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com I have an opinion that system logs /var/log/messages are valuable info to investigate the real system issues at the data center. People handling data center issues do spend a lot time and efforts analyzing messages files. Having usage error logged into

Re: [PATCH 1/2] btrfs: label should not contain return char

2014-05-20 Thread Anand Jain
On 20/05/14 01:16, Eric Sandeen wrote: On 5/19/14, 12:04 PM, Anand Jain wrote: From: Anand Jain anand.j...@oracle.com generally if you use echo test /sys/fs/btrfs/fsid/label it would introduce return char at the end and it can not be part of the label. The correct command is echo -n

Re: [PATCH 1/2] btrfs: label should not contain return char

2014-05-20 Thread Anand Jain
On 20/05/14 01:19, Roman Mamedov wrote: On Tue, 20 May 2014 01:04:30 +0800 Anand Jain anand.j...@oracle.com wrote: From: Anand Jain anand.j...@oracle.com generally if you use echo test /sys/fs/btrfs/fsid/label it would introduce return char at the end and it can not be part of the

[PATCH] btrfs-progs: Improve the parse_size() error message.

2014-05-20 Thread Qu Wenruo
When using parse_size(), even non-numeric value is passed, it will only give error message ERROR: size value is empty, which is quite confusing for end users. This patch will introduce more meaningful error message for the following new cases 1) Invalid size string (non-numeric string) 2) Minus

btrfs-progs: add arg_strtou64 break library btrfs

2014-05-20 Thread Dennis Lan (dlan)
progs # cat btrfs_mini_tst.c /* * gcc -o btrfs_mini_tst btrfs_mini_tst.c -lbtrfs */ char btrfs_read_and_process_send_stream (); int main () { return btrfs_read_and_process_send_stream (); } progs # gcc -o btrfs_mini_tst btrfs_mini_tst.c -lbtrfs

Re: btrfs-progs: add arg_strtou64 break library btrfs

2014-05-20 Thread Arvin Schnell
On Tue, May 20, 2014 at 06:15:49PM +0800, Dennis Lan (dlan) wrote: progs # cat btrfs_mini_tst.c /* * gcc -o btrfs_mini_tst btrfs_mini_tst.c -lbtrfs */ char btrfs_read_and_process_send_stream (); int main () { return btrfs_read_and_process_send_stream (); } progs # gcc -o

Re: btrfs-progs: add arg_strtou64 break library btrfs

2014-05-20 Thread Dennis Lan (dlan)
On Tue, May 20, 2014 at 6:20 PM, Arvin Schnell aschn...@suse.de wrote: On Tue, May 20, 2014 at 06:15:49PM +0800, Dennis Lan (dlan) wrote: progs # cat btrfs_mini_tst.c /* * gcc -o btrfs_mini_tst btrfs_mini_tst.c -lbtrfs */ char btrfs_read_and_process_send_stream (); int main ()

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

2014-05-20 Thread David Sterba
On Tue, May 20, 2014 at 08:34:19AM +0800, Qu Wenruo wrote: Proposed changes: - format all subcommands as bold instead of italic ('' - **) - add all missing ... - find a way how to add '...' around ... (xsl or sed or whatever) Does that work for you? That is OK for me, I'll investigate it.

Re: send/receive and bedup

2014-05-20 Thread Scott Middleton
On 20 May 2014 06:07, Konstantinos Skarlatos k.skarla...@gmail.com wrote: On 19/5/2014 8:38 μμ, Mark Fasheh wrote: Well, after having good results with duperemove with a few gigs of data, i tried it on a 500gb subvolume. After it scanned all files, it is stuck at 100% of one cpu core for

[PATCH] Btrfs: free tmp ulist for qgroup rescan

2014-05-20 Thread Josef Bacik
Memory leaks are bad mmkay? Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/qgroup.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 73f706c..cf5aead 100644 --- a/fs/btrfs/qgroup.c +++ b/fs/btrfs/qgroup.c @@ -2478,6 +2478,7 @@ static void

btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

2014-05-20 Thread Astro Xe
On my box, the used value in the output of btrfs filesystem df is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute btrfs filesystem sync, in order to update the value. Could someone fix this, please? I suspect the cause is that subvolumes

btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

2014-05-20 Thread Astro Xe
On my box, the used value in the output of btrfs filesystem df is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute btrfs filesystem sync, in order to update the value. How do I fix this? Or, could someone fix this in btrfs-progs, please?

Re: ditto blocks on ZFS

2014-05-20 Thread Austin S Hemmelgarn
On 2014-05-19 22:07, Russell Coker wrote: On Mon, 19 May 2014 23:47:37 Brendan Hide wrote: This is extremely difficult to measure objectively. Subjectively ... see below. [snip] *What other failure modes* should we guard against? I know I'd sleep a /little/ better at night knowing that a

Re: btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

2014-05-20 Thread Russell Coker
On Tue, 20 May 2014 14:50:10 Astro Xe wrote: On my box, the used value in the output of btrfs filesystem df is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute btrfs filesystem sync, in order to update the value. How do I fix this? Or,

Re: btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

2014-05-20 Thread Hugo Mills
On Tue, May 20, 2014 at 02:50:10PM +0100, Astro Xe wrote: On my box, the used value in the output of btrfs filesystem df is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute btrfs filesystem sync, in order to update the value. How do

Better handling of stale scrub status

2014-05-20 Thread Holger Hoffstätte
As Marc Merlin recently wrote in his blog [1] scrub can sometimes leave a stale state file behind, making cancel/resume complain. I took a peek at the code and found most of the scrub state file handling fairly straightforward, so before I go off and start hacking, what would a better

[PATCH] [btrfs] add volid to failed csum messages

2014-05-20 Thread Alexandre Oliva
The failed csum messages generated by btrfs mention the inode number, but on filesystems with multiple subvolumes, that's not enough to identify the file. I've added the inode number to the messages so that they're more complete. I also noticed that the extent/offset information printed for the

Re: [PATCH 1/2 v2] btrfs: label should not contain return char

2014-05-20 Thread Eric Sandeen
On 5/20/14, 1:36 AM, Anand Jain wrote: From: Anand Jain anand.j...@oracle.com generally if you use echo test /sys/fs/btrfs/fsid/label it would introduce return char at the end and it can not be part of the label. The correct command is echo -n test /sys/fs/btrfs/fsid/label This

Re: [PATCH 1/2 v2] btrfs: label should not contain return char

2014-05-20 Thread David Sterba
On Tue, May 20, 2014 at 02:36:48PM +0800, Anand Jain wrote: From: Anand Jain anand.j...@oracle.com generally if you use echo test /sys/fs/btrfs/fsid/label it would introduce return char at the end and it can not be part of the label. The correct command is echo -n test

Re: [PATCH 2/2 v2] btrfs: usage error should not be logged into system log

2014-05-20 Thread David Sterba
On Tue, May 20, 2014 at 02:38:11PM +0800, Anand Jain wrote: From: Anand Jain anand.j...@oracle.com I have an opinion that system logs /var/log/messages are valuable info to investigate the real system issues at the data center. People handling data center issues do spend a lot time and

Re: [PATCH 1/2 v2] btrfs: label should not contain return char

2014-05-20 Thread Eric Sandeen
On 5/20/14, 11:33 AM, David Sterba wrote: On Tue, May 20, 2014 at 02:36:48PM +0800, Anand Jain wrote: From: Anand Jain anand.j...@oracle.com generally if you use echo test /sys/fs/btrfs/fsid/label it would introduce return char at the end and it can not be part of the label. The correct

Re: What do the wait_current_trans messages mean I see on my raspberry pi?

2014-05-20 Thread David Sterba
On Sun, May 18, 2014 at 09:48:17PM +0200, Clemens Eisserer wrote: Over the weekend I tried to copy one external usb drive (on ext4) to another one formatted with btrfs. Now I came back, and 48h later only ~300GB were copied and I found messages like the following on syslog: That's about 1

Re: btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

2014-05-20 Thread Calvin Walton
On Tue, 2014-05-20 at 14:46 +0100, Astro Xe wrote: On my box, the used value in the output of btrfs filesystem df is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute btrfs filesystem sync, in order to update the value. Could someone fix

btrfs check, btrfsck, fsck.btrfs

2014-05-20 Thread Chris Murphy
Should the initrd/initramfs no longer include btrfsck, and instead include btrfs and fsck.btrfs? In btrfs-progs 3.14 there is now a 1K /sbin/fsck.btrfs placeholder file. btrfs and btrfsck files are the same binary, the difference is btrfsck only can do check/repair. Including btrfs instead

[GIT PULL] Btrfs

2014-05-20 Thread Chris Mason
Hi Linus Please pull my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus It has two fixes that we've been testing for 3.16, but since both are safe and fix real bugs, it makes sense to send for 3.15 instead Liu Bo (1) commits (+5/-1): Btrfs:

Re: ditto blocks on ZFS

2014-05-20 Thread Brendan Hide
On 2014/05/20 04:07 PM, Austin S Hemmelgarn wrote: On 2014-05-19 22:07, Russell Coker wrote: [snip] As an aside, I'd really like to be able to set RAID levels by subtree. I'd like to use RAID-1 with ditto blocks for my important data and RAID-0 for unimportant data. But the proposed changes

Re: problem with degraded boot and systemd

2014-05-20 Thread Goffredo Baroncelli
On 05/19/2014 02:54 AM, Chris Murphy wrote: Summary: It's insufficient to pass rootflags=degraded to get the system root to mount when a device is missing. It looks like when a device is missing, udev doesn't create the dev-disk-by-uuid linkage that then causes systemd to change the device

Re: problem with degraded boot and systemd

2014-05-20 Thread Hugo Mills
On Wed, May 21, 2014 at 12:00:24AM +0200, Goffredo Baroncelli wrote: On 05/19/2014 02:54 AM, Chris Murphy wrote: Summary: It's insufficient to pass rootflags=degraded to get the system root to mount when a device is missing. It looks like when a device is missing, udev doesn't create

Re: send/receive and bedup

2014-05-20 Thread Mark Fasheh
On Tue, May 20, 2014 at 01:07:50AM +0300, Konstantinos Skarlatos wrote: Duperemove will be shipping as supported software in a major SUSE release so it will be bug fixed, etc as you would expect. At the moment I'm very busy trying to fix qgroup bugs so I haven't had much time to add features,

Re: send/receive and bedup

2014-05-20 Thread Konstantinos Skarlatos
On 21/5/2014 1:37 πμ, Mark Fasheh wrote: On Tue, May 20, 2014 at 01:07:50AM +0300, Konstantinos Skarlatos wrote: Duperemove will be shipping as supported software in a major SUSE release so it will be bug fixed, etc as you would expect. At the moment I'm very busy trying to fix qgroup bugs so I

Re: btrfs check, btrfsck, fsck.btrfs

2014-05-20 Thread Duncan
Chris Murphy posted on Tue, 20 May 2014 10:56:26 -0600 as excerpted: Should the initrd/initramfs no longer include btrfsck, and instead include btrfs and fsck.btrfs? btrfs (the program) should be included in any case as btrfs device scan should be run (normally triggered via udev rules when a

Re: [PATCH] [btrfs] add volid to failed csum messages

2014-05-20 Thread Duncan
Alexandre Oliva posted on Tue, 20 May 2014 12:37:07 -0300 as excerpted: The failed csum messages generated by btrfs mention the inode number, but on filesystems with multiple subvolumes, that's not enough to identify the file. I've added the inode number to the messages ?? s/added the inode

Re: Formalizing the use of Boot Area B

2014-05-20 Thread H. Peter Anvin
On 05/14/2014 05:01 PM, H. Peter Anvin wrote: It turns out that the primary 64K Boot Area A is too small for some applications and/or some architectures. When I discussed this with Chris Mason, he pointed out that the area beyond the superblock is also unused, up until at least the megabyte

Re: Formalizing the use of Boot Area B

2014-05-20 Thread Chris Mason
On 05/20/2014 07:29 PM, H. Peter Anvin wrote: On 05/14/2014 05:01 PM, H. Peter Anvin wrote: It turns out that the primary 64K Boot Area A is too small for some applications and/or some architectures. When I discussed this with Chris Mason, he pointed out that the area beyond the superblock

Re: Formalizing the use of Boot Area B

2014-05-20 Thread H. Peter Anvin
On 05/20/2014 04:37 PM, Chris Mason wrote: On 05/20/2014 07:29 PM, H. Peter Anvin wrote: On 05/14/2014 05:01 PM, H. Peter Anvin wrote: It turns out that the primary 64K Boot Area A is too small for some applications and/or some architectures. When I discussed this with Chris Mason, he

Re: problem with degraded boot and systemd

2014-05-20 Thread Duncan
Hugo Mills posted on Tue, 20 May 2014 23:26:09 +0100 as excerpted: On Wed, May 21, 2014 at 12:00:24AM +0200, Goffredo Baroncelli wrote: On 05/19/2014 02:54 AM, Chris Murphy wrote: It's insufficient to pass rootflags=degraded to get the system root to mount when a device is missing. It looks

Re: Formalizing the use of Boot Area B

2014-05-20 Thread H. Peter Anvin
On 05/20/2014 04:37 PM, Chris Mason wrote: Hi Peter, We do leave the first 1MB of each device alone. Can we do 256K-1024K for the boot loader? We don't have an immediate need for the extra space, but I'd like to reserve a little more than the extra 64KB. Incidentally, the current

Re: btrfs check, btrfsck, fsck.btrfs

2014-05-20 Thread Chris Murphy
On May 20, 2014, at 5:02 PM, Duncan 1i5t5.dun...@cox.net wrote: Chris Murphy posted on Tue, 20 May 2014 10:56:26 -0600 as excerpted: Should the initrd/initramfs no longer include btrfsck, and instead include btrfs and fsck.btrfs? btrfs (the program) should be included in any case as

Re: problem with degraded boot and systemd

2014-05-20 Thread Chris Murphy
On May 20, 2014, at 6:03 PM, Duncan 1i5t5.dun...@cox.net wrote: I'd actually argue that's functioning as it should, since I see forced manual intervention in ordered to mount degraded as a FEATURE, NOT A BUG. Manual intervention is OK for now, when it takes the form of dropping to a

Re: send/receive and bedup

2014-05-20 Thread Chris Murphy
On May 20, 2014, at 4:56 PM, Konstantinos Skarlatos k.skarla...@gmail.com wrote: On 21/5/2014 1:37 πμ, Mark Fasheh wrote: On Tue, May 20, 2014 at 01:07:50AM +0300, Konstantinos Skarlatos wrote: Duperemove will be shipping as supported software in a major SUSE release so it will be bug

Re: ditto blocks on ZFS

2014-05-20 Thread Russell Coker
On Tue, 20 May 2014 07:56:41 ashf...@whisperpc.com wrote: 1. There will be more disk space used by the metadata. I've been aware of space allocation issues in BTRFS for more than three years. If the use of ditto blocks will make this issue worse, then it's probably not a good idea to

Re: [RFC PATCH 1/2] btrfs: Add missing device check in dev_info/rm_dev ioctl

2014-05-20 Thread Qu Wenruo
I'm sorry to bother your but it has been about 2 weeks after your last reply. Is there any problem? Thanks, Qu Original Message Subject: Re: [RFC PATCH 1/2] btrfs: Add missing device check in dev_info/rm_dev ioctl From: Anand Jain anand.j...@oracle.com To: Qu Wenruo

Re: [RFC PATCH 1/2] btrfs-progs: Follow kernel changes to add new member of btrfs_ioctl_dev_info_args.

2014-05-20 Thread Anand Jain
Hi Qu, in-line below.. On 06/05/14 14:33, Qu Wenruo wrote: Follow the kernel header changes to add new member of btrfs_ioctl_dev_info_args. This change will use special bit to keep backward compatibility, so even on old kernels this will not screw anything up. Signed-off-by: Qu Wenruo

Re: [RFC PATCH 2/2] btrfs-progs: Add userspace support for kernel missing dev detection.

2014-05-20 Thread Anand Jain
Hi Qu, First of all. With this patch, we don't need the old btrfs-progs workaround fix [1] anymore. David should back-out if you are ok. [1] commit 206efb60cbe3049e0d44c6da3c1909aeee18f813 Author: Qu Wenruo quwen...@cn.fujitsu.com Date: Fri Feb 7 15:07:19 2014 +0800 btrfs-progs:

Re: [RFC PATCH 1/2] btrfs: Add missing device check in dev_info/rm_dev ioctl

2014-05-20 Thread Anand Jain
Hi Qu, Either we need to determine missing disk when disk disappear or when we hit by -ENODEV during IO. Discovering the same at the time when user run 'btrfs fi show' or 'btrfs dev del', is just not correct. You might use interface as proposed here. [PATCH RFC] btrfs: revamp

Re: [RFC PATCH 1/2] btrfs: Add missing device check in dev_info/rm_dev ioctl

2014-05-20 Thread Anand Jain
Hi Qu, Thanks for checking with me. sorry for the delay. The preliminary RFC patch which was sent and mentioned in the other emails took time more than expected. Further on top of your check_missing patch I am writing code to to handle disk reappear. I should be sending them all soon.

Re: historical backups with hardlinks vs cp --reflink vs snapshots

2014-05-20 Thread Marc MERLIN
On Mon, May 19, 2014 at 06:01:25PM +0200, Brendan Hide wrote: On 19/05/14 15:00, Scott Middleton wrote: On 19 May 2014 09:07, Marc MERLIN m...@merlins.org wrote: On Wed, May 14, 2014 at 11:36:03PM +0800, Scott Middleton wrote: I read so much about BtrFS that I mistaked Bedup with Duperemove.

Re: [RFC PATCH 1/2] btrfs: Add missing device check in dev_info/rm_dev ioctl

2014-05-20 Thread Qu Wenruo
Original Message Subject: Re: [RFC PATCH 1/2] btrfs: Add missing device check in dev_info/rm_dev ioctl From: Anand Jain anand.j...@oracle.com To: Qu Wenruo quwen...@cn.fujitsu.com Date: 2014年05月21日 11:54 Hi Qu, Thanks for checking with me. sorry for the delay. The