Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Ian Armstrong
On Fri, 21 Nov 2014 10:45:21 -0700 Chris Murphy wrote: > On Fri, Nov 21, 2014 at 5:55 AM, Ian Armstrong > wrote: > > > In my situation what I've found is that if I scrub & let it fix the > > errors then a second pass immediately after will show no errors. If > > I then leave it a few days & try

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Zygo Blaxell
On Fri, Nov 21, 2014 at 11:06:19AM -0700, Chris Murphy wrote: > On Fri, Nov 21, 2014 at 10:42 AM, Zygo Blaxell > wrote: > > > I run 'smartctl -t long' from cron overnight (or whenever the drives > > are most idle). You can also set up smartd.conf to launch the self > > tests; however, the synta

Re: [PATCH] Btrfs: ctree: reduce args where only fs_info used

2014-11-21 Thread David Sterba
On Sat, Nov 22, 2014 at 01:03:32AM +0900, Daniel Dressler wrote: > No problem, I'll redo everything so it is one function per patch. Now > fair warning: there are about 102 functions to cleanup. I was a bit > worried that many patches would cause too much maintainer overhead but > it is no problem

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Duncan posted on Fri, 21 Nov 2014 23:41:49 + as excerpted: > Duncan posted on Fri, 21 Nov 2014 22:49:06 + as excerpted: > >> Chris Murphy posted... > >>> That's not true for thin volume snapshots. They take up next to no >>> space upon creation, they don't need space reserved in advance.

Re: [PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread David Sterba
On Sat, Nov 22, 2014 at 01:37:10AM +0900, Daniel Dressler wrote: > What would a cover letter be like? Would that be a separate email to > the list, or maybe the first email in a patch series? It's a separate mail that does not carry any diff but an overview of the following patches. The patches ar

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Duncan posted on Fri, 21 Nov 2014 22:49:06 + as excerpted: > Chris Murphy posted... >> That's not true for thin volume snapshots. They take up next to no >> space upon creation, they don't need space reserved in advance. > > Thus the mention of compression if necessary. Thin-volume snapshot

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Zygo Blaxell posted on Fri, 21 Nov 2014 12:56:23 -0500 as excerpted: > It's not a bug as long as I can completely control which devices are > searched for UUIDs, and the system behaves sanely when multiple UUIDs > are found through automatic discovery; otherwise, it's not only a bug, > it's a DoS

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Chris Murphy posted on Fri, 21 Nov 2014 11:23:45 -0700 as excerpted: > On Thu, Nov 20, 2014 at 11:22 PM, Duncan <1i5t5.dun...@cox.net> wrote: > > >> When I have such a filesystem level problem, I simply dd from the >> backing device to some other location, generally to a file that's on a >> diff

[RFC][PATCH] dm: add dm-power-fail target

2014-11-21 Thread Josef Bacik
Hello, I'm hoping some FS guys can weigh in and verify my approach for testing power fail conditions, and the DM guys to of course verify I didn't completely fail at making a DM target. All suggestions welcome, I want to have a nice robust tool for testing power fail so we can be sure our fsync/c

Re: [PATCH 1/4] fs: split update_time() into update_time() and write_time()

2014-11-21 Thread Theodore Ts'o
Out of curiosity, why does btrfs_update_time() need to call btrfs_root_readonly()? Why can't it just depend on the __mnt_want_write() call in touch_atime()? Surely if there are times when it's not OK to write into a btrfs file system and mnt_is_readonly() returns false, the VFS is going to get ve

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Robert White
On 11/21/2014 01:12 PM, Robert White wrote: > (wrong links included in post...) Dangit... those two links were bad... wrong clipboard... /sigh... I'll just stand on the pasted text from the driver. 8-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a mess

Re: [PATCH 3/4] vfs: don't let the dirty time inodes get more than a day stale

2014-11-21 Thread Theodore Ts'o
On Fri, Nov 21, 2014 at 02:19:07PM -0600, Andreas Dilger wrote: > > - if (inode->i_sb->s_flags & MS_LAZYTIME) { > > + if ((inode->i_sb->s_flags & MS_LAZYTIME) && > > + (!inode->i_ts_dirty_day || > > +inode->i_ts_dirty_day == days_since_boot)) { > > spin_lock(&inode->i_

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Robert White
On 11/21/2014 07:11 AM, Phillip Susi wrote: On 11/20/2014 5:45 PM, Robert White wrote: If you search for "ACPI ide" you'll find people complaining in 2008-2010 about windows error messages indicating the device is present in their system but no OS driver is available. Nope... not finding it.

Re: [PATCH 3/4] vfs: don't let the dirty time inodes get more than a day stale

2014-11-21 Thread Andreas Dilger
On Nov 21, 2014, at 1:59 PM, Theodore Ts'o wrote: > Guarantee that the on-disk timestamps will be no more than 24 hours > stale. > > Signed-off-by: Theodore Ts'o > --- > fs/fs-writeback.c | 1 + > fs/inode.c | 7 ++- > include/linux/fs.h | 1 + > 3 files changed, 8 insertions(+), 1 del

Re: [PATCH 1/4] fs: split update_time() into update_time() and write_time()

2014-11-21 Thread Chris Mason
On Fri, Nov 21, 2014 at 2:59 PM, Theodore Ts'o wrote: In preparation for adding support for the lazytime mount option, we need to be able to separate out the update_time() and write_time() inode operations. Currently, only btrfs and xfs uses update_time(). We needed to preserve update_time() b

[PATCH 3/4] vfs: don't let the dirty time inodes get more than a day stale

2014-11-21 Thread Theodore Ts'o
Guarantee that the on-disk timestamps will be no more than 24 hours stale. Signed-off-by: Theodore Ts'o --- fs/fs-writeback.c | 1 + fs/inode.c | 7 ++- include/linux/fs.h | 1 + 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c inde

[PATCH 0/4] add support for a lazytime mount option

2014-11-21 Thread Theodore Ts'o
This is an updated version of what had originally been an ext4-specific patch which significantly improves performance by lazily writing timestamp updates (and in particular, mtime updates) to disk. The in-memory timestamps are always correct, but they are only written to disk when required for cor

[PATCH 4/4] ext4: add support for a lazytime mount option

2014-11-21 Thread Theodore Ts'o
Add an optimization for the MS_LAZYTIME mount option so that we will opportunistically write out any inodes with the I_DIRTY_TIME flag set in a particular inode table block when we need to update some inode in that inode table block anyway. Also add some temporary code so that we can set the lazyt

[PATCH 1/4] fs: split update_time() into update_time() and write_time()

2014-11-21 Thread Theodore Ts'o
In preparation for adding support for the lazytime mount option, we need to be able to separate out the update_time() and write_time() inode operations. Currently, only btrfs and xfs uses update_time(). We needed to preserve update_time() because btrfs wants to have a special btrfs_root_readonly(

[PATCH 2/4] vfs: add support for a lazytime mount option

2014-11-21 Thread Theodore Ts'o
Add a new mount option which enables a new "lazytime" mode. This mode causes atime, mtime, and ctime updates to only be made to the in-memory version of the inode. The on-disk times will only get updated when (a) if the inode needs to be updated for some non-time related change, (b) if userspace

[PATCH] Btrfs: make sure logged extents complete in the current transaction V3

2014-11-21 Thread Josef Bacik
Liu Bo pointed out that my previous fix would lose the generation update in the scenario I described. It is actually much worse than that, we could lose the entire extent if we lose power right after the transaction commits. Consider the following write extent 0-4k log extent in log tree commit

Re: [PATCH v2 5/5] btrfs: enable swap file support

2014-11-21 Thread Filipe David Manana
On Fri, Nov 21, 2014 at 6:00 PM, David Sterba wrote: > On Fri, Nov 21, 2014 at 02:08:31AM -0800, Omar Sandoval wrote: >> Implement the swap file a_ops on btrfs. Activation simply checks for a usable >> swap file: it must be fully allocated (no holes), support direct I/O (so no >> compressed or inl

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Chris Murphy
On Thu, Nov 20, 2014 at 11:22 PM, Duncan <1i5t5.dun...@cox.net> wrote: > > When I have such a filesystem level problem, I simply dd from the backing > device to some other location, generally to a file that's on a different > filesystem (preferrably non-btrfs, I use reiserfs as I've found it very

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Chris Murphy
On Fri, Nov 21, 2014 at 10:42 AM, Zygo Blaxell wrote: > I run 'smartctl -t long' from cron overnight (or whenever the drives > are most idle). You can also set up smartd.conf to launch the self > tests; however, the syntax for test scheduling is byzantine compared to > cron (and that's saying so

Re: [PATCH v2 5/5] btrfs: enable swap file support

2014-11-21 Thread David Sterba
On Fri, Nov 21, 2014 at 02:08:31AM -0800, Omar Sandoval wrote: > Implement the swap file a_ops on btrfs. Activation simply checks for a usable > swap file: it must be fully allocated (no holes), support direct I/O (so no > compressed or inline extents) and should be nocow (I'm not sure about that

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Zygo Blaxell
On Fri, Nov 21, 2014 at 06:22:57AM +, Duncan wrote: > After all, an LVM block-level snapshot takes the same space as a file > containing the same raw data, and if there's room for the data in an LVM > snapshot, given a different layout, there's room for exactly the same > amount of data as a

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Zygo Blaxell
On Fri, Nov 21, 2014 at 09:05:32AM +0200, Brendan Hide wrote: > On 2014/11/21 06:58, Zygo Blaxell wrote: > >You have one reallocated sector, so the drive has lost some data at some > >time in the last 49000(!) hours. Normally reallocations happen during > >writes so the data that was "lost" was da

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Chris Murphy
On Fri, Nov 21, 2014 at 5:55 AM, Ian Armstrong wrote: > In my situation what I've found is that if I scrub & let it fix the > errors then a second pass immediately after will show no errors. If I > then leave it a few days & try again there will be errors, even in > old files which have not been

Re: [PATCH v2 4/5] btrfs: don't allow -C or +c chattrs on a swap file

2014-11-21 Thread David Sterba
On Fri, Nov 21, 2014 at 02:08:30AM -0800, Omar Sandoval wrote: > @@ -293,14 +293,21 @@ static int btrfs_ioctl_setflags(struct file *file, void > __user *arg) > } > } else { You can put the condition here, instead of shifting the nested block. } else if (!IS_SWAPFILE(i

Re: [PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread Daniel Dressler
Thank you David this is helpful feedback. What would a cover letter be like? Would that be a separate email to the list, or maybe the first email in a patch series? Sorry I've twice looked for the integration repo. I found some that look like it could be but those had older commits. Could you dir

Re: [PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread David Sterba
On Fri, Nov 21, 2014 at 05:15:07PM +0900, Daniel Dressler wrote: > This is the 3rd independent patch of a larger > project to cleanup btrfs's internal usage of > btrfs_root. Many functions take btrfs_root > only to grab the fs_info struct. > > By requiring a root these functions cause > programmer

Re: [PATCH] Btrfs: ctree: reduce args where only fs_info used

2014-11-21 Thread Daniel Dressler
Ah thanks David for looking at this. Sorry for the thin paragraphs my vim was warming too early about long lines. I will reformat it to break at 74 chars. No problem, I'll redo everything so it is one function per patch. Now fair warning: there are about 102 functions to cleanup. I was a bit worr

Re: [PATCH] Btrfs: delayed-inode: replace root args iff only fs_info used

2014-11-21 Thread David Sterba
On Mon, Nov 17, 2014 at 10:05:02PM +0900, Daniel Dressler wrote: > This is the second independent patch of a larger > project to cleanup btrfs's internal usage of > btrfs_root. Many functions take btrfs_root > only to grab the fs_info struct. > > By requiring a root these functions cause > program

Re: [PATCH] Btrfs: ctree: reduce args where only fs_info used

2014-11-21 Thread David Sterba
On Wed, Nov 12, 2014 at 01:43:09PM +0900, Daniel Dressler wrote: > This patch is part of a larger project to cleanup > btrfs's internal usage of struct btrfs_root. Many > functions take btrfs_root only to grab a pointer > to fs_info. Thanks for picking up the project. A mere formality, can you pl

Re: [PATCH v2] Btrfs: qgroup: add BUILD_BUG to report pointer cast breakage

2014-11-21 Thread David Sterba
On Thu, Nov 13, 2014 at 12:54:39AM +0900, Daniel Dressler wrote: > Our ulist data structure stores at max 64bit > values. qgroup has used this structure to store > pointers. In the future when we upgrade to 128bit > this casting of pointers to uint64_t will break. What are we going to upgrade to 1

Re: [PATCH v2 1/5] direct-io: don't dirty ITER_BVEC pages on read

2014-11-21 Thread Dave Kleikamp
On 11/21/2014 04:08 AM, Omar Sandoval wrote: > Reads through the iov_iter infrastructure for kernel pages shouldn't be > dirtied > by the direct I/O code. > > This is based on Dave Kleikamp's and Ming Lei's previously posted patches. Acked-by: Dave Kleikamp > Cc: Ming Lei > Signed-off-by: Oma

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Phillip Susi
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 6:08 PM, Robert White wrote: > Well you should have _actually_ trimmed your response down to not > pressing send. > > _Many_ motherboards have complete RAID support at levels 0, 1, 10, > and five 5. A few have RAID6. > > Some of them e

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Phillip Susi
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 5:45 PM, Robert White wrote: > Nice attempt at saving face, but wrong as _always_. > > The CONFIG_PATA_ACPI option has been in the kernel since 2008 and > lots of people have used it. > > If you search for "ACPI ide" you'll find people

Re: soft lockup - CPU#0 stuck - Kernel 3.17.2

2014-11-21 Thread Chris Mason
On Fri, Nov 21, 2014 at 8:01 AM, Patrick Schmid wrote: On 15.11.2014 00:47, Chris Mason wrote: Ok, I think this is related to the new fair read/writer lock implementation in the generic kernel code. btrfs_clear_path_blocking() will end up taking locks outside of the strict locking orde

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Ian Armstrong
On Fri, 21 Nov 2014 09:05:32 +0200, Brendan Hide wrote: > On 2014/11/21 06:58, Zygo Blaxell wrote: > > I also notice you are not running regular SMART self-tests (e.g. > > by smartctl -t long) and the last (and first, and only!) self-test > > the drive ran was ~12000 hours ago. That means most o

Re: soft lockup - CPU#0 stuck - Kernel 3.17.2

2014-11-21 Thread Patrick Schmid
On 15.11.2014 00:47, Chris Mason wrote: > > Ok, I think this is related to the new fair read/writer lock > implementation in the generic kernel code. btrfs_clear_path_blocking() > will end up taking locks outside of the strict locking order the rest > of Btrfs uses. This used to be fine becau

Re: btrfs send and an existing backup

2014-11-21 Thread Austin S Hemmelgarn
On 2014-11-20 09:10, Duncan wrote: Bardur Arantsson posted on Thu, 20 Nov 2014 14:17:52 +0100 as excerpted: If you have no other backups, I would really recommend that you *don't* use btrfs for your backup, or at least have a *third* backup which isn't on btrfs -- there are *still* problems wit

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Robert White posted on Fri, 21 Nov 2014 03:35:05 -0800 as excerpted: > On 11/20/2014 10:22 PM, Duncan wrote: >> But while other filesystems might allow un-UUIDs (heh, UUUIDs or U3IDs >> =:^), because they're no longer unique, requiring them to be unique >> just as the label says cannot be consider

Re: Changing label few times killed filesystem?

2014-11-21 Thread Duncan
Chris Murphy posted on Thu, 20 Nov 2014 19:20:22 -0700 as excerpted: > On Thu, Nov 20, 2014 at 6:27 PM, Boris Chernov > wrote: > >> Since it failed after "checking extents" I decided to try >> --init-extent-tree: > > There might be hope yet if you didn't use --repair which is said on the >

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Robert White
On 11/20/2014 10:22 PM, Duncan wrote: But while other filesystems might allow un-UUIDs (heh, UUUIDs or U3IDs =:^), because they're no longer unique, requiring them to be unique just as the label says cannot be considered a bug. It's simply stricter enforcement of the rules, which are, after all,

Re: [PATCH] btrfs: remove empty fs_devices to prevent memory runout

2014-11-21 Thread Anand Jain
Hi Gui, Thanks for attempting this. There was this patch previously attempted for the same problem, which I had to nack.. [PATCH 1/2] btrfs: device list could grow infinite I haven't seen your fix in full yet, But looks like you are using dev_t to suffice the problem of identifying uniq

Re: [PATCH v2 0/5] btrfs: implement swap file support

2014-11-21 Thread Christoph Hellwig
On Fri, Nov 21, 2014 at 02:15:31AM -0800, Omar Sandoval wrote: > Sorry for the noise, looks like Christoph got back to me on the previous RFC > just before I sent this out -- disregard this for now. If the NFS people are fine with this version I'd certainly welcome it as a first step. Additional

Re: [PATCH v2 0/5] btrfs: implement swap file support

2014-11-21 Thread Omar Sandoval
Sorry for the noise, looks like Christoph got back to me on the previous RFC just before I sent this out -- disregard this for now. -- Omar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http:

[PATCH v2 0/5] btrfs: implement swap file support

2014-11-21 Thread Omar Sandoval
This patch series, based on 3.18-rc5, implements support for swap files on BTRFS. The standard swap file implementation uses the filesystem's implementation of bmap() to get a list of physical blocks on disk, which the swap file code then does I/O on directly without going through the filesystem.

[PATCH v2 3/5] swap: use direct I/O for SWP_FILE swap_readpage

2014-11-21 Thread Omar Sandoval
On Mon, Nov 17, 2014 at 07:48:17AM -0800, Christoph Hellwig wrote: > With the new iov_iter infrastructure that supprots direct I/O to kernel > pages please get rid of the ->readpage hack first. I'm still utterly > disapoined that this crap ever got merged. Cc: Mel Gorman Signed-off-by: Omar Sand

[PATCH v2 4/5] btrfs: don't allow -C or +c chattrs on a swap file

2014-11-21 Thread Omar Sandoval
swap_activate will check for a compressed or copy-on-write file; we shouldn't allow it to become either once it has already been activated. Signed-off-by: Omar Sandoval --- fs/btrfs/ioctl.c | 50 +++--- 1 file changed, 31 insertions(+), 19 deletions(-)

Re: 3.18.0-rc4 - WARNING: CPU: 3 PID: 3889 at fs/btrfs/extent-tree.c:3785 btrfs_free_reserved_data_space+0xc8/0xe0 [btrfs]()

2014-11-21 Thread Torbjørn
On 12. nov. 2014 09:29, Torbjørn wrote: Hi, After upgrade to 3.18.0-rc4 get a lot of these warnings. The system seems to otherwise behave normally. I have not observed any actual issues. --snip-- Just to close off this email thread. After upgrading to 3.18.0-rc5 these messages have disappea

Re: [RFC PATCH 0/6] btrfs: implement swap file support

2014-11-21 Thread Omar Sandoval
On Fri, Nov 21, 2014 at 02:06:57AM -0800, Christoph Hellwig wrote: > On Tue, Nov 18, 2014 at 11:22:35PM -0800, Omar Sandoval wrote: > > Here's a nice little bit of insanity I put together in that direction -- > > consider it a discussion point more than a patch. It does two things: > > > > - Uses

[PATCH v2 1/5] direct-io: don't dirty ITER_BVEC pages on read

2014-11-21 Thread Omar Sandoval
Reads through the iov_iter infrastructure for kernel pages shouldn't be dirtied by the direct I/O code. This is based on Dave Kleikamp's and Ming Lei's previously posted patches. Cc: Dave Kleikamp Cc: Ming Lei Signed-off-by: Omar Sandoval --- fs/direct-io.c | 8 +--- 1 file changed, 5 ins

[PATCH v2 5/5] btrfs: enable swap file support

2014-11-21 Thread Omar Sandoval
Implement the swap file a_ops on btrfs. Activation simply checks for a usable swap file: it must be fully allocated (no holes), support direct I/O (so no compressed or inline extents) and should be nocow (I'm not sure about that last one). Signed-off-by: Omar Sandoval --- fs/btrfs/inode.c | 71 +

[PATCH v2 2/5] nfs: don't dirty ITER_BVEC pages read through direct I/O

2014-11-21 Thread Omar Sandoval
As with the generic blockdev code, kernel pages shouldn't be dirtied by the direct I/O path. Signed-off-by: Omar Sandoval --- fs/nfs/direct.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 10bf072..a67fa2c 100644 --- a/fs/nfs/direc

Re: [RFC PATCH 0/6] btrfs: implement swap file support

2014-11-21 Thread Christoph Hellwig
On Tue, Nov 18, 2014 at 11:22:35PM -0800, Omar Sandoval wrote: > Here's a nice little bit of insanity I put together in that direction -- > consider it a discussion point more than a patch. It does two things: > > - Uses an ITER_BVEC iov_iter to do direct_IO for swap_readpage. This makes > swap_

Re: Changing label few times killed filesystem?

2014-11-21 Thread Boris Chernov
On 2014-11-21 04:35, Roman Mamedov wrote: On Fri, 21 Nov 2014 01:27:17 + Boris Chernov wrote: I have changed file system label few times in total. When I tried to mount it after that, it became not mountable: # mount /dev/sdb1 /mnt mount: Not a directory I'd say that implies somethi

[PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread Daniel Dressler
This is the 3rd independent patch of a larger project to cleanup btrfs's internal usage of btrfs_root. Many functions take btrfs_root only to grab the fs_info struct. By requiring a root these functions cause programmer overhead. That these functions can accept any valid root is not obvious until