Re: [PATCH 5/5] btrfs: move error handling code together in ctree.h
On Wed, Mar 16, 2016 at 04:43:08PM +0800, Anand Jain wrote: > Looks like we added the incompatible defines in between the error > handling defines in the file ctree.h. Now group them back. > > Signed-off-by: Anand JainReviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: remove unused function btrfs_assert()
On Wed, Mar 16, 2016 at 04:43:07PM +0800, Anand Jain wrote: > Apparently looks like ASSERT does the same intended job, > as intended btrfs_assert(). > > Signed-off-by: Anand JainReviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: rename btrfs_std_error() to btrfs_handle_fs_error()
On Wed, Mar 16, 2016 at 04:43:06PM +0800, Anand Jain wrote: > btrfs_std_error() handles errors, puts FS into readonly mode > (as of now). So its good idea to rename it to btrfs_handle_fs_error(). > So that bugs like > btrfs: pass the error code to the btrfs_std_error and log ret > (in the ML) may be avoided. > > Signed-off-by: Anand JainReviewed-by: David Sterba btrfs_handle_fs_error is much better. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: rename save_error_info to set_fs_error
On Wed, Mar 16, 2016 at 04:43:05PM +0800, Anand Jain wrote: > @@ -171,7 +171,7 @@ void __btrfs_std_error(struct btrfs_fs_info *fs_info, > const char *function, > #endif > > /* Don't go through full error handling during mount */ > - save_error_info(fs_info); > + set_fs_error(fs_info); The helper is too trivial, can you please drop it and replace by the it's code here? > if (sb->s_flags & MS_BORN) > btrfs_handle_error(fs_info); > } -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[4.4.1] btrfs-transacti frequent high CPU usage despite little fragmentation
Hi, on my box, frequently, mostly while using firefox, any process doing disk IO freezes while btrfs-transacti has a spike in CPU usage for more than a minute. I know about btrfs' fragmentation issue, but have a couple of questions: * While btrfs-transacti is spiking, can I trace which files are the culprit somehow? * On my setup, with measured fragmentation, are the CPU spike durations and freezes normal? * Can I alleviate the situation by anything except defragmentation? Any insight is appreciated. Details: I have a 1TB SSD with a large btrfs partition: # btrfs filesystem usage / Overall: Device size: 915.32GiB Device allocated:915.02GiB Device unallocated: 306.00MiB Device missing: 0.00B Used:152.90GiB Free (estimated):751.96GiB (min: 751.96GiB) Data ratio: 1.00 Metadata ratio: 1.00 Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:901.01GiB, Used:149.35GiB /dev/sda2 901.01GiB Metadata,single: Size:14.01GiB, Used:3.55GiB /dev/sda2 14.01GiB System,single: Size:4.00MiB, Used:128.00KiB /dev/sda2 4.00MiB Unallocated: /dev/sda2 306.00MiB I've done the obvious and defragmented files. Some files were defragmented from 10k+ to still more than 100 extents. But the problem persisted or came back very quickly. Just now i re-ran defragmentation with the following results (only showing files with more than 100 extents before fragmentation): extents before / extents after / anonymized path 103 / 1 /home/foo/.mozilla/firefox/foo.default/formhistory.sqlite: 133 / 1 /home/foo/.thunderbird/foo.default/ImapMail/imap.example.org/ml-btrfs: 155 / 1 /var/log/messages: 158 / 30 /home/foo/.thunderbird/foo.default/ImapMail/mail.example.org/INBOX: 160 / 32 /home/foo/.thunderbird/foo.default/calendar-data/cache.sqlite: 255 / 255 /var/lib/docker/devicemapper/devicemapper/data: 550 / 1 /home/foo/.cache/chromium/Default/Cache/data_1: 627 / 1 /home/foo/.cache/chromium/Default/Cache/data_2: 1738 / 25 /home/foo/.cache/chromium/Default/Cache/data_3: 1764 / 77 /home/foo/.mozilla/firefox/foo.default/places.sqlite: 4414 / 284 /home/foo/.digikam/thumbnails-digikam.db: 6576 / 3 /home/foo/.digikam/digikam4.db: So fragmentation came back quickly, and the firefox places.sqlite file could explain why the system freezes while browsing. BTW: I did a VACUUM on the sqlite db and afterwards it had 1 extent. Expected, just saying that vacuuming seems to be a good measure for defragmenting sqlite databases. I am using snapper and have about 40 snapshots going back for some months. Those are read only. Could that have any effect? Cheers, Ole -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: use btrfs-debugfs to fetch block group information
On Mon, Jan 11, 2016 at 12:10:22PM -0800, Liu Bo wrote: > From: liub.li...@gmail.com> > This aims to decide whether a balance can reduce the number of > data block groups and if it is, this shows the '-dvrange' block > group's objectid. > > Signed-off-by: Liu Bo Applied. I consider btrfs-debugfs as a debugging utility only, without guarantees of a stable output or commandline arugments. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2 v2] btrfs-progs: Fix a regression that "property" with -t option doesn't work
On Wed, Mar 16, 2016 at 11:15:42AM +0900, Satoru Takeuchi wrote: > Sorry, I forgot to add "btrfs-progs: " in the subject line. > --- > "property" is considered as working without any options > from the following commit. > > commit 176aeca9a148 ("btrfs-progs: add getopt stubs where needed") > > However, we can pass -t option to this command. > > * actual result > > == > $ ./btrfs prop list -t f /btrfs > btrfs property list: invalid option -- 't' > usage: btrfs property list [-t ] > > Lists available properties with their descriptions for the given object. > > Please see the help of 'btrfs property get' for a description of > objects and object types. > > == > > * expected result > > == > $ ./btrfs prop list -t f /btrfs > label Set/get label of device. > == > > Signed-off-by: Satoru Takeuchi
Re: [PATCH 1/2] btrfs-progs: Describe optarg of -m option in the manpage of receive
On Wed, Mar 16, 2016 at 11:01:11AM +0900, Satoru Takeuchi wrote: > Signed-off-by: Satoru TakeuchiApplied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Incompat features: raid56 ... when creating a RAID6?
Am Mittwoch, 16. März 2016, 08:52:52 CET schrieb Anand Jain: > Yes looks like. As it was there when I read your posting. > > > >>> AndiN‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±nÚß²)í…æèw*jg¬±¨¶‰šŽŠÝ¢j/ê > >>> äz > >>> ¹Þ–Šà2ŠÞ™¨èÚ&¢)ß¡«a¶Úþø®G«éh®æj:+v‰¨Šwè†Ù¥ I'll have a look at this. I have no idea where this came from. But it seems this stuff is invisible on the list archives on the web on my initial message... N�r��yb�X��ǧv�^�){.n�+{�n�߲)w*jg����ݢj/���z�ޖ��2�ޙ&�)ߡ�a�����G���h��j:+v���w��٥
Re: task btrfs-cleaner:770 blocked for more than 120 seconds.
On Tue, 15 Mar 2016 09:18:14 +0500 Михаил Гавриловwrote: > Yesterday btrfs again hangs on kernel 4.4.5 > It's the same or another problem? Seems different to me, as there's also defrag involved. Try removing and not using the defrag mount option. > This messages say that restore failed? So you can't mount the FS anymore? Did you try btrfsck, what does it report? -- With respect, Roman pgpXO1jr0gUvC.pgp Description: OpenPGP digital signature
[PATCH] btrfs: rename save_error_info to set_fs_error
Actually save_error_info() is setting the FS state to error. So rename is more appropriate. Signed-off-by: Anand Jain--- fs/btrfs/super.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 63a74303da9e..a11823b317ac 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -97,7 +97,7 @@ const char *btrfs_decode_error(int errno) return errstr; } -static void save_error_info(struct btrfs_fs_info *fs_info) +static void set_fs_error(struct btrfs_fs_info *fs_info) { /* * today we only save the error info into ram. Long term we'll @@ -171,7 +171,7 @@ void __btrfs_std_error(struct btrfs_fs_info *fs_info, const char *function, #endif /* Don't go through full error handling during mount */ - save_error_info(fs_info); + set_fs_error(fs_info); if (sb->s_flags & MS_BORN) btrfs_handle_error(fs_info); } -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: rename btrfs_std_error() to btrfs_handle_fs_error()
btrfs_std_error() handles errors, puts FS into readonly mode (as of now). So its good idea to rename it to btrfs_handle_fs_error(). So that bugs like btrfs: pass the error code to the btrfs_std_error and log ret (in the ML) may be avoided. Signed-off-by: Anand Jain--- fs/btrfs/ctree.c | 6 +++--- fs/btrfs/ctree.h | 6 +++--- fs/btrfs/disk-io.c | 8 fs/btrfs/extent-tree.c | 2 +- fs/btrfs/inode-item.c | 2 +- fs/btrfs/ioctl.c | 2 +- fs/btrfs/relocation.c | 2 +- fs/btrfs/root-tree.c | 4 ++-- fs/btrfs/super.c | 6 +++--- fs/btrfs/transaction.c | 2 +- fs/btrfs/tree-log.c| 8 fs/btrfs/volumes.c | 14 +++--- 12 files changed, 31 insertions(+), 31 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 769e0ff1b4ce..b12d0815a754 100644 --- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c @@ -1011,7 +1011,7 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans, return ret; if (refs == 0) { ret = -EROFS; - btrfs_std_error(root->fs_info, ret, NULL); + btrfs_handle_fs_error(root->fs_info, ret, NULL); return ret; } } else { @@ -1927,7 +1927,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans, child = read_node_slot(root, mid, 0); if (!child) { ret = -EROFS; - btrfs_std_error(root->fs_info, ret, NULL); + btrfs_handle_fs_error(root->fs_info, ret, NULL); goto enospc; } @@ -2030,7 +2030,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans, */ if (!left) { ret = -EROFS; - btrfs_std_error(root->fs_info, ret, NULL); + btrfs_handle_fs_error(root->fs_info, ret, NULL); goto enospc; } wret = balance_node_right(trans, root, mid, left); diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index c9fb7b9ca8a4..d6ae2d324080 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -4286,7 +4286,7 @@ static inline void assfail(char *expr, char *file, int line) #define btrfs_assert() __printf(5, 6) __cold -void __btrfs_std_error(struct btrfs_fs_info *fs_info, const char *function, +void __btrfs_handle_fs_error(struct btrfs_fs_info *fs_info, const char *function, unsigned int line, int errno, const char *fmt, ...); const char *btrfs_decode_error(int errno); @@ -4429,9 +4429,9 @@ do { \ __LINE__, (errno)); \ } while (0) -#define btrfs_std_error(fs_info, errno, fmt, args...) \ +#define btrfs_handle_fs_error(fs_info, errno, fmt, args...)\ do { \ - __btrfs_std_error((fs_info), __func__, __LINE__,\ + __btrfs_handle_fs_error((fs_info), __func__, __LINE__, \ (errno), fmt, ##args);\ } while (0) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index f76b0d96ffdb..b59c98c723f7 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2410,7 +2410,7 @@ static int btrfs_replay_log(struct btrfs_fs_info *fs_info, /* returns with log_tree_root freed on success */ ret = btrfs_recover_log_trees(log_tree_root); if (ret) { - btrfs_std_error(tree_root->fs_info, ret, + btrfs_handle_fs_error(tree_root->fs_info, ret, "Failed to recover log tree"); free_extent_buffer(log_tree_root->node); kfree(log_tree_root); @@ -3631,7 +3631,7 @@ static int write_all_supers(struct btrfs_root *root, int max_mirrors) if (ret) { mutex_unlock( >fs_info->fs_devices->device_list_mutex); - btrfs_std_error(root->fs_info, ret, + btrfs_handle_fs_error(root->fs_info, ret, "errors while submitting device barriers."); return ret; } @@ -3671,7 +3671,7 @@ static int write_all_supers(struct btrfs_root *root, int max_mirrors) mutex_unlock(>fs_info->fs_devices->device_list_mutex); /* FUA is masked off if unsupported and can't be the reason */ - btrfs_std_error(root->fs_info, -EIO, + btrfs_handle_fs_error(root->fs_info, -EIO, "%d errors while writing supers", total_errors); return -EIO; } @@ -3689,7 +3689,7 @@ static int write_all_supers(struct btrfs_root *root, int max_mirrors)
[PATCH] btrfs: remove unused function btrfs_assert()
Apparently looks like ASSERT does the same intended job, as intended btrfs_assert(). Signed-off-by: Anand Jain--- fs/btrfs/ctree.h | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index d6ae2d324080..bb5a1114da7c 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -4283,7 +4283,6 @@ static inline void assfail(char *expr, char *file, int line) #define ASSERT(expr) ((void)0) #endif -#define btrfs_assert() __printf(5, 6) __cold void __btrfs_handle_fs_error(struct btrfs_fs_info *fs_info, const char *function, -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/5] btrfs: move error handling code together in ctree.h
Looks like we added the incompatible defines in between the error handling defines in the file ctree.h. Now group them back. Signed-off-by: Anand Jain--- fs/btrfs/ctree.h | 78 +--- 1 file changed, 40 insertions(+), 38 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index bb5a1114da7c..d5848b8b67be 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -4295,6 +4295,46 @@ void __btrfs_abort_transaction(struct btrfs_trans_handle *trans, struct btrfs_root *root, const char *function, unsigned int line, int errno); +/* + * Call btrfs_abort_transaction as early as possible when an error condition is + * detected, that way the exact line number is reported. + */ +#define btrfs_abort_transaction(trans, root, errno)\ +do { \ + /* Report first abort since mount */\ + if (!test_and_set_bit(BTRFS_FS_STATE_TRANS_ABORTED, \ + &((root)->fs_info->fs_state))) {\ + WARN(1, KERN_DEBUG \ + "BTRFS: Transaction aborted (error %d)\n", \ + (errno)); \ + } \ + __btrfs_abort_transaction((trans), (root), __func__,\ + __LINE__, (errno)); \ +} while (0) + +#define btrfs_handle_fs_error(fs_info, errno, fmt, args...)\ +do { \ + __btrfs_handle_fs_error((fs_info), __func__, __LINE__, \ + (errno), fmt, ##args);\ +} while (0) + +__printf(5, 6) +__cold +void __btrfs_panic(struct btrfs_fs_info *fs_info, const char *function, + unsigned int line, int errno, const char *fmt, ...); +/* + * If BTRFS_MOUNT_PANIC_ON_FATAL_ERROR is in mount_opt, __btrfs_panic + * will panic(). Otherwise we BUG() here. + */ +#define btrfs_panic(fs_info, errno, fmt, args...) \ +do { \ + __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \ + BUG(); \ +} while (0) + + +/* compatibility and incompatibility defines */ + #define btrfs_set_fs_incompat(__fs_info, opt) \ __btrfs_set_fs_incompat((__fs_info), BTRFS_FEATURE_INCOMPAT_##opt) @@ -4411,44 +4451,6 @@ static inline int __btrfs_fs_compat_ro(struct btrfs_fs_info *fs_info, u64 flag) return !!(btrfs_super_compat_ro_flags(disk_super) & flag); } -/* - * Call btrfs_abort_transaction as early as possible when an error condition is - * detected, that way the exact line number is reported. - */ -#define btrfs_abort_transaction(trans, root, errno)\ -do { \ - /* Report first abort since mount */\ - if (!test_and_set_bit(BTRFS_FS_STATE_TRANS_ABORTED, \ - &((root)->fs_info->fs_state))) {\ - WARN(1, KERN_DEBUG \ - "BTRFS: Transaction aborted (error %d)\n", \ - (errno)); \ - } \ - __btrfs_abort_transaction((trans), (root), __func__,\ - __LINE__, (errno)); \ -} while (0) - -#define btrfs_handle_fs_error(fs_info, errno, fmt, args...)\ -do { \ - __btrfs_handle_fs_error((fs_info), __func__, __LINE__, \ - (errno), fmt, ##args);\ -} while (0) - -__printf(5, 6) -__cold -void __btrfs_panic(struct btrfs_fs_info *fs_info, const char *function, - unsigned int line, int errno, const char *fmt, ...); - -/* - * If BTRFS_MOUNT_PANIC_ON_FATAL_ERROR is in mount_opt, __btrfs_panic - * will panic(). Otherwise we BUG() here. - */ -#define btrfs_panic(fs_info, errno, fmt, args...) \ -do { \ - __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \ - BUG(); \ -} while (0) - /* acl.c */ #ifdef CONFIG_BTRFS_FS_POSIX_ACL struct posix_acl *btrfs_get_acl(struct inode *inode, int type); -- 2.7.0 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Major HDD performance degradation on btrfs receive
On Tue, Mar 15, 2016 at 10:23 PM, Nazar Mokrynskyiwrote: > Sounds like a really good idea! > > I'll try to implement in in my backup tool, but it might take some time to > see real benefit from it (or no benefit:)). There is a catch. I'm not sure how much testing deleting 100 subvolumes at once gets. It should work. I haven't looked in xfstests to see how much of this is being tested. So it's possible you're testing it. So be ready. Also since it's batched, consider doing it at night when it's not used. The cleaner task will always slow things down because it has to decrement reference counts and then find out what to actually delete and then update metadata to reflect it. You could also add in a 5 minute delay after subvolume deletes then issue sysrq+w in case there's significant blocked tasks, the logs will have extra debug info. Another idea is maybe graphically modeling (seekwatcher) the normal write pattern, and see how it changes when even a single subvolume is deleted (the current every 15 minute method). That might give an idea how significantly cleaner tasks affect your particular workload. The results might support batching not only to avoid fragmentation by getting larger contiguous free space, but to avoid the IOPS hit during the day time from too aggressive (?) of a cleaner task. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stupid (?) Idea about extent lifetimes.
It occurs to me that it would be desirable to mark extents as "least favoured nations" and so all new writes would like to not be written there and any data written there would have a desire to be somewhere else. So lets say the wholly unallocated space has a natural status of 100. Allocated blocks would normally have statuses less than that by a trivial amount, such as 99. One could then marks blocks with higher numbers for being less favoured, or lower numbers for being more favoured as desired. Basically this would create a gravity map of sorts that would be factored into allocation decisions. So say you just converted an ext4 to btrfs. It's got all those oddly sized and placed extents. You could give them all higher numbers in hopes that the data would naturally migrate away. Say just number them really large with no two numbers the same. Now the largest number would naturally become vacant and likely to be freed. Likewise you could weight your data to migrate spindle-ward or such in the weeks before a reorg. Similarly changes in geometry could simply mark segments as ill-favoured where the old geometry doesn't match the new and data would migrate under pressure. One could reverse the age induced entropy of a file system by just periodically increasing the disfavour values of all the blocks, causing the oldest blocks to be the least favoured of all, and so creating a slowly rolling pattern. So say new blocks start life as 50s by default. And empty space is 100. And every so often every block gets an increment (say 3 so that 100 is naturally skipped over. Young blocks are now very magnetic. As they age they lose favour. Eventually they pass the value for unallocated space. Then they start losing data and eventually, in a system with 100 percent turnover the blocks get deallocated. defragging and occasional balancing take care of the files that "never" change. Very high numbers could also be reserved for pinning. Specially flagged files would have reverse gravity. A desire to stay put. So say a NOCOW files or Swap Files might have reverse gravity and one could use some tool to allocate blocks at the cold end of the disk with those sorts of numbers. Effectively segregating the the static from the churning data. Fresh files would thereby tend to vacate extents full of snapshot data and freeing static (Read only) snapshot data would tend to release contiguous space. As the disk runs out of space the system naturally breaks down into the existing best-fit allocation. It's less than a defrag or autodefrag, or balance, and it would tend to be more like digestive peristalsis, at the extreme end (where people are taking way too many snapshots) it becomes an elevator sort by extent age. (If this is a tired idea, my apologies. I took cough medicine a little while ago and this thought that's been rattling around in my head for months bubbled out of its cauldron.) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html