Re: Broken filesystem, looking for guidance

2018-11-18 Thread Qu Wenruo
On 2018/11/19 上午3:37, Sébastien Luttringer wrote: > Hello, > > I opened a bug report[1] Normally mail list will give you a quick response. > about a situation I have with a btrfs filesystem on my > backup server. Looks like there is internal errors inside the filesytem which > cannot be fixed

Important

2018-11-18 Thread Reem Al-Hashimi
Hello, My name is ms. Reem Al-Hashimi. The UAE minister of state for international cooparation. I got your contact from a certain email database from your country while i was looking for someone to handle a huge financial transaction for me in confidence. Can you receive and invest on behalf

Broken filesystem, looking for guidance

2018-11-18 Thread Sébastien Luttringer
Hello, I opened a bug report[1] about a situation I have with a btrfs filesystem on my backup server. Looks like there is internal errors inside the filesytem which cannot be fixed by btrfs check. Currently, each time I cat a specific file, the filesystem is remounted read-only. How can I get

Re: bad tree block start, want 705757184 have 82362368

2018-11-18 Thread Anand Jain
On 11/18/2018 03:56 PM, Stephan Olbrich wrote: Am Sonntag, 18. November 2018, 01:30:14 CET schrieb Qu Wenruo: Late on I got the same errors for my /home partition (on the same drive) as well. I have snapshots of all partitions on another drive made by btrbk. To get a working system, I made

Re: bad tree block start, want 705757184 have 82362368

2018-11-17 Thread Stephan Olbrich
Am Sonntag, 18. November 2018, 01:30:14 CET schrieb Qu Wenruo: > >>> Late on I got the same errors for my /home partition (on the same drive) > >>> as well. I have snapshots of all partitions on another drive made by > >>> btrbk. To get a working system, I made new (rw) snapshots of the most > >>>

Re: bad tree block start, want 705757184 have 82362368

2018-11-17 Thread Qu Wenruo
On 2018/11/18 上午4:28, Stephan Olbrich wrote: > Am Samstag, 17. November 2018, 09:03:04 CET schrieb Qu Wenruo: >> On 2018/11/17 上午12:17, Stephan Olbrich wrote: >>> Hi, >>> >>> a few days ago my root file system (simple btrfs on a SSD, no RAID or >>> anything) suddenly became read only. Looking at

Re: bad tree block start, want 705757184 have 82362368

2018-11-17 Thread Stephan Olbrich
Am Freitag, 16. November 2018, 17:44:35 CET schrieb Nikolay Borisov: > On 16.11.18 г. 18:17 ч., Stephan Olbrich wrote: > > Hi, > > > > a few days ago my root file system (simple btrfs on a SSD, no RAID or > > anything) suddenly became read only. Looking at dmsg, I found this: > > > > [

Re: bad tree block start, want 705757184 have 82362368

2018-11-17 Thread Stephan Olbrich
Am Samstag, 17. November 2018, 09:03:04 CET schrieb Qu Wenruo: > On 2018/11/17 上午12:17, Stephan Olbrich wrote: > > Hi, > > > > a few days ago my root file system (simple btrfs on a SSD, no RAID or > > anything) suddenly became read only. Looking at dmsg, I found this: > > > > [ 19.285020]

Re: btrfs convert problem

2018-11-17 Thread Qu Wenruo
On 2018/11/17 下午5:49, Serhat Sevki Dincer wrote: > Hi, > > On my second attempt to convert my 698 GiB usb HDD from ext4 to btrfs > with btrfs-progs 4.19 from Manjaro (kernel 4.14.80): > > I identified bad files with > $ find . -type f -exec cat {} > /dev/null \; > This revealed 6 corrupted

Re: btrfs convert problem

2018-11-17 Thread Serhat Sevki Dincer
Hi, On my second attempt to convert my 698 GiB usb HDD from ext4 to btrfs with btrfs-progs 4.19 from Manjaro (kernel 4.14.80): I identified bad files with $ find . -type f -exec cat {} > /dev/null \; This revealed 6 corrupted files, I deleted them. Tried it again with no error message. Then I

Re: bad tree block start, want 705757184 have 82362368

2018-11-17 Thread Qu Wenruo
On 2018/11/17 上午12:17, Stephan Olbrich wrote: > Hi, > > a few days ago my root file system (simple btrfs on a SSD, no RAID or > anything) suddenly became read only. > Looking at dmsg, I found this: > > [ 19.285020] BTRFS error (device sda2): bad tree block start, want > 705757184 have

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: btrfs_can_relocate returns 0 when it concludes the given chunk can be relocated and -1 otherwise. Since this function is used as a predicated and it return a binary value it makes no sense to have it's return value as an int so change it to bool.

Re: [PATCH 5/5] btrfs: Replace BUG_ON with ASSERT in find_lock_delalloc_range

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: lock_delalloc_pages should only return 2 values - 0 in case of success and -EAGAIN if the range of pages to be locked should be shrunk due to some of gone. Manual inspections confirms that this is indeed the case since __process_pages_contig is

Re: [PATCH 4/5] btrfs: Sink find_lock_delalloc_range's 'max_bytes' argument

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: All callers of this function pass BTRFS_MAX_EXTENT_SIZE (128M) so let's reduce the argument count and make that a local variable. No functional changes. Signed-off-by: Nikolay Borisov Reviewed-by: Anand Jain Thanks, Anand ---

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-16 Thread Chris Murphy
On Thu, Nov 15, 2018 at 10:39 AM Juan Alberto Cirez wrote: > > Is BTRFS mature enough to be deployed on a production system to underpin > the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? > > Based on our limited experience with BTRFS (1+ year) under the above > scenario the

Re: [PATCH 3/5] btrfs: Remove superfluous check form btrfs_remove_chunk

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: It's unnecessary to check map->stripes[i].dev for NULL given its value is already set and dereferenced above the the check. No functional changes. Signed-off-by: Nikolay Borisov Reviewed-by: Anand Jain Thanks, Anand ---

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-16 Thread David Sterba
On Fri, Nov 16, 2018 at 08:06:36PM +0800, Anand Jain wrote: > > > On 11/15/2018 11:35 PM, David Sterba wrote: > > On Sun, Nov 11, 2018 at 10:22:22PM +0800, Anand Jain wrote: > >> When we successfully cancel the replace its scrub returns -ECANCELED, > >> which then passed to

Re: bad tree block start, want 705757184 have 82362368

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 18:17 ч., Stephan Olbrich wrote: > Hi, > > a few days ago my root file system (simple btrfs on a SSD, no RAID or > anything) suddenly became read only. > Looking at dmsg, I found this: > > [ 19.285020] BTRFS error (device sda2): bad tree block start, want > 705757184 have

bad tree block start, want 705757184 have 82362368

2018-11-16 Thread Stephan Olbrich
Hi, a few days ago my root file system (simple btrfs on a SSD, no RAID or anything) suddenly became read only. Looking at dmsg, I found this: [ 19.285020] BTRFS error (device sda2): bad tree block start, want 705757184 have 82362368 [ 19.285042] BTRFS: error (device sda2) in

Re: [PATCH 0/5] Misc cleanups in balance code

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 17:18 ч., David Sterba wrote: > On Fri, Oct 26, 2018 at 02:43:16PM +0300, Nikolay Borisov wrote: >> While investigating the balance hang I came across various inconsistencies >> in >> the source. This series aims to fix those. >> >> The first patch is (I believe) a fix to a

Re: [PATCH] Btrfs: allow clear_extent_dirty() to receive a cached extent state record

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 15:04 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > We can have a lot freed extents during the life span of transaction, so > the red black tree that keeps track of the ranges of each freed extent > (fs_info->freed_extents[]) can get quite big. When finishing a

Re: [PATCH 0/5] Misc cleanups in balance code

2018-11-16 Thread David Sterba
On Fri, Oct 26, 2018 at 02:43:16PM +0300, Nikolay Borisov wrote: > While investigating the balance hang I came across various inconsistencies in > the source. This series aims to fix those. > > The first patch is (I believe) a fix to a longstanding bug that could cause > balance to fail due to

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-16 Thread Anand Jain
On 11/16/2018 03:51 AM, Nikolay Borisov wrote: On 15.11.18 г. 20:39 ч., Juan Alberto Cirez wrote: Is BTRFS mature enough to be deployed on a production system to underpin the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? Based on our limited experience with BTRFS (1+

[PATCH] Btrfs: allow clear_extent_dirty() to receive a cached extent state record

2018-11-16 Thread fdmanana
From: Filipe Manana We can have a lot freed extents during the life span of transaction, so the red black tree that keeps track of the ranges of each freed extent (fs_info->freed_extents[]) can get quite big. When finishing a transaction commit we find each range, process it (discard the

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-16 Thread Austin S. Hemmelgarn
On 2018-11-15 13:39, Juan Alberto Cirez wrote: Is BTRFS mature enough to be deployed on a production system to underpin the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? For NVR, I'd say no. BTRFS does pretty horribly with append-only workloads, even if they are WORM

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-16 Thread Anand Jain
On 11/15/2018 11:35 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:22PM +0800, Anand Jain wrote: When we successfully cancel the replace its scrub returns -ECANCELED, which then passed to btrfs_dev_replace_finishing(), it cleans up based on the scrub returned status and propagates the

[PATCH] Btrfs: bring back key search optimization to btrfs_search_old_slot()

2018-11-16 Thread fdmanana
From: Filipe Manana Commit d7396f07358a ("Btrfs: optimize key searches in btrfs_search_slot"), dated from August 2013, introduced an optimization to search for keys in a node/leaf to both btrfs_search_slot() and btrfs_search_old_slot(). For the later, it ended up being reverted in commit

Re: [PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-16 Thread Anand Jain
On 11/16/2018 06:29 PM, Anand Jain wrote: On 11/15/2018 11:31 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:23PM +0800, Anand Jain wrote: As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. This has probably some user

Re: [PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-16 Thread Anand Jain
On 11/15/2018 11:31 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:23PM +0800, Anand Jain wrote: As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. This has probably some user visible effect or threre are steps to reproduce the

Re: [PATCH 0/9 v2] fix replace-start and replace-cancel racing

2018-11-16 Thread Anand Jain
On 11/15/2018 11:41 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:15PM +0800, Anand Jain wrote: v1->v2: 2/9: Drop writeback required 3/9: Drop writeback required 7/9: Use the condition within the WARN_ON() 6/9: Use the condition within the ASSERT() Replace-start and

Re: [PATCH v1.1 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 10:22 ч., Qu Wenruo wrote: > GCC 8.2.1 will report the following warning with "make W=1": > > ctree.c: In function 'btrfs_next_sibling_tree_block': > ctree.c:2990:21: warning: 'slot' may be used uninitialized in this function > [-Wmaybe-uninitialized] >

[PATCH v1.1 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-11-16 Thread Qu Wenruo
GCC 8.2.1 will report the following warning with "make W=1": ctree.c: In function 'btrfs_next_sibling_tree_block': ctree.c:2990:21: warning: 'slot' may be used uninitialized in this function [-Wmaybe-uninitialized] path->slots[level] = slot; ~~~^~ The culprit is

Re: [PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-11-16 Thread Qu Wenruo
On 2018/11/16 下午4:13, Nikolay Borisov wrote: > > > On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: >> The only location is the following code: >> >> int level = path->lowest_level + 1; >> BUG_ON(path->lowest_level + 1 >= BTRFS_MAX_LEVEL); >> while(level < BTRFS_MAX_LEVEL) { >>

Re: [PATCH v1.1 9/9] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 10:04 ч., Qu Wenruo wrote: > The following missing prototypes will be fixed: > 1) btrfs.c::handle_special_globals() > 2) check/mode-lowmem.c::repair_ternary_lowmem() > 3) extent-tree.c::btrfs_search_overlap_extent() > Above 3 can be fixed by making them static > > 4)

Re: [PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: > The only location is the following code: > > int level = path->lowest_level + 1; > BUG_ON(path->lowest_level + 1 >= BTRFS_MAX_LEVEL); > while(level < BTRFS_MAX_LEVEL) { > slot = path->slots[level] + 1; >

Re: [PATCH 5/9] btrfs-progs: Fix Wimplicit-fallthrough warning

2018-11-16 Thread Qu Wenruo
On 2018/11/16 下午4:04, Nikolay Borisov wrote: > > > On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: >> Although most fallthrough case is pretty obvious, we still need to teach >> the dumb compiler that it's an explicit fallthrough. >> >> Also reformat the code to use common indent. >> >>

Re: [PATCH 8/9] btrfs-progs: Fix Wtype-limits warning

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: > The only hit is the following code: > > tlv_len = le16_to_cpu(tlv_hdr->tlv_len); > > if (tlv_type == 0 || tlv_type > BTRFS_SEND_A_MAX > || tlv_len > BTRFS_SEND_BUF_SIZE) { >

Re: [PATCH 6/9] btrfs-progs: Fix Wsuggest-attribute=format warning

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: > Add __attribute__ ((format (printf, 4, 0))) to fix the vprintf calling > function. > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- > string-table.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/string-table.c

Re: [PATCH 5/9] btrfs-progs: Fix Wimplicit-fallthrough warning

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: > Although most fallthrough case is pretty obvious, we still need to teach > the dumb compiler that it's an explicit fallthrough. > > Also reformat the code to use common indent. > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov Is this

[PATCH v1.1 9/9] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes

2018-11-16 Thread Qu Wenruo
The following missing prototypes will be fixed: 1) btrfs.c::handle_special_globals() 2) check/mode-lowmem.c::repair_ternary_lowmem() 3) extent-tree.c::btrfs_search_overlap_extent() Above 3 can be fixed by making them static 4) utils.c::btrfs_check_nodesize() Fixed by moving it to

Re: [PATCH 4/9] btrfs-progs: Fix Wempty-body warning

2018-11-16 Thread Nikolay Borisov
On 16.11.18 г. 9:54 ч., Qu Wenruo wrote: > messages.h:49:24: warning: suggest braces around empty body in an 'if' > statement [-Wempty-body] > PRINT_TRACE_ON_ERROR;\ > > Just extra braces would solve the problem. > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- >

[PATCH v1.1 3/9] btrfs-progs: Makefile.extrawarn: Don't warn on sign compare

2018-11-16 Thread Qu Wenruo
Under most case, we are just using 'int' for 'unsigned int', and doesn't care about the sign. The Wsign-compare is causing tons of false alerts. Suppressing it would make W=1 less noisy. Signed-off-by: Qu Wenruo --- changelog: v1.1: Use cc-disable-warning to provide much better compatibility

[PATCH 5/9] btrfs-progs: Fix Wimplicit-fallthrough warning

2018-11-15 Thread Qu Wenruo
Although most fallthrough case is pretty obvious, we still need to teach the dumb compiler that it's an explicit fallthrough. Also reformat the code to use common indent. Signed-off-by: Qu Wenruo --- utils.c | 30 ++ 1 file changed, 22 insertions(+), 8 deletions(-)

[PATCH 2/9] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-11-15 Thread Qu Wenruo
From: Su Yanjun When using gcc8 compiles utils.c, it complains as below: utils.c:852:45: warning: '%s' directive output may be truncated writing up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] snprintf(path, sizeof(path), "/dev/mapper/%s", name);

[PATCH 8/9] btrfs-progs: Fix Wtype-limits warning

2018-11-15 Thread Qu Wenruo
The only hit is the following code: tlv_len = le16_to_cpu(tlv_hdr->tlv_len); if (tlv_type == 0 || tlv_type > BTRFS_SEND_A_MAX || tlv_len > BTRFS_SEND_BUF_SIZE) { error("invalid tlv in cmd tlv_type = %hu, tlv_len = %hu",

[PATCH 1/9] btrfs-progs: Makefile.extrawarn: Import cc-disable-warning

2018-11-15 Thread Qu Wenruo
We imported cc-option but forgot to import cc-disable-warning. Fixes: b556a992c3ad ("btrfs-progs: build: allow to build with various compiler warnings") Signed-off-by: Qu Wenruo --- Makefile.extrawarn | 6 ++ 1 file changed, 6 insertions(+) diff --git a/Makefile.extrawarn

[PATCH 0/9] btrfs-progs: Make W=1 clean (no "again")

2018-11-15 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/warning_fixes Which is based on v4.19 tag. This patchset will make "make W=1" reports no warning. This patch will first introduce fix to Makefile.extrawarn to make "cc-disable-warning" works, then disable

[PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-11-15 Thread Qu Wenruo
The only location is the following code: int level = path->lowest_level + 1; BUG_ON(path->lowest_level + 1 >= BTRFS_MAX_LEVEL); while(level < BTRFS_MAX_LEVEL) { slot = path->slots[level] + 1; ... } path->slots[level] = slot;

[PATCH 6/9] btrfs-progs: Fix Wsuggest-attribute=format warning

2018-11-15 Thread Qu Wenruo
Add __attribute__ ((format (printf, 4, 0))) to fix the vprintf calling function. Signed-off-by: Qu Wenruo --- string-table.c | 1 + 1 file changed, 1 insertion(+) diff --git a/string-table.c b/string-table.c index 95833768960d..455285702d51 100644 --- a/string-table.c +++ b/string-table.c @@

[PATCH 3/9] btrfs-progs: Makefile.extrawarn: Don't warn on sign compare

2018-11-15 Thread Qu Wenruo
Under most case, we are just using 'int' for 'unsigned int', and doesn't care about the sign. The Wsign-compare is causing tons of false alerts. Suppressing it would make W=1 less noisy. Signed-off-by: Qu Wenruo --- Makefile.extrawarn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[PATCH 4/9] btrfs-progs: Fix Wempty-body warning

2018-11-15 Thread Qu Wenruo
messages.h:49:24: warning: suggest braces around empty body in an 'if' statement [-Wempty-body] PRINT_TRACE_ON_ERROR;\ Just extra braces would solve the problem. Signed-off-by: Qu Wenruo --- messages.h | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git

[PATCH 9/9] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Qu Wenruo
The following missing prototypes will be fixed: 1) btrfs.c::handle_special_globals() 2) check/mode-lowmem.c::repair_ternary_lowmem() 3) extent-tree.c::btrfs_search_overlap_extent() Above 3 can be fixed by making them static 4) utils.c::btrfs_check_nodesize() Fixed by moving it to

Re: [PATCH] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-11-15 Thread Qu Wenruo
On 2018/10/26 上午9:58, Su Yanjun wrote: > When using gcc8 compiles utils.c, it complains as below: > > utils.c:852:45: warning: '%s' directive output may be truncated writing > up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] >snprintf(path, sizeof(path), "/dev/mapper/%s",

Re: [PATCH] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Qu Wenruo
On 2018/11/16 下午2:39, Nikolay Borisov wrote: > > > On 16.11.18 г. 8:13 ч., Qu Wenruo wrote: >> The following missing prototypes will be fixed: >> 1) btrfs.c::handle_special_globals() >> 2) check/mode-lowmem.c::repair_ternary_lowmem() >> 3) extent-tree.c::btrfs_search_overlap_extent() >>

Re: [PATCH] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Nikolay Borisov
On 16.11.18 г. 8:13 ч., Qu Wenruo wrote: > The following missing prototypes will be fixed: > 1) btrfs.c::handle_special_globals() > 2) check/mode-lowmem.c::repair_ternary_lowmem() > 3) extent-tree.c::btrfs_search_overlap_extent() >Above 3 can be fixed by making them static > > 4)

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-15 Thread Anand Jain
On 11/15/2018 11:27 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:20PM +0800, Anand Jain wrote: In btrfs_dev_replace_cancel() we should check if the btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread Anand Jain
On 11/15/2018 11:25 PM, David Sterba wrote: On Thu, Nov 15, 2018 at 03:00:21PM +0100, David Sterba wrote: On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: mutex_unlock(_replace->lock_finishing_cancel_unmount); return result; There's a compiler warning:

[PATCH] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes except free space tree

2018-11-15 Thread Qu Wenruo
The following missing prototypes will be fixed: 1) btrfs.c::handle_special_globals() 2) check/mode-lowmem.c::repair_ternary_lowmem() 3) extent-tree.c::btrfs_search_overlap_extent() Above 3 can be fixed by making them static 4) utils.c::btrfs_check_nodesize() Fixed by moving it to

Re: [PATCH v2] btrfs: add zstd compression level support

2018-11-15 Thread Nick Terrell
> On Nov 13, 2018, at 5:29 AM, Timofey Titovets wrote: > > вт, 13 нояб. 2018 г. в 04:52, Nick Terrell : >> >> >> >>> On Nov 12, 2018, at 4:33 PM, David Sterba wrote: >>> >>> On Wed, Oct 31, 2018 at 11:11:08AM -0700, Nick Terrell wrote: From: Jennifer Liu Adds zstd

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-15 Thread Roman Mamedov
On Thu, 15 Nov 2018 11:39:58 -0700 Juan Alberto Cirez wrote: > Is BTRFS mature enough to be deployed on a production system to underpin > the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? What are you looking to gain from using Btrfs on an NVR system? It doesn't sound like

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-15 Thread Nikolay Borisov
On 15.11.18 г. 20:39 ч., Juan Alberto Cirez wrote: > Is BTRFS mature enough to be deployed on a production system to underpin > the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? > > Based on our limited experience with BTRFS (1+ year) under the above > scenario the answer

BTRFS on production: NVR 16+ IP Cameras

2018-11-15 Thread Juan Alberto Cirez
Is BTRFS mature enough to be deployed on a production system to underpin the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? Based on our limited experience with BTRFS (1+ year) under the above scenario the answer seems to be no; but I wanted you ask the community at large

Re: [PATCH] Btrfs: send, fix infinite loop due to directory rename dependencies

2018-11-15 Thread David Sterba
On Wed, Nov 14, 2018 at 06:32:37PM +, fdman...@kernel.org wrote: > From: Robbie Ko > > When doing an incremental send, due to the need of delaying directory move > (rename) operations we can end up in infinite loop at > apply_children_dir_moves(). > > An example scenario that triggers this

Re: [PATCH 0/9 v2] fix replace-start and replace-cancel racing

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:15PM +0800, Anand Jain wrote: > v1->v2: > 2/9: Drop writeback required > 3/9: Drop writeback required > 7/9: Use the condition within the WARN_ON() > 6/9: Use the condition within the ASSERT() > > Replace-start and replace-cancel threads can race to create a

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:22PM +0800, Anand Jain wrote: > When we successfully cancel the replace its scrub returns -ECANCELED, > which then passed to btrfs_dev_replace_finishing(), it cleans up based > on the scrub returned status and propagates the same -ECANCELED back > the parent function.

Re: [PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:23PM +0800, Anand Jain wrote: > As of now only user requested replace cancel can cancel the replace-scrub > so no need to log error for it. This has probably some user visible effect or threre are steps to reproduce the message even if it's not expected (ie. the case

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-15 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:20PM +0800, Anand Jain wrote: > In btrfs_dev_replace_cancel() we should check if the > btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return > BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user can try to > cancel the replace again. I've

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 03:00:21PM +0100, David Sterba wrote: > On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: > > >> mutex_unlock(_replace->lock_finishing_cancel_unmount); > > >> return result; > > > > > > There's a compiler warning: > > > > > >

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread David Sterba
On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: > >>mutex_unlock(_replace->lock_finishing_cancel_unmount); > >>return result; > > > > There's a compiler warning: > > > > fs/btrfs/dev-replace.c: In function ‘btrfs_dev_replace_cancel’: > > fs/btrfs/dev-replace.c:865:9: warning:

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 01:23:25PM +0800, Qu Wenruo wrote: > > > On 2018/11/15 上午3:05, David Sterba wrote: > > On Tue, Nov 13, 2018 at 05:58:14PM +, Filipe Manana wrote: > >> That's the infinite loop issue fixed by one of the patches submitted > >> for 4.20-rc2: > >> > >>

Re: [PATCH 0/6] btrfs: fix compiler warning with make W=1

2018-11-15 Thread David Sterba
On Thu, Nov 15, 2018 at 09:30:42AM +0800, Qu Wenruo wrote: > BTW, could we just make "-Wunused-but-set-variable" default for btrfs > module instead of following the global setting? We'd need to fix all the warnings first, there are more left. We could make it on by default for the debugging build

Re: [PATCH 5/6] btrfs: remove unused variable tree in end_compressed_bio_write()

2018-11-15 Thread Johannes Thumshirn
On 14/11/2018 22:04, David Sterba wrote: > The fix has been folded to the original patch already, same the other > one removing 'tree'. If you used linux-next to run the build, then it's > probably lagging behind the development branches that are synced to > kernel.org after some testing. The

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-14 Thread Qu Wenruo
On 2018/11/15 上午3:05, David Sterba wrote: > On Tue, Nov 13, 2018 at 05:58:14PM +, Filipe Manana wrote: >> That's the infinite loop issue fixed by one of the patches submitted >> for 4.20-rc2: >> >>

Re: [PATCH RFC] btrfs: harden agaist duplicate fsid

2018-11-14 Thread Anand Jain
On 11/13/2018 11:47 PM, Anand Jain wrote: On 11/13/2018 11:31 PM, David Sterba wrote: On Mon, Oct 01, 2018 at 09:31:04PM +0800, Anand Jain wrote: +    /* + * we are going to replace the device path, make sure its the + * same device if the device mounted + */ + 

Re: [PATCH 0/6] btrfs: fix compiler warning with make W=1

2018-11-14 Thread Qu Wenruo
On 2018/11/14 下午9:35, Johannes Thumshirn wrote: > This patchset fixes most of the compiler warnings encountered when building > btrfs with make W=1. > > There are two more compiler warnings left in raid56.c: > CC [M] fs/btrfs/raid56.o > fs/btrfs/raid56.c: In function ‘finish_rmw’: >

Re: [PATCH] btrfs: Fix suspicious RCU usage warning in device_list_add

2018-11-14 Thread Lu Fengqi
On Wed, Nov 14, 2018 at 05:05:48PM +0100, David Sterba wrote: >On Wed, Nov 14, 2018 at 03:24:56PM +0800, Lu Fengqi wrote: >> = >> WARNING: suspicious RCU usage >> 4.20.0-rc2+ #23 Tainted: G O >> - >> fs/btrfs/volumes.c:886

Re: [PATCH 6/6] btrfs: unconditionally provide function prototypes from free-space-tree.h

2018-11-14 Thread David Sterba
On Wed, Nov 14, 2018 at 11:05:16PM +0200, Nikolay Borisov wrote: > > > On 14.11.18 г. 21:53 ч., David Sterba wrote: > > On Wed, Nov 14, 2018 at 02:54:37PM +0100, Johannes Thumshirn wrote: > >> On 14/11/2018 14:52, Nikolay Borisov wrote: > >>> I agree with this patch, however you go into the gray

Re: [PATCH 5/6] btrfs: remove unused variable tree in end_compressed_bio_write()

2018-11-14 Thread David Sterba
On Wed, Nov 14, 2018 at 02:35:19PM +0100, Johannes Thumshirn wrote: > Commit 2922040236f9 (btrfs: Remove extent_io_ops::writepage_end_io_hook) > removed the indirection to extent_io_ops::writepage_end_io_hook but didn't > remove the tree variable which then became unused. > > Remove 'tree' as

Re: [PATCH 6/6] btrfs: unconditionally provide function prototypes from free-space-tree.h

2018-11-14 Thread Nikolay Borisov
On 14.11.18 г. 21:53 ч., David Sterba wrote: > On Wed, Nov 14, 2018 at 02:54:37PM +0100, Johannes Thumshirn wrote: >> On 14/11/2018 14:52, Nikolay Borisov wrote: >>> I agree with this patch, however you go into the gray area of >>> "everything which is exported should have btrfs_ prefix". It's

Re: [PATCH 6/6] btrfs: unconditionally provide function prototypes from free-space-tree.h

2018-11-14 Thread David Sterba
On Wed, Nov 14, 2018 at 02:54:37PM +0100, Johannes Thumshirn wrote: > On 14/11/2018 14:52, Nikolay Borisov wrote: > > I agree with this patch, however you go into the gray area of > > "everything which is exported should have btrfs_ prefix". It's up to > > David to see if he is content with this

Re: [PATCH 5/6] btrfs: remove unused variable tree in end_compressed_bio_write()

2018-11-14 Thread Omar Sandoval
On Wed, Nov 14, 2018 at 02:35:19PM +0100, Johannes Thumshirn wrote: > Commit 2922040236f9 (btrfs: Remove extent_io_ops::writepage_end_io_hook) > removed the indirection to extent_io_ops::writepage_end_io_hook but didn't > remove the tree variable which then became unused. > > Remove 'tree' as

Re: [PATCH 4/6] btrfs: remove unused variable tree in bio_readpage_error()

2018-11-14 Thread Omar Sandoval
On Wed, Nov 14, 2018 at 02:35:18PM +0100, Johannes Thumshirn wrote: > Commit 2922040236f9 (btrfs: Remove extent_io_ops::writepage_end_io_hook) > removed the indirection to extent_io_ops::writepage_end_io_hook but didn't > remove the tree variable which then became unused. > > Remove 'tree' as

Re: [PATCH 3/6] btrfs: remove unused function btrfs_sysfs_feature_update()

2018-11-14 Thread Omar Sandoval
On Wed, Nov 14, 2018 at 02:35:17PM +0100, Johannes Thumshirn wrote: > btrfs_sysfs_feature_update() was introduced with commit 444e75169872 (btrfs: > sysfs: introduce helper for syncing bits with sysfs files) to provide a helper > which was used in 14e46e04958d (btrfs: synchronize incompat feature

Re: [PATCH 1/6] btrfs: remove unused drop_on_err in btrfs_mkdir()

2018-11-14 Thread Omar Sandoval
On Wed, Nov 14, 2018 at 02:35:15PM +0100, Johannes Thumshirn wrote: > Up to commit 32955c5422a8 (btrfs: switch to discard_new_inode()) the > drop_on_err variable in btrfs_mkdir() was used to check whether the inode had > to be dropped via iput(). > > After commit 32955c5422a8 (btrfs: switch to

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-14 Thread David Sterba
On Tue, Nov 13, 2018 at 05:58:14PM +, Filipe Manana wrote: > That's the infinite loop issue fixed by one of the patches submitted > for 4.20-rc2: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v4.20-rc2=11023d3f5fdf89bba5e1142127701ca6e6014587 > > The branch

[PATCH] btrfs: test send after radical changes in a complex directory hierarchy

2018-11-14 Thread fdmanana
From: Filipe Manana Test an incremental send operation in a scenario where the relationship of ancestor-descendant between multiple directories is inversed, and where multiple directories that were previously ancestors of another directory now become descendents of multiple directories that used

[PATCH] Btrfs: send, fix infinite loop due to directory rename dependencies

2018-11-14 Thread fdmanana
From: Robbie Ko When doing an incremental send, due to the need of delaying directory move (rename) operations we can end up in infinite loop at apply_children_dir_moves(). An example scenario that triggers this problem is described below, where directory names correspond to the numbers of

Re: [PATCH] Btrfs: ensure path name is null terminated at btrfs_control_ioctl

2018-11-14 Thread David Sterba
On Wed, Nov 14, 2018 at 11:35:24AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > We were using the path name received from user space without checking that > it is null terminated. While btrfs-progs is well behaved and does proper > validation and null termination, someone could

Re: [PATCH] btrfs: Fix suspicious RCU usage warning in device_list_add

2018-11-14 Thread David Sterba
On Wed, Nov 14, 2018 at 03:24:56PM +0800, Lu Fengqi wrote: > = > WARNING: suspicious RCU usage > 4.20.0-rc2+ #23 Tainted: G O > - > fs/btrfs/volumes.c:886 suspicious rcu_dereference_check() usage! > > Use btrfs_info_in_rcu instead

Re: [PATCH 6/6] btrfs: unconditionally provide function prototypes from free-space-tree.h

2018-11-14 Thread Johannes Thumshirn
On 14/11/2018 14:52, Nikolay Borisov wrote: > > I agree with this patch, however you go into the gray area of > "everything which is exported should have btrfs_ prefix". It's up to > David to see if he is content with this change. Yep you're right. I think I should change it, but I'll wait for

Re: [PATCH 6/6] btrfs: unconditionally provide function prototypes from free-space-tree.h

2018-11-14 Thread Nikolay Borisov
On 14.11.18 г. 15:35 ч., Johannes Thumshirn wrote: > Currently the function prototypes of: > * search_free_space_info() > * convert_free_space_to_bitmaps() > * convert_free_space_to_extents() > * free_space_test_bit() > * __remove_from_free_space_tree() > * __add_to_free_space_tree() > are

Re: [PATCH 2/6] btrfs: remove set but not used variable err in btrfs_add_link

2018-11-14 Thread Nikolay Borisov
On 14.11.18 г. 15:35 ч., Johannes Thumshirn wrote: > err holds the return value of either btrfs_del_root_ref() or > btrfs_del_inode_ref() but it hasn't been checked since it's introduction with > commit fe66a05a0679 (Btrfs: improve error handling for btrfs_insert_dir_item > callers) in 2012. >

Re: [PATCH 0/6] btrfs: fix compiler warning with make W=1

2018-11-14 Thread Nikolay Borisov
On 14.11.18 г. 15:35 ч., Johannes Thumshirn wrote: > This patchset fixes most of the compiler warnings encountered when building > btrfs with make W=1. > > There are two more compiler warnings left in raid56.c: > CC [M] fs/btrfs/raid56.o > fs/btrfs/raid56.c: In function ‘finish_rmw’: >

[PATCH 0/6] btrfs: fix compiler warning with make W=1

2018-11-14 Thread Johannes Thumshirn
This patchset fixes most of the compiler warnings encountered when building btrfs with make W=1. There are two more compiler warnings left in raid56.c: CC [M] fs/btrfs/raid56.o fs/btrfs/raid56.c: In function ‘finish_rmw’: fs/btrfs/raid56.c:1185:6: warning: variable ‘p_stripe’ set but not used

[PATCH 6/6] btrfs: unconditionally provide function prototypes from free-space-tree.h

2018-11-14 Thread Johannes Thumshirn
Currently the function prototypes of: * search_free_space_info() * convert_free_space_to_bitmaps() * convert_free_space_to_extents() * free_space_test_bit() * __remove_from_free_space_tree() * __add_to_free_space_tree() are hidden behind CONFIG_BTRFS_FS_RUN_SANITY_TESTS. If

[PATCH 4/6] btrfs: remove unused variable tree in bio_readpage_error()

2018-11-14 Thread Johannes Thumshirn
Commit 2922040236f9 (btrfs: Remove extent_io_ops::writepage_end_io_hook) removed the indirection to extent_io_ops::writepage_end_io_hook but didn't remove the tree variable which then became unused. Remove 'tree' as well to silence the warning when -Wunused-but-set-variable is used to compile

[PATCH 2/6] btrfs: remove set but not used variable err in btrfs_add_link

2018-11-14 Thread Johannes Thumshirn
err holds the return value of either btrfs_del_root_ref() or btrfs_del_inode_ref() but it hasn't been checked since it's introduction with commit fe66a05a0679 (Btrfs: improve error handling for btrfs_insert_dir_item callers) in 2012. As the error value hasn't been of any interest for 6 years we

[PATCH 3/6] btrfs: remove unused function btrfs_sysfs_feature_update()

2018-11-14 Thread Johannes Thumshirn
btrfs_sysfs_feature_update() was introduced with commit 444e75169872 (btrfs: sysfs: introduce helper for syncing bits with sysfs files) to provide a helper which was used in 14e46e04958d (btrfs: synchronize incompat feature bits with sysfs files). But commit e410e34fad91 (Revert "btrfs:

[PATCH 5/6] btrfs: remove unused variable tree in end_compressed_bio_write()

2018-11-14 Thread Johannes Thumshirn
Commit 2922040236f9 (btrfs: Remove extent_io_ops::writepage_end_io_hook) removed the indirection to extent_io_ops::writepage_end_io_hook but didn't remove the tree variable which then became unused. Remove 'tree' as well to silence the warning when -Wunused-but-set-variable is used to compile

[PATCH 1/6] btrfs: remove unused drop_on_err in btrfs_mkdir()

2018-11-14 Thread Johannes Thumshirn
Up to commit 32955c5422a8 (btrfs: switch to discard_new_inode()) the drop_on_err variable in btrfs_mkdir() was used to check whether the inode had to be dropped via iput(). After commit 32955c5422a8 (btrfs: switch to discard_new_inode()) discard_new_inode() is called when err is set and inode is

<    3   4   5   6   7   8   9   10   11   12   >