[PATCH 1/2] btrfs-progs: introduce test_issubvolname() for simplicity
From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
There are many duplicated codes to check if the given string is
correct subvolume name. Introduce test_issubvolname() for this
purpose for simplicity.
Signed-off-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
---
cmds-subvolume.c | 21 +++--
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 639fb10..b7bfb3e 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -43,6 +43,18 @@ static const char * const subvolume_cmd_group_usage[] = {
};
/*
+ * test if name is a correct subvolume name
+ * this function return
+ * 0- name is not a correct subvolume name
+ * 1- name is a correct subvolume name
+ */
+static int test_issubvolname(char *name)
+{
+ return name[0] != '\0' !strchr(name, '/')
+ strcmp(name, .) strcmp(name, ..);
+}
+
+/*
* test if path is a directory
* this function return
* 0- path exists but it is not a directory
@@ -127,8 +139,7 @@ static int cmd_subvol_create(int argc, char **argv)
dupdir = strdup(dst);
dstdir = dirname(dupdir);
- if (!strcmp(newname, .) || !strcmp(newname, ..) ||
-strchr(newname, '/') ){
+ if (!test_issubvolname(newname)) {
fprintf(stderr, ERROR: uncorrect subvolume name ('%s')\n,
newname);
goto out;
@@ -302,8 +313,7 @@ again:
vname = basename(dupvname);
free(cpath);
- if (!strcmp(vname, .) || !strcmp(vname, ..) ||
-strchr(vname, '/')) {
+ if (!test_issubvolname(vname)) {
fprintf(stderr, ERROR: incorrect subvolume name ('%s')\n,
vname);
ret = 1;
@@ -711,8 +721,7 @@ static int cmd_snapshot(int argc, char **argv)
dstdir = dirname(dupdir);
}
- if (!strcmp(newname, .) || !strcmp(newname, ..) ||
-strchr(newname, '/') ){
+ if (!test_issubvolname(newname)) {
fprintf(stderr, ERROR: incorrect snapshot name ('%s')\n,
newname);
goto out;
--
1.9.3
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] btrfs-progs: Unify the messy error message formats
From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
- There are many format to show snapshot name in error messages,
'%s', '%s, %s, ('%s'), and ('%s). Since it's messy,
unify these to '%s' format.
- Fix a type: s/uncorrect/incorrect/
Signed-off-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
---
cmds-subvolume.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index b7bfb3e..ce38503 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -140,14 +140,14 @@ static int cmd_subvol_create(int argc, char **argv)
dstdir = dirname(dupdir);
if (!test_issubvolname(newname)) {
- fprintf(stderr, ERROR: uncorrect subvolume name ('%s')\n,
+ fprintf(stderr, ERROR: incorrect subvolume name '%s'\n,
newname);
goto out;
}
len = strlen(newname);
if (len == 0 || len = BTRFS_VOL_NAME_MAX) {
- fprintf(stderr, ERROR: subvolume name too long ('%s)\n,
+ fprintf(stderr, ERROR: subvolume name too long '%s'\n,
newname);
goto out;
}
@@ -314,7 +314,7 @@ again:
free(cpath);
if (!test_issubvolname(vname)) {
- fprintf(stderr, ERROR: incorrect subvolume name ('%s')\n,
+ fprintf(stderr, ERROR: incorrect subvolume name '%s'\n,
vname);
ret = 1;
goto out;
@@ -322,7 +322,7 @@ again:
len = strlen(vname);
if (len == 0 || len = BTRFS_VOL_NAME_MAX) {
- fprintf(stderr, ERROR: snapshot name too long ('%s)\n,
+ fprintf(stderr, ERROR: too long snapshot name '%s'\n,
vname);
ret = 1;
goto out;
@@ -722,14 +722,14 @@ static int cmd_snapshot(int argc, char **argv)
}
if (!test_issubvolname(newname)) {
- fprintf(stderr, ERROR: incorrect snapshot name ('%s')\n,
+ fprintf(stderr, ERROR: incorrect snapshot name '%s'\n,
newname);
goto out;
}
len = strlen(newname);
if (len == 0 || len = BTRFS_VOL_NAME_MAX) {
- fprintf(stderr, ERROR: snapshot name too long ('%s)\n,
+ fprintf(stderr, ERROR: snapshot name too long '%s'\n,
newname);
goto out;
}
@@ -778,7 +778,7 @@ static int cmd_snapshot(int argc, char **argv)
res = ioctl(fddst, BTRFS_IOC_SNAP_CREATE_V2, args);
if (res 0) {
- fprintf( stderr, ERROR: cannot snapshot %s - %s\n,
+ fprintf( stderr, ERROR: cannot snapshot '%s' - %s\n,
subvol_descr, strerror(errno));
goto out;
}
@@ -991,7 +991,7 @@ static int cmd_subvol_show(int argc, char **argv)
ret = find_mount_root(fullpath, mnt);
if (ret 0) {
- fprintf(stderr, ERROR: find_mount_root failed on %s:
+ fprintf(stderr, ERROR: find_mount_root failed on '%s':
%s\n, fullpath, strerror(-ret));
goto out;
}
--
1.9.3
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/4 v3] fiemap: add EXTENT_DATA_COMPRESSED flag
On Fri, Jul 25, 2014 at 4:04 AM, Andreas Dilger adil...@dilger.ca wrote: On Jul 24, 2014, at 1:22 PM, David Sterba dste...@suse.cz wrote: On Thu, Jul 17, 2014 at 12:07:57AM -0600, Andreas Dilger wrote: any progress on this patch series? I'm sorry I got distracted at the end of year and did not finish the series. I never saw an updated version of this patch series after the last round of reviews, but it would be great to move it forward. I have filefrag patches in my e2fsprogs tree waiting for an updated version of your patch. I recall the main changes were: - add FIEMAP_EXTENT_PHYS_LENGTH flag to indicate if fe_phys_length was valid fe_phys_length will be always valid, so other the flags are set only if it's not equal to the logical length. - rename fe_length to fe_logi_length and #define fe_length fe_logi_length - always fill in fe_phys_length (= fe_logi_length for uncompressed files) and set FIEMAP_EXTENT_PHYS_LENGTH whether the extent is compressed or not This is my understanding and contradicts the first point. I think Dave Chinner's former point was that having fe_phys_length validity depend on FIEMAP_EXTENT_DATA_COMPRESSED is a non-intuitive interface. It is not true that fe_phys_length would always be valid, since that is not the case for older kernels that currently always set this field to 0, so they need some flag to indicate if fe_phys_length is valid. Alternately, userspace could do: if (ext-fe_phys_length == 0) ext-fe_phys_length = ext-fe_logi_length; but that pre-supposes that fe_phys_length == 0 is never a valid value when fe_logi_length is non-zero, and this might introduce errors in some cases. I could imagine that some compression methods might not allocate any space at all if it was all zeroes, and just store a bit in the blockpointer or extent, so having a separate FIEMAP_EXTENT_PHYS_LENGTH is probably safer in the long run. zfs is an example of this. That opens up the question of whether a written zero filled space that gets compressed away is different from a hole, but I'd prefer to just return whatever the file mapping is than interpret it. Cheers, Andreas - add WARN_ONCE() in fiemap_fill_next_extent() as described below I don't know if there was any clear statement about whether there should be separate FIEMAP_EXTENT_PHYS_LENGTH and FIEMAP_EXTENT_DATA_COMPRESSED flags, or if the latter should be implicit? Probably makes sense to have separate flags. It should be fine to use: #define FIEMAP_EXTENT_PHYS_LENGTH0x0010 since this flag was never used. I've kept only FIEMAP_EXTENT_DATA_COMPRESSED, I don't see a need for FIEMAP_EXTENT_PHYS_LENGTH and this would be yet another flag because the FIEMAP_EXTENT_DATA_ENCODED is also implied. I'll send V4, we can discuss the PHYS_LENGTH flag then. Cheers, Andreas Regards, Rohan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] mkfs.btrfs: round all device sizes to sectorsize
On 07/25/2014 12:27 PM, Eric Sandeen wrote: make_btrfs() rounds down the first device size to a multiple of sectorsize: num_bytes = (num_bytes / sectorsize) * sectorsize; but subsequent device adds don't. This seems a bit odd inconsistent, and it makes xfstest btrfs/011 _notrun(), because it explicitly checks that devices are the same size. I don't know that there is anything inherently wrong with having a few device bytes extend past the last block, but to be consistent, it seems like btrfs_add_to_fsid() should round the size in the same way. And now btrfs/011 runs more consistently; the test devices don't have to be sectorsize multiples in order for all mkfs'd device sizes to match. Signed-off-by: Eric Sandeen sand...@redhat.com --- ideally this might go into btrfs_device_size(), but we don't have the chosen sector size anywhere near there... diff --git a/utils.c b/utils.c index e130849..4d7ee35 100644 --- a/utils.c +++ b/utils.c @@ -554,7 +554,7 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, device-sector_size = sectorsize; device-fd = fd; device-writeable = 1; - device-total_bytes = block_count; + device-total_bytes = (block_count / sectorsize) * sectorsize; device-bytes_used = 0; device-total_ios = 0; device-dev_root = root-fs_info-dev_root; its better to do this at the function btrfs_prepare_device() itself, so that it would include first and the rest added. or we need to apply this on the block_count instead, as we do add that to the total_bytes. Thanks, Anand -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: Return right extent when fiemap gives unaligned offset and len.
Hi Qu, (2014/07/25 10:49), Qu Wenruo wrote: When page aligned start and len passed to extent_fiemap(), the result is good, but when start and len is not aligned, e.g. start = 1 and len = 4095 is passed to extent_fiemap(), it returns no extent. The problem is that start and len is all rounded up which causes the problem. This patch will round down start and round up (start + len) to return right extent. Reported-by: Chandan Rajendra chan...@linux.vnet.ibm.com Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com --- changelog: v2: reword the description(ALIGN rounds up, not rounds down). --- fs/btrfs/extent_io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index a389820..1c70cff 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4213,8 +4213,8 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, return -ENOMEM; path-leave_spinning = 1; - start = ALIGN(start, BTRFS_I(inode)-root-sectorsize); - len = ALIGN(len, BTRFS_I(inode)-root-sectorsize); + start = round_down(start, BTRFS_I(inode)-root-sectorsize); + len = round_up(max, BTRFS_I(inode)-root-sectorsize) - start; I have one question. Why isn't it len = round_up(len, BTRFS_I(inode)-root-sectorsize); ? Old behavior make len sectorsize aligned. However, if start is not aligned to sectorsize, new bahavior make len not sectorsize aligned. Does it an expected behavior? CMIIW. I'm not good at extent code. Thanks, Satoru /* * lookup the last file extent. We're not using i_size here -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 01/10] Btrfs: Fix the problem that the replace destroys the seed filesystem
Thanks for nailing down most of the seed related bugs, scratching
off few from my list.
On 07/24/2014 11:37 AM, Miao Xie wrote:
The seed filesystem was destroyed by the device replace, the reproduce
method is:
# mkfs.btrfs -f dev0
# btrfstune -S 1 dev0
# mount dev0 mnt
# btrfs device add dev1 mnt
# umount mnt
# mount dev1 mnt
# btrfs replace start -f dev0 dev2 mnt
# umount mnt
# mount dev0 mnt
It is because we erase the super block on the seed device. It is wrong,
we should not change anything on the seed device.
nice fix.
Reviewed-by: Anand Jain anand.j...@oracle.com
Signed-off-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/volumes.c | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 2776070f..19188df 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1853,8 +1853,12 @@ void btrfs_rm_dev_replace_srcdev(struct btrfs_fs_info
*fs_info,
if (srcdev-bdev) {
fs_info-fs_devices-open_devices--;
- /* zero out the old super */
- btrfs_scratch_superblock(srcdev);
+ /*
+* zero out the old super if it is not writable
+* (e.g. seed device)
+*/
+ if (srcdev-writeable)
+ btrfs_scratch_superblock(srcdev);
}
call_rcu(srcdev-rcu, free_device);
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: use IS_ALIGNED() for assertion in btrfs_lookup_csums_range() for simplicity
From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com btrfs_lookup_csums_range() uses ALIGN() to check if start and end + 1 are aligned to root-sectorsize. It's better to replace these with IS_ALIGNED() for simplicity. Signed-off-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com --- fs/btrfs/file-item.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c index f46cfe4..4b9e40f 100644 --- a/fs/btrfs/file-item.c +++ b/fs/btrfs/file-item.c @@ -329,8 +329,8 @@ int btrfs_lookup_csums_range(struct btrfs_root *root, u64 start, u64 end, u64 csum_end; u16 csum_size = btrfs_super_csum_size(root-fs_info-super_copy); - ASSERT(start == ALIGN(start, root-sectorsize) - (end + 1) == ALIGN(end + 1, root-sectorsize)); + ASSERT(IS_ALIGNED(start, root-sectorsize) + IS_ALIGNED(end + 1, root-sectorsize)); path = btrfs_alloc_path(); if (!path) -- 1.9.3 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: Return right extent when fiemap gives unaligned offset and len.
Original Message Subject: Re: [PATCH v2] btrfs: Return right extent when fiemap gives unaligned offset and len. From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com To: Qu Wenruo quwen...@cn.fujitsu.com, linux-btrfs@vger.kernel.org Date: 2014年07月25日 15:52 Hi Qu, (2014/07/25 10:49), Qu Wenruo wrote: When page aligned start and len passed to extent_fiemap(), the result is good, but when start and len is not aligned, e.g. start = 1 and len = 4095 is passed to extent_fiemap(), it returns no extent. The problem is that start and len is all rounded up which causes the problem. This patch will round down start and round up (start + len) to return right extent. Reported-by: Chandan Rajendra chan...@linux.vnet.ibm.com Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com --- changelog: v2: reword the description(ALIGN rounds up, not rounds down). --- fs/btrfs/extent_io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index a389820..1c70cff 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4213,8 +4213,8 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, return -ENOMEM; path-leave_spinning = 1; - start = ALIGN(start, BTRFS_I(inode)-root-sectorsize); - len = ALIGN(len, BTRFS_I(inode)-root-sectorsize); + start = round_down(start, BTRFS_I(inode)-root-sectorsize); + len = round_up(max, BTRFS_I(inode)-root-sectorsize) - start; I have one question. Why isn't it len = round_up(len, BTRFS_I(inode)-root-sectorsize); ? Old behavior make len sectorsize aligned. However, if start is not aligned to sectorsize, new bahavior make len not sectorsize aligned. Does it an expected behavior? In fact, in my code, len is also aligned. The calculation is somewhat easy, round_down original 'start' and round_up original end(in the code, max is original unaligned end position), then calculate 'len' by minus aligned end with aligned 'start'. Since the *new* 'start' is already aligned and the new end is aligned, the len will also be aligned. Thanks, Qu CMIIW. I'm not good at extent code. Thanks, Satoru /* * lookup the last file extent. We're not using i_size here -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/5 v4] fiemap: introduce EXTENT_DATA_COMPRESSED flag
The original FIEMAP patch did not define this bit, btrfs will make use of it. The defined constant maintains the same value as originally proposed. Currently, the 'filefrag' utility has no way to recognize and denote a compressed extent. As implemented in btrfs right now, the compression step splits a big extent into smaller chunks and this is reported as a heavily fragmented file. Adding the flag to filefrag will at least give some explanation why, this has been confusing users for some time already. fiemap_fill_next_extent is extended and takes argument to fill the physical length. V4: The physical length is always set and equal to logical, or different and then sets the COMPRESSED flag. fiemap_extent::fe_length renamed to fe_logi_length V3: Based on feedback from Andreas, implement #1 from V2, current users of fiemap_fill_next_extent (fs/, ext4, gfs2, ocfs2, nilfs2, xfs) updated accordingly, no functional change. V2: Based on feedback from Andreas, the fiemap_extent is now able to hold the physical extent length, to be filled by the filesystem callback. 1) extend fiemap_fill_next_extent to take phys_length and update all users (ext4, gfs2, ocfs2, nilfs2, xfs) David Sterba (5): fiemap: fix comment at EXTENT_DATA_ENCRYPTED fiemap: add EXTENT_DATA_COMPRESSED flag btrfs: set FIEMAP_EXTENT_DATA_COMPRESSED for compressed extents Documentation/fiemap: Document the DATA_COMPRESSED flag fiemap: rename fe_length to fe_logi_length Documentation/filesystems/fiemap.txt | 19 +++ fs/btrfs/extent_io.c | 8 ++-- fs/ext4/extents.c| 3 ++- fs/ext4/inline.c | 2 +- fs/gfs2/inode.c | 2 +- fs/ioctl.c | 29 ++--- fs/nilfs2/inode.c| 8 +--- fs/ocfs2/extent_map.c| 4 ++-- fs/xfs/xfs_iops.c| 2 +- include/linux/fs.h | 2 +- include/uapi/linux/fiemap.h | 13 ++--- 11 files changed, 66 insertions(+), 26 deletions(-) -- 1.8.4.5 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/5] fiemap: add EXTENT_DATA_COMPRESSED flag
This flag was not accepted when fiemap was proposed [2] due to lack of
in-kernel users. Btrfs has compression for a long time and we'd like to
see that an extent is compressed in the output of 'filefrag' utility
once it's taught about it.
For that purpose, a reserved field from fiemap_extent is used to let the
filesystem store along the physcial extent length when the flag is set.
This keeps compatibility with applications that use FIEMAP.
Extend arguments of fiemap_fill_next_extent and update all users.
[1] http://article.gmane.org/gmane.comp.file-systems.ext4/8871
[2] http://thread.gmane.org/gmane.comp.file-systems.ext4/8870
[3] http://thread.gmane.org/gmane.linux.file-systems/77632 (v1)
[4] http://www.spinics.net/lists/linux-fsdevel/msg69078.html (v2)
Cc: Al Viro v...@zeniv.linux.org.uk
CC: Andreas Dilger adil...@dilger.ca
CC: Chris Mason c...@fb.com
CC: Christoph Hellwig h...@infradead.org
CC KONISHI Ryusuke konishi.ryus...@lab.ntt.co.jp
CC: Mark Fasheh mfas...@suse.com
CC: Steven Whitehouse swhit...@redhat.com
CC: Theodore Ts'o ty...@mit.edu
CC: Ben Myers b...@sgi.com
Signed-off-by: David Sterba dste...@suse.cz
---
fs/btrfs/extent_io.c| 2 +-
fs/ext4/extents.c | 3 ++-
fs/ext4/inline.c| 2 +-
fs/gfs2/inode.c | 2 +-
fs/ioctl.c | 27 +--
fs/nilfs2/inode.c | 8 +---
fs/ocfs2/extent_map.c | 4 ++--
fs/xfs/xfs_iops.c | 2 +-
include/linux/fs.h | 2 +-
include/uapi/linux/fiemap.h | 8 +++-
10 files changed, 42 insertions(+), 18 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index a389820d158b..eec118bf77ae 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -4357,7 +4357,7 @@ int extent_fiemap(struct inode *inode, struct
fiemap_extent_info *fieinfo,
end = 1;
}
ret = fiemap_fill_next_extent(fieinfo, em_start, disko,
- em_len, flags);
+ em_len, em_len, flags);
if (ret)
goto out_free;
}
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 4da228a0e6d0..0bdd173ac728 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -2253,6 +2253,7 @@ static int ext4_fill_fiemap_extents(struct inode *inode,
(__u64)es.es_lblk blksize_bits,
(__u64)es.es_pblk blksize_bits,
(__u64)es.es_len blksize_bits,
+ (__u64)es.es_len blksize_bits,
flags);
if (err 0)
break;
@@ -5125,7 +5126,7 @@ static int ext4_xattr_fiemap(struct inode *inode,
if (physical)
error = fiemap_fill_next_extent(fieinfo, 0, physical,
- length, flags);
+ length, length, flags);
return (error 0 ? error : 0);
}
diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c
index 645205d8ada6..3825ff9dc40d 100644
--- a/fs/ext4/inline.c
+++ b/fs/ext4/inline.c
@@ -1825,7 +1825,7 @@ int ext4_inline_data_fiemap(struct inode *inode,
if (physical)
error = fiemap_fill_next_extent(fieinfo, 0, physical,
- length, flags);
+ length, length, flags);
brelse(iloc.bh);
out:
up_read(EXT4_I(inode)-xattr_sem);
diff --git a/fs/gfs2/inode.c b/fs/gfs2/inode.c
index e62e59477884..5b45cf4e5465 100644
--- a/fs/gfs2/inode.c
+++ b/fs/gfs2/inode.c
@@ -1931,7 +1931,7 @@ static int gfs2_fiemap(struct inode *inode, struct
fiemap_extent_info *fieinfo,
len = size - start;
if (start size)
ret = fiemap_fill_next_extent(fieinfo, start, phys,
- len, flags);
+ len, len, flags);
if (ret == 1)
ret = 0;
} else {
diff --git a/fs/ioctl.c b/fs/ioctl.c
index 8ac3fad36192..24a9d912d1e6 100644
--- a/fs/ioctl.c
+++ b/fs/ioctl.c
@@ -70,20 +70,26 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
* @logical: Extent logical start offset, in bytes
* @phys: Extent physical start offset, in bytes
* @len: Extent length, in bytes
+ * @phys_len: Physical extent length in bytes
* @flags: FIEMAP_EXTENT flags that describe this extent
*
* Called from file system -fiemap callback. Will populate extent
* info as passed in via arguments and copy to user memory. On
* success, extent count on fieinfo is incremented.
*
+ * Extents without any encoding must set the physical and logical length
+ * to the same value. Otherwise, set flags to
Re: BTRFS hang with 3.16-rc5 (and also with 3.16-rc4)
[this time, to the mailing list as well] On Fri, Jul 25, 2014 at 09:02:44AM +0100, Hugo Mills wrote: On Thu, Jul 24, 2014 at 11:06:34PM -0400, Nick Krause wrote: On Thu, Jul 24, 2014 at 10:32 PM, Duncan 1i5t5.dun...@cox.net wrote: [snip] Hey Duncan and others , I have read this and this seems to need some working on. If you want my help please ask , I am new to the kernel so I may ask a dumb question or two but if that's fine with you I have no problem helping out here. I would like a log of printk statements leading to the hand if that's not too much work in order for me to trace this back. Note that btrfs is complex -- there's something around 100k lines of code in it. My first piece of kernel work in btrfs was simply documenting the way that the on-disk data structures related to each other[1]. That on its own took me two to three weeks of solid full-time effort, reading the code to find where each structure was used and how its elements related to other structures. You can't just wander up and dive in without putting in the effort of learning first. Whilst people will help you (come over to #btrfs on Freenode for more real-time interaction), they can't do the basic work of sitting down and understanding the code in detail for you. Chris, who designed and wrote the filesystem, has spent the last couple of weeks tracking down this particular problem. Do you think it's appropriate to leap into the middle of the discussion on this subtle bug as someone with absolutely no experience in the area? Your first task is to reproduce the bug on your own machine. If you can do that, _then_ you might be able to start tracking down its cause. But I wouldn't recommend doing that, as (a) it's a nasty subtle bug, and (b) Chris seems to be close to tracking it down anyway. My recommendations for you, if you want to work on btrfs, are: * Build and install the latest kernel from Linus's git repo * Read and understand the user documentation [2] * Create one or several btrfs filesystems with different configurations and learn how they work in userspace -- what are the features, what are the problems you see? Actually use at least one of the filesystems you created for real data in daily use (with backups) * Build the userspace tools from git * Pick up one of the userspace projects from [3] and implement that. If you pick the right one(s), you'll have to learn about some of the internal structures of the FS anyway. Compile and test your patch. If you're adding a new feature, write an automated xfstest for it as well. * Get that patch accepted. This will probably involve a sequence of revisions to it, multiple versions over a period of several weeks or more, with a review process. You should also send your test to xfstests and get that accepted. * Do the above again, until you get used to the processes involved, and have demonstrated that you can work well with the other people in the subsystem, and are generally producing useful and sane code. It's all about trust -- can you be trusted to mostly do the right thing? (So far on linux-kernel, you've rather demonstrated the opposite: your intentions are good, but your execution leaves a lot to be desired) * Use the documentation at [4], and the output of btrfs-debug-tree to understand the internal structure of the FS * Pick up one of the smaller, more self-contained ideas from the projects page [5] (say, [6] or [7]) and try to implement it. Again: build, write test code, test thoroughly, submit patch for review, modify as suggested by reviewers, and repeat as often as necessary Hugo. [1] https://btrfs.wiki.kernel.org/index.php/Data_Structures [2] https://btrfs.wiki.kernel.org/index.php/Main_Page#Guides_and_usage_information [3] https://btrfs.wiki.kernel.org/index.php/Project_ideas#Userspace_tools_projects [4] https://btrfs.wiki.kernel.org/index.php/Main_Page#Developer_documentation [5] https://btrfs.wiki.kernel.org/index.php/Project_ideas [6] https://btrfs.wiki.kernel.org/index.php/Project_ideas#Cancellable_operations [7] https://btrfs.wiki.kernel.org/index.php/Project_ideas#Implement_new_FALLOC_FL_.2A_modes -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- But people have always eaten people, / what else is there to --- eat? / If the Juju had meant us not to eat people / he wouldn't have made us of meat. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- ORLY? IÄ! R'LYH! --- signature.asc Description: Digital signature
Re: [PATCH 02/10] Btrfs: don't write any data into a readonly device when scrub
On 07/24/2014 11:37 AM, Miao Xie wrote:
We should not write data into a readonly device especially seed device when
doing scrub, skip those devices.
Reviewed-by: Anand Jain anand.j...@oracle.com
Signed-off-by: Miao Xie mi...@cn.fujitsu.com
---
fs/btrfs/scrub.c | 11 +++
1 file changed, 11 insertions(+)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index b6d198f..23d3f6e 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -2904,6 +2904,7 @@ int btrfs_scrub_dev(struct btrfs_fs_info *fs_info, u64
devid, u64 start,
struct scrub_ctx *sctx;
int ret;
struct btrfs_device *dev;
+ struct rcu_string *name;
if (btrfs_fs_closing(fs_info))
return -EINVAL;
@@ -2965,6 +2966,16 @@ int btrfs_scrub_dev(struct btrfs_fs_info *fs_info, u64
devid, u64 start,
return -ENODEV;
}
+ if (!is_dev_replace !readonly !dev-writeable) {
just reading the commit message would ask question what
about readonly scrub anyway. Nice readonly scrub case
is taken care as well.
+ mutex_unlock(fs_info-fs_devices-device_list_mutex);
+ rcu_read_lock();
+ name = rcu_dereference(dev-name);
+ btrfs_err(fs_info, scrub: device %s is not writable,
+ name-str);
+ rcu_read_unlock();
+ return -EROFS;
+ }
+
mutex_lock(fs_info-scrub_lock);
if (!dev-in_fs_metadata || dev-is_tgtdev_for_dev_replace) {
mutex_unlock(fs_info-scrub_lock);
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix compressed write corruption on enospc
Am Donnerstag, 24. Juli 2014, 22:48:05 schrieben Sie: When failing to allocate space for the whole compressed extent, we'll fallback to uncompressed IO, but we've forgotten to redirty the pages which belong to this compressed extent, and these 'clean' pages will simply skip 'submit' part and go to endio directly, at last we got data corruption as we write nothing. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c | 12 1 file changed, 12 insertions(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 3668048..8ea7610 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -709,6 +709,18 @@ retry: unlock_extent(io_tree, async_extent-start, async_extent-start + async_extent-ram_size - 1); + + /* + * we need to redirty the pages if we decide to + * fallback to uncompressed IO, otherwise we + * will not submit these pages down to lower + * layers. + */ + extent_range_redirty_for_io(inode, + async_extent-start, + async_extent-start + + async_extent-ram_size - 1); + goto retry; } goto out_free; I am testing this currently. So far no lockup. Lets see. Still has not filled the the block device with trees completely after I balanced them: Label: 'home' uuid: […] Total devices 2 FS bytes used 125.57GiB devid1 size 160.00GiB used 153.00GiB path /dev/dm-0 devid2 size 160.00GiB used 153.00GiB path /dev/mapper/sata-home I believe the lockups happen more easily if the trees occupy all of disk space. Well I will do some compiling of some KDE components which may let BTRFS fill all space again. This patch will mean it when it can´t make enough free space in the (fragmented) tree it will write uncompressed? This would mean that one would have a defragment trees regularily to allow for writes to happen compressed at all times. Well… of course still better than lockup or corruption. Ciao, -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: fix compressed write corruption on enospc
Am Freitag, 25. Juli 2014, 09:00:19 schrieb Liu Bo: On Thu, Jul 24, 2014 at 10:55:47AM -0400, Chris Mason wrote: On 07/24/2014 10:48 AM, Liu Bo wrote: When failing to allocate space for the whole compressed extent, we'll fallback to uncompressed IO, but we've forgotten to redirty the pages which belong to this compressed extent, and these 'clean' pages will simply skip 'submit' part and go to endio directly, at last we got data corruption as we write nothing. This fallback code was my #1 suspect for the hangs people have been seeing since 3.15. I changed things around to trigger the fallback randomly and wasn't able to trigger problems, but I was looking for hangs and not corruptions. So now you're able to trigger the hang without changing the fallback code? I tried raid1 and raid0 with fsmark and rsync in different ways but still fails to reproduce the hang :-( The most weird thing is who the hell holds the free space inode's page, is it possible to share pages with other inode? (My answer is NO, but I'm not sure now...) Can you try doing this on a BTRFS filesystem that has all of block device free space filled with its trees already? My suspicion is that this highly contributes to the likelyhood of this hang to happen (see my other mail). This really seems had to hit in a test scenario, while it happens regularily on some production filesystems. Making a kernel package with make-kpkg (from kernel-package package in Debian) triggers lockup quite reliably here. That said, with 3.16.0-rc6-tp520-fixcompwrite+ no lockup so far, but after balancing trees do not yet fill whole device again. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS hang with 3.16-rc5 (and also with 3.16-rc4)
Am Freitag, 25. Juli 2014, 02:32:17 schrieb Duncan: Martin Steigerwald posted on Thu, 24 Jul 2014 20:49:37 +0200 as excerpted: It may take some time tough cause during compiling the kernel BTRFS hung again, which caused loss of KDE Baloo desktop search file index and parts of a mail I wrote in KMail. Heh. While I do run a kde(-lite) desktop, at least I don't have those problems to deal with. As a gentooer I have the option to build kde without the semantic-desktop junk and I've taken that option, so no baloo or the like here. And after the akonadified kmail lost one too many mails and I was going to need to reset the data store to retrieve them once again, I asked myself why I was putting up with it, after all, email is a decades old technology that should NOT be rocket-science any longer, and soon enough I was NOT putting up with it any longer, as I'd switched to claws-mail. Actually, killing with fire kdepim and anything akonadi related was what allowed me to kill semantic-desktop as well instead of just run-time disabling it, since akonadi is part of the steaming pile. Heh, I don´t want to make it a discussion about KDE, but I like to make a few points about it: 1) It was kmail save letter feature which saved me from *typing* all of the mail again. It still had a previous version open. 2) I didn´t see dataloss in KMail for months. 3) Since the recents developments (running stuff partly from Git) I am quite satisfied, IMAP performance / handling improved a lot. Even usable with our Exchange server now, and barely noticable with my own Dovecot IMAP setup. It was a tough ride. It still haves rough edges. But it got a lot better to a point I am using it on all my systems again. I failed back to other mail clients at work for a moment cause IMAP access was barely usable. There is still one data loss bug with filtering mails through CRM114 mailfilter spamfilter I didn´t dare to retry with most recent versions. But it may have been fixed already. Interesting this came up here just now, too, as there's a current xmodulo post about baloo and milou in kde4 and the carryover to kde-frameworks5 and plasma5, too, with an ongoing discussion. http://xmodulo.com/2014/07/kde-semantic-desktop-nepomuk-baloo.html Baloo is a ton of a improvement. Only two thing I have with it at the moment is: 1) File indexing database is fragile in case of sudden write interruption… but here with BTRFS compressed extents corruption it may not even be at fault. 2) Sometimes krunner crashes while entering a search topic. Other than that I have grown to love the marvellous mail address auto completion from all indexed mails, the almost instant full text mail search and the almost instant fulltext file search. I suggest we continues this privately or on a KDE related list like kdepim- users as its not really BTRFS related. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS hang with 3.16-rc5 (and also with 3.16-rc4)
On 25. juli 2014 11:28, Liu Bo wrote:
Hi Torbjørn,
On Fri, Jul 25, 2014 at 06:51:44AM +0200, Torbjørn wrote:
On 07/24/2014 04:58 PM, Chris Mason wrote:
snip
Liu Bo has a promising patch:
https://patchwork.kernel.org/patch/4618421/
Please give it a shot. There's a second deadlock reading the free space
cache, I'm still working on that one too.
-chris
I (as expected, my hang was with free space cache) still get hangs
with this applied on top of 3.16-rc6.
Looking forward to the free space cache patch.
I have not been able to trigger the same hang as I had with 3.15 on
any of the 3.16-rc6 kernels so far.
Seems that you can run into the hang quite stably, I have a stupid debugging
patch here with adding a WARN_ON() to __load_free_space_inode(), would you
please give it a shot and show us the output of dmesg?
(This may make 'dmesg' a mess, but it will show who the hell holds the free
space inode's page just at the time before hang)
thanks,
-liubo
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 2b0a627..e364888 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -350,6 +350,8 @@ static int io_ctl_prepare_pages(struct io_ctl *io_ctl,
struct inode *inode,
gfp_t mask = btrfs_alloc_write_mask(inode-i_mapping);
int i;
+ WARN_ON(1);
+
for (i = 0; i io_ctl-num_pages; i++) {
page = find_or_create_page(inode-i_mapping, i, mask);
if (!page) {
That patch did not apply cleanly on top of current linus master.
Not sure what branch you are at, so I added that WARN_ON(1); manually in
free-space-cache.c.
I reverted all the other patches before manually applying this one.
If you need me to test from some other branch or with some other patch
included, please let me know.
I'm doing a build now. Will run my rsync job and report back.
Thanks.
--
Torbjørn
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS hang with 3.16-rc5 (and also with 3.16-rc4)
On 25. juli 2014 13:09, Torbjørn wrote:
On 25. juli 2014 12:22, Torbjørn wrote:
On 25. juli 2014 11:28, Liu Bo wrote:
Hi Torbjørn,
On Fri, Jul 25, 2014 at 06:51:44AM +0200, Torbjørn wrote:
On 07/24/2014 04:58 PM, Chris Mason wrote:
snip
Liu Bo has a promising patch:
https://patchwork.kernel.org/patch/4618421/
Please give it a shot. There's a second deadlock reading the free
space
cache, I'm still working on that one too.
-chris
I (as expected, my hang was with free space cache) still get hangs
with this applied on top of 3.16-rc6.
Looking forward to the free space cache patch.
I have not been able to trigger the same hang as I had with 3.15 on
any of the 3.16-rc6 kernels so far.
Seems that you can run into the hang quite stably, I have a stupid
debugging
patch here with adding a WARN_ON() to __load_free_space_inode(),
would you
please give it a shot and show us the output of dmesg?
(This may make 'dmesg' a mess, but it will show who the hell holds
the free
space inode's page just at the time before hang)
thanks,
-liubo
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 2b0a627..e364888 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -350,6 +350,8 @@ static int io_ctl_prepare_pages(struct io_ctl
*io_ctl,
struct inode *inode,
gfp_t mask = btrfs_alloc_write_mask(inode-i_mapping);
int i;
+WARN_ON(1);
+
for (i = 0; i io_ctl-num_pages; i++) {
page = find_or_create_page(inode-i_mapping, i, mask);
if (!page) {
That patch did not apply cleanly on top of current linus master.
Not sure what branch you are at, so I added that WARN_ON(1); manually
in free-space-cache.c.
I reverted all the other patches before manually applying this one.
If you need me to test from some other branch or with some other
patch included, please let me know.
I'm doing a build now. Will run my rsync job and report back.
Thanks.
--
Torbjørn
Back with results.
That's output from dmesg. If you need more, I can send netconsole log
as well.
Seems like the list did not accept the huge email.
dmesg is available from:
https://gist.github.com/anonymous/dba1f53211a1d482c3e0
--
Torbjørn
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: integration tree updated
On 07/24/2014 10:16 PM, Qu Wenruo wrote: Hi chris, It seems that two of my wrong patches got merged in integration branch: 6068d17c8ab5bce946e9678ed2064e9f966cbe62 btrfs: Merge default subvolume mount codes into btrfs_mount_subvol(). 8a2166332e332541f13b34b7248c0f14f575731e btrfs: Call mount_subtree() even 'subvolid=' mount option is given. These two patches does not completely work, and the successor patch is under review.(the show_path one) If it is OK for you, please remove these two patches. Thanks! these are removed. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs: replace seed device followed by unmount causes kernel WARNING
After the seed device has been replaced the new target device
is no more a seed device. So we need to bring that state in
the fs_devices.
reproducer:
mount /dev/sdb /btrfs
btrfs dev add /dev/sdc /btrfs
btrfs rep start -B /dev/sdb /dev/sdd /btrfs
umount /btrfs
WARNING: CPU: 0 PID: 12661 at fs/btrfs/volumes.c:891
__btrfs_close_devices+0x1b0/0x200 [btrfs]()
::
__btrfs_close_devices()
::
WARN_ON(fs_devices-open_devices);
WARN_ON(fs_devices-rw_devices);
per the btrfs-devlist tool (to dump fs_devices and
btrfs_device from the kernel) the num_device, open_devices,
rw_devices are still at 1 but the total_device is at 2,
even after the seed device has been replaced in the above example.
Signed-off-by: Anand Jain anand.j...@oracle.com
---
fs/btrfs/dev-replace.c | 13 +
1 file changed, 13 insertions(+)
diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index eea26e1..a144bb1 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -569,6 +569,19 @@ static int btrfs_dev_replace_finishing(struct
btrfs_fs_info *fs_info,
btrfs_rm_dev_replace_blocked(fs_info);
+ /*
+* if we are replacing a seed device with a writable device
+* then FS won't be a seeding FS any more.
+*/
+ if (src_device-fs_devices-seeding !src_device-writeable) {
+ fs_info-fs_devices-rw_devices++;
+ fs_info-fs_devices-num_devices++;
+ fs_info-fs_devices-open_devices++;
+
+ fs_info-fs_devices-seeding = 0;
+ fs_info-fs_devices-seed = NULL;
+ }
+
btrfs_rm_dev_replace_srcdev(fs_info, src_device);
btrfs_rm_dev_replace_unblocked(fs_info);
--
2.0.0.153.g79d
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH not for integration] btrfs-devlist: dumps btrfs_device and btrfs_fs_devices from kernel
This would dump the following info: fs_address dev_address dev_root_addr root_fsid fsid name uuid (seed_fsid@seed_addr sprout_fsid@sprout_addr) (fs_num_devices fs_open_devices fs_rw_devices fs_missing_devices fs_total_devices) fs_total_rw_bytes fs_num_can_discard devid gen total_bytes disk_total_bytes bytes_used type io_align io_width sector_size fmode not_fs_Mounted|not_fs_Seeding|not_fs_Rotating not_Writable|not_MD|not_Missing|not_Discard|not_Replace_tgt|not_Run_pending|not_Nobarriers|not_Stat_valid|not_Bdev Applies on Chris integration branch now Anand Jain (1): btrfs: introduce BTRFS_IOC_GET_DEVS fs/btrfs/super.c | 86 +++ fs/btrfs/volumes.c | 145 + fs/btrfs/volumes.h | 3 + include/uapi/linux/btrfs.h | 53 - 4 files changed, 286 insertions(+), 1 deletion(-) Anand Jain (1): btrfs-progs: introduce btrfs-devlist .gitignore | 1 + Makefile| 4 +- btrfs-devlist.c | 268 ioctl.h | 52 +++ 4 files changed, 323 insertions(+), 2 deletions(-) create mode 100644 btrfs-devlist.c -- 2.0.0.153.g79d -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/1] btrfs-progs: introduce btrfs-devlist
This is a small (debug) program to dump the device list
in the raw format from the btrfs kernel.
here I use ioctl which was introduced in the below kernel
patch
btrfs: introduce BTRFS_IOC_GET_DEVS
Signed-off-by: Anand Jain anand.j...@oracle.com
---
.gitignore | 1 +
Makefile| 4 +-
btrfs-devlist.c | 268
ioctl.h | 52 +++
4 files changed, 323 insertions(+), 2 deletions(-)
create mode 100644 btrfs-devlist.c
diff --git a/.gitignore b/.gitignore
index fc8c07a..760eaeb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,3 +35,4 @@ libbtrfs.a
libbtrfs.so
libbtrfs.so.0
libbtrfs.so.0.1
+btrfs-devlist
diff --git a/Makefile b/Makefile
index 9f805ac..ec6011c 100644
--- a/Makefile
+++ b/Makefile
@@ -48,7 +48,7 @@ MAKEOPTS = --no-print-directory Q=$(Q)
progs = mkfs.btrfs btrfs-debug-tree btrfsck \
btrfs btrfs-map-logical btrfs-image btrfs-zero-log btrfs-convert \
- btrfs-find-root btrfstune btrfs-show-super
+ btrfs-find-root btrfstune btrfs-show-super btrfs-devlist
# external libs required by various binaries; for btrfs-foo,
# specify btrfs_foo_libs = list of libs; see $($(subst...)) rules below
@@ -229,7 +229,7 @@ clean: $(CLEANDIRS)
@echo Cleaning
$(Q)rm -f $(progs) cscope.out *.o *.o.d btrfs-convert btrfs-image
btrfs-select-super \
btrfs-zero-log btrfstune dir-test ioctl-test quick-test send-test
btrfsck \
- btrfs.static mkfs.btrfs.static btrfs-calc-size \
+ btrfs.static mkfs.btrfs.static btrfs-calc-size btrfs-devlist\
version.h $(check_defs) \
$(libs) $(lib_links)
diff --git a/btrfs-devlist.c b/btrfs-devlist.c
new file mode 100644
index 000..740957a
--- /dev/null
+++ b/btrfs-devlist.c
@@ -0,0 +1,268 @@
+/*
+ * Copyright (C) 2014 Oracle. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include unistd.h
+#include getopt.h
+#include fcntl.h
+#include sys/ioctl.h
+#include uuid/uuid.h
+
+#include ctree.h
+#include ioctl.h
+#include commands.h
+#include utils.h
+
+void print_header(void)
+{
+ printf(
+ fs_address dev_address dev_root_addr root_fsid\n\
+ fsid \
+ name \
+ uuid \
+ (seed_fsid@seed_addr \
+ sprout_fsid@sprout_addr) \n\
+ \t(fs_num_devices \
+ fs_open_devices \
+ fs_rw_devices \
+ fs_missing_devices \
+ fs_total_devices) \
+ fs_total_rw_bytes \
+ fs_num_can_discard\n\
+ \tdevid \
+ gen \
+ total_bytes \
+ disk_total_bytes \
+ bytes_used \
+ type \
+ io_align \
+ io_width \
+ sector_size \
+ fmode \n\
+ \tfs_flags\n\
+ \tdev_flags\n);
+}
+
+void print_dev(struct btrfs_ioctl_devlist *dev)
+{
+ char uuid[BTRFS_UUID_UNPARSED_SIZE];
+ char fsid[BTRFS_UUID_UNPARSED_SIZE];
+ char seed_fsid[BTRFS_UUID_UNPARSED_SIZE];
+ char sprout_fsid[BTRFS_UUID_UNPARSED_SIZE];
+ char root_fsid[BTRFS_UUID_UNPARSED_SIZE];
+
+ memset(seed_fsid, '0', BTRFS_UUID_UNPARSED_SIZE);
+ strcpy(seed_fsid, null);
+ memset(sprout_fsid, '0', BTRFS_UUID_UNPARSED_SIZE);
+ strcpy(sprout_fsid, null);
+ memset(root_fsid, '0', BTRFS_UUID_UNPARSED_SIZE);
+ strcpy(root_fsid, null);
+
+ uuid_unparse(dev-uuid, uuid);
+ uuid_unparse(dev-fsid, fsid);
+ if (! uuid_is_null(dev-seed_fsid))
+ uuid_unparse(dev-seed_fsid, seed_fsid);
+ if (! uuid_is_null(dev-sprout_fsid))
+ uuid_unparse(dev-sprout_fsid, sprout_fsid);
+ if (! uuid_is_null(dev-root_fsid))
+ uuid_unparse(dev-root_fsid, root_fsid);
+
+ printf(\n%llX %llX %llX %s\n\
+ %s %s %s (%s@%llX %s@%llX)\n\
+ \t(%llu %llu %llu %llu %llu) %llu %llu\n\
+ \t%llu %llu %llu %llu %llu %llu %u %u %u 0x%llX\n\
+ \t%s|%s|%s\n\
+ \t%s|%s|%s|%s|%s|%s|%s|%s|%s\n,
+ dev-fs_addr,
+ dev-dev_addr,
+ dev-dev_root_addr,
+ root_fsid,
+ fsid,
+
[PATCH 1/1] btrfs: introduce BTRFS_IOC_GET_DEVS
The user land progs needs a simple way to see
the raw list of disks and its parameters as seen
by the btrfs kernel.
As of now btrfs-devlist uses this ioctl.
Signed-off-by: Anand Jain anand.j...@oracle.com
---
fs/btrfs/super.c | 86 +++
fs/btrfs/volumes.c | 145 +
fs/btrfs/volumes.h | 3 +
include/uapi/linux/btrfs.h | 53 -
4 files changed, 286 insertions(+), 1 deletion(-)
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 59172ed..f3660f5 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1957,6 +1957,89 @@ static struct file_system_type btrfs_fs_type = {
};
MODULE_ALIAS_FS(btrfs);
+static int btrfs_ioc_get_devlist(void __user *arg)
+{
+ int ret = 0;
+ u64 sz_devlist_arg;
+ u64 sz_devlist;
+ u64 u_sz_devlist_arg;
+ u64 u_sz_devlist;
+ u64 sz_out;
+
+ struct btrfs_ioctl_devlist_args *devlist_arg;
+ struct btrfs_ioctl_devlist_args *tmp_devlist_arg;
+ struct btrfs_ioctl_devlist *devlist;
+
+ u64 cnt = 0, ucnt;
+
+ sz_devlist_arg = sizeof(*devlist_arg);
+ sz_devlist = sizeof(*devlist);
+
+ /* Check if the user land and kernel ioctl struct match */
+ if (copy_from_user(u_sz_devlist_arg,
+ (struct btrfs_ioctl_devlist_args __user *)(arg +
+ offsetof(struct btrfs_ioctl_devlist_args, sz_self)),
+ sizeof(u_sz_devlist_arg)))
+ return -EFAULT;
+
+ if (copy_from_user(u_sz_devlist,
+ (struct btrfs_ioctl_devlist_args __user *)(arg +
+ sz_devlist_arg +
+ offsetof(struct btrfs_ioctl_devlist, sz_self)),
+ sizeof(u_sz_devlist)))
+ return -EFAULT;
+
+ if (u_sz_devlist_arg != sz_devlist_arg ||
+ u_sz_devlist != sz_devlist) {
+ if (copy_to_user(arg +
+ offsetof(struct btrfs_ioctl_devlist_args, sz_self),
+ sz_devlist_arg, sizeof(sz_devlist_arg)))
+ return -EFAULT;
+ if (copy_to_user(arg +
+ sz_devlist_arg +
+ offsetof(struct btrfs_ioctl_devlist, sz_self),
+ sz_devlist, sizeof(sz_devlist)))
+ return -EFAULT;
+ return 2;
+ }
+
+ if (copy_from_user(ucnt,
+ (struct btrfs_ioctl_devlist_args __user *)(arg +
+ offsetof(struct btrfs_ioctl_devlist_args, count)),
+ sizeof(ucnt)))
+ return -EFAULT;
+
+ cnt = btrfs_get_devlist_cnt();
+
+ if (cnt ucnt) {
+ if (copy_to_user(arg +
+ offsetof(struct btrfs_ioctl_devlist_args, count),
+ cnt, sizeof(cnt)))
+ return -EFAULT;
+ return 1;
+ }
+
+ sz_out = sz_devlist_arg + sz_devlist * cnt;
+
+ tmp_devlist_arg = devlist_arg = memdup_user(arg, sz_out);
+ if (IS_ERR(devlist_arg))
+ return PTR_ERR(devlist_arg);
+
+ devlist = (struct btrfs_ioctl_devlist *) (++tmp_devlist_arg);
+ cnt = btrfs_get_devlist(devlist, cnt);
+ devlist_arg-count = cnt;
+
+ if (copy_to_user(arg, devlist_arg, sz_out)) {
+ ret = -EFAULT;
+ goto out;
+ }
+ ret = 0;
+out:
+ kfree(devlist_arg);
+ return ret;
+
+}
+
static int btrfs_ioc_get_fslist(void __user *arg)
{
int ret = 0;
@@ -2019,6 +2102,9 @@ static long btrfs_control_ioctl(struct file *file,
unsigned int cmd,
return -EPERM;
switch (cmd) {
+ case BTRFS_IOC_GET_DEVS:
+ ret = btrfs_ioc_get_devlist(argp);
+ break;
case BTRFS_IOC_SCAN_DEV:
vol = memdup_user((void __user *)arg, sizeof(*vol));
if (IS_ERR(vol))
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 92b3d4d..15c533d 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -229,6 +229,151 @@ error:
return ret;
}
+int btrfs_get_devlist_cnt(void)
+{
+ int cnt = 0;
+ struct btrfs_device *device;
+ struct btrfs_fs_devices *fs_devices;
+ struct btrfs_fs_devices *cur_fs_devices;
+
+ mutex_lock(uuid_mutex);
+ list_for_each_entry(cur_fs_devices, fs_uuids, list) {
+
+ fs_devices = cur_fs_devices;
+again_dev_cnt:
+ list_for_each_entry(device, fs_devices-devices, dev_list)
+ cnt++;
+
+ fs_devices = fs_devices-seed;
+ if (fs_devices)
+ goto again_dev_cnt;
+ }
+
+ mutex_unlock(uuid_mutex);
+
+ return cnt;
+}
+
+u64 btrfs_get_devlist(struct btrfs_ioctl_devlist *dev, u64 alloc_cnt)
+{
+ u64 cnt = 0;
+
+ struct btrfs_device *device;
+ struct btrfs_fs_devices
[PATCH v3] btrfs-progs: Add uninstall targets to Makefiles.
From: Nils Steinger g...@n-st.de Signed-off-by: Nils Steinger g...@n-st.de --- v2: As Satoru Takeuchi pointed out, I forgot to remove $(libs) in the main Makefile. v3: David Sterba suggested not to remove the $(bindir) and $(libdir), since those might be needed by other programs, even if the btrfs uninstallation leaves them empty. Documentation/Makefile | 4 Makefile | 7 +++ 2 files changed, 11 insertions(+) diff --git a/Documentation/Makefile b/Documentation/Makefile index 45299bb..5c9780b 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -76,6 +76,10 @@ install-man: man $(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir) $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir) +uninstall: + cd $(DESTDIR)$(man8dir); rm -f btrfs-check.8.gz $(GZ_MAN8) + rmdir -p --ignore-fail-on-non-empty $(DESTDIR)$(man8dir) + clean: $(RM) *.xml *.xml+ *.8 *.8.gz diff --git a/Makefile b/Makefile index 76565e8..de5bd9e 100644 --- a/Makefile +++ b/Makefile @@ -257,6 +257,13 @@ $(INSTALLDIRS): @echo Making install in $(patsubst install-%,%,$@) $(Q)$(MAKE) $(MAKEOPTS) -C $(patsubst install-%,%,$@) install +uninstall: + $(Q)$(MAKE) $(MAKEOPTS) -C Documentation uninstall + cd $(DESTDIR)$(incdir); rm -f $(headers) + rmdir -p --ignore-fail-on-non-empty $(DESTDIR)$(incdir) + cd $(DESTDIR)$(libdir); rm -f $(lib_links) $(libs) + cd $(DESTDIR)$(bindir); rm -f btrfsck fsck.btrfs $(progs) + ifneq ($(MAKECMDGOALS),clean) -include $(objects:.o=.o.d) $(cmd-objects:.o=.o.d) $(subst .btrfs,, $(filter-out btrfsck.o.d, $(progs:=.o.d))) endif -- 1.9.1 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs-progs: Add uninstall targets to Makefiles.
I apologize for the long hiatus. On Fri, Jun 27, 2014 at 02:32:01PM +0200, David Sterba wrote: On Wed, Jun 25, 2014 at 09:40:40PM +0200, Nils Steinger wrote: On my system, man8dir didn't exist prior to the installation, so I thought it would be reasonable to have the uninstallation routine remove it. According to the FHS [1] the manX directories do not have to exsit, so this part shall stay. [1] http://www.pathname.com/fhs/pub/fhs-2.3.html#USRSHAREMANMANUALPAGES and then note #32 bindir and libdir will exist by default on most systems, so that's a different case… So, should we really keep the directories around, even if they were created by the installation and are now empty (if they aren't, they won't be removed anyway)? But we don't track if the directories were created by the installation or not. Normally the directories would exist anyway (/usr or /usr/local as prefix) and are expected to exist at the locations. Installation to arbitraty directory works, but managing the directories is IMO up to the user. So are you ok with keeping bindir and libdir only (ie. removing only man8dir)? Yes, that should be sensible. I've updated my patch accordingly and resubmitted it few minutes ago. Regards, Nils -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS hang with 3.16-rc5 (and also with 3.16-rc4)
On 07/25/2014 01:37 PM, Torbjørn wrote:
On 25. juli 2014 13:09, Torbjørn wrote:
On 25. juli 2014 12:22, Torbjørn wrote:
On 25. juli 2014 11:28, Liu Bo wrote:
Hi Torbjørn,
On Fri, Jul 25, 2014 at 06:51:44AM +0200, Torbjørn wrote:
On 07/24/2014 04:58 PM, Chris Mason wrote:
snip
Liu Bo has a promising patch:
https://patchwork.kernel.org/patch/4618421/
Please give it a shot. There's a second deadlock reading the
free space
cache, I'm still working on that one too.
-chris
I (as expected, my hang was with free space cache) still get hangs
with this applied on top of 3.16-rc6.
Looking forward to the free space cache patch.
I have not been able to trigger the same hang as I had with 3.15 on
any of the 3.16-rc6 kernels so far.
Seems that you can run into the hang quite stably, I have a stupid
debugging
patch here with adding a WARN_ON() to __load_free_space_inode(),
would you
please give it a shot and show us the output of dmesg?
(This may make 'dmesg' a mess, but it will show who the hell holds
the free
space inode's page just at the time before hang)
thanks,
-liubo
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 2b0a627..e364888 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -350,6 +350,8 @@ static int io_ctl_prepare_pages(struct io_ctl
*io_ctl,
struct inode *inode,
gfp_t mask = btrfs_alloc_write_mask(inode-i_mapping);
int i;
+WARN_ON(1);
+
for (i = 0; i io_ctl-num_pages; i++) {
page = find_or_create_page(inode-i_mapping, i, mask);
if (!page) {
That patch did not apply cleanly on top of current linus master.
Not sure what branch you are at, so I added that WARN_ON(1);
manually in free-space-cache.c.
I reverted all the other patches before manually applying this one.
If you need me to test from some other branch or with some other
patch included, please let me know.
I'm doing a build now. Will run my rsync job and report back.
Thanks.
--
Torbjørn
Back with results.
That's output from dmesg. If you need more, I can send netconsole log
as well.
Seems like the list did not accept the huge email.
dmesg is available from:
https://gist.github.com/anonymous/dba1f53211a1d482c3e0
--
Torbjørn
If it's of any help; I was unable to trigger the hang when running
without compress=lzo
--
Torbjørn
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] mkfs.btrfs: round all device sizes to sectorsize
On Thu, Jul 24, 2014 at 11:27:32PM -0500, Eric Sandeen wrote: make_btrfs() rounds down the first device size to a multiple of sectorsize: ^^^ - device-total_bytes = block_count; + device-total_bytes = (block_count / sectorsize) * sectorsize; kerncompat.h:#define round_down(x, y) ((x) ~__round_mask(x, y)) - z -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] mkfs.btrfs: round all device sizes to sectorsize
On 7/25/14, 12:12 PM, Zach Brown wrote: On Thu, Jul 24, 2014 at 11:27:32PM -0500, Eric Sandeen wrote: make_btrfs() rounds down the first device size to a multiple of sectorsize: ^^^ -device-total_bytes = block_count; +device-total_bytes = (block_count / sectorsize) * sectorsize; kerncompat.h:#define round_down(x, y) ((x) ~__round_mask(x, y)) - z yeah yeah ;) this isn't copied kernel code but sure, that'd be better. I'm trying to clean up this whole we say blocks when we mean bytes! thing, and I'll include round_down() as well. Thanks, -Eric -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] mkfs.btrfs: round all device sizes to sectorsize
On 7/25/14, 12:25 PM, Eric Sandeen wrote: On 7/25/14, 12:12 PM, Zach Brown wrote: On Thu, Jul 24, 2014 at 11:27:32PM -0500, Eric Sandeen wrote: make_btrfs() rounds down the first device size to a multiple of sectorsize: ^^^ - device-total_bytes = block_count; + device-total_bytes = (block_count / sectorsize) * sectorsize; kerncompat.h:#define round_down(x, y) ((x) ~__round_mask(x, y)) - z yeah yeah ;) this isn't copied kernel code but sure, that'd be better. I'm trying to clean up this whole we say blocks when we mean bytes! thing, and I'll include round_down() as well. meh. And half this is done in kernelspace for device add/replace (device size setting etc) so TBH I'm increasingly inclined to just back away slowly here. :( (IOWs device_add calls btrfs_prepare_device(), but the size it finds is never used; the kernel does: device-total_bytes = i_size_read(bdev-bd_inode); so changing prepare_device doesn't catch add/replace cases...) Perhaps the simpler option is to remove the rounding which is only done on the first device added in userspace, but I honestly don't know what the design plan is, or what the ramifications of that might be ... -Eric -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Help with Project on brtfs wiki
Hey brtfs devolopers. I am new so I think this project,Implement new FALLOC_FL_* modes needs more information for me to write if for you guys. I am wondering what is fallocate and how you want me to write this, define statements or as functions in a certain file? I am not asking to hold my hand just point me to some documentation somewhere and I will read it when I have time as I am working on directory speed work for ext4. Cheers Nick -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
