Re: Standards Problems [Was: [PATCH v2 1/3] Btrfs: get more accurate output in df command.]

2014-12-18 Thread Duncan
Robert White posted on Wed, 17 Dec 2014 20:07:27 -0800 as excerpted:

  We have room for 1 more metadata extent on each
 drive, but if we allocate two more metadat extents on each drive we will
 burn up 1.25 GiB by reducing it to 0.75GiB.

FWIW, at least the last chunk assignment can be smaller than normal.  I 
believe I've seen it happen both here and on posted reports.  The 0.75 
GiB could thus be allocated as data if needed.

I'm not actually sure how the allocator works with the last few GiB.  
Some developer comments have hinted that it starts carving smaller chunks 
before it actually has to, and I could imagine it dropping data chunks to 
a half gig, then a quarter gig, than 128 MiB, then 64 MiB... as space 
gets tight, and metadata chunks similarly but of course from a quarter 
gig down.  That I'm not sure about.  But I'm quite sure it will actually 
use the last little bit (provided it can properly fill its raid policy 
when doing so), as I'm quite sure I've seen it do it.

I know for sure it does that in mixed-mode, as I have a 256 MiB mixed-
mode dup /boot (and a backup /boot of the same size on the other device, 
so I can select the one that's booted from the BIOS), and they tend to be 
fully chunk-allocated.   Note that even with mixed-mode, which defaults 
to metadata-sized-chunks, thus 256 MiB, on a 256 MiB device, by the time 
overhead, system, and reserve chunks are allocated, there's definitely 
NOT 256 MiB left for a data/metadata-mixed chunk, so if it couldn't 
allocate smaller bits it couldn't allocate even ONE chunk, let alone a 
pair (dup mode).

And I think I've seen it happen on my larger (not mixed) filesystems of 
several GiB as well, tho I don't actually tend to fill them up quite so 
routinely, so it's more difficult to say for sure.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2] Fix wrong memory free on check_is_root

2014-12-18 Thread Gui Hecheng
On Thu, 2014-12-18 at 16:39 +0900, Satoru Takeuchi wrote:
 From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
 
 When / is Btrfs, btrfs property subcommand / regards it
 as non-root by mistake.
 
 check_is_root() regards @object as a file system root if
 the following two conditions are satisfied.
 
  a) Both @object and its parent directory are Btrfs object
 (file system root, subvolume, inode, and device
 used for Btrfs).
  b) fsid of the above mentioned two objects are different.
 
 It doesn't work if @object is / because, in this case,
 fsid of / and its parent (it's also /), are the same.
 
 * Test environment
 
 Two Btrfs file system (not subvolume) / and /home/sat/mnt.
 
 * How to reproduce
 
 Submit btrfs prop get against the above mentioned file systems.
 
 * Test Result
 
 ** Actual result (without my patch)
 
 ==
 # btrfs prop get /home/sat/mnt/
 ro=false
 label= # label is displayed because it's a file system root
 # btrfs prop get /
 ro=false   # label is not displayed even if it's a file system 
 root
 ==
 ** Expected result (with my patch)
 
 ==
 # ./btrfs-new prop get btrfs-auto-test/
 # ./btrfs-new prop get /home/sat/mnt/
 ro=false
 label=
 # ./btrfs-new prop get / 
 ro=false
 label=foo# label is displayed
 ===
 
 Signed-off-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
 Reported-by: Naohiro Aota na...@elisp.net

Looks good to me.
Reviewed-by: Gui Hecheng guihc.f...@cn.fujitsu.com

Thanks,
Gui

 ---
 changelog
 v2: Return a correct error code when realpath() fails.
 Thank you Gui Hecheng for pointing my mistake out.
 https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg40204.html
 ---
  cmds-property.c | 14 +-
  1 file changed, 13 insertions(+), 1 deletion(-)
 
 diff --git a/cmds-property.c b/cmds-property.c
 index a764293..6501338 100644
 --- a/cmds-property.c
 +++ b/cmds-property.c
 @@ -124,7 +124,18 @@ static int check_is_root(const char *object)
   int ret;
   u8 fsid[BTRFS_FSID_SIZE];
   u8 fsid2[BTRFS_FSID_SIZE];
 - char *tmp;
 + char *tmp = NULL;
 + char *rp;
 +
 + rp = realpath(object, NULL);
 + if (!rp) {
 + ret = -errno;
 + goto out;
 + }
 + if (!strcmp(rp, /)) {
 + ret = 0;
 + goto out;
 + }
  
   tmp = malloc(strlen(object) + 5);
   if (!tmp) {
 @@ -165,6 +176,7 @@ static int check_is_root(const char *object)
  
  out:
   free(tmp);
 + free(rp);
   return ret;
  }
  


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Terminology (was Re: [PATCH 6/7] Print the summary)

2014-12-18 Thread Hugo Mills
On Wed, Dec 17, 2014 at 09:44:43PM -0800, Robert White wrote:
 On 12/16/2014 01:05 AM, Hugo Mills wrote:
 On Mon, Dec 15, 2014 at 07:47:06PM -0800, Robert White wrote:
 I prefer slice, not that I am totally happy with that word either.
 But by the time you get through loopback devices, memory map
 devices, the device files that represent parts of partitioned
 devices, logical volumes, and god only knows what future
 thingies we will end up dealing with... I like the terms that
 remove false hints.
 
 All this agreed, but slice will be largely viewed as a neologism
 by anyone who hasn't used (IIRC) BSD or Solaris. On the other hand,
 for all their other flaws, device is much better than disk here,
 as you could he talking about a partition, a flash memory card (which
 many people don't view as disks), or a network block device. You
 could refer to a block device to clarify, because that's really what
 we're talking about here, but it could get a bit cumbersome.
 
 That's part of why I'm not perfectly happy with slice either.
 
 But just last week I was trying to explain the whole thing of being
 out of raw space to allocate storage extents compared to being out
 of managed space to allocate file, um, extents...
 
 And I'm currently going around in circles over whole when is a
 sector free as far as /bin/df is concerned...
 
 
 we've got devices as we view them which are parts of devices as the ...
 
 We've got storage extents and we are dealing with people who are
 used to seeing extent used as the name of contiguously stored
 sections of files, but our extents store segments of the devices.

   ...our extents store segments of the devices _as well_, and that
term doesn't tend to leak out into public discussions. They're also
arked with a different record type in the extent tree, IIRC.

 We've also got those two-disk raid-5(s) which we've established are
 mathematically correct, but which will trip up everyone that is used
 to the three-or-more rule.

   I don't buy this one. You're the first person who's complained
about the use of the parity RAID names. However, we've had plenty of
people complaining about the RAID-1 term.

   As I said at the time, maybe we need to revisit the RAID vocabulary
completely and move to a more flexible terminology.

   David -- would you accept the cNsMpP patches if I revisted them?

 Then using the variable bytenr instead of byte_number (when all the
 rest of the variables are spelled out) and I'm not sure if thats in
 the logical view or the physical layout view, or both.

   Everything is in the logical view, unless you're looking at the
chunk tree, which does that mapping.

 In short (too late) there is a huge lexicon problem to newcomers
 here because of noun-overloading (in the programmatic sense).

   The extents and bytenr concerns don't leak out into the public
(user) arena. RAID is a known problem with a proposed solution. I
don't see this as being *huge* -- not in the same way that space
reporting to users is huge.

   Hugo.

-- 
Hugo Mills | Always be sincere, whether you mean it or not.
hugo@... carfax.org.uk |
http://carfax.org.uk/  |  Flanders  Swann
PGP: 65E74AC0  |The Reluctant Cannibal


signature.asc
Description: Digital signature


Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation.

2014-12-18 Thread Qu Wenruo


 Original Message 
Subject: Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to 
reduce ENOSPC caused by unbalanced data/metadata allocation.

From: Liu Bo bo.li@oracle.com
To: Qu Wenruo quwen...@cn.fujitsu.com
Date: 2014年10月30日 17:39

On Thu, Oct 30, 2014 at 11:46:18AM +0800, Qu Wenruo wrote:

[snipped]

Okay, I buy this.


Thanks,
Qu

Thanks,
Qu

[snipped]
  /*
+ * Try not to occupy more than half of the unallocated space.
+ * When run short of space and alloc all the space to
+ * data/metadata will cause ENOSPC to be
triggered more easily.
+ *
+ * And since the minimum chunk size is 16M, the
half-half will cause
+ * 16M allocated from 20M available space and
reset 4M will not be
+ * used ever. In that case(16~32M), allocate all directly.
+ */
+if (total_avail_space  32 * 1024 * 1024 
+total_avail_space  16 * 1024 * 1024)
+max_chunk_size = total_avail_space;
+else
+max_chunk_size = min(total_avail_space / 2,
max_chunk_size);
+max_chunk_size = min(total_avail_space / 2, max_chunk_size);

   

Why another one?  This won't make it use all space within [16M, 32M].

thanks,
-liubo

Sorry for the later reply, I didn't notice the mail for a long time.

Yes, that's my mistake setting the size twice...
Will fix it soon.

Thanks,
Qu



+
+/*
   * now sort the devices by hole size / available space
   */
  sort(devices_info, ndevs, sizeof(struct btrfs_device_info),
--
2.1.2

--
To unsubscribe from this list: send the line
unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v2] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation.

2014-12-18 Thread Qu Wenruo
When btrfs allocate a chunk, it will try to alloc up to 1G for data and
256M for metadata, or 10% of all the writeable space if there is enough
space for the stripe on device.

However, when we run out of space, this allocation may cause unbalanced
chunk allocation.
For example, there are only 1G unallocated space, and request for
allocate DATA chunk is sent, and all the space will be allocated as data
chunk, making later metadata chunk alloc request unable to handle, which
will cause ENOSPC.
This is the one of the common complains from end users about why ENOSPC
happens but there is still available space.

This patch will try not to alloc chunk which is more than half of the
unallocated space, making the last space more balanced at a small cost
of more fragmented chunk at the last 1G.

Some easy example:
Preallocate 17.5G on a 20G empty btrfs fs:
[Before]
 # btrfs fi show /mnt/test
Label: none  uuid: da8741b1-5d47-4245-9e94-bfccea34e91e
Total devices 1 FS bytes used 17.50GiB
devid1 size 20.00GiB used 20.00GiB path /dev/sdb
All space is allocated. No space later metadata space.

[After]
 # btrfs fi show /mnt/test
Label: none  uuid: e6935aeb-a232-4140-84f9-80aab1f23d56
Total devices 1 FS bytes used 17.50GiB
devid1 size 20.00GiB used 19.77GiB path /dev/sdb
About 230M is still available for later metadata allocation.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
Changelog:
v2:
   Fix a dead judgement which will cause last 16~32M size unavailable.
---
 fs/btrfs/volumes.c | 17 +
 1 file changed, 17 insertions(+)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 0144790..1cd0256 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -4236,6 +4236,7 @@ static int __btrfs_alloc_chunk(struct btrfs_trans_handle 
*trans,
int ret;
u64 max_stripe_size;
u64 max_chunk_size;
+   u64 total_avail_space = 0;
u64 stripe_size;
u64 num_bytes;
u64 raid_stripe_len = BTRFS_STRIPE_LEN;
@@ -4348,10 +4349,26 @@ static int __btrfs_alloc_chunk(struct 
btrfs_trans_handle *trans,
devices_info[ndevs].max_avail = max_avail;
devices_info[ndevs].total_avail = total_avail;
devices_info[ndevs].dev = device;
+   total_avail_space += total_avail;
++ndevs;
}
 
/*
+* Try not to occupy more than half of the unallocated space.
+* When run short of space and alloc all the space to
+* data/metadata will cause ENOSPC to be triggered more easily.
+*
+* And since the minimum chunk size is 16M, the half-half will cause
+* 16M allocated from 20M available space and reset 4M will not be
+* used ever. In that case(16~32M), allocate all directly.
+*/
+   if (total_avail_space  32 * 1024 * 1024 
+   total_avail_space  16 * 1024 * 1024)
+   max_chunk_size = total_avail_space;
+   else
+   max_chunk_size = min(total_avail_space / 2, max_chunk_size);
+
+   /*
 * now sort the devices by hole size / available space
 */
sort(devices_info, ndevs, sizeof(struct btrfs_device_info),
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC] btrfs-progs: Support for musl libc (and perhaps also uclibc)

2014-12-18 Thread Merlijn Wajer
Hi,

On 17/12/14 17:49, David Sterba wrote:
 On Sat, Dec 13, 2014 at 03:35:09PM +0100, Merlijn Wajer wrote:
 [snip]
 
 Attached are the two patches generated with git format-patch. I am aware
 that this may not be required format for submitting patches -- but
 please give me some time to get used to the etiquette. :-)
 
 Thanks, there are minor things that I won't bother to point out to
 occasional contributors and fix them myself. The only formal requirement
 is the Signed-off-by tag, you can find the description eg. here
 
 http://lxr.free-electrons.com/source/Documentation/SubmittingPatches#L358

Alright, cool. I've rebased my two patches and used --signoff when
amending the commit. Please find the new patches attached.


 Please let me know if musl-libc (or any other libc) is a supported
 platform, and if so, if and how I can improve on said patches.
 
 I'm not aware of non-glibc users, but I don't see any problem to add
 support for more libc implementations. However, I won't regularly verify
 that it builds so it might break.

That is fine; at this point there are not a lot of people using an
alternative libc, so this is common practice. I'll be here with a new
round of patches if it breaks in a future release; it shouldn't really
break too often. :)


 Seems that only standardized library calls are used in btrfs-progs so
 any kind of support is probably going to be satisfied by #ifdefs.
 Your patches are simple so I'll try to schedule them to some 3.18.x
 update.

That would be great; thank you.

Regards,
Merlijn
From 0109fd7ce62eee8b16aa5bdf78f3de54011e00f3 Mon Sep 17 00:00:00 2001
From: Merlijn Wajer merl...@wizzup.org
Date: Sat, 13 Dec 2014 15:07:25 +0100
Subject: [PATCH 1/2] Include headers required for musl-libc.

This fixes various compilation errors where PATH_MAX and XATTR_SIZE_MAX
were missing. To my knowledge, this should have no bad side effects.

Signed-off-by: Merlijn Wajer merl...@wizzup.org
---
 btrfs-convert.c | 1 +
 help.c  | 1 +
 mkfs.c  | 2 ++
 3 files changed, 4 insertions(+)

diff --git a/btrfs-convert.c b/btrfs-convert.c
index 02c5e94..7b69a13 100644
--- a/btrfs-convert.c
+++ b/btrfs-convert.c
@@ -31,6 +31,7 @@
 #include fcntl.h
 #include unistd.h
 #include uuid/uuid.h
+#include linux/limits.h
 
 #include ctree.h
 #include disk-io.h
diff --git a/help.c b/help.c
index fab942b..56aaf9c 100644
--- a/help.c
+++ b/help.c
@@ -17,6 +17,7 @@
 #include stdio.h
 #include stdlib.h
 #include string.h
+#include limits.h
 
 #include commands.h
 #include utils.h
diff --git a/mkfs.c b/mkfs.c
index e10e62d..6343831 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -35,6 +35,8 @@
 #include uuid/uuid.h
 #include ctype.h
 #include sys/xattr.h
+#include limits.h
+#include linux/limits.h
 #include blkid/blkid.h
 #include ftw.h
 #include ctree.h
-- 
2.0.4

From 5ae5cfa3af473625ef5b31daf5f229f12be753d2 Mon Sep 17 00:00:00 2001
From: Merlijn Wajer merl...@wizzup.org
Date: Sat, 13 Dec 2014 15:08:43 +0100
Subject: [PATCH 2/2] Disable backtrace and define __always_inline

Disable backtrace and define __always_inline when glibc is not used as
libc. This, together with some header changes allows btrfs-progs to
compile with musl-libc.

Signed-off-by: Merlijn Wajer merl...@wizzup.org
---
 kerncompat.h | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/kerncompat.h b/kerncompat.h
index 8afadc8..05823a7 100644
--- a/kerncompat.h
+++ b/kerncompat.h
@@ -29,6 +29,12 @@
 #include stddef.h
 #include linux/types.h
 #include stdint.h
+
+#ifndef __glibc__
+#define BTRFS_DISABLE_BACKTRACE
+#define __always_inline __inline __attribute__ ((__always_inline__))
+#endif
+
 #ifndef BTRFS_DISABLE_BACKTRACE
 #include execinfo.h
 #endif
-- 
2.0.4



signature.asc
Description: OpenPGP digital signature


Re: btrfs-prog: improve build-system by autoconf

2014-12-18 Thread Karel Zak
On Wed, Dec 17, 2014 at 03:07:26PM +0100, David Sterba wrote:
 On Fri, Dec 12, 2014 at 01:35:14PM +0100, Karel Zak wrote:
  This is first step to make btrfs-progs build system more conventional
  for userspace users and developers. All is implemented by small incremental
  patches to keep things review-able.
 
 Thanks. I went through the patches and haven't found major problems. The
 changes are affecting build system and this will need a longer period
 before all distros have a chance to adapt to that, so I'm postponing it
 to 3.19.

Cool, I'll try to prepare next set of patches with automake.

BTW, I have good experience with build-system changes -- downstream
distributions (maintainers) are usually pretty flexible :-)

  Note that there is also strange unused btrfs_convert_libs, btrfs_image_libs 
  and
  btrfs_fragments_libs variables with things like -lgd -lpng -ljpeg 
  -lfreetype.
  I guess it's some legacy, right? I didn't touch these variables as I have no
  clue about sense of this stuff.
 
 No, it's part of the macro magic. There are pattern rules that accept
 any source in the form btrfs-something.c and also pick the libraries for
 that from variable btrfs_something_libs:
 
 btrfs-%: $(objects) $(libs) btrfs-%.o
 @echo [LD] $@
 $(Q)$(CC) $(CFLAGS) -o $@ $(objects) $@.o $(LDFLAGS) $(LIBS) 
 $($(subst -,_,$@-libs))
 
 This is for convenience, if this turns out to be hard to do with in 
 combination
 with autotools, I don't insist on keeping it but it has simplified the 
 Makefile
 significantly.

OK, so -ljpeg in the Makefile is just example, right?


Anyway, for these things is better to introduce extra autoconf
AC_ARG_VAR() variables, keep is empty by default and use it in
Makefile. The advantage is that the variables are documented and
visible by ./configure --help.

For example in util-linux we have many {SUID,DAEMON,SOLIB,...}_CFLAGS
and LDFLAGS for distributions that require extensions like -fPIE,
-Wl,-z,relro etc. The same is possible to do with $LIBS.

Karel

-- 
 Karel Zak  k...@redhat.com
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/6] btrfs-progs: fi usage, change option for tabular output to T

2014-12-18 Thread David Sterba
We're going to use -t for tera- units prefix, same as 'fi df' does.

Signed-off-by: David Sterba dste...@suse.cz
---
 cmds-fi-disk_usage.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/cmds-fi-disk_usage.c b/cmds-fi-disk_usage.c
index 6a33b5d1da51..77ab8b4c1604 100644
--- a/cmds-fi-disk_usage.c
+++ b/cmds-fi-disk_usage.c
@@ -848,7 +848,7 @@ const char * const cmd_filesystem_usage_usage[] = {
Show in which disk the chunks are allocated.,
,
-b\tSet byte as unit,
-   -t\tShow data in tabular format,
+   -T\tShow data in tabular format,
NULL
 };
 
@@ -861,7 +861,7 @@ int cmd_filesystem_usage(int argc, char **argv)
 
optind = 1;
while (1) {
-   int c = getopt(argc, argv, bt);
+   int c = getopt(argc, argv, bT);
 
if (c  0)
break;
@@ -870,7 +870,7 @@ int cmd_filesystem_usage(int argc, char **argv)
case 'b':
mode = UNITS_RAW;
break;
-   case 't':
+   case 'T':
tabular = 1;
break;
default:
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 0/6] Btrfs-progs, add units to 'usage' subcommands

2014-12-18 Thread David Sterba
Copy from 'fi df' to 'fi usage' and 'dev usage'.

David Sterba (6):
  btrfs-progs: fi usage, change option for tabular output to T
  btrfs-progs: fi usage, add switches to set output units
  btrfs-progs: fi usage, update manpage
  btrfs-progs: dev usage, add switches to set output units
  btrfs-progs: dev usage, update manpage
  btrfs-progs: unify unit mode parameters and variables

 Documentation/btrfs-device.txt |  26 
 Documentation/btrfs-filesystem.txt |  28 
 cmds-device.c  |  66 ++
 cmds-fi-disk_usage.c   | 133 -
 cmds-fi-disk_usage.h   |   4 +-
 5 files changed, 197 insertions(+), 60 deletions(-)

-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 5/6] btrfs-progs: dev usage, update manpage

2014-12-18 Thread David Sterba
Signed-off-by: David Sterba dste...@suse.cz
---
 Documentation/btrfs-device.txt | 26 ++
 1 file changed, 26 insertions(+)

diff --git a/Documentation/btrfs-device.txt b/Documentation/btrfs-device.txt
index 46c54b94413c..4cd633fe6c03 100644
--- a/Documentation/btrfs-device.txt
+++ b/Documentation/btrfs-device.txt
@@ -98,6 +98,32 @@ identified by path or for a single device.
 -z
 Reset stats to zero after reading them.
 
+*usage* [options] path [path...]::
+Show detailed information about internal allocations in devices.
++
+`Options`
++
+-b|--raw
+raw numbers in bytes, without the 'B' suffix
+-h
+print human friendly numbers, base 1024, this is the default
+-H
+print human friendly numbers, base 1000
+--iec
+select the 1024 base for the following options, according to the IEC standard
+--si
+select the 1000 base for the following options, according to the SI standard
+-k|--kbytes
+show sizes in KiB, or kB with --si
+-m|--mbytes
+show sizes in MiB, or MB with --si
+-g|--gbytes
+show sizes in GiB, or GB with --si
+-t|--tbytes
+show sizes in TiB, or TB with --si
+
+If conflicting options are passed, the last one takes precedence.
+
 EXIT STATUS
 ---
 *btrfs device* returns a zero exit status if it succeeds. Non zero is
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/6] btrfs-progs: fi usage, add switches to set output units

2014-12-18 Thread David Sterba
Same set of options as 'fi df': binary and decimal bases, human readable
options etc.

Signed-off-by: David Sterba dste...@suse.cz
---
 cmds-fi-disk_usage.c | 64 +++-
 1 file changed, 53 insertions(+), 11 deletions(-)

diff --git a/cmds-fi-disk_usage.c b/cmds-fi-disk_usage.c
index 77ab8b4c1604..b3b70cc23fd1 100644
--- a/cmds-fi-disk_usage.c
+++ b/cmds-fi-disk_usage.c
@@ -21,6 +21,7 @@
 #include sys/ioctl.h
 #include errno.h
 #include stdarg.h
+#include getopt.h
 
 #include utils.h
 #include kerncompat.h
@@ -844,31 +845,72 @@ out:
 }
 
 const char * const cmd_filesystem_usage_usage[] = {
-   btrfs filesystem usage [-b][-t] path [path..],
-   Show in which disk the chunks are allocated.,
-   ,
-   -b\tSet byte as unit,
-   -T\tShow data in tabular format,
+   btrfs filesystem usage [options] path [path..],
+   Show detailed information about internal filesystem usage .,
+   -b|--raw   raw numbers in bytes,
+   -h human friendly numbers, base 1024 (default),
+   -H human friendly numbers, base 1000,
+   --iec  use 1024 as a base (KiB, MiB, GiB, TiB),
+   --si   use 1000 as a base (kB, MB, GB, TB),
+   -k|--kbytesshow sizes in KiB, or kB with --si,
+   -m|--mbytesshow sizes in MiB, or MB with --si,
+   -g|--gbytesshow sizes in GiB, or GB with --si,
+   -t|--tbytesshow sizes in TiB, or TB with --si,
+   -T show data in tabular format,
NULL
 };
 
 int cmd_filesystem_usage(int argc, char **argv)
 {
-   int mode = UNITS_HUMAN;
+   unsigned unit_mode = UNITS_DEFAULT;
int ret = 0;
int i, more_than_one = 0;
int tabular = 0;
 
optind = 1;
while (1) {
-   int c = getopt(argc, argv, bT);
+   int long_index;
+   static const struct option long_options[] = {
+   { raw, no_argument, NULL, 'b'},
+   { kbytes, no_argument, NULL, 'k'},
+   { mbytes, no_argument, NULL, 'm'},
+   { gbytes, no_argument, NULL, 'g'},
+   { tbytes, no_argument, NULL, 't'},
+   { si, no_argument, NULL, 256},
+   { iec, no_argument, NULL, 257},
+   };
+   int c = getopt_long(argc, argv, bhHkmgtT, long_options,
+   long_index);
 
if (c  0)
break;
-
switch (c) {
case 'b':
-   mode = UNITS_RAW;
+   unit_mode = UNITS_RAW;
+   break;
+   case 'k':
+   units_set_base(unit_mode, UNITS_KBYTES);
+   break;
+   case 'm':
+   units_set_base(unit_mode, UNITS_MBYTES);
+   break;
+   case 'g':
+   units_set_base(unit_mode, UNITS_GBYTES);
+   break;
+   case 't':
+   units_set_base(unit_mode, UNITS_TBYTES);
+   break;
+   case 'h':
+   unit_mode = UNITS_HUMAN_BINARY;
+   break;
+   case 'H':
+   unit_mode = UNITS_HUMAN_DECIMAL;
+   break;
+   case 256:
+   units_set_mode(unit_mode, UNITS_DECIMAL);
+   break;
+   case 257:
+   units_set_mode(unit_mode, UNITS_BINARY);
break;
case 'T':
tabular = 1;
@@ -905,12 +947,12 @@ int cmd_filesystem_usage(int argc, char **argv)
goto cleanup;
 
ret = print_filesystem_usage_overall(fd, chunkinfo, chunkcount,
-   devinfo, devcount, argv[i], mode);
+   devinfo, devcount, argv[i], unit_mode);
if (ret)
goto cleanup;
printf(\n);
ret = print_filesystem_usage_by_chunk(fd, chunkinfo, chunkcount,
-   devinfo, devcount, argv[i], mode, tabular);
+   devinfo, devcount, argv[i], unit_mode, tabular);
 cleanup:
close_file_or_dir(fd, dirstream);
free(chunkinfo);
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 6/6] btrfs-progs: unify unit mode parameters and variables

2014-12-18 Thread David Sterba
Use unsigned type and a common name everywhere.

Signed-off-by: David Sterba dste...@suse.cz
---
 cmds-device.c|  6 ++---
 cmds-fi-disk_usage.c | 69 ++--
 cmds-fi-disk_usage.h |  4 +--
 3 files changed, 40 insertions(+), 39 deletions(-)

diff --git a/cmds-device.c b/cmds-device.c
index 13458d850c26..9dd52a4b9fac 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -465,7 +465,7 @@ const char * const cmd_device_usage_usage[] = {
NULL
 };
 
-static int _cmd_device_usage(int fd, char *path, int mode)
+static int _cmd_device_usage(int fd, char *path, unsigned unit_mode)
 {
int i;
int ret = 0;
@@ -481,9 +481,9 @@ static int _cmd_device_usage(int fd, char *path, int mode)
 
for (i = 0; i  devcount; i++) {
printf(%s, ID: %llu\n, devinfo[i].path, devinfo[i].devid);
-   print_device_sizes(fd, devinfo[i], mode);
+   print_device_sizes(fd, devinfo[i], unit_mode);
print_device_chunks(fd, devinfo[i], chunkinfo, chunkcount,
-   mode);
+   unit_mode);
printf(\n);
}
 
diff --git a/cmds-fi-disk_usage.c b/cmds-fi-disk_usage.c
index b3b70cc23fd1..d81f01155e6c 100644
--- a/cmds-fi-disk_usage.c
+++ b/cmds-fi-disk_usage.c
@@ -310,7 +310,7 @@ static void get_raid56_used(int fd, struct chunk_info 
*chunks, int chunkcount,
 #defineMIN_UNALOCATED_THRESH   (16 * 1024 * 1024)
 static int print_filesystem_usage_overall(int fd, struct chunk_info *chunkinfo,
int chunkcount, struct device_info *devinfo, int devcount,
-   char *path, int mode)
+   char *path, unsigned unit_mode)
 {
struct btrfs_ioctl_space_args *sargs = 0;
int i;
@@ -450,30 +450,30 @@ static int print_filesystem_usage_overall(int fd, struct 
chunk_info *chunkinfo,
free_min += r_total_unused / max_data_ratio;
}
 
-   if (mode != UNITS_HUMAN)
+   if (unit_mode != UNITS_HUMAN)
width = 18;
 
printf(Overall:\n);
 
printf(Device size:\t\t%*s\n, width,
-   pretty_size_mode(r_total_size, mode));
+   pretty_size_mode(r_total_size, unit_mode));
printf(Device allocated:\t\t%*s\n, width,
-   pretty_size_mode(r_total_chunks, mode));
+   pretty_size_mode(r_total_chunks, unit_mode));
printf(Device unallocated:\t\t%*s\n, width,
-   pretty_size_mode(r_total_unused, mode));
+   pretty_size_mode(r_total_unused, unit_mode));
printf(Used:\t\t\t%*s\n, width,
-   pretty_size_mode(r_total_used, mode));
+   pretty_size_mode(r_total_used, unit_mode));
printf(Free (estimated):\t\t%*s\t(,
width,
-   pretty_size_mode(free_estimated, mode));
-   printf(min: %s)\n, pretty_size_mode(free_min, mode));
+   pretty_size_mode(free_estimated, unit_mode));
+   printf(min: %s)\n, pretty_size_mode(free_min, unit_mode));
printf(Data ratio:\t\t\t%*.2f\n,
width, data_ratio);
printf(Metadata ratio:\t\t%*.2f\n,
width, metadata_ratio);
printf(Global reserve:\t\t%*s\t(used: %s)\n, width,
-   pretty_size_mode(l_global_reserve, mode),
-   pretty_size_mode(l_global_reserve_used, mode));
+   pretty_size_mode(l_global_reserve, unit_mode),
+   pretty_size_mode(l_global_reserve_used, unit_mode));
 
 exit:
 
@@ -601,7 +601,7 @@ static u64 calc_chunk_size(struct chunk_info *ci)
  *  This function print the results of the command btrfs fi usage
  *  in tabular format
  */
-static void _cmd_filesystem_usage_tabular(int mode,
+static void _cmd_filesystem_usage_tabular(unsigned unit_mode,
struct btrfs_ioctl_space_args *sargs,
struct chunk_info *chunks_info_ptr,
int chunks_info_count,
@@ -678,7 +678,7 @@ static void _cmd_filesystem_usage_tabular(int mode,
 
if (size)
table_printf(matrix, col, i+3,
-   %s, pretty_size_mode(size, mode));
+   %s, pretty_size_mode(size, 
unit_mode));
else
table_printf(matrix, col, i+3, -);
 
@@ -690,7 +690,7 @@ static void _cmd_filesystem_usage_tabular(int mode,
- total_allocated;
 
table_printf(matrix, sargs-total_spaces + 1, i + 3,
-  %s, pretty_size_mode(unused, mode));
+  %s, pretty_size_mode(unused, unit_mode));
total_unused += unused;
 
}
@@ -702,15 +702,15 @@ static void _cmd_filesystem_usage_tabular(int mode,

[PATCH 4/6] btrfs-progs: dev usage, add switches to set output units

2014-12-18 Thread David Sterba
Same set of options as 'fi df': binary and decimal bases, human readable
options etc.

Signed-off-by: David Sterba dste...@suse.cz
---
 cmds-device.c | 60 +--
 1 file changed, 50 insertions(+), 10 deletions(-)

diff --git a/cmds-device.c b/cmds-device.c
index 33f1311a546c..13458d850c26 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -451,10 +451,17 @@ out:
 }
 
 const char * const cmd_device_usage_usage[] = {
-   btrfs device usage [-b] path [path..],
-   Show which chunks are in a device.,
-   ,
-   -b\tSet byte as unit,
+   btrfs device usage [options] path [path..],
+   Show detailed information about internal allocations in devices.,
+   -b|--raw   raw numbers in bytes,
+   -h human friendly numbers, base 1024 (default),
+   -H human friendly numbers, base 1000,
+   --iec  use 1024 as a base (KiB, MiB, GiB, TiB),
+   --si   use 1000 as a base (kB, MB, GB, TB),
+   -k|--kbytesshow sizes in KiB, or kB with --si,
+   -m|--mbytesshow sizes in MiB, or MB with --si,
+   -g|--gbytesshow sizes in GiB, or GB with --si,
+   -t|--tbytesshow sizes in TiB, or TB with --si,
NULL
 };
 
@@ -489,21 +496,54 @@ out:
 
 int cmd_device_usage(int argc, char **argv)
 {
-
-   int mode = UNITS_HUMAN;
+   unsigned unit_mode = UNITS_DEFAULT;
int ret = 0;
int i, more_than_one = 0;
 
optind = 1;
while (1) {
-   int c = getopt(argc, argv, b);
+   int long_index;
+   static const struct option long_options[] = {
+   { raw, no_argument, NULL, 'b'},
+   { kbytes, no_argument, NULL, 'k'},
+   { mbytes, no_argument, NULL, 'm'},
+   { gbytes, no_argument, NULL, 'g'},
+   { tbytes, no_argument, NULL, 't'},
+   { si, no_argument, NULL, 256},
+   { iec, no_argument, NULL, 257},
+   };
+   int c = getopt_long(argc, argv, bhHkmgt, long_options,
+   long_index);
 
if (c  0)
break;
-
switch (c) {
case 'b':
-   mode = UNITS_RAW;
+   unit_mode = UNITS_RAW;
+   break;
+   case 'k':
+   units_set_base(unit_mode, UNITS_KBYTES);
+   break;
+   case 'm':
+   units_set_base(unit_mode, UNITS_MBYTES);
+   break;
+   case 'g':
+   units_set_base(unit_mode, UNITS_GBYTES);
+   break;
+   case 't':
+   units_set_base(unit_mode, UNITS_TBYTES);
+   break;
+   case 'h':
+   unit_mode = UNITS_HUMAN_BINARY;
+   break;
+   case 'H':
+   unit_mode = UNITS_HUMAN_DECIMAL;
+   break;
+   case 256:
+   units_set_mode(unit_mode, UNITS_DECIMAL);
+   break;
+   case 257:
+   units_set_mode(unit_mode, UNITS_BINARY);
break;
default:
usage(cmd_device_usage_usage);
@@ -527,7 +567,7 @@ int cmd_device_usage(int argc, char **argv)
goto out;
}
 
-   ret = _cmd_device_usage(fd, argv[i], mode);
+   ret = _cmd_device_usage(fd, argv[i], unit_mode);
close_file_or_dir(fd, dirstream);
 
if (ret)
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/6] btrfs-progs: fi usage, update manpage

2014-12-18 Thread David Sterba
Signed-off-by: David Sterba dste...@suse.cz
---
 Documentation/btrfs-filesystem.txt | 28 
 1 file changed, 28 insertions(+)

diff --git a/Documentation/btrfs-filesystem.txt 
b/Documentation/btrfs-filesystem.txt
index a8f2972a0e1a..85a94eb52569 100644
--- a/Documentation/btrfs-filesystem.txt
+++ b/Documentation/btrfs-filesystem.txt
@@ -123,6 +123,34 @@ Show or update the label of a filesystem.
 If a newlabel optional argument is passed, the label is changed.
 NOTE: the maximum allowable length shall be less than 256 chars
 
+*usage* [options] path [path...]::
+Show detailed information about internal filesystem usage.
++
+`Options`
++
+-b|--raw
+raw numbers in bytes, without the 'B' suffix
+-h
+print human friendly numbers, base 1024, this is the default
+-H
+print human friendly numbers, base 1000
+--iec
+select the 1024 base for the following options, according to the IEC standard
+--si
+select the 1000 base for the following options, according to the SI standard
+-k|--kbytes
+show sizes in KiB, or kB with --si
+-m|--mbytes
+show sizes in MiB, or MB with --si
+-g|--gbytes
+show sizes in GiB, or GB with --si
+-t|--tbytes
+show sizes in TiB, or TB with --si
+-T
+show data in tabular format
+
+If conflicting options are passed, the last one takes precedence.
+
 EXIT STATUS
 ---
 *btrfs filesystem* returns a zero exit status if it succeeds. Non zero is
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


btrfs is using 25% more disk than it should

2014-12-18 Thread Daniele Testa
Hey,

I am hoping you guys can shed some light on my issue. I know that it's
a common question that people see differences in the disk used when
running different calculations, but I still think that my issue is
weird.

root@s4 / # mount
/dev/md3 on /opt/drives/ssd type btrfs
(rw,noatime,compress=zlib,discard,nospace_cache)

root@s4 / # btrfs filesystem df /opt/drives/ssd
Data: total=407.97GB, used=404.08GB
System, DUP: total=8.00MB, used=52.00KB
System: total=4.00MB, used=0.00
Metadata, DUP: total=1.25GB, used=672.21MB
Metadata: total=8.00MB, used=0.00

root@s4 /opt/drives/ssd # ls -alhs
total 302G
4.0K drwxr-xr-x 1 root root   42 Dec 18 14:34 .
4.0K drwxr-xr-x 4 libvirt-qemu libvirt-qemu 4.0K Dec 18 14:31 ..
302G -rw-r--r-- 1 libvirt-qemu libvirt-qemu 315G Dec 18 14:49 disk_208.img
   0 drwxr-xr-x 1 libvirt-qemu libvirt-qemu0 Dec 18 10:08 snapshots

root@s4 /opt/drives/ssd # du -h
0   ./snapshots
302G.

As seen above, I have a 410GB SSD mounted at /opt/drives/ssd. On
that partition, I have one single starse file, taking 302GB of space
(max 315GB). The snapshots directory is completely empty.

However, for some weird reason, btrfs seems to think it takes 404GB.
The big file is a disk that I use in a virtual server and when I write
stuff inside that virtual server, the disk-usage of the btrfs
partition on the host keeps increasing even if the sparse-file is
constant at 302GB. I even have 100GB of free disk-space inside that
virtual disk-file. Writing 1GB inside the virtual disk-file seems to
increase the usage about 4-5GB on the outside.

Does anyone have a clue on what is going on? How can the difference
and behaviour be like this when I just have one single file? Is it
also normal to have 672MB of metadata for a single file?

Regards,
Daniele
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Extra info

2014-12-18 Thread Daniele Testa
Sorry, did not read the guidelines correctly. Here comes more info:

root@s4 /opt/drives/ssd # uname -a
Linux s4.podnix.com 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 x86_64 GNU/Linux

root@s4 /opt/drives/ssd # btrfs --version
Btrfs Btrfs v0.19

root@s4 /opt/drives/ssd # btrfs fi show
Label: none  uuid: 752ed11b-defc-4717-b4c9-a9e08ad64ba6
Total devices 1 FS bytes used 404.74GB
devid1 size 410.50GB used 410.50GB path /dev/md3

Regards,
Daniele
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Extra info

2014-12-18 Thread Hugo Mills
On Thu, Dec 18, 2014 at 11:02:34PM +0800, Daniele Testa wrote:
 Sorry, did not read the guidelines correctly. Here comes more info:
 
 root@s4 /opt/drives/ssd # uname -a
 Linux s4.podnix.com 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 x86_64 
 GNU/Linux

   This is your problem. I think the difficulty is that writes into
the middle of an extent didn't split the extent and allow the
overwritten area to be reclaimed, so the whole extent still takes up
space.

   IIRC, josef fixed this about 18 months ago. You should upgrade your
kernel to something that isn't written in cueniform (like 3.18, say),
and defrag the file in question. I think that should fix the problem.

 root@s4 /opt/drives/ssd # btrfs --version
 Btrfs Btrfs v0.19

   This is also an antique, and probably needs an upgrade too
(although it's less critical than the kernel).

   Hugo.

 root@s4 /opt/drives/ssd # btrfs fi show
 Label: none  uuid: 752ed11b-defc-4717-b4c9-a9e08ad64ba6
 Total devices 1 FS bytes used 404.74GB
 devid1 size 410.50GB used 410.50GB path /dev/md3
 
 Regards,
 Daniele

-- 
Hugo Mills | Python is executable pseudocode; perl is executable
hugo@... carfax.org.uk | line-noise.
http://carfax.org.uk/  |
PGP: 65E74AC0  |Ben Burton


signature.asc
Description: Digital signature


Re: [PATCH 1/2] btrfs-progs: Add support for btrfs-image + corrupt script fsck test case.

2014-12-18 Thread David Sterba
On Mon, Dec 15, 2014 at 02:09:02PM +0800, Qu Wenruo wrote:
 It seems the second patch, which about 225K including a btrfs-image 
 dump, can't pass the ml's size limit.

I've cherry-picked the commit with the test image.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Extra info

2014-12-18 Thread Daniele Testa
I am running latest Debian stable. However, I used backports to update
the kernel to 3.16.

root@s4 /opt/drives/ssd # uname -a
Linux s4.podnix.com 3.16.0-0.bpo.4-amd64 #1 SMP Debian
3.16.7-ckt2-1~bpo70+1 (2014-12-08) x86_64 GNU/Linux

root@s4 /opt/drives/ssd # btrfs --version
Btrfs v3.14.1

It still reports over-use, so I am running a defrag on the file:

root@s4 /opt/drives/ssd # btrfs filesystem defragment
/opt/drives/ssd/disk_208.img

But I see it slowly eats even more disk space durring the defrag. I
had about 7GB before. When it went down close
to 1GB, I cancelled it as I'm afraid it will corrupt the file if it
runs out of space.

Do you know how btrfs behaves if it runs out of space durring a
defrag? Any other ideas how I can solve it?

Regards,
Daniele


2014-12-18 23:35 GMT+08:00 Hugo Mills h...@carfax.org.uk:
 On Thu, Dec 18, 2014 at 11:02:34PM +0800, Daniele Testa wrote:
 Sorry, did not read the guidelines correctly. Here comes more info:

 root@s4 /opt/drives/ssd # uname -a
 Linux s4.podnix.com 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 x86_64 
 GNU/Linux

This is your problem. I think the difficulty is that writes into
 the middle of an extent didn't split the extent and allow the
 overwritten area to be reclaimed, so the whole extent still takes up
 space.

IIRC, josef fixed this about 18 months ago. You should upgrade your
 kernel to something that isn't written in cueniform (like 3.18, say),
 and defrag the file in question. I think that should fix the problem.

 root@s4 /opt/drives/ssd # btrfs --version
 Btrfs Btrfs v0.19

This is also an antique, and probably needs an upgrade too
 (although it's less critical than the kernel).

Hugo.

 root@s4 /opt/drives/ssd # btrfs fi show
 Label: none  uuid: 752ed11b-defc-4717-b4c9-a9e08ad64ba6
 Total devices 1 FS bytes used 404.74GB
 devid1 size 410.50GB used 410.50GB path /dev/md3

 Regards,
 Daniele

 --
 Hugo Mills | Python is executable pseudocode; perl is executable
 hugo@... carfax.org.uk | line-noise.
 http://carfax.org.uk/  |
 PGP: 65E74AC0  |Ben Burton
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC] btrfs-progs: Support for musl libc (and perhaps also uclibc)

2014-12-18 Thread David Sterba
On Thu, Dec 18, 2014 at 11:29:40AM +0100, Merlijn Wajer wrote:
 That is fine; at this point there are not a lot of people using an
 alternative libc, so this is common practice. I'll be here with a new
 round of patches if it breaks in a future release; it shouldn't really
 break too often. :)

Good. I've put the patches to 3.18 queue.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-prog: improve build-system by autoconf

2014-12-18 Thread Eric Sandeen
On 12/18/14 7:31 AM, Karel Zak wrote:
 On Wed, Dec 17, 2014 at 03:07:26PM +0100, David Sterba wrote:
 On Fri, Dec 12, 2014 at 01:35:14PM +0100, Karel Zak wrote:
 This is first step to make btrfs-progs build system more conventional
 for userspace users and developers. All is implemented by small incremental
 patches to keep things review-able.

 Thanks. I went through the patches and haven't found major problems. The
 changes are affecting build system and this will need a longer period
 before all distros have a chance to adapt to that, so I'm postponing it
 to 3.19.
 
 Cool, I'll try to prepare next set of patches with automake.
 
 BTW, I have good experience with build-system changes -- downstream
 distributions (maintainers) are usually pretty flexible :-)
 
 Note that there is also strange unused btrfs_convert_libs, btrfs_image_libs 
 and
 btrfs_fragments_libs variables with things like -lgd -lpng -ljpeg 
 -lfreetype.
 I guess it's some legacy, right? I didn't touch these variables as I have no
 clue about sense of this stuff.

 No, it's part of the macro magic. There are pattern rules that accept
 any source in the form btrfs-something.c and also pick the libraries for
 that from variable btrfs_something_libs:

 btrfs-%: $(objects) $(libs) btrfs-%.o
 @echo [LD] $@
 $(Q)$(CC) $(CFLAGS) -o $@ $(objects) $@.o $(LDFLAGS) $(LIBS) 
 $($(subst -,_,$@-libs))

 This is for convenience, if this turns out to be hard to do with in 
 combination
 with autotools, I don't insist on keeping it but it has simplified the 
 Makefile
 significantly.

Yeah, I did those long ago IIRC. It obfuscates things a bit, but cut down on 
tons of
cut and paste in the Makefile...

 OK, so -ljpeg in the Makefile is just example, right?

It's been specified for the btrfs-framgnets tool since .. forever.

commit 6d37fbfc1f83c34f00df7c9d8e5b60e49d9db48d
Author: Arne Jansen sensi...@gmx.net
Date:   Mon Nov 28 17:12:30 2011 +0100

Btrfs-progs: tool to visualize fragmentation
...
+btrfs-fragments: $(objects) $(libs) fragments.o
+   @echo [LD] $@
+   $(Q)$(CC) $(CFLAGS) -o btrfs-fragments $(objects) fragments.o 
$(LDFLAGS) $(LIBS) -lgd -lpng -ljpeg -lfreetype

but the code only does:

gdImagePng(im, pngout);

so it's probably not needed...

 
 Anyway, for these things is better to introduce extra autoconf
 AC_ARG_VAR() variables, keep is empty by default and use it in
 Makefile. The advantage is that the variables are documented and
 visible by ./configure --help.
 
 For example in util-linux we have many {SUID,DAEMON,SOLIB,...}_CFLAGS
 and LDFLAGS for distributions that require extensions like -fPIE,
 -Wl,-z,relro etc. The same is possible to do with $LIBS.

Once auto$FOO is implemented, I am sure there are better ways to do it
than what I did.  :)

Thanks,
-Eric

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


btrfs-progs: integration-20141218 possible corruption test regression

2014-12-18 Thread WorMzy Tykashi
Hi guys, 

The latest integration fails 'make test' with the following output:

 [TEST]013-leaf-corruption-no-extent-data.tar.xz
btrfs check should have detected corruption
Makefile:144: recipe for target 'test' failed
make: *** [test] Error 1
rm btrfs-corrupt-block.o


I've only built the progs on a single machine so far, but it consistently fails 
at this test. 

Cheers, 


WorMzy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-progs: integration-20141218 possible corruption test regression

2014-12-18 Thread WorMzy Tykashi
On 18 December 2014 18:35:16 GMT+00:00, WorMzy Tykashi 
wormzy.tyka...@gmail.com wrote:
Hi guys, 

The latest integration fails 'make test' with the following output:

 [TEST]013-leaf-corruption-no-extent-data.tar.xz
btrfs check should have detected corruption
Makefile:144: recipe for target 'test' failed
make: *** [test] Error 1
rm btrfs-corrupt-block.o


I've only built the progs on a single machine so far, but it
consistently fails at this test. 

Cheers, 


WorMzy

Regression is present in the 3.18.x branch too.

Cheers, 


WorMzy

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: How to deal with The chunks in memory can not match the metadata of the fs. Repair failed.?

2014-12-18 Thread Karl-Philipp Richter
Hi,
Just to give you a really late feedback on this: I got it fixed with
btrfs-progs 3.17.3n on Linux 3.17.6, i.e. `btrfs rescue chunk-recover`
just did the job. Thanks for your advice.

-Kalle

Am 09/16/2014 um 03:46 AM schrieb Gui Hecheng:
 On Mon, 2014-09-15 at 16:59 +0200, Karl-Philipp Richter wrote:
 Hi together,
 I'm currently encountering another issue with `btrfs rescue 
 chunk-recover`. After some seconds of maximal I/O activity, the program 
 is idle for some seconds and then prints

  The chunks in memory can not match the metadata of the fs. Repair 
 failed.
  Fail to recover the chunk tree.

 I don't find any mention of this output except in the source code (this 
 is as well the only thing $SEARCH_ENGINE points me to). What is the 
 recommended next step? It would be nice to see the situation (The chunks 
 in memory can not match the metadata of the fs) or the output mentioned 
 in `man btrfs rescue`.

 Any help is appreciated.

 -Kalle
 
 Hi Kalle,
 If you are encountering the above msg, it means the chunk tree has some
 inconsistent stuff with other trees like device tree  extent tree.
 You can specify the '-v' option for chunk-recover to see more details.
 
 And also, the btrfs check cmd may tell more about the story.
 
 Thanks,
 Gui
 
 
 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 
 
 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: Extra info

2014-12-18 Thread Paul Jones
Another way to defrag the file is to move the file to another disk and then 
move it back. I've had trouble with virtual machine disks before (Windows 
server raw) and this has fixed the problem.

FYI 3.17.2 and beyond seems much better now. No crazy slow downs.

Paul.

-Original Message-
From: linux-btrfs-ow...@vger.kernel.org 
[mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Daniele Testa
Sent: Friday, 19 December 2014 4:33 AM
To: Hugo Mills; Daniele Testa; linux-btrfs@vger.kernel.org
Subject: Re: Extra info

I am running latest Debian stable. However, I used backports to update the 
kernel to 3.16.

root@s4 /opt/drives/ssd # uname -a
Linux s4.podnix.com 3.16.0-0.bpo.4-amd64 #1 SMP Debian
3.16.7-ckt2-1~bpo70+1 (2014-12-08) x86_64 GNU/Linux

root@s4 /opt/drives/ssd # btrfs --version Btrfs v3.14.1

It still reports over-use, so I am running a defrag on the file:

root@s4 /opt/drives/ssd # btrfs filesystem defragment 
/opt/drives/ssd/disk_208.img

But I see it slowly eats even more disk space durring the defrag. I had about 
7GB before. When it went down close to 1GB, I cancelled it as I'm afraid it 
will corrupt the file if it runs out of space.

Do you know how btrfs behaves if it runs out of space durring a defrag? Any 
other ideas how I can solve it?

Regards,
Daniele


2014-12-18 23:35 GMT+08:00 Hugo Mills h...@carfax.org.uk:
 On Thu, Dec 18, 2014 at 11:02:34PM +0800, Daniele Testa wrote:
 Sorry, did not read the guidelines correctly. Here comes more info:

 root@s4 /opt/drives/ssd # uname -a
 Linux s4.podnix.com 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 
 x86_64 GNU/Linux

This is your problem. I think the difficulty is that writes into 
 the middle of an extent didn't split the extent and allow the 
 overwritten area to be reclaimed, so the whole extent still takes up 
 space.

IIRC, josef fixed this about 18 months ago. You should upgrade your 
 kernel to something that isn't written in cueniform (like 3.18, say), 
 and defrag the file in question. I think that should fix the problem.

 root@s4 /opt/drives/ssd # btrfs --version Btrfs Btrfs v0.19

This is also an antique, and probably needs an upgrade too 
 (although it's less critical than the kernel).

Hugo.

 root@s4 /opt/drives/ssd # btrfs fi show
 Label: none  uuid: 752ed11b-defc-4717-b4c9-a9e08ad64ba6
 Total devices 1 FS bytes used 404.74GB
 devid1 size 410.50GB used 410.50GB path /dev/md3

 Regards,
 Daniele

 --
 Hugo Mills | Python is executable pseudocode; perl is executable
 hugo@... carfax.org.uk | line-noise.
 http://carfax.org.uk/  |
 PGP: 65E74AC0  |Ben Burton
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in the 
body of a message to majord...@vger.kernel.org More majordomo info at  
http://vger.kernel.org/majordomo-info.html
N�r��yb�X��ǧv�^�)޺{.n�+{�n�߲)w*jg����ݢj/���z�ޖ��2�ޙ�)ߡ�a�����G���h��j:+v���w��٥

Re: Extra info

2014-12-18 Thread Chris Murphy
The other thing going on here is that every write is going to be COW,
which isn't ideal for VM images. Another thing that I wonder is what
SSD this is and if it's supporting deterministic TRIM, since discard
is enabled. And I'm not sure that md by default is passing down trim
(?). This is on md raid? What level? If it's 56 then there's a
separate enabler for raid56 discard. In general if this isn't an SSD
that supports queued trim commands you're better off without discard
mount option and just running fstrim manually once a week or even once
a day is OK if it's a very busy SSD, just schedule it for a time when
it's typically idle.

Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Extra info

2014-12-18 Thread Chris Murphy
On Thu, Dec 18, 2014 at 6:57 PM, Chris Murphy li...@colorremedies.com wrote:
 The other thing going on here is that every write is going to be COW,
 which isn't ideal for VM images.

Reminder goes here for setting xattr +C on VM images. You can't set it
after the fact though. Only at the time the file is created. A reflink
copy into a folder with +C will fail. And there's not enough space on
the volume to fully copy it.


-- 
Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 1/3] Btrfs: get more accurate output in df command.

2014-12-18 Thread Zygo Blaxell
On Sun, Dec 14, 2014 at 10:06:50PM -0800, Robert White wrote:
 ABSTRACT:: Stop being clever, just give the raw values. That's what
 you should be doing anyway. There are no other correct values to
 give that doesn't blow someone's paradigm somewhere.

The trouble is a lot of existing software can't cope with the raw values
without configuration changes and access to a bunch of out-of-band data.
Nor should it.

I thank Robert for providing so many pathological examples in this thread.
They illustrate nicely why it's so important to provide adequately cooked
values through statvfs, especially for f_bavail!

 ITEM #1 :: In my humble opinion (ha ha) the size column should never
 change unless you add or remove actual storage. It should
 approximate the raw block size of the device on initial creation,
 and it should adjust to the size changes that happen when you
 semantically resize the filesystem with e.g. btrfs resize.

The units for f_blocks and f_bavail should be roughly similar because
software does calculate the ratio of those values (i.e. percentage of
disk used); however, there is no strong accuracy requirement--it could
be off by a few percent, and most software won't care.

Some software will start to misbehave if the ratio error is too large,
e.g. btrfs-RAID1 reporting the total disk size instead of the stored
data size.  This makes it necessary to scale f_blocks according to
chunk profiles, at least to within a few percent of actual size.

One critical rule is that (f_blocks - f_bavail) and (f_blocks - f_bfree)
should never be negative, or software will break in assorted ways.

 ITEM #4 :: Blocks available to unprivileged users is pretty iffy
 since unprivileged users cannot write to the filesystem. This datum
 doesn't have a plain reading. I'd start with filesystem total
 blocks, then subtract the total blocks used by all trees nodes in
 all trees. (e.g. Nodes * 0x1000 or whatever node size is) then shave
 off the N superblocks, then subtract the number of blocks already
 allocated in data extents. And you're done.

Blocks available (f_bavail) should be the intersection of all sets of
blocks currently available for allocation by all users (ignoring quota
for now).  It must be an underestimate, so that ENOSPC implies 
f_bavail == 0.  It is acceptable to report f_bavail = 0 but allow writes
and allocations to succeed for a subset of cases (e.g. inline extents,
metadata modifications).

Historically, filesystems reserved arbitrary quantities of blocks that
were available to unprivileged users, but not counted in f_bavail, in
order to ensure the relation holds even in corner cases.  Existing
application software is well adapted to this behavior.

Compressing filesystems underestimate free space on the filesystem,
in the sense that N blocks written may result in M fewer free blocks
where M  N.  Compression in filesystems has been well tolerated by
application software for years now.

ENOSPC when f_bavail  0 is very bad.  Depending on how large f_bavail
was at the ENOSPC failure, it means:

- admins didn't get alerts triggered by a low space threshold
and did not take action to avoid ENOSPC

- software that garbage-collects based on available space wasn't
activated in time to prevent ENOSPC

- service applications have been starting transactions they
believe they can finish because f_bavail is large enough,
but failing when they hit ENOSPC instead

f_bavail should be the minimum of the free data space and the free
metadata space (counting free space in mixed-mode chunks and non-chunk
disk space that can be used by the current profile in both sets).
This provides a more accurate indication of the amount of free space in
cases where metadata blocks are more scarce than data blocks.  This breaks
a common trap for btrfs users who are blindsided by metadata ENOSPC with
gigabytes free in df.

f_bavail should count only free space in chunks that are already allocated
in the current allocation profile, or unallocated space that _could_
become a chunk of the currently selected profile (separately for data and
metadata, then the lower of the two numbers kept).

Free space not allocated to chunks could be difficult to measure
precisely, so guess low.  

(Aside:  would it make sense to have btrfs preallocate all
available space just to make this calculation easier?  We can
now reallocate empty chunks on demand, so one reason _not_
to do this is already gone)

Chunks of non-current profiles (e.g. raid0 when the current profile is
raid1) should not be counted in f_bavail because such space can only
become available for use after a balance/convert operation.  Such space
should appear in f_bavail after that operation occurs.

This means that f_bavail will hit zero long before writes to the
filesystem start to fail due to lack of space, so there will be plenty of
warning for the administrator and software before space really runs 

Re: [PATCH 3/6] btrfs-progs: fi usage, update manpage

2014-12-18 Thread Satoru Takeuchi
Hi David,

On 2014/12/18 23:27, David Sterba wrote:
 Signed-off-by: David Sterba dste...@suse.cz
 ---
   Documentation/btrfs-filesystem.txt | 28 
   1 file changed, 28 insertions(+)
 
 diff --git a/Documentation/btrfs-filesystem.txt 
 b/Documentation/btrfs-filesystem.txt
 index a8f2972a0e1a..85a94eb52569 100644
 --- a/Documentation/btrfs-filesystem.txt
 +++ b/Documentation/btrfs-filesystem.txt
 @@ -123,6 +123,34 @@ Show or update the label of a filesystem.
   If a newlabel optional argument is passed, the label is changed.
   NOTE: the maximum allowable length shall be less than 256 chars
   
 +*usage* [options] path [path...]::
 +Show detailed information about internal filesystem usage.

Options from -b to -t are the completely same as btrfs fi df's ones.
So how about pointing the df's options as follows?

===
...
+
`Options`
+
-T
Show data in tabular format
+
There are some option to set unit. See description of *df*'s options
from '-b' to '-t'.
+
If conflicting options are passed, the last one takes precedence.
...
===

I consider it can prevent mistakes caused by further changes.

Thanks,
Satoru

 ++
 +`Options`
 ++
 +-b|--raw
 +raw numbers in bytes, without the 'B' suffix
 +-h
 +print human friendly numbers, base 1024, this is the default
 +-H
 +print human friendly numbers, base 1000
 +--iec
 +select the 1024 base for the following options, according to the IEC standard
 +--si
 +select the 1000 base for the following options, according to the SI standard
 +-k|--kbytes
 +show sizes in KiB, or kB with --si
 +-m|--mbytes
 +show sizes in MiB, or MB with --si
 +-g|--gbytes
 +show sizes in GiB, or GB with --si
 +-t|--tbytes
 +show sizes in TiB, or TB with --si
 +-T
 +show data in tabular format
 +
 +If conflicting options are passed, the last one takes precedence.
 +
   EXIT STATUS
   ---
   *btrfs filesystem* returns a zero exit status if it succeeds. Non zero is
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs-progs: cleanup: avoid to use literal for getopt val

2014-12-18 Thread Satoru Takeuchi
From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com

---
Tested with dev/integration-20141218 + this patch
---
 cmds-fi-disk_usage.c | 8 
 cmds-filesystem.c| 8 
 utils.h  | 3 +++
 3 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/cmds-fi-disk_usage.c b/cmds-fi-disk_usage.c
index d81f011..5e8e39b 100644
--- a/cmds-fi-disk_usage.c
+++ b/cmds-fi-disk_usage.c
@@ -877,8 +877,8 @@ int cmd_filesystem_usage(int argc, char **argv)
{ mbytes, no_argument, NULL, 'm'},
{ gbytes, no_argument, NULL, 'g'},
{ tbytes, no_argument, NULL, 't'},
-   { si, no_argument, NULL, 256},
-   { iec, no_argument, NULL, 257},
+   { si, no_argument, NULL, GETOPT_VAL_SI},
+   { iec, no_argument, NULL, GETOPT_VAL_IEC},
};
int c = getopt_long(argc, argv, bhHkmgtT, long_options,
long_index);
@@ -907,10 +907,10 @@ int cmd_filesystem_usage(int argc, char **argv)
case 'H':
unit_mode = UNITS_HUMAN_DECIMAL;
break;
-   case 256:
+   case GETOPT_VAL_SI:
units_set_mode(unit_mode, UNITS_DECIMAL);
break;
-   case 257:
+   case GETOPT_VAL_IEC:
units_set_mode(unit_mode, UNITS_BINARY);
break;
case 'T':
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 7eaccb9..8f037dd 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -216,8 +216,8 @@ static int cmd_filesystem_df(int argc, char **argv)
{ mbytes, no_argument, NULL, 'm'},
{ gbytes, no_argument, NULL, 'g'},
{ tbytes, no_argument, NULL, 't'},
-   { si, no_argument, NULL, 256},
-   { iec, no_argument, NULL, 257},
+   { si, no_argument, NULL, GETOPT_VAL_SI},
+   { iec, no_argument, NULL, GETOPT_VAL_IEC},
};
int c = getopt_long(argc, argv, bhHkmgt, long_options,
long_index);
@@ -245,10 +245,10 @@ static int cmd_filesystem_df(int argc, char **argv)
case 'H':
unit_mode = UNITS_HUMAN_DECIMAL;
break;
-   case 256:
+   case GETOPT_VAL_SI:
units_set_mode(unit_mode, UNITS_DECIMAL);
break;
-   case 257:
+   case GETOPT_VAL_IEC:
units_set_mode(unit_mode, UNITS_BINARY);
break;
default:
diff --git a/utils.h b/utils.h
index 0464c2d..34fd5e4 100644
--- a/utils.h
+++ b/utils.h
@@ -40,6 +40,9 @@
 
 #define ARGV0_BUF_SIZE PATH_MAX
 
+#define GETOPT_VAL_SI  256
+#define GETOPT_VAL_IEC 257
+
 int check_argc_exact(int nargs, int expected);
 int check_argc_min(int nargs, int expected);
 int check_argc_max(int nargs, int expected);
-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 0/5] Cleanup warnings from clang

2014-12-18 Thread Qu Wenruo
Cleanup warning when compile btrfs-progs with clang.

Clang analyser also reports about 60+ errors, but it will be another
patchset fixing it later.

Qu Wenruo (5):
  btrfs-progs: Makefile: Move linker only option to LDFLAGS
  btrfs-progs: Fix a clang dead-judgement warning in disk-io.c.
  btrfs-progs: Remove a unused function root_gtp_mask().
  btrfs-progs: Remove a unused function offset_to_bitmap()
  btrfs-progs: Remove deprecated _BSD_SOURCE macro.

 Makefile   |  3 ++-
 cmds-receive.c |  2 +-
 disk-io.c  |  9 ++---
 free-space-cache.c | 16 
 radix-tree.c   |  5 -
 5 files changed, 9 insertions(+), 26 deletions(-)

-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 4/5] btrfs-progs: Remove a unused function offset_to_bitmap()

2014-12-18 Thread Qu Wenruo
Fix the following clang warning:

free-space-cache.c:464:19: warning: unused function 'offset_to_bitmap'
[-Wunused-function]
static inline u64 offset_to_bitmap(struct btrfs_free_space_ctl *ctl,
  ^
1 warning generated.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
 free-space-cache.c | 16 
 1 file changed, 16 deletions(-)

diff --git a/free-space-cache.c b/free-space-cache.c
index 220449e..99ad420 100644
--- a/free-space-cache.c
+++ b/free-space-cache.c
@@ -461,22 +461,6 @@ static inline unsigned long bytes_to_bits(u64 bytes, u32 
unit)
return (unsigned long)(bytes / unit);
 }
 
-static inline u64 offset_to_bitmap(struct btrfs_free_space_ctl *ctl,
-  u64 offset)
-{
-   u64 bitmap_start;
-   u64 bytes_per_bitmap;
-   u32 sectorsize = ctl-sectorsize;
-
-   bytes_per_bitmap = BITS_PER_BITMAP(sectorsize) * ctl-unit;
-   bitmap_start = offset - ctl-start;
-   bitmap_start = bitmap_start / bytes_per_bitmap;
-   bitmap_start *= bytes_per_bitmap;
-   bitmap_start += ctl-start;
-
-   return bitmap_start;
-}
-
 static int tree_insert_offset(struct rb_root *root, u64 offset,
  struct rb_node *node, int bitmap)
 {
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 5/5] btrfs-progs: Remove deprecated _BSD_SOURCE macro.

2014-12-18 Thread Qu Wenruo
Fix the following gcc(4.9) and clang warning:

In file included from cmds-receive.c:24:
In file included from ./kerncompat.h:22:
In file included from /usr/include/stdio.h:27:
/usr/include/features.h:148:3: warning: _BSD_SOURCE and _SVID_SOURCE
are deprecated, use _DEFAULT_SOURCE [-W#warnings]
  ^
1 warning generated.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
 cmds-receive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-receive.c b/cmds-receive.c
index 358df1f..5bc8f8c 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -19,7 +19,7 @@
 #define _GNU_SOURCE
 #define _POSIX_C_SOURCE 200809
 #define _XOPEN_SOURCE 700
-#define _BSD_SOURCE
+#define _DEFAILT_SOURCE
 
 #include kerncompat.h
 
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/5] btrfs-progs: Makefile: Move linker only option to LDFLAGS

2014-12-18 Thread Qu Wenruo
Move the linker only option -rdynamic to LDFLAGS.
This resolve lots of the following warning if using clang as CC:
clang: warning: argument unused during compilation: '-rdynamic'

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index e1b0a88..be0f48b 100644
--- a/Makefile
+++ b/Makefile
@@ -5,7 +5,8 @@ CC = gcc
 LN = ln
 AR = ar
 AM_CFLAGS = -Wall -D_FILE_OFFSET_BITS=64 -DBTRFS_FLAT_INCLUDES 
-fno-strict-aliasing -fPIC
-CFLAGS = -g -O1 -fno-strict-aliasing -rdynamic
+CFLAGS = -g -O1 -fno-strict-aliasing
+LDFLAGS = -rdynamic
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
  extent-cache.o extent_io.o volumes.o utils.o repair.o \
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/5] btrfs-progs: Fix a clang dead-judgement warning in disk-io.c.

2014-12-18 Thread Qu Wenruo
When compiled with clang, the following warning is outputted.

disk-io.c:1017:15: warning: comparison of unsigned expression  0 is
always false [-Wtautological-compare]
if (dev_size  0)
 ^ ~
1 warning generated.

This is because dev_size is defined as unsigned type, but lseek() will
return singed valued.
So the judgement will always to false.

Use temporary off_t return value to solve it.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
 disk-io.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/disk-io.c b/disk-io.c
index 03edf8e..2bf8586 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -1009,13 +1009,16 @@ int btrfs_scan_fs_devices(int fd, const char *path,
 {
u64 total_devs;
u64 dev_size;
+   off_t seek_ret;
int ret;
if (!sb_bytenr)
sb_bytenr = BTRFS_SUPER_INFO_OFFSET;
 
-   dev_size = lseek(fd, 0, SEEK_END);
-   if (dev_size  0)
-   return (int)(dev_size);
+   seek_ret = lseek(fd, 0, SEEK_END);
+   if (seek_ret  0)
+   return -errno;
+
+   dev_size = seek_ret;
lseek(fd, 0, SEEK_SET);
if (sb_bytenr  dev_size) {
fprintf(stderr, Superblock bytenr is larger than device 
size\n);
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/5] btrfs-progs: Remove a unused function root_gtp_mask().

2014-12-18 Thread Qu Wenruo
Fix the following clang warning when compiling btrfs-progs:

radix-tree.c:78:21: warning: unused function 'root_gfp_mask'
[-Wunused-function]
static inline gfp_t root_gfp_mask(struct radix_tree_root *root)
^
1 warning generated.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
 radix-tree.c | 5 -
 1 file changed, 5 deletions(-)

diff --git a/radix-tree.c b/radix-tree.c
index 7457944..f259ab5 100644
--- a/radix-tree.c
+++ b/radix-tree.c
@@ -75,11 +75,6 @@ struct radix_tree_preload {
 };
 static struct radix_tree_preload radix_tree_preloads = { 0, };
 
-static inline gfp_t root_gfp_mask(struct radix_tree_root *root)
-{
-   return root-gfp_mask  __GFP_BITS_MASK;
-}
-
 static int internal_nodes = 0;
 /*
  * This assumes that the caller has performed appropriate preallocation, and
-- 
2.1.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-progs: integration-20141218 possible corruption test regression

2014-12-18 Thread Qu Wenruo


 Original Message 
Subject: Re: btrfs-progs: integration-20141218 possible corruption test 
regression

From: WorMzy Tykashi wormzy.tyka...@gmail.com
To: linux-btrfs@vger.kernel.org
Date: 2014年12月19日 02:39

On 18 December 2014 18:35:16 GMT+00:00, WorMzy Tykashi 
wormzy.tyka...@gmail.com wrote:

Hi guys,

The latest integration fails 'make test' with the following output:

 [TEST]013-leaf-corruption-no-extent-data.tar.xz
btrfs check should have detected corruption
Makefile:144: recipe for target 'test' failed
make: *** [test] Error 1
rm btrfs-corrupt-block.o


I've only built the progs on a single machine so far, but it
consistently fails at this test.

Cheers,


WorMzy

Regression is present in the 3.18.x branch too.

Cheers,


WorMzy

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

In fact, it's not a regression.

The 013 testcase is a special case that uses a script to corrupt the 
image and then do the btrfsck test.


There is a patch before the commit, to allow btrfs-progs test script 
call corruption script.
But since there is still some discussion about the corruption script and 
maybe later verify script,

the previous patch is not picked.

So btrfsck is ran on the *FINE* image without corrupting it, so you 
failed the test.


This testcase seems not to be included into 3.18.x soon, so it's not a 
problem.


If you still want to test it, please add the following patch as a 
temporary fix:

https://patchwork.kernel.org/patch/5490241/

Thanks,
Qu
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs-progs: Enhance the document of btrfs property

2014-12-18 Thread Satoru Takeuchi
From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com

Enhance the document of btrfs property as follows.

- Add the description about the candidates of name.
- Enrich the description of object.
- Fix typos and some redundancy.

Signed-off-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com
Reported-by: Naohiro Aota na...@elisp.net

---
 Documentation/btrfs-property.txt | 32 +++-
 1 file changed, 19 insertions(+), 13 deletions(-)

diff --git a/Documentation/btrfs-property.txt b/Documentation/btrfs-property.txt
index 3a71315..f8dcbab 100644
--- a/Documentation/btrfs-property.txt
+++ b/Documentation/btrfs-property.txt
@@ -11,8 +11,9 @@ SYNOPSIS
 
 DESCRIPTION
 ---
-*btrfs property* is used to get/set/list property, like lable or compression
-state, for given btrfs object.
+*btrfs property* is used to get/set/list property for given btrfs object.
+See the description of *get* subcommand for more information about
+both btrfs object and property.
 
 *btrfs property* provides an unified and user-friendly method to tune different
 btrfs properties instead of using the traditional method like `chattr`(1) or
@@ -23,31 +24,36 @@ SUBCOMMAND
 *get* [-t type] object [name]::
 Gets a property from a btrfs object.
 +
-If no name is specified, all properties for the given object are
-printed.
-A filesystem object can be a the filesystem itself, a subvolume,
-an inode or a device.
+A btrfs object, which is set by object, can be a btrfs filesystem
+itself, a btrfs subvolume, an inode(file or directory) inside btrfs,
+or a device on which a btrfs exists.
 +
+Set the name of property by 'name'. If no 'name' is specified,
+all properties for the given object are printed. 'name' is one of
+the followings.
++
+ro
+read-only flag of subvolume: true or false
+label
+label of device
+compression
+compression setting for an inode: lzo, zlib, or  (empty string)
+
 The '-t type' option can be used to explicitly
 specify what type of object you meant. This is only needed when a
 property could be set for more then one object type.
 +
 Possible types are 's[ubvol]', 'f[ilesystem]', 'i[node]' and 'd[evice]'.
-+
-The object can be path of btrfs device, btrfs mount point, or any
-directories/files inside btrfs.
 
 *set* [-t type] object name value::
 Sets a property on a btrfs object.
 +
-See description of *get* subcommand for a description of objects and object
-types.
+See the description of *get* subcommand for the meaning of each option.
 
 *list* [-t type] object::
 Lists available properties with their descriptions for the given object.
 +
-Please see the help of *btrfs property get* for a description of
-objects and object types.
+See the description of *get* subcommand for the meaning of each option.
 
 EXIT STATUS
 ---
-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html