Re: [PATCH v3 1/2] xfstests: add fssum tool
On 8/13/13 10:28 AM, Josef Bacik wrote: On Sun, Aug 11, 2013 at 09:54:41PM -0500, Eric Sandeen wrote: On Aug 11, 2013, at 8:15 PM, Dave Chinner da...@fromorbit.com wrote: On Thu, Aug 08, 2013 at 12:40:55PM -0500, Eric Sandeen wrote: On 8/8/13 3:17 AM, Jan Schmidt wrote: SUBDIRS = LLDLIBS = $(LIBATTR) $(LIBHANDLE) $(LIBACL) +OPT_LDLIBS = -lssl -lcrypto Hm, new deps. I guess it's not a huge problem, these should always be available, right? Please, let's keep crypto out of xfstests if we can. That's just going to add a nightmare of US export compliance garbage to any distro that wants to package and ship this It's ssl that's (optionally) required for check summing I believe. But it's optional if done properly... It doesn't build without -lcrypto for me, so either we have an external dependancy (which I'm fine with), or we add it to xfstests (which is what you wanted). Does a config option alleviate your concerns? And I it's just linking it in at build time, I don't think we have to work about export compliance as long as we aren't shipping actual crytpo stuff right? Anybody who is going to be running this stuff is going to have libssl installed, which installs libcrypto (at least on my box) so I don't think it's a huge deal to have us linking it. Thanks, Yeah I think linking in a consumer is not at all the same as exporting the functionality. Looks like my Makefile patch needs to add both -lssl -lcrypto after all, but again, yeah, just linking against it _if_ it's there. Oh, and here's my m4/package_ssldev.m4, sorry about that: AC_DEFUN([AC_PACKAGE_WANT_SSL], [ AC_CHECK_HEADERS(openssl/md5.h, [ have_ssl=true ], [ have_ssl=false ]) AC_SUBST(have_ssl) ]) -Eric Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/2] xfstests: add fssum tool
On Thu, Aug 08, 2013 at 12:40:55PM -0500, Eric Sandeen wrote: On 8/8/13 3:17 AM, Jan Schmidt wrote: SUBDIRS = LLDLIBS = $(LIBATTR) $(LIBHANDLE) $(LIBACL) +OPT_LDLIBS = -lssl -lcrypto Hm, new deps. I guess it's not a huge problem, these should always be available, right? Please, let's keep crypto out of xfstests if we can. That's just going to add a nightmare of US export compliance garbage to any distro that wants to package and ship this Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/2] xfstests: add fssum tool
On Aug 11, 2013, at 8:15 PM, Dave Chinner da...@fromorbit.com wrote: On Thu, Aug 08, 2013 at 12:40:55PM -0500, Eric Sandeen wrote: On 8/8/13 3:17 AM, Jan Schmidt wrote: SUBDIRS = LLDLIBS = $(LIBATTR) $(LIBHANDLE) $(LIBACL) +OPT_LDLIBS = -lssl -lcrypto Hm, new deps. I guess it's not a huge problem, these should always be available, right? Please, let's keep crypto out of xfstests if we can. That's just going to add a nightmare of US export compliance garbage to any distro that wants to package and ship this It's ssl that's (optionally) required for check summing I believe. But it's optional if done properly... -Eric Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 1/2] xfstests: add fssum tool
fssum is a tool to build a recursive checksum for a file system. The home
repository of fssum is
git://git.kernel.org/pub/scm/linux/kernel/git/arne/far-progs.git
It is added as an optional target, because it depends on glibc = 2.15 for
SEEK_HOLE / SEEK_DATA. The test to be added using fssum will just be skipped
if fssum wasn't built.
Signed-off-by: Jan Schmidt list@jan-o-sch.net
---
.gitignore|1 +
common/config |2 +
src/Makefile | 11 +-
src/fssum.c | 819 +
4 files changed, 832 insertions(+), 1 deletions(-)
create mode 100644 src/fssum.c
diff --git a/.gitignore b/.gitignore
index 11594aa..c2fc6e3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -45,6 +45,7 @@
/src/fill
/src/fill2
/src/fs_perms
+/src/fssum
/src/fstest
/src/fsync-tester
/src/ftrunc
diff --git a/common/config b/common/config
index 67c1498..c8bee29 100644
--- a/common/config
+++ b/common/config
@@ -146,6 +146,8 @@ export SED_PROG=`set_prog_path sed`
export BC_PROG=`set_prog_path bc`
[ $BC_PROG = ] _fatal bc not found
+export FSSUM_PROG=`set_prog_path fssum $here/src/fssum`
+
export PS_ALL_FLAGS=-ef
export DF_PROG=`set_prog_path df`
diff --git a/src/Makefile b/src/Makefile
index cc679e8..10a4d3c 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -20,10 +20,14 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize
preallo_rw_pattern_reader \
stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
seek_copy_test t_readdir_1 t_readdir_2 fsync-tester
+OPT_TARGETS = fssum
+
SUBDIRS =
LLDLIBS = $(LIBATTR) $(LIBHANDLE) $(LIBACL)
+OPT_LDLIBS = -lssl -lcrypto
+
ifeq ($(HAVE_XLOG_ASSIGN_LSN), true)
LINUX_TARGETS += loggen
endif
@@ -60,7 +64,7 @@ CFILES = $(TARGETS:=.c)
LDIRT = $(TARGETS)
-default: depend $(TARGETS) $(SUBDIRS)
+default: depend $(TARGETS) $(OPT_TARGETS) $(SUBDIRS)
depend: .dep
@@ -70,11 +74,16 @@ $(TARGETS): $(LIBTEST)
@echo [CC]$@
$(Q)$(LTLINK) $@.c -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS) $(LIBTEST)
+$(OPT_TARGETS): $(LIBTEST)
+ @echo [CC]$@
+ -$(Q)$(LTLINK) $@.c -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS) $(OPT_LDLIBS)
$(LIBTEST)
+
LINKTEST = $(LTLINK) $@.c -o $@ $(CFLAGS) $(LDFLAGS)
install: default $(addsuffix -install,$(SUBDIRS))
$(INSTALL) -m 755 -d $(PKG_LIB_DIR)/src
$(LTINSTALL) -m 755 $(TARGETS) $(PKG_LIB_DIR)/src
+ -$(LTINSTALL) -m 755 $(OPT_TARGETS) $(PKG_LIB_DIR)/src
$(LTINSTALL) -m 755 fill2attr fill2fs fill2fs_check scaleread.sh
$(PKG_LIB_DIR)/src
$(LTINSTALL) -m 644 dumpfile $(PKG_LIB_DIR)/src
diff --git a/src/fssum.c b/src/fssum.c
new file mode 100644
index 000..ecddb6a
--- /dev/null
+++ b/src/fssum.c
@@ -0,0 +1,819 @@
+/*
+ * Copyright (C) 2012 STRATO AG. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#define _BSD_SOURCE
+#define _LARGEFILE64_SOURCE
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include stdio.h
+#include stdlib.h
+#include unistd.h
+#include string.h
+#include fcntl.h
+#include dirent.h
+#include errno.h
+#include sys/types.h
+#include sys/stat.h
+#ifdef __SOLARIS__
+#include sys/mkdev.h
+#endif
+#include openssl/md5.h
+#include netinet/in.h
+#include inttypes.h
+#include assert.h
+
+#define CS_SIZE 16
+#define CHUNKS 128
+
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define htonll(x) __bswap_64 (x)
+#endif
+
+/* TODO: add hardlink recognition */
+/* TODO: add xattr/acl */
+
+struct excludes {
+ char *path;
+ int len;
+};
+
+typedef struct _sum {
+ MD5_CTX md5;
+ unsigned char out[16];
+} sum_t;
+
+typedef int (*sum_file_data_t)(int fd, sum_t *dst);
+
+int gen_manifest = 0;
+int in_manifest = 0;
+char *checksum = NULL;
+struct excludes *excludes;
+int n_excludes = 0;
+int verbose = 0;
+FILE *out_fp;
+FILE *in_fp;
+
+enum _flags {
+ FLAG_UID,
+ FLAG_GID,
+ FLAG_MODE,
+ FLAG_ATIME,
+ FLAG_MTIME,
+ FLAG_CTIME,
+ FLAG_DATA,
+ FLAG_OPEN_ERROR,
+ FLAG_STRUCTURE,
+ NUM_FLAGS
+};
+
+const char flchar[] = ugoamcdes;
+char line[65536];
+
+int flags[NUM_FLAGS] = {1, 1, 1, 1, 1, 0, 1, 0, 0};
+
+char *
+getln(char *buf, int size, FILE *fp)
+{
+ char *p;
+ int l;
+
+ p = fgets(buf, size, fp);
+ if (!p)
+ return NULL;
+
+ l
Re: [PATCH v3 1/2] xfstests: add fssum tool
On 8/8/13 3:17 AM, Jan Schmidt wrote: fssum is a tool to build a recursive checksum for a file system. The home repository of fssum is git://git.kernel.org/pub/scm/linux/kernel/git/arne/far-progs.git It is added as an optional target, because it depends on glibc = 2.15 for SEEK_HOLE / SEEK_DATA. The test to be added using fssum will just be skipped if fssum wasn't built. Signed-off-by: Jan Schmidt list@jan-o-sch.net --- .gitignore|1 + common/config |2 + src/Makefile | 11 +- src/fssum.c | 819 + 4 files changed, 832 insertions(+), 1 deletions(-) create mode 100644 src/fssum.c diff --git a/.gitignore b/.gitignore index 11594aa..c2fc6e3 100644 --- a/.gitignore +++ b/.gitignore @@ -45,6 +45,7 @@ /src/fill /src/fill2 /src/fs_perms +/src/fssum /src/fstest /src/fsync-tester /src/ftrunc diff --git a/common/config b/common/config index 67c1498..c8bee29 100644 --- a/common/config +++ b/common/config @@ -146,6 +146,8 @@ export SED_PROG=`set_prog_path sed` export BC_PROG=`set_prog_path bc` [ $BC_PROG = ] _fatal bc not found +export FSSUM_PROG=`set_prog_path fssum $here/src/fssum` So this will pick up a local copy of fssum if it exists; is that really desired? (If there's any difference in behavior, then the one in src/ presumably would need to be fixed...) + export PS_ALL_FLAGS=-ef export DF_PROG=`set_prog_path df` diff --git a/src/Makefile b/src/Makefile index cc679e8..10a4d3c 100644 --- a/src/Makefile +++ b/src/Makefile @@ -20,10 +20,14 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \ seek_copy_test t_readdir_1 t_readdir_2 fsync-tester +OPT_TARGETS = fssum + I'm not sure how this helps . . . SUBDIRS = LLDLIBS = $(LIBATTR) $(LIBHANDLE) $(LIBACL) +OPT_LDLIBS = -lssl -lcrypto Hm, new deps. I guess it's not a huge problem, these should always be available, right? ifeq ($(HAVE_XLOG_ASSIGN_LSN), true) LINUX_TARGETS += loggen endif @@ -60,7 +64,7 @@ CFILES = $(TARGETS:=.c) LDIRT = $(TARGETS) -default: depend $(TARGETS) $(SUBDIRS) +default: depend $(TARGETS) $(OPT_TARGETS) $(SUBDIRS) Anyway, OPT_TARGETS isn't optional, because you still build it by default. :) depend: .dep @@ -70,11 +74,16 @@ $(TARGETS): $(LIBTEST) @echo [CC]$@ $(Q)$(LTLINK) $@.c -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS) $(LIBTEST) +$(OPT_TARGETS): $(LIBTEST) + @echo [CC]$@ + -$(Q)$(LTLINK) $@.c -o $@ $(CFLAGS) $(LDFLAGS) $(LDLIBS) $(OPT_LDLIBS) $(LIBTEST) Oh, I see, you ignore the error. Well, that's still pretty ugly. I'd really rather you just add the #defines as I suggested in my reply to [PATCH 0/2], so it'll build for everyone. Thanks, -Eric + LINKTEST = $(LTLINK) $@.c -o $@ $(CFLAGS) $(LDFLAGS) install: default $(addsuffix -install,$(SUBDIRS)) $(INSTALL) -m 755 -d $(PKG_LIB_DIR)/src $(LTINSTALL) -m 755 $(TARGETS) $(PKG_LIB_DIR)/src + -$(LTINSTALL) -m 755 $(OPT_TARGETS) $(PKG_LIB_DIR)/src $(LTINSTALL) -m 755 fill2attr fill2fs fill2fs_check scaleread.sh $(PKG_LIB_DIR)/src $(LTINSTALL) -m 644 dumpfile $(PKG_LIB_DIR)/src -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
