Hello community, here is the log from the commit of package fdupes for openSUSE:Factory checked in at 2016-11-07 12:21:17 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/fdupes (Old) and /work/SRC/openSUSE:Factory/.fdupes.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "fdupes" Changes: -------- --- /work/SRC/openSUSE:Factory/fdupes/fdupes.changes 2015-09-19 07:56:24.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.fdupes.new/fdupes.changes 2016-11-07 12:21:18.000000000 +0100 @@ -1,0 +2,25 @@ +Fri Nov 4 14:33:59 UTC 2016 - [email protected] + +- Drop 50_bts284274_hardlinkreplace.dpatch. The --linkhard option + added by this patch has an implementation bug that can cause data + loss. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=677419 + has more details. + +------------------------------------------------------------------- +Fri Nov 4 13:47:27 UTC 2016 - [email protected] + +- Update to version 1.6.1. The following patches have been applied + upstream and were dropped: + * 0001-restore-pristine-code.patch + * 0002-Added-to-escape-minus-signs-in-manpage-lintian-warni.patch + * 0003-Fix-a-typo-in-a-manpage-bts353789.patch + * 0005-add-summarize-to-manpage-bts481809.patch + * 0006-add-nohidden-support-bts511702.patch + * 0007-Disambiguate-the-options-recurse-and-recurse-bts5371.patch + * 0008-speedup-the-file-compare.patch + * 0009-glibc-endianness-check-in-md5.patch + * 0010-add-permissions-mode.patch + * 0011-add-an-option-to-sort-duplicate-files-by-name.patch +- 50_bts284274_hardlinkreplace.dpatch had to be refreshed. + +------------------------------------------------------------------- Old: ---- 0001-restore-pristine-code.patch 0002-Added-to-escape-minus-signs-in-manpage-lintian-warni.patch 0003-Fix-a-typo-in-a-manpage-bts353789.patch 0005-add-summarize-to-manpage-bts481809.patch 0006-add-nohidden-support-bts511702.patch 0007-Disambiguate-the-options-recurse-and-recurse-bts5371.patch 0008-speedup-the-file-compare.patch 0009-glibc-endianness-check-in-md5.patch 0010-add-permissions-mode.patch 0011-add-an-option-to-sort-duplicate-files-by-name.patch 50_bts284274_hardlinkreplace.dpatch fdupes-1.51.tar.gz New: ---- fdupes-1.6.1.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ fdupes.spec ++++++ --- /var/tmp/diff_new_pack.wIyG34/_old 2016-11-07 12:21:19.000000000 +0100 +++ /var/tmp/diff_new_pack.wIyG34/_new 2016-11-07 12:21:19.000000000 +0100 @@ -1,7 +1,7 @@ # # spec file for package fdupes # -# Copyright (c) 2015 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -17,61 +17,25 @@ Name: fdupes -Version: 1.51 +Version: 1.6.1 Release: 0 Summary: Identifying or deleting duplicate files License: MIT Group: Productivity/Archiving/Compression Url: https://github.com/adrianlopezroche/fdupes -Source0: https://github.com/adrianlopezroche/%{name}/archive/%{name}-%{version}.tar.gz +Source0: https://github.com/adrianlopezroche/fdupes/archive/v%{version}.tar.gz#/%{name}-%{version}.tar.gz Source1: macros.fdupes #PATCH-FIX-SUSE: fix patch according distro's needs Patch0: fdupes-makefile.patch -# I'v imported all patches from Debian (Fedora and Gentoo has the subset only) -# to git and added there as individual patches -# TODO: export the git repo or upstream it -#PATCH-FIX-DEBIAN: obsoletes partly the speedup patch -Patch1: 0001-restore-pristine-code.patch -#PATCH-FIX-DEBIAN: minor typo in man page -Patch2: 0002-Added-to-escape-minus-signs-in-manpage-lintian-warni.patch -#PATCH-FIX-DEBIAN: dtto -Patch3: 0003-Fix-a-typo-in-a-manpage-bts353789.patch -#PATCH-FIX-DEBIAN: manpage fix -Patch5: 0005-add-summarize-to-manpage-bts481809.patch -#PATCH-FIX-DEBIAN: dtto -Patch6: 0006-add-nohidden-support-bts511702.patch -#PATCH-FIX-DEBIAN: manpage fix -Patch7: 0007-Disambiguate-the-options-recurse-and-recurse-bts5371.patch -#PATCH-FIX-OPENSUSE: speedup the compare -Patch8: 0008-speedup-the-file-compare.patch -#PATCH-FIX-OPENSUSE: endianness check in md5.h -Patch9: 0009-glibc-endianness-check-in-md5.patch -#PATCH-FIX-OPENSUSE: -p/--permissions mode -Patch10: 0010-add-permissions-mode.patch -#PATCH-FIX-OPENSUSE: -o/--order mode -Patch11: 0011-add-an-option-to-sort-duplicate-files-by-name.patch -#PATCH-FIX-DEBIAN: add -L/--linkhard -Patch20: 50_bts284274_hardlinkreplace.dpatch BuildRoot: %{_tmppath}/%{name}-%{version}-build %description FDUPES is a program for identifying or deleting duplicate files -residing within specified directories +residing within specified directories. %prep -%setup -q -n %{name}-%{name}-%{version} +%setup -q %patch0 -%patch1 -p1 -%patch2 -p1 -%patch3 -p1 -%patch5 -p1 -%patch6 -p1 -%patch7 -p1 -%patch8 -p1 -%patch9 -p1 -%patch10 -p1 -%patch11 -p1 -%patch20 -p1 %build make %{?_smp_mflags} COMPILER_OPTIONS="%{optflags}" ++++++ fdupes-1.51.tar.gz -> fdupes-1.6.1.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/.gitignore new/fdupes-1.6.1/.gitignore --- old/fdupes-fdupes-1.51/.gitignore 1970-01-01 01:00:00.000000000 +0100 +++ new/fdupes-1.6.1/.gitignore 2016-08-21 06:54:46.000000000 +0200 @@ -0,0 +1,2 @@ +fdupes +*.o diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/CHANGES new/fdupes-1.6.1/CHANGES --- old/fdupes-fdupes-1.51/CHANGES 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/CHANGES 2016-08-21 06:54:46.000000000 +0200 @@ -6,6 +6,25 @@ those who've otherwise worked on that item. For a list of contributors names and identifiers please see the CONTRIBUTORS file. +Changes from 1.6.0 to 1.6.1: + +- Fix 'invalid option' error for -I. [AL] + +Changes from 1.51 to 1.6.0: + +- Add --nohidden option. [MC] +- Add --permissions option. [MV] +- Add --order option. [SB] +- Add --reverse option. [AL] +- Add --immediate option. [AL] +- Speed up file comparison. [SB, ST] +- Fix bug where fdupes fails to consistently ignore hardlinks, depending on + file processing order, when F_CONSIDERHARDLINKS flag is not set. [AL] +- Fix bug in is_hardlink(): Proper traversal of duplicates is done via + dupe->duplicates, not via dupe->next. [AL] +- Revise md5.c to compile under OSX. [TH] +- Remove EXTERNAL_MD5 configuration option. [ID] +- Various other changes (see git log). Changes from 1.50 to 1.51 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/CONTRIBUTORS new/fdupes-1.6.1/CONTRIBUTORS --- old/fdupes-fdupes-1.51/CONTRIBUTORS 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/CONTRIBUTORS 2016-08-21 06:54:46.000000000 +0200 @@ -2,17 +2,22 @@ of fdupes. Please see the CHANGES file for detailed information on their contributions. Names are listed in alphabetical order. - [AB] Adrian Bridgett ([email protected]) - [AL] Adrian Lopez ([email protected]) -[CHL] Charles Longeau ([email protected]) + [AB] Adrian Bridgett <[email protected]> + [AL] Adrian Lopez <[email protected]> +[CHL] Charles Longeau <[email protected]> [FD] Frank DENIS, a.k.a. Jedi/Sector One, a.k.a. - DJ Chrysalis ([email protected]) - [JB] Jean-Baptiste () - [KK] Kresimir Kukulj ([email protected]) - [LB] Laurent Bonnaud ([email protected]) - [LM] Luca Montecchiani ([email protected]) - [LR] Lukas Ruf ([email protected]) - [PB] Peter Bray (Sydney, Australia) -[SSD] Steven S. Dick ([email protected]) - + DJ Chrysalis <[email protected]> + [JB] Jean-Baptiste + [KK] Kresimir Kukulj <[email protected]> + [LB] Laurent Bonnaud <[email protected]> + [LM] Luca Montecchiani <[email protected]> + [LR] Lukas Ruf <[email protected]> + [PB] Peter Bray (Sydney, Australia) +[SSD] Steven S. Dick <[email protected]> + [MC] Maximiliano Curia <[email protected]> + [MV] Michal Vyskocil <[email protected]> + [SB] Stefan Brüns <[email protected]> + [ST] Sandro Tosi <[email protected]> + [ID] Ivan Diorditsa <[email protected]> + [TH] Tom Hoover <[email protected]> \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/Makefile new/fdupes-1.6.1/Makefile --- old/fdupes-fdupes-1.51/Makefile 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/Makefile 2016-08-21 06:54:46.000000000 +0200 @@ -25,13 +25,6 @@ # #OMIT_GETOPT_LONG = -DOMIT_GETOPT_LONG -# -# To use the md5sum program for calculating signatures (instead of the -# built in MD5 message digest routines) uncomment the following -# line (try this if you're having trouble with built in code). -# -#EXTERNAL_MD5 = -DEXTERNAL_MD5=\"md5sum\" - ##################################################################### # Developer Configuration Section # ##################################################################### @@ -56,7 +49,7 @@ # MAN_DIR indicates directory where the fdupes man page is to be # installed. Suggested value is "$(PREFIX)/man/man1" # -MAN_BASE_DIR = $(PREFIX)/man +MAN_BASE_DIR = $(PREFIX)/share/man MAN_DIR = $(MAN_BASE_DIR)/man1 MAN_EXT = 1 @@ -77,10 +70,10 @@ # # Make Configuration # -CC = gcc +CC ?= gcc COMPILER_OPTIONS = -Wall -O -g -CFLAGS= $(COMPILER_OPTIONS) -I. -DVERSION=\"$(VERSION)\" $(EXTERNAL_MD5) $(OMIT_GETOPT_LONG) $(FILEOFFSET_64BIT) +CFLAGS= $(COMPILER_OPTIONS) -I. -DVERSION=\"$(VERSION)\" $(OMIT_GETOPT_LONG) $(FILEOFFSET_64BIT) INSTALL_PROGRAM = $(INSTALL) -c -m 0755 INSTALL_DATA = $(INSTALL) -c -m 0644 @@ -100,15 +93,15 @@ all: fdupes fdupes: $(OBJECT_FILES) - $(CC) $(CFLAGS) -o fdupes $(OBJECT_FILES) + $(CC) $(CFLAGS) $(LDFLAGS) -o fdupes $(OBJECT_FILES) installdirs: - test -d $(BIN_DIR) || $(MKDIR) $(BIN_DIR) - test -d $(MAN_DIR) || $(MKDIR) $(MAN_DIR) + test -d $(DESTDIR)$(BIN_DIR) || $(MKDIR) $(DESTDIR)$(BIN_DIR) + test -d $(DESTDIR)$(MAN_DIR) || $(MKDIR) $(DESTDIR)$(MAN_DIR) install: fdupes installdirs - $(INSTALL_PROGRAM) fdupes $(BIN_DIR)/$(PROGRAM_NAME) - $(INSTALL_DATA) fdupes.1 $(MAN_DIR)/$(PROGRAM_NAME).$(MAN_EXT) + $(INSTALL_PROGRAM) fdupes $(DESTDIR)$(BIN_DIR)/$(PROGRAM_NAME) + $(INSTALL_DATA) fdupes.1 $(DESTDIR)$(MAN_DIR)/$(PROGRAM_NAME).$(MAN_EXT) clean: $(RM) $(OBJECT_FILES) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/Makefile.inc/VERSION new/fdupes-1.6.1/Makefile.inc/VERSION --- old/fdupes-fdupes-1.51/Makefile.inc/VERSION 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/Makefile.inc/VERSION 2016-08-21 06:54:46.000000000 +0200 @@ -2,4 +2,4 @@ # VERSION determines the program's version number. # -VERSION = 1.51 +VERSION = 1.6.1 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/README new/fdupes-1.6.1/README --- old/fdupes-fdupes-1.51/README 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/README 2016-08-21 06:54:46.000000000 +0200 @@ -11,15 +11,18 @@ -r --recurse for every directory given follow subdirectories encountered within -R --recurse: for each directory given after this option follow - subdirectories encountered within + subdirectories encountered within (note the ':' at + the end of the option, manpage for more details) -s --symlinks follow symlinks -H --hardlinks normally, when two or more files point to the same disk area they are treated as non-duplicates; this option will change this behavior -n --noempty exclude zero-length files from consideration + -A --nohidden exclude hidden files from consideration -f --omitfirst omit the first file in each set of matches -1 --sameline list each set of matches on a single line -S --size show size of duplicate files + -m --summarize summarize dupe information -q --quiet hide progress indicator -d --delete prompt user for files to preserve and delete all others; important: under particular circumstances, @@ -27,6 +30,16 @@ with -s or --symlinks, or when specifying a particular directory more than once; refer to the fdupes documentation for additional information + -N --noprompt together with --delete, preserve the first file in + each set of duplicates and delete the rest without + prompting the user + -I --immediate delete duplicates as they are encountered, without + grouping into sets; implies --noprompt + -p --permissions don't consider files with different owner/group or + permission bits as duplicates + -o --order=BY select sort order for output, linking and deleting; by + mtime (BY='time'; default) or filename (BY='name') + -i --reverse reverse order while sorting -v --version display fdupes version -h --help display this help message diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/fdupes.1 new/fdupes-1.6.1/fdupes.1 --- old/fdupes-fdupes-1.51/fdupes.1 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/fdupes.1 2016-08-21 06:54:46.000000000 +0200 @@ -23,7 +23,8 @@ .TP .B -R --recurse: for each directory given after this option follow subdirectories -encountered within +encountered within (note the ':' at the end of option; see the +Examples section below for further explanation) .TP .B -s --symlinks follow symlinked directories @@ -38,12 +39,18 @@ .B -f --omitfirst omit the first file in each set of matches .TP +.B -A --nohidden +exclude hidden files from consideration +.TP .B -1 --sameline list each set of matches on a single line .TP .B -S --size show size of duplicate files .TP +.B -m --summarize +summarize duplicate files information +.TP .B -q --quiet hide progress indicator .TP @@ -53,9 +60,23 @@ below) .TP .B -N --noprompt -when used together with --delete, preserve the first file in each +when used together with \-\-delete, preserve the first file in each set of duplicates and delete the others without prompting the user .TP +.B -I --immediate +delete duplicates as they are encountered, without +grouping into sets; implies --noprompt +.TP +.B -p --permissions +don't consider files with different owner/group or permission bits as duplicates +.TP +.B -o --order\fR=\fIWORD\fR +order files according to WORD: +time - sort by mtime, name - sort by filename +.TP +.B -i --reverse +reverse order while sorting +.TP .B -v --version display fdupes version .TP @@ -79,12 +100,21 @@ .B --sameline is specified, spaces and backslash characters (\fB\e\fP) appearing in a filename are preceded by a backslash character. + +.SH EXAMPLES +.TP +.B fdupes a --recurse: b +will follow subdirectories under b, but not those under a. +.TP +.B fdupes a --recurse b +will follow subdirectories under both a and b. + .SH CAVEATS If fdupes returns with an error message such as .B fdupes: error invoking md5sum it means the program has been compiled to use an external program to calculate MD5 signatures (otherwise, fdupes uses -interal routines for this purpose), and an error has occurred +internal routines for this purpose), and an error has occurred while attempting to execute it. If this is the case, the specified program should be properly installed prior to running fdupes. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/fdupes.c new/fdupes-1.6.1/fdupes.c --- old/fdupes-fdupes-1.51/fdupes.c 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/fdupes.c 2016-08-21 06:54:46.000000000 +0200 @@ -31,10 +31,9 @@ #endif #include <string.h> #include <errno.h> +#include <libgen.h> -#ifndef EXTERNAL_MD5 #include "md5/md5.h" -#endif #define ISFLAG(a,b) ((a & b) == b) #define SETFLAG(a,b) (a |= b) @@ -51,6 +50,15 @@ #define F_RECURSEAFTER 0x0200 #define F_NOPROMPT 0x0400 #define F_SUMMARIZEMATCHES 0x0800 +#define F_EXCLUDEHIDDEN 0x1000 +#define F_PERMISSIONS 0x2000 +#define F_REVERSE 0x4000 +#define F_IMMEDIATE 0x8000 + +typedef enum { + ORDER_TIME = 0, + ORDER_NAME +} ordertype_t; char *program_name; @@ -62,6 +70,8 @@ #define PARTIAL_MD5_SIZE 4096 +#define MD5_DIGEST_LENGTH 16 + /* TODO: Partial sums (for working with very large files). @@ -83,8 +93,8 @@ typedef struct _file { char *d_name; off_t size; - char *crcpartial; - char *crcsignature; + md5_byte_t *crcpartial; + md5_byte_t *crcsignature; dev_t device; ino_t inode; time_t mtime; @@ -240,6 +250,7 @@ struct stat linfo; static int progress = 0; static char indicator[] = "-\\|/"; + char *fullname, *name; cd = opendir(dir); @@ -285,6 +296,17 @@ strcat(newfile->d_name, "/"); strcat(newfile->d_name, dirinfo->d_name); + if (ISFLAG(flags, F_EXCLUDEHIDDEN)) { + fullname = strdup(newfile->d_name); + name = basename(fullname); + if (name[0] == '.' && strcmp(name, ".") && strcmp(name, "..") ) { + free(newfile->d_name); + free(newfile); + continue; + } + free(fullname); + } + if (filesize(newfile->d_name) == 0 && ISFLAG(flags, F_EXCLUDEEMPTY)) { free(newfile->d_name); free(newfile); @@ -325,20 +347,13 @@ return filecount; } -#ifndef EXTERNAL_MD5 - -/* If EXTERNAL_MD5 is not defined, use L. Peter Deutsch's MD5 library. - */ -char *getcrcsignatureuntil(char *filename, off_t max_read) +md5_byte_t *getcrcsignatureuntil(char *filename, off_t max_read) { - int x; off_t fsize; off_t toread; md5_state_t state; - md5_byte_t digest[16]; + static md5_byte_t digest[MD5_DIGEST_LENGTH]; static md5_byte_t chunk[CHUNK_SIZE]; - static char signature[16*2 + 1]; - char *sigp; FILE *file; md5_init(&state); @@ -356,7 +371,7 @@ } while (fsize > 0) { - toread = (fsize % CHUNK_SIZE) ? (fsize % CHUNK_SIZE) : CHUNK_SIZE; + toread = (fsize >= CHUNK_SIZE) ? CHUNK_SIZE : fsize; if (fread(chunk, toread, 1, file) != 1) { errormsg("error reading from file %s\n", filename); fclose(file); @@ -368,71 +383,44 @@ md5_finish(&state, digest); - sigp = signature; - - for (x = 0; x < 16; x++) { - sprintf(sigp, "%02x", digest[x]); - sigp = strchr(sigp, '\0'); - } - fclose(file); - return signature; + return digest; } -char *getcrcsignature(char *filename) +md5_byte_t *getcrcsignature(char *filename) { return getcrcsignatureuntil(filename, 0); } -char *getcrcpartialsignature(char *filename) +md5_byte_t *getcrcpartialsignature(char *filename) { return getcrcsignatureuntil(filename, PARTIAL_MD5_SIZE); } -#endif /* [#ifndef EXTERNAL_MD5] */ - -#ifdef EXTERNAL_MD5 - -/* If EXTERNAL_MD5 is defined, use md5sum program to calculate signatures. - */ -char *getcrcsignature(char *filename) +int md5cmp(const md5_byte_t *a, const md5_byte_t *b) { - static char signature[256]; - char *command; - char *separator; - FILE *result; - - command = (char*) malloc(strlen(filename)+strlen(EXTERNAL_MD5)+2); - if (command == NULL) { - errormsg("out of memory\n"); - exit(1); - } - - sprintf(command, "%s %s", EXTERNAL_MD5, filename); + int x; - result = popen(command, "r"); - if (result == NULL) { - errormsg("error invoking %s\n", EXTERNAL_MD5); - exit(1); + for (x = 0; x < MD5_DIGEST_LENGTH; ++x) + { + if (a[x] < b[x]) + return -1; + else if (a[x] > b[x]) + return 1; } - - free(command); - if (fgets(signature, 256, result) == NULL) { - errormsg("error generating signature for %s\n", filename); - return NULL; - } - separator = strchr(signature, ' '); - if (separator) *separator = '\0'; + return 0; +} - pclose(result); +void md5copy(md5_byte_t *to, const md5_byte_t *from) +{ + int x; - return signature; + for (x = 0; x < MD5_DIGEST_LENGTH; ++x) + to[x] = from[x]; } -#endif /* [#ifdef EXTERNAL_MD5] */ - void purgetree(filetree_t *checktree) { if (checktree->left != NULL) purgetree(checktree->left); @@ -467,10 +455,51 @@ return 1; } +int same_permissions(char* name1, char* name2) +{ + struct stat s1, s2; + + if (stat(name1, &s1) != 0) return -1; + if (stat(name2, &s2) != 0) return -1; + + return (s1.st_mode == s2.st_mode && + s1.st_uid == s2.st_uid && + s1.st_gid == s2.st_gid); +} + +int is_hardlink(filetree_t *checktree, file_t *file) +{ + file_t *dupe; + ino_t inode; + dev_t device; + + inode = getinode(file->d_name); + device = getdevice(file->d_name); + + if ((inode == checktree->file->inode) && + (device == checktree->file->device)) + return 1; + + if (checktree->file->hasdupes) + { + dupe = checktree->file->duplicates; + + do { + if ((inode == dupe->inode) && + (device == dupe->device)) + return 1; + + dupe = dupe->duplicates; + } while (dupe != NULL); + } + + return 0; +} + file_t **checkmatch(filetree_t **root, filetree_t *checktree, file_t *file) { int cmpresult; - char *crcsignature; + md5_byte_t *crcsignature; off_t fsize; /* If device and inode fields are equal one of the files is a @@ -479,9 +508,8 @@ duplicates unless the user specifies otherwise. */ - if (!ISFLAG(flags, F_CONSIDERHARDLINKS) && (getinode(file->d_name) == - checktree->file->inode) && (getdevice(file->d_name) == - checktree->file->device)) return NULL; + if (!ISFLAG(flags, F_CONSIDERHARDLINKS) && is_hardlink(checktree, file)) + return NULL; fsize = filesize(file->d_name); @@ -489,32 +517,42 @@ cmpresult = -1; else if (fsize > checktree->file->size) cmpresult = 1; + else + if (ISFLAG(flags, F_PERMISSIONS) && + !same_permissions(file->d_name, checktree->file->d_name)) + cmpresult = -1; else { if (checktree->file->crcpartial == NULL) { crcsignature = getcrcpartialsignature(checktree->file->d_name); - if (crcsignature == NULL) return NULL; + if (crcsignature == NULL) { + errormsg ("cannot read file %s\n", checktree->file->d_name); + return NULL; + } - checktree->file->crcpartial = (char*) malloc(strlen(crcsignature)+1); + checktree->file->crcpartial = (md5_byte_t*) malloc(MD5_DIGEST_LENGTH * sizeof(md5_byte_t)); if (checktree->file->crcpartial == NULL) { errormsg("out of memory\n"); exit(1); } - strcpy(checktree->file->crcpartial, crcsignature); + md5copy(checktree->file->crcpartial, crcsignature); } if (file->crcpartial == NULL) { crcsignature = getcrcpartialsignature(file->d_name); - if (crcsignature == NULL) return NULL; + if (crcsignature == NULL) { + errormsg ("cannot read file %s\n", file->d_name); + return NULL; + } - file->crcpartial = (char*) malloc(strlen(crcsignature)+1); + file->crcpartial = (md5_byte_t*) malloc(MD5_DIGEST_LENGTH * sizeof(md5_byte_t)); if (file->crcpartial == NULL) { errormsg("out of memory\n"); exit(1); } - strcpy(file->crcpartial, crcsignature); + md5copy(file->crcpartial, crcsignature); } - cmpresult = strcmp(file->crcpartial, checktree->file->crcpartial); + cmpresult = md5cmp(file->crcpartial, checktree->file->crcpartial); /*if (cmpresult != 0) errormsg(" on %s vs %s\n", file->d_name, checktree->file->d_name);*/ if (cmpresult == 0) { @@ -522,27 +560,27 @@ crcsignature = getcrcsignature(checktree->file->d_name); if (crcsignature == NULL) return NULL; - checktree->file->crcsignature = (char*) malloc(strlen(crcsignature)+1); + checktree->file->crcsignature = (md5_byte_t*) malloc(MD5_DIGEST_LENGTH * sizeof(md5_byte_t)); if (checktree->file->crcsignature == NULL) { errormsg("out of memory\n"); exit(1); } - strcpy(checktree->file->crcsignature, crcsignature); + md5copy(checktree->file->crcsignature, crcsignature); } if (file->crcsignature == NULL) { crcsignature = getcrcsignature(file->d_name); if (crcsignature == NULL) return NULL; - file->crcsignature = (char*) malloc(strlen(crcsignature)+1); + file->crcsignature = (md5_byte_t*) malloc(MD5_DIGEST_LENGTH * sizeof(md5_byte_t)); if (file->crcsignature == NULL) { errormsg("out of memory\n"); exit(1); } - strcpy(file->crcsignature, crcsignature); + md5copy(file->crcsignature, crcsignature); } - cmpresult = strcmp(file->crcsignature, checktree->file->crcsignature); + cmpresult = md5cmp(file->crcsignature, checktree->file->crcsignature); /*if (cmpresult != 0) errormsg("P on %s vs %s\n", file->d_name, checktree->file->d_name); else errormsg("P F on %s vs %s\n", file->d_name, @@ -577,8 +615,8 @@ int confirmmatch(FILE *file1, FILE *file2) { - unsigned char c1 = 0; - unsigned char c2 = 0; + unsigned char c1[CHUNK_SIZE]; + unsigned char c2[CHUNK_SIZE]; size_t r1; size_t r2; @@ -586,14 +624,13 @@ fseek(file2, 0, SEEK_SET); do { - r1 = fread(&c1, sizeof(c1), 1, file1); - r2 = fread(&c2, sizeof(c2), 1, file2); + r1 = fread(c1, sizeof(unsigned char), sizeof(c1), file1); + r2 = fread(c2, sizeof(unsigned char), sizeof(c2), file2); - if (c1 != c2) return 0; /* file contents are different */ - } while (r1 && r2); + if (r1 != r2) return 0; /* file lengths are different */ + if (memcmp (c1, c2, r1)) return 0; /* file contents are different */ + } while (r2); - if (r1 != r2) return 0; /* file lengths are different */ - return 1; } @@ -629,7 +666,7 @@ if (numbytes < 1024.0) printf("%d duplicate files (in %d sets), occupying %.0f bytes.\n\n", numfiles, numsets, numbytes); else if (numbytes <= (1000.0 * 1000.0)) - printf("%d duplicate files (in %d sets), occupying %.1f kylobytes\n\n", numfiles, numsets, numbytes / 1000.0); + printf("%d duplicate files (in %d sets), occupying %.1f kilobytes\n\n", numfiles, numsets, numbytes / 1000.0); else printf("%d duplicate files (in %d sets), occupying %.1f megabytes\n\n", numfiles, numsets, numbytes / (1000.0 * 1000.0)); @@ -643,7 +680,7 @@ while (files != NULL) { if (files->hasdupes) { if (!ISFLAG(flags, F_OMITFIRST)) { - if (ISFLAG(flags, F_SHOWSIZE)) printf("%lld byte%seach:\n", files->size, + if (ISFLAG(flags, F_SHOWSIZE)) printf("%lld byte%seach:\n", (long long int)files->size, (files->size != 1) ? "s " : " "); if (ISFLAG(flags, F_DSAMELINE)) escapefilename("\\ ", &files->d_name); printf("%s%c", files->d_name, ISFLAG(flags, F_DSAMELINE)?' ':'\n'); @@ -796,7 +833,7 @@ do { printf("Set %d of %d, preserve files [1 - %d, all]", curgroup, groups, counter); - if (ISFLAG(flags, F_SHOWSIZE)) printf(" (%lld byte%seach)", files->size, + if (ISFLAG(flags, F_SHOWSIZE)) printf(" (%lld byte%seach)", (long long int)files->size, (files->size != 1) ? "s " : " "); printf(": "); fflush(stdout); @@ -869,21 +906,26 @@ int sort_pairs_by_arrival(file_t *f1, file_t *f2) { if (f2->duplicates != 0) - return 1; + return !ISFLAG(flags, F_REVERSE) ? 1 : -1; - return -1; + return !ISFLAG(flags, F_REVERSE) ? -1 : 1; } int sort_pairs_by_mtime(file_t *f1, file_t *f2) { if (f1->mtime < f2->mtime) - return -1; + return !ISFLAG(flags, F_REVERSE) ? -1 : 1; else if (f1->mtime > f2->mtime) - return 1; + return !ISFLAG(flags, F_REVERSE) ? 1 : -1; return 0; } +int sort_pairs_by_filename(file_t *f1, file_t *f2) +{ + return strcmp(f1->d_name, f2->d_name); +} + void registerpair(file_t **matchlist, file_t *newmatch, int (*comparef)(file_t *f1, file_t *f2)) { @@ -930,6 +972,38 @@ } } +void deletesuccessor(file_t **existing, file_t *duplicate, + int (*comparef)(file_t *f1, file_t *f2)) +{ + file_t *to_keep; + file_t *to_delete; + + if (comparef(duplicate, *existing) >= 0) + { + to_keep = *existing; + to_delete = duplicate; + } + else + { + to_keep = duplicate; + to_delete = *existing; + + *existing = duplicate; + } + + if (!ISFLAG(flags, F_HIDEPROGRESS)) fprintf(stderr, "\r%40s\r", " "); + + printf(" [+] %s\n", to_keep->d_name); + if (remove(to_delete->d_name) == 0) { + printf(" [-] %s\n", to_delete->d_name); + } else { + printf(" [!] %s ", to_delete->d_name); + printf("-- unable to delete file!\n"); + } + + printf("\n"); +} + void help_text() { printf("Usage: fdupes [options] DIRECTORY...\n\n"); @@ -937,12 +1011,14 @@ printf(" -r --recurse \tfor every directory given follow subdirectories\n"); printf(" \tencountered within\n"); printf(" -R --recurse: \tfor each directory given after this option follow\n"); - printf(" \tsubdirectories encountered within\n"); + printf(" \tsubdirectories encountered within (note the ':' at\n"); + printf(" \tthe end of the option, manpage for more details)\n"); printf(" -s --symlinks \tfollow symlinks\n"); printf(" -H --hardlinks \tnormally, when two or more files point to the same\n"); printf(" \tdisk area they are treated as non-duplicates; this\n"); printf(" \toption will change this behavior\n"); printf(" -n --noempty \texclude zero-length files from consideration\n"); + printf(" -A --nohidden \texclude hidden files from consideration\n"); printf(" -f --omitfirst \tomit the first file in each set of matches\n"); printf(" -1 --sameline \tlist each set of matches on a single line\n"); printf(" -S --size \tshow size of duplicate files\n"); @@ -958,6 +1034,13 @@ printf(" -N --noprompt \ttogether with --delete, preserve the first file in\n"); printf(" \teach set of duplicates and delete the rest without\n"); printf(" \tprompting the user\n"); + printf(" -I --immediate \tdelete duplicates as they are encountered, without\n"); + printf(" \tgrouping into sets; implies --noprompt\n"); + printf(" -p --permissions \tdon't consider files with different owner/group or\n"); + printf(" \tpermission bits as duplicates\n"); + printf(" -o --order=BY \tselect sort order for output, linking and deleting; by\n"); + printf(" \tmtime (BY='time'; default) or filename (BY='name')\n"); + printf(" -i --reverse \treverse order while sorting\n"); printf(" -v --version \tdisplay fdupes version\n"); printf(" -h --help \tdisplay this help message\n\n"); #ifdef OMIT_GETOPT_LONG @@ -978,6 +1061,7 @@ int progress = 0; char **oldargv; int firstrecurse; + ordertype_t ordertype = ORDER_TIME; #ifndef OMIT_GETOPT_LONG static struct option long_options[] = @@ -994,12 +1078,17 @@ { "hardlinks", 0, 0, 'H' }, { "relink", 0, 0, 'l' }, { "noempty", 0, 0, 'n' }, + { "nohidden", 0, 0, 'A' }, { "delete", 0, 0, 'd' }, { "version", 0, 0, 'v' }, { "help", 0, 0, 'h' }, { "noprompt", 0, 0, 'N' }, + { "immediate", 0, 0, 'I'}, { "summarize", 0, 0, 'm'}, { "summary", 0, 0, 'm' }, + { "permissions", 0, 0, 'p' }, + { "order", 1, 0, 'o' }, + { "reverse", 0, 0, 'i' }, { 0, 0, 0, 0 } }; #define GETOPT getopt_long @@ -1011,7 +1100,7 @@ oldargv = cloneargs(argc, argv); - while ((opt = GETOPT(argc, argv, "frRq1Ss::HlndvhNm" + while ((opt = GETOPT(argc, argv, "frRq1SsHlnAdvhNImpo:i" #ifndef OMIT_GETOPT_LONG , long_options, NULL #endif @@ -1044,6 +1133,9 @@ case 'n': SETFLAG(flags, F_EXCLUDEEMPTY); break; + case 'A': + SETFLAG(flags, F_EXCLUDEHIDDEN); + break; case 'd': SETFLAG(flags, F_DELETEFILES); break; @@ -1056,9 +1148,28 @@ case 'N': SETFLAG(flags, F_NOPROMPT); break; + case 'I': + SETFLAG(flags, F_IMMEDIATE); + break; case 'm': SETFLAG(flags, F_SUMMARIZEMATCHES); break; + case 'p': + SETFLAG(flags, F_PERMISSIONS); + break; + case 'o': + if (!strcasecmp("name", optarg)) { + ordertype = ORDER_NAME; + } else if (!strcasecmp("time", optarg)) { + ordertype = ORDER_TIME; + } else { + errormsg("invalid value for --order: '%s'\n", optarg); + exit(1); + } + break; + case 'i': + SETFLAG(flags, F_REVERSE); + break; default: fprintf(stderr, "Try `fdupes --help' for more information.\n"); @@ -1134,11 +1245,12 @@ } if (confirmmatch(file1, file2)) { - registerpair(match, curfile, sort_pairs_by_mtime); - - /*match->hasdupes = 1; - curfile->duplicates = match->duplicates; - match->duplicates = curfile;*/ + if (ISFLAG(flags, F_DELETEFILES) && ISFLAG(flags, F_IMMEDIATE)) + deletesuccessor(match, curfile, + (ordertype == ORDER_TIME) ? sort_pairs_by_mtime : sort_pairs_by_filename ); + else + registerpair(match, curfile, + (ordertype == ORDER_TIME) ? sort_pairs_by_mtime : sort_pairs_by_filename ); } fclose(file1); @@ -1164,7 +1276,12 @@ } else { - stdin = freopen("/dev/tty", "r", stdin); + if (freopen("/dev/tty", "r", stdin) == 0) + { + errormsg("could not open terminal for input\n"); + exit(1); + } + deletefiles(files, 1, stdin); } } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/fdupes-fdupes-1.51/md5/md5.c new/fdupes-1.6.1/md5/md5.c --- old/fdupes-fdupes-1.51/md5/md5.c 2013-04-20 20:21:46.000000000 +0200 +++ new/fdupes-1.6.1/md5/md5.c 2016-08-21 06:54:46.000000000 +0200 @@ -35,7 +35,9 @@ contributors chl - Charles Longeau <[email protected]> + tom - Tom Hoover <[email protected]> + 2015-02-04 tom Added #ifdef __APPLE__ for proper compilation under OSX. 2002-05-31 chl Relocated string.h to avoid memcpy warning. 1999-11-04 lpd Edited comments slightly for automatic TOC extraction. 1999-10-18 lpd Fixed typo in header comment (ansi2knr rather than md5). @@ -45,6 +47,19 @@ #include "md5.h" #include <string.h> +/* endianness check using glibc endian.h */ +#ifdef __APPLE__ +# include <machine/endian.h> +#else +# include <endian.h> +#endif + +#if __BYTE_ORDER == __BIG_ENDIAN +# define ARCH_IS_BIG_ENDIAN 1 +#elif __BYTE_ORDER == __LITTLE_ENDIAN +# define ARCH_IS_BIG_ENDIAN 0 +#endif + #ifdef TEST /* * Compile with -DTEST to create a self-contained executable test program. ++++++ fdupes-makefile.patch ++++++ --- /var/tmp/diff_new_pack.wIyG34/_old 2016-11-07 12:21:20.000000000 +0100 +++ /var/tmp/diff_new_pack.wIyG34/_new 2016-11-07 12:21:20.000000000 +0100 @@ -1,7 +1,7 @@ Index: Makefile =================================================================== ---- Makefile.orig -+++ Makefile +--- Makefile.orig 2016-08-21 06:54:46.000000000 +0200 ++++ Makefile 2016-11-04 13:46:21.037423459 +0100 @@ -11,7 +11,7 @@ # determination of the actual installation directories. # Suggested values are "/usr/local", "/usr", "/pkgs/fdupes-$(VERSION)" @@ -11,12 +11,3 @@ # # When compiling for 32-bit systems, FILEOFFSET_64BIT must be enabled -@@ -56,7 +56,7 @@ BIN_DIR = $(PREFIX)/bin - # MAN_DIR indicates directory where the fdupes man page is to be - # installed. Suggested value is "$(PREFIX)/man/man1" - # --MAN_BASE_DIR = $(PREFIX)/man -+MAN_BASE_DIR = $(PREFIX)/share/man - MAN_DIR = $(MAN_BASE_DIR)/man1 - MAN_EXT = 1 -
