On 10/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > I've started testing the detect-renamed patch with 2.6.9 and soon 3.0.0pre1. > I have an unique situation where I'm rsync'ing to a HSM based filesystem. > I've found that the detect-renamed patch works but it appears to do a copy of > the file to the new destination. This is particular slow since the file in > the HSM based filesystem may only be a stub and all the data is only resident > on tape. The copy waits for the datq to be recalled from tape which > depending on the file size can take a long time. I've looked through the > patch code and am wondering if there is an easy way to have rsync do a move > from the ~.tmp. directory.
This is easy to do, and I have implemented a --trust-detect-renamed option to do it in the attached patch to the current CVS rsync. However, it is risky because a false rename detection could cause rsync to substitute an unrelated but similar-looking destination file for a new source file. Don't use the option unless you are prepared for the consequences. Matt
In combination with detect-renamed.diff, this patch adds an option --trust-detect-renamed that adopts an apparently unrenamed destination file without verifying that its data matches that of the source file. This is risky but it is what Greg <[EMAIL PROTECTED]> wanted: http://lists.samba.org/archive/rsync/2007-October/018827.html This patch is EXPERIMENTAL, though it did work correctly in my single test. -- Matt McCutchen <[EMAIL PROTECTED]> --- old/generator.c +++ new/generator.c @@ -80,6 +80,7 @@ extern int compare_dest; extern int copy_dest; extern int link_dest; extern int detect_renamed; +extern int trust_detect_renamed; extern int whole_file; extern int list_only; extern int read_batch; @@ -1828,6 +1829,22 @@ static void recv_generator(char *fname, fnamecmp = partialptr; fnamecmp_type = FNAMECMP_PARTIAL_DIR; statret = 0; + if (detect_renamed && trust_detect_renamed + && unchanged_file(fnamecmp, file, &sx.st)) { + /* Adopt the partial file. */ + finish_transfer(fname, fnamecmp, NULL, NULL, file, 1, 1); + handle_partial_dir(partialptr, PDIR_DELETE); + if (itemizing) + itemize(fnamecmp, file, ndx, -1, &sx, + ITEM_LOCAL_CHANGE, fnamecmp_type, NULL); +#ifdef SUPPORT_HARD_LINKS + if (preserve_hard_links && F_IS_HLINKED(file)) + finish_hard_link(file, fname, ndx, &sx.st, itemizing, code, -1); +#endif + if (remove_source_files == 1) + goto return_with_success; + goto cleanup; + } } if (!do_xfers) --- old/options.c +++ new/options.c @@ -81,6 +81,7 @@ int am_starting_up = 1; int relative_paths = -1; int implied_dirs = 1; int detect_renamed = 0; +int trust_detect_renamed = 0; int numeric_ids = 0; int allow_8bit_chars = 0; int force_delete = 0; @@ -385,6 +386,7 @@ void usage(enum logcode F) rprintf(F," -T, --temp-dir=DIR create temporary files in directory DIR\n"); rprintf(F," -y, --fuzzy find similar file for basis if no dest file\n"); rprintf(F," --detect-renamed try to find renamed files to speed up the transfer\n"); + rprintf(F," --trust-detect-renamed ... and assume identical to source files (risky!)\n"); rprintf(F," --compare-dest=DIR also compare destination files relative to DIR\n"); rprintf(F," --copy-dest=DIR ... and include copies of unchanged files\n"); rprintf(F," --link-dest=DIR hardlink to files in DIR when unchanged\n"); @@ -564,6 +566,7 @@ static struct poptOption long_options[] {"copy-dest", 0, POPT_ARG_STRING, 0, OPT_COPY_DEST, 0, 0 }, {"link-dest", 0, POPT_ARG_STRING, 0, OPT_LINK_DEST, 0, 0 }, {"detect-renamed", 0, POPT_ARG_NONE, &detect_renamed, 0, 0, 0 }, + {"trust-detect-renamed",0,POPT_ARG_NONE, &trust_detect_renamed, 0, 0, 0 }, {"fuzzy", 'y', POPT_ARG_NONE, &fuzzy_basis, 0, 0, 0 }, {"compress", 'z', POPT_ARG_NONE, 0, 'z', 0, 0 }, {"no-compress", 0, POPT_ARG_VAL, &do_compression, 0, 0, 0 }, @@ -1895,8 +1898,12 @@ void server_options(char **args, int *ar } } /* Both sides need to know in case this disables incremental recursion. */ - if (detect_renamed) + if (detect_renamed) { args[ac++] = "--detect-renamed"; + /* But the addition of --trust-detect-renamed is only the receiver's business. */ + if (am_sender && trust_detect_renamed) + args[ac++] = "--trust-detect-renamed"; + } if (modify_window_set) { if (asprintf(&arg, "--modify-window=%d", modify_window) < 0) --- old/rsync.yo +++ new/rsync.yo @@ -385,6 +385,7 @@ to the detailed description below for a -T, --temp-dir=DIR create temporary files in directory DIR -y, --fuzzy find similar file for basis if no dest file --detect-renamed try to find renamed files to speed the xfer + --trust-detect-renamed .. & assume identical to src files (risky!) --compare-dest=DIR also compare received files relative to DIR --copy-dest=DIR ... and include copies of unchanged files --link-dest=DIR hardlink to files in DIR when unchanged @@ -1440,6 +1441,14 @@ the bf(--partial-dir) option, that direc potential alternate-basis files will be removed as the transfer progresses. This option conflicts with bf(--inplace) and bf(--append). +dit(bf(--trust-detect-renamed)) In combination with bf(--detect-renamed), this +option makes rsync hard-link em(dest/D) to em(dest/S) without verifying that +em(src/S) and em(dest/S) have the same data. This poses a significant risk +of corrupting the destination by representing a new source file by an +unrelated destination file that coincidentally passes the quick check with the +source file. Use this option only if you accept the risk and disk I/O is a +bottleneck. + dit(bf(--compare-dest=DIR)) This option instructs rsync to use em(DIR) on the destination machine as an additional hierarchy to compare destination files against doing transfers (if the files are missing in the destination
-- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
