Hello community, here is the log from the commit of package convmv for openSUSE:Factory checked in at 2015-12-17 15:54:51 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/convmv (Old) and /work/SRC/openSUSE:Factory/.convmv.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "convmv" Changes: -------- --- /work/SRC/openSUSE:Factory/convmv/convmv.changes 2014-10-07 16:01:28.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.convmv.new/convmv.changes 2015-12-17 15:54:54.000000000 +0100 @@ -1,0 +2,14 @@ +Wed Dec 16 13:47:38 UTC 2015 - [email protected] + +- updated to 2.0: + * fix checks for NFD conversion, where convmv could run into a + "resulting filename is ... bytes long (max: 255) error message + for no obvious reason. + * the --preserve-mtimes option is the default now + * fix a bug where mtimes might not be restored in some cases + * add --map option to support additional character mappings like + to mapping Microsoft's illegal NTFS characters + * issue warning if we cannot traverse a directory in recursive + mode + +------------------------------------------------------------------- Old: ---- convmv-1.15.tar.gz New: ---- convmv-2.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ convmv.spec ++++++ --- /var/tmp/diff_new_pack.FNtN78/_old 2015-12-17 15:54:55.000000000 +0100 +++ /var/tmp/diff_new_pack.FNtN78/_new 2015-12-17 15:54:55.000000000 +0100 @@ -1,7 +1,7 @@ # # spec file for package convmv # -# Copyright (c) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany. +# Copyright (c) 2015 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -17,7 +17,7 @@ Name: convmv -Version: 1.15 +Version: 2.0 Release: 0 Url: http://j3e.de/linux/convmv/ Summary: Converts File Names from One Encoding to Another ++++++ convmv-1.15.tar.gz -> convmv-2.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/convmv-1.15/CREDITS new/convmv-2.0/CREDITS --- old/convmv-1.15/CREDITS 2011-07-18 03:11:57.000000000 +0200 +++ new/convmv-2.0/CREDITS 2015-03-16 01:51:35.000000000 +0100 @@ -7,3 +7,4 @@ Kuang-che Wu Hansjörg MAURER Gabor PAPP +Bruno HAIBLE diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/convmv-1.15/Changes new/convmv-2.0/Changes --- old/convmv-1.15/Changes 2011-08-20 23:35:09.000000000 +0200 +++ new/convmv-2.0/Changes 2015-04-24 23:56:06.000000000 +0200 @@ -1,3 +1,10 @@ +2.0 +- fix checks for NFD conversion, where convmv could run into a "resulting filename is ... bytes long (max: 255) error message for no obvious reason. +- the --preserve-mtimes option is the default now +- fix a bug where mtimes might not be restored in some cases +- add --map option to support additional character mappings like to mapping Microsoft's illegal NTFS characters +- issue warning if we cannot traverse a directory in recursive mode + 1.15 - add --dump-options to make it easier to allow people to write a bash completion file. Let me know when you have one ready :-) - fix a bug where on case-insensitive filesystems --upper and --lower did not work because we thought the new file would already exist (stat "foo" and stat "FOO" are the same, yeah. The obvious workaround to use "--replace" would delete the files on case-insensitive filesystems instead, which was even worse. Case conversion should be working well now, I tested it on Darwin and on case-insensitive XFS. The latter one is a bit buggy by the way, see https://bugzilla.kernel.org/show_bug.cgi?id=39512. convmv works around that bug, too. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/convmv-1.15/SHA256sums new/convmv-2.0/SHA256sums --- old/convmv-1.15/SHA256sums 2011-08-20 23:37:56.000000000 +0200 +++ new/convmv-2.0/SHA256sums 2015-05-16 01:36:51.000000000 +0200 @@ -1,18 +1,18 @@ -----BEGIN PGP SIGNED MESSAGE----- -Hash: RIPEMD160 +Hash: SHA1 +58ecf5c6fd30a4dbc6398cdf8b0dc503dcf879a47a51eccd0d8b86f83d5672ac ./Changes +42b2cfe802e3bac5bdc2462a0d61de002991125053e41c18d5cd7799b4cebf2b ./CREDITS a2a9cfa48ba7f7453edf1c43bb8bb1aef5fcd1b87dde11420edceee2e2528db0 ./GPL2 10f96f4cf2f61333c289f603cedd4c0f5e523d2da05486d9bea670f3877d8592 ./Makefile 31a04f0b0584237d6f81fab7e77b0d43e307230102f09db947e20b2a3103a108 ./testsuite.tar 6cd57b452e9e2be5a9ca992831f96e4a8317c1bc37171f237094d8dc0a6e0d27 ./TODO -2a7832af2969e717ea4cbf8ea1a04161903e301b393bbdfa86359cdb0161dabe ./VERSION -a9d530edd9481a14a9b36c2d619f967ac4d6fbc8df97cc9a7c4f0c37a4f0dcc3 ./CREDITS -c98797a886244bb703337d460d30902f4085534d0d1ca91f656d106cf0d060e4 ./Changes -9f1c27d3b0c3d2449d57e3053b9eb5a48366ce467ace7c26d1c9fc85ba8d928d ./convmv +d526eb4e878a23ef26ae190031b4efd2d58ed66789ac049ea3dbaf74c9df7402 ./VERSION +10260daf5535038858393f47f0b700c0e7164f02092eb8131eada2d62e97cd3b ./convmv -----BEGIN PGP SIGNATURE----- -Version: GnuPG v2.0.18 (GNU/Linux) +Version: GnuPG v2 -iEYEAREDAAYFAk5QKTQACgkQdoo0s+hIejmflQCeKACFVw8WOI2uxV9M1Ew2hVUP -jGUAoOsIyNUA7VjPd8klIOZRHbLP2slm -=Gb6H +iEYEARECAAYFAlVWgxEACgkQdoo0s+hIejnq4gCeOExEQwBJ1fsTDykLtt3atHC4 +OYMAoPTvQqEUNSM63RGlL/niYZuSN3kF +=srw5 -----END PGP SIGNATURE----- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/convmv-1.15/VERSION new/convmv-2.0/VERSION --- old/convmv-1.15/VERSION 2011-08-20 23:37:50.000000000 +0200 +++ new/convmv-2.0/VERSION 2015-05-16 01:36:49.000000000 +0200 @@ -1 +1 @@ -1.15 +2.0 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/convmv-1.15/convmv new/convmv-2.0/convmv --- old/convmv-1.15/convmv 2011-08-20 23:22:34.000000000 +0200 +++ new/convmv-2.0/convmv 2015-05-16 01:36:37.000000000 +0200 @@ -1,6 +1,6 @@ #!/usr/bin/perl -# convmv 1.15 - converts filenames from one encoding to another -# Copyright © 2003-2011 Bjoern JACKE <[email protected]> +# convmv 2.0 - converts filenames from one encoding to another +# Copyright © 2003-2015 Bjoern JACKE <[email protected]> # # This program comes with ABSOLUTELY NO WARRANTY; it may be copied or modified # under the terms of the GNU General Public License version 2 or 3 as @@ -106,12 +106,13 @@ column is separated by \0\n (nullbyte newline). Each row (one action) is separated by \0\0\n (nullbyte nullbyte newline). -=item B<--preserve-mtimes> +=item B<--no-preserve-mtimes> modifying filenames usually causes the parent directory's mtime being updated. -This option allows to reset the mtime to the old value. If your filesystem -supports sub-second resolution the sub-second part of the atime and mtime -will be lost as Perl does not yet support that. +Since version 2 convmv by default resets the mtime to the old value. If your +filesystem supports sub-second resolution the sub-second part of the atime and +mtime will be lost as Perl does not yet support that. With this option you can +B<disable> the preservation of the mtimes. =item B<--replace> @@ -130,6 +131,20 @@ turn filenames into all upper or all lower case. When the file is not ASCII-encoded, convmv expects a charset to be entered via the -f switch. +=item B<--map=>some-extra-mapping + +apply some custom character mappings, currently supported are: + +ntfs-sfm(-undo), ntfs-sfu(-undo) for the mapping of illegal ntfs characters for +Linux or Macintosh cifs clients (see MS KB 117258 also mapchars mount option of +mount.cifs on Linux). + +ntfs-pretty(-undo) for for the mapping of illegal ntfs characters to pretty +legal Japanese versions of them. + +See the map_get_newname() function how to easily add own mappings if needed. +Let me know if you think convmv is missing some useful mapping here. + =item B<--dotlessi> care about the dotless i/I issue. A lowercase version of "I" will also be @@ -257,6 +272,10 @@ no bugs or fleas known +=head1 DONATE + +You can support convmv by doing a donation, see L<https://www.j3e.de/donate.html> + =head1 AUTHOR Bjoern JACKE @@ -280,6 +299,7 @@ binmode STDOUT, ":bytes"; binmode STDERR, ":bytes"; +my $opt_mtimes = 1; # default 1 since convmv 2.0 my %opts = ( 'nfc'=>\$opt_nfc, 'nfd'=>\$opt_nfd, @@ -302,13 +322,17 @@ 'dotlessi'=>\$opt_dotlessi, 'parsable'=>\$opt_parsable, 'fixdouble'=>\$opt_fixdouble, - 'preserve-mtimes'=>\$opt_mtimes, + 'preserve-mtimes!'=>\$opt_mtimes, 'dump-options'=>\$opt_dumpoptions, + 'undo-script=s'=>\$opt_undo_script, + 'map=s'=>\$opt_map, ); GetOptions %opts or exit 1; use File::Compare; $errors_occurred=0; $warnings_occurred=0; +$ops=0; +$mytime = time(); $maxfilenamelength=255; # $maxpathlength=4096; # this might be used somehow, somewhere? @@ -331,6 +355,10 @@ } } +if ($opt_replace and $opt_undo_script) { + die "--replace and --undo-script can't work together!\n"; +} + if ($opt_unescape) { die "No charset conversion when unescaping!\n" if ($opt_f or $opt_t); $checkenc=\&unescape_checkenc; @@ -341,6 +369,12 @@ $checkenc=\&upperlower_checkenc; $get_newname=\&upperlower_get_newname; $opt_f="ascii" unless ($opt_f); +} elsif ($opt_map) { + if ($opt_t or $opt_f or $opt_upper or $opt_lower or $opt_unescape) { + die "--map parameter not allowed with other character conversion parameters\n"; + } + $checkenc=\&dummy; + $get_newname=\&map_get_newname; } else { if (not ($opt_f and $opt_f=Encode::resolve_alias($opt_f))) { die "wrong/unknown \"from\" encoding!\n"; @@ -391,14 +425,23 @@ undef @ARGV; for (@args) { + s/\/\.\//\/\//g; # normalize "/./" to "/" + s/\/[\/]+/\//g; # normalize "//" to "/" die "file or directory not found: $_\n" unless (-e or -l); } - if ($opt_parsable) { $outerr=NUL; } else { $outerr=STDERR; } +if ($opt_undo_script) { + die "undo-script file already exists, exiting.\n" if (-e $opt_undo_script); + open(UNDOLOG, ">", $opt_undo_script) or die "couldn't open undo-script for writing. Aborting.\n"; + print UNDOLOG "# this is a per undo script generated by convmv.\n", + "# Please check if this looks reasonable before running!\n"; + print UNDOLOG "# Example: perl $opt_undo_script\n"; + print UNDOLOG "chdir $pwd;\n"; +} ## do {print ord($_)."_" for (split(//,$_));print "\n"; } for (@args); # debug print @@ -422,14 +465,15 @@ &$myfind({wanted=>\&process_main,bydepth=>1,no_chdir=>1}, @args); # check for unintentionally left files -for (keys %dir_time_hash) { - print $outerr "error: left in %dir_time_hash: $_\n"; -} +#for (keys %dir_time_hash) { +# print $outerr "error: left in %dir_time_hash: $_\n"; +#} +$mytime = time() - $mytime; if ($opt_notest) { - print $outerr "Ready!\n", + print $outerr "Ready! I converted $ops files in $mytime seconds.\n", } else { - print $outerr "No changes to your files done. Use --notest to finally rename the files.\n"; + print $outerr "No changes to your files done. Would have converted $ops files in $mytime seconds.\nUse --notest to finally rename the files.\n"; } ##### @@ -455,6 +499,10 @@ # print "dir: $arg in $dir\n"; $inod_fullname{(stat $arg)[1]}=$dir."/".$arg if (!$opt_lowmem); if (not defined(&$checkenc($arg))) { $errors_occurred=1 }; + if ($opt_r and not (-x $arg or -r $arg)) { + print $outerr "WARNING: cannot traverse ",&$from_print($dir."/".$arg),"\n"; + $warnings_occurred=1; + } } elsif (-f $arg) { # print "file: $arg in $dir\n"; $inod_fullname{(stat $arg)[1]}=$dir."/".$arg if (!$opt_lowmem); @@ -481,10 +529,13 @@ if ($opt_notest) { unlink $arg; symlink ($newname, $arg); + print UNDOLOG "unlink \"".$File::Find::name."\";\n"; + print UNDOLOG "symlink (\"$oldlink\", \"".$File::Find::name."\");\n"; } elsif ($opt_parsable) { print "unlink".$del.$File::Find::name.$fin_del; print "symlink".$del.$newname.$del.$File::Find::name.$fin_del; } + $ops++; } else { print $outerr "link target \"",&$from_print($oldlink),"\" of \"",&$from_print($dir."/".$arg),"\" not in subtree, left untouched!\n"; } @@ -510,6 +561,7 @@ if ($newname and $newname ne $arg) { &renameit($arg,$newname); } + &restore_times_if_any($dir,$arg,$newname) if ($opt_mtimes); } elsif (-f $arg) { # $type="file"; $newname=&$get_newname($arg); @@ -517,7 +569,7 @@ &renameit($arg,$newname); } } - &restore_times_if_any($dir,$arg,$newname) if ($opt_mtimes); + # &restore_times_if_any($dir,$arg,$newname) if ($opt_mtimes); # only in -d case needed !? -> moved up! chdir $pwd; } @@ -595,6 +647,8 @@ if (! -e $try) { #print $outerr "(via temp rename to \"$try\")\n"; rename ($oldfile, $try); + $opt++; + print UNDOLOG "rename (\"$dir/$try\", \"$dir/$oldfile\");\n"; $oldfile=$try; last; } @@ -624,17 +678,22 @@ system($cmd); } else { rename ($oldfile, $newname) or print $outerr "Error: $!\n"; + print UNDOLOG "rename (\"$dir/$newname\", \"$dir/$oldfile\");\n"; } } elsif ($opt_parsable) { print "rename".$del.$dir."/".$oldfile.$del.$dir."/".$newname.$fin_del; } } + $ops++; } sub save_parent_mtime() { my $dir=shift; + $dir =~ s/^\.\///; + $dir =~ s/\.\/$//; +# return if ($dir eq "."); # broken !? return if (exists $dir_time_hash{$dir}); - #print $outerr "Putting $dir in %dir_time_hash\n"; + #print $outerr "Putting \"$dir\" in %dir_time_hash\n"; # debug print @{$dir_time_hash{$dir}}=(stat("."))[8..10]; } @@ -648,14 +707,16 @@ $dir .= "/"; } $dir .= $old; - ## print $outerr "Trying to delete $dir now $new from %dir_time_hash\n"; # debug print + # print $outerr "Trying to delete \"$dir\" now \"$new\" from %dir_time_hash\n"; # debug print if (exists $dir_time_hash{$dir}) { if ($opt_notest) { utime ${$dir_time_hash{$dir}}[0], ${$dir_time_hash{$dir}}[1], $new or print $outerr "Could not run utime() on $new: $!\n"; + print UNDOLOG "utime ".${$dir_time_hash{$dir}}[0].", ".${$dir_time_hash{$dir}}[1].", ".$new." or print \"Could not run utime() on $new: \$!\n\""; } elsif ($opt_parsable) { print "utime".$del.$dir.$del.${$dir_time_hash{$dir}}[0].$del.${$dir_time_hash{$dir}}[1].$del.${$dir_time_hash{$dir}}[2].$fin_del; } delete $dir_time_hash{$dir}; + #print $outerr "done\n"; # debug print } } @@ -706,7 +767,7 @@ # $new is utf-8 now and $oldfile's encoding was valid ... my $filenamelength; if ($to_is_utf8) { - $new = &$norm($new); + $new = &$norm(decode_utf8($new)); $filenamelength=length($new); } else { $new=encode_utf8(NFC(decode_utf8($new))); @@ -731,8 +792,8 @@ sub printusage { &check_for_perl_bugs; print <<END; -convmv 1.15 - converts filenames from one encoding to another -Copyright (C) 2003-2011 Bjoern JACKE <bjoern\@j3e.de> +convmv 2.0 - converts filenames from one encoding to another +Copyright (C) 2003-2015 Bjoern JACKE <bjoern\@j3e.de> This program comes with ABSOLUTELY NO WARRANTY; it may be copied or modified under the terms of the GNU General Public License version 2 or 3 as published @@ -750,6 +811,7 @@ --exec c execute command instead of rename (use #1 and #2 and see man page) --list list all available encodings --lowmem keep memory footprint low (see man page) +--map m apply an additional character mapping --nosmart ignore if files already seem to be UTF-8 and convert if posible --notest actually do rename the files --replace will replace files if they are equal @@ -852,6 +914,54 @@ } } +sub map_get_newname() { + $_ = shift; + return $_ if ($_ eq "." or $_ eq ".."); + _utf8_on($_); # this is needed for tr/multibyte/non-multibyte/ to work! Otherwise we would + # have to make a s/// for each character, grrr... + if ($opt_map eq "ntfs-sfm") { # see MS KB 117258 + tr/\x01-\x1f\"\*\/\<\>\?\\\|/\x{f001}-\x{f027}/; + s/ $/\x{f028}/; # Space, only if occurring as the last character of the name + s/\.$/\x{f029}/; # period, only if occurring as the last character of the name + } elsif ($opt_map eq "ntfs-sfm-undo") { + tr/\x{f001}-\x{f027}/\x01-\x1f"*\/<>?\\| /; + s/\x{f028}$/ /; # Space, only if occurring as the last character of the name + s/\x{f029}$/./; # period, only if occurring as the last character of the name + } elsif ($opt_map eq "ntfs-sfu") { # +0xF000, see MS KB ???? anyone knows a link or has archived an old one? + tr/\x01-\x1f\"\*\/\<\>\?\\\|/\x{f001}-\x{f01f}\x{f022}\x{f02a}\x{f02f}\x{f03c}\x{f03e}\x{f03f}\x{f05c}\x{f07c}/; + #??? s/ $/space/; # Space, only if occurring as the last character of the name + #??? s/\.$/period/; # period, only if occurring as the last character of the name + } elsif ($opt_map eq "ntfs-sfu-undo") { + tr/\x{f001}-\x{f01f}\x{f022}\x{f02a}\x{f02f}\x{f03c}\x{f03e}\x{f03f}\x{f05c}\x{f07c}/\x01-\x1f"*\/<>?\\|/; + #??? s/space$/ /; # Space, only if occurring as the last character of the name + #??? s/period$/./; # period, only if occurring as the last character of the name + } elsif ($opt_map eq "ntfs-pretty") { + s/\"/”/g; # U+201D + s/\*/∗/g; # U+2731 + s/\?/？/g; # U+FF1F + s/\:/꞉/g; # U+A789 + s/\</＜/g; # U+FF1C + s/\>/＞/g; # U+FF1E + s/\|/❘/g; # U+2758 + s/\\/＼/g; # U+FF3C + } elsif ($opt_map eq "ntfs-pretty-undo") { + s/”/"/g; # U+201D + s/∗/*/g; # U+2731 + s/？/?/g; # U+FF1F + s/꞉/:/g; # U+A789 + s/＜/</g; # U+FF1C + s/＞/>/g; # U+FF1E + s/❘/|/g; # U+2758 + s/＼/\\/g; # U+FF3C + } else { + die "map parameter \"$opt_map\" not supported. Use one of ", + "ntfs-sfm, ntfs-sfm-undo, ", + "ntfs-sfu, ntfs-sfu-undo, ", + "ntfs-pretty, ntfs-pretty-undo\n"; + } + return $_; +} + sub unescape_get_newname() { # return undef on error, string otherwise my $newname = shift; # $newname =~ s/([^a-zA-Z0-9_.-])/uc sprintf("%%%02x",ord($1))/eg; # this was done before
