The problem with these specific files seems to have gone away after upgrading to 4.3.1 so indeed perhaps was not a Charset issue as you and I have both started to include, but rather was/is part of the same root cause(s) that I have highlighted in my other bug reports.
That being said, after upgrading I still have a mix of (other) errors of the types highlighted before -- will follow up more in a separate email Craig Barratt wrote at about 15:05:24 -0700 on Friday, July 5, 2019: > Jeff, > > I tried recreating this but wasn't able to get the error you reported. The > file name I created piped to "od -c" prints 'r 351 p o n s e . > d o c' (the 351 is octal, hex e9). > > When I set ClientCharset to utf8, I do indeed get your first error (cannot > convert filename). But when ClientCharset is empty, it backs up correctly > without an error message. I tried this with rsync_bpc 3.0.9.13 and > rsync_bpc 3.1.2.0 (although these have the the BPC_FTYPE_DELETED fix > <https://github.com/backuppc/rsync-bpc/commit/07b18a3469caefc72c01546515973dd1dc21deb4> > I committed last week). > > So maybe this indeed is another symptom of the other issue, as you first > suggested. > > Craig > > On Tue, Jun 25, 2019 at 8:07 AM <backu...@kosowsky.org> wrote: > > > Craig Barratt wrote at about 23:00:28 -0700 on Monday, June 24, 2019: > > > Jeff, > > > > > > I don't think it's the same issue, even though the final state looks > > > similar. In this case, since the file name isn't encoded correctly, I > > > suspect that when rsync goes back to actually open it, it does appear > > to be > > > missing since the file name byte string doesn't point to the file (ie, > > it > > > (incorrectly) looks like the file doesn't exist). > > > > Note that when I manually run 'rsync -av' (without iconv) on either the > > file itself or the > > directory containing it, the transfer occurs just fine. > > > > > Let me try to re-create this (should be easy) and I'll try some > > experiments > > > to see if there is a sensible behavior that is easy to implement. > > > > > > Craig > > > > > > On Mon, Jun 24, 2019 at 10:46 PM <backu...@kosowsky.org> wrote: > > > > > > > Hi Craig, > > > > > > > > I just tried a backup with: > > > > $Conf{ClientCharset} = ''; > > > > > > > > But now instead of the previous error: > > > > [sender] cannot convert filename: user1/docs/r\#351ponse.doc > > (Invalid > > > > or incomplete multibyte or wide character) > > > > I now get the (familiar?) error: > > > > rsync_bpc: fstat "user1/docs/r\#351ponse.doc" failed: No such > > file or > > > > directory (2) > > > > > > > > I suspect this is due to the same/similar bug I have been chasing > > down with > > > > you regarding the handling of deleted (or spuriously deleted) files. > > > > > > > > Indeed, attribPrint shows: > > > > $VAR1 = { > > > > 'r�ponse.doc' => { > > > > 'compress' => 3, > > > > 'digest' => '', > > > > 'gid' => 0, > > > > 'inode' => 0, > > > > 'mode' => 0, > > > > 'mtime' => 0, > > > > 'name' => 'r�ponse.doc', > > > > 'nlinks' => 0, > > > > 'size' => 0, > > > > 'type' => 10, > > > > 'uid' => 0 > > > > } } > > > > }; > > > > > > > > Craig Barratt wrote at about 21:43:38 -0700 on Monday, June 24, 2019: > > > > > Jeff, > > > > > > > > > > I suspect you'll get the same error if you use native rsync and > > tell it > > > > to > > > > > convert utf8 to utf8 (eg, add '--iconv=utf8,utf8' to your native > > rsync > > > > > command). > > > > > > > > > > Since you don't need any conversion, you should set > > > > $Conf{ClientCharset} to > > > > > an empty string. That should cause rsync to no longer attempt any > > > > charset > > > > > conversion. > > > > > > > > > > Craig > > > > > > > > > > On Sun, Jun 23, 2019 at 6:13 PM <backu...@kosowsky.org> wrote: > > > > > > > > > > > Craig Barratt via BackupPC-users wrote at about 13:59:46 -0700 on > > > > Sunday, > > > > > > June 23, 2019: > > > > > > > The error you are getting is when rsync_bpc is trying to use > > iconv > > > > to > > > > > > > convert the client file name encoding to utf8 (which is the > > native > > > > > > format > > > > > > > that BackupPC on the server). > > > > > > > > > > > > > > What is $Conf{ClientCharset} set to for this host? If it is > > empty > > > > then > > > > > > > rsync_bpc should skip any charset conversion, so I assume > > it's not > > > > > > > empty. > > > > > > I have it set to UTF-8 > > > > > > In my notes I have that this should be set to the output of the > > > > > > command 'locale charmap' which is UTF-8 on Ubuntu 18.04 -- but I > > > > > > can't remember where/why I determined that. > > > > > > > > > > > > > Next, it would appear that 'r'$'\351''ponse.doc' is not a > > valid > > > > encoded > > > > > > > name in that Charset. > > > > > > > > > > > > Well, when I type 'ls' on the server, it shows the file as: > > > > > > -rwx------ 1 user1 user1 28672 May 11 2005 > > 'r'$'\351''ponse.doc' > > > > > > > > > > > > Indeed when I run: > > > > > > echo 'r'$'\351''ponse.doc' | iconv -f UTF-8 -t UTF-8 > > > > > > I get: > > > > > > riconv: illegal input sequence at position 1 > > > > > > Not sure why this happens if I have UTF-8 as my 'locale charmap' > > > > > > > > > > > > Even so, shouldn't backuppc treat such an error more gracefully. > > > > > > For example, > > > > > > - If iconv gives an error, try without charset conversion (like > > > > > > BackupPC 3.x) > > > > > > > > > > > > At least this would be better than the current setup where lack > > of > > > > > > charset conversion causes the file to fail to be backed up at > > all - > > > > > > which in many ways is the worst thing possible... > > > > > > > > > > > > > > > > > > > > In BackupPC 3.x, the xfer methods didn't support any charset > > > > conversion > > > > > > > (you could, for example, have smbclient do it instead). > > > > > > > > > > > > > > Craig > > > > > > > > > > > > > > On Sat, Jun 15, 2019 at 8:56 PM <backu...@kosowsky.org> > > wrote: > > > > > > > > > > > > > > > Michael Stowe wrote at about 03:35:13 +0000 on Sunday, June > > 16, > > > > 2019: > > > > > > > > > On 2019-06-15 19:20, backu...@kosowsky.org wrote: > > > > > > > > > > I am running backuppc 4.3.0 on Linux Ubuntu 18.04 > > > > > > > > > > > > > > > > > > > > I have a file copied over from an old Windows > > installation > > > > to my > > > > > > Linux > > > > > > > > > > server with name: > > > > > > > > > > > > > > > > > > > > -rwx------ 1 user1 user1 28672 May 11 2005 > > > > > > 'r'$'\351''ponse.doc' > > > > > > > > > > > > > > > > > > > > This file gives an error under backuppc as follows: > > > > > > > > > > [sender] cannot convert filename: > > > > > > > > > > user1/docs/r\#351ponse.doc (Invalid or incomplete > > > > multibyte > > > > > > or > > > > > > > > > > wide character) > > > > > > > > > > > > > > > > > > > > Note that rsync itself has no trouble copying this file > > > > when I > > > > > > run > > > > > > > > > > rsync manually. > > > > > > > > > > > > > > > > > > > > Also, interestingly, this same file backed up fine on > > my old > > > > > > Fedora > > > > > > > > 12 > > > > > > > > > > server > > > > > > > > > > running backuppc 3.x. > > > > > > > > > > > > > > > > > > > > Any ideas why backuppc is having trouble with this > > file? > > > > > > > > > > > > > > > > > > Almost definitely because the encoding is invalid > > relative to > > > > the > > > > > > > > > locale. The simplest thing to do is to rename the file > > so > > > > that > > > > > > it's a > > > > > > > > > valid filename. > > > > > > > > > > > > > > > > I could do that of course, but i would like to "fix" it if > > > > possible. > > > > > > > > If the native filesystem and 'rsync' are both able to deal > > with > > > > the > > > > > > > > file, it seems to me that backuppc should also be able to > > deal > > > > with > > > > > > > > it. > > > > > > > > > > > > > > > > Backup should be maximally tolerant so as to duplicate the > > files > > > > as > > > > > > > > permissively as possible... > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > BackupPC-users mailing list > > > > > > > > BackupPC-users@lists.sourceforge.net > > > > > > > > List: > > > > https://lists.sourceforge.net/lists/listinfo/backuppc-users > > > > > > > > Wiki: http://backuppc.wiki.sourceforge.net > > > > > > > > Project: http://backuppc.sourceforge.net/ > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > BackupPC-users mailing list > > > > > > > BackupPC-users@lists.sourceforge.net > > > > > > > List: > > > > https://lists.sourceforge.net/lists/listinfo/backuppc-users > > > > > > > Wiki: http://backuppc.wiki.sourceforge.net > > > > > > > Project: http://backuppc.sourceforge.net/ > > > > > > > > > > > > _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
