backu...@kosowsky.org wrote at about 23:12:32 -0400 on Saturday, April 6, 2019:
> backu...@kosowsky.org wrote at about 14:58:10 -0400 on Sunday, March 31,
> 2019:
> >
> > When I run 'BackupPC_migrateV3toV4' (v4.3.0), I get occasional errors
> > of form:
> > BackupPC_migrateV3toV4: can't read attribute file
> ./f%2fdata/fmedia/f0/fAndroid/fdata/fcom.ebay.redlaser/fcache/fhttp/attrib
> >
> > Plus an associated error:
> > bpc_attrib_dirRead: got unreasonable file name length 852354
> >
> > Similarly if I try to read the attrib file directly using
> BackupPC_attribPrint, I get:
> > BackupPC_attribPrint: cannot read attrib file
> <path-to-file>/f%2fdata/fmedia/f0/fAndroid/fdata/fcom.ebay.redlaser/fcache/fhttp/attrib
> >
> > HOWEVER, if I use the v3.1 version of BackupPC_attribPrint, the attrib
> file prints out just fine!!!
> > SO, the issue seems to be with how v4.x reads attrib files
> >
> > And none of the filenames are longer than 255 characters...
> > For example the longest is:
> >
> fcache_http%253A%252F%252Fwww.swap.com%252Fi%252FM%252FAMIfv95PqwhrAhpZvHMI9wTeAMmp74SepSA1FcgE-HwOBvARxlhxJ2MNyBqHSNkxbLWiKa50yAM6dEAP7CA_DpByvhW4MB4wp6KPcmAzKpcUSgAliHCnZ4OTQ8NbA3xOUfehqKnpdFXq-xHdSJ3aU09RXP-0pfnFVDH9YnkSDLTv8IeTU3tbAvE-1000%254080.jpg
> >
> > In other cases, I get the same error even though the longest filename is
> less than 100 chars. So that is not the issue...
> >
> > NOTE: that the error is rare -- I migrated about 2 million pool files
> repeated across 100 backups... yet I only encountered about a dozen
> "unreadable" attrib files
> > Also, the "got unreasonable file name length" error is always associated
> with the "can't read attribute file" error (and vice-versa)
> >
> > I tried to troubleshoot, but the problem seems to be in the
> > BackupPC::XS:Attrib read C-code...
> >
>
> OK... I dug into the C-code a bit...
>
> The problem occurs in bpc_attrib.c in the lines:
> } else if ( magic == BPC_ATTRIB_TYPE_UNIX ) {
> while ( bufP < buf + nRead ) {
> uint fileNameLen;
> char *fileName;
> bpc_attrib_file *file;
> int64 sizeDiv4GB;
> uint type;
>
> if ( nRead == sizeof(buf) && bufP > buf + nRead - 2 *
> BPC_MAXPATHLEN
> && read_more_data(&fd, buf, sizeof(buf), &nRead, &bufP,
> attribPath) ) {
> bpc_fileZIO_close(&fd);
> printf("JJK: magic 2\n");
> return -1;
> }
>
> fileNameLen = getVarInt(&bufP, buf + nRead);
> if ( fileNameLen > 2 * BPC_MAXPATHLEN - 16 ) {
> bpc_logErrf("bpc_attrib_dirRead: got unreasonable file name
> length %d\n", fileNameLen);
> bpc_fileZIO_close(&fd);
> return -1;
> }
>
>
> For example, fileNameLen comes back as 852354 which is obviously a
> ridiculous number.
>
> I also played with the attrib file itself and found the following:
> Specifically, when I break the attrib file into parts:
> 1. Every file is individually readable when broken into separate attrib
> files with just one entry
> 2. Only certain combinations of files reconstructed into an attrib file are
> not readable. For example, in one case, files 7-118 are not readable but
> files 6-117 or 8-118 are readable.
> 2. In v3.x is always readable.
>
> I suspect that Perl is storing longer hashes in a way that is not readable
> by the C-code in v4.x.
>
> Happy to share the above examples if anyone is willing to help me
> troubleshoot...
>
> Jeff
Here is a test attrib hash that is readable in 3.x but not 4.x
$VAR1 = {
'temp' => {
'uid' => 1023,
'mtime' => 1386359561,
'sizeDiv4GB' => 0,
'size' => 24090,
'sizeMod4GB' => 24090,
'mode' => 33204,
'gid' => 1023,
'type' => 0
},
'c.shld.net%2Frpx%2Fi%2Fs%2Fpi%2Fmp%2F3962%2F7417620111p%3Fsrc%3Dhttp%253A%252F%252Fwww.petraimages.com%252F500x500%252FBSH260224.jpg'
=> {
'mode' => 33204,
'gid' => 1023,
'type' => 0,
'mtime' => 1385712394,
'uid' => 1023,
'sizeDiv4GB' => 0,
'size' => 0,
'sizeMod4GB' => 3330
}
};
Perhaps the problem is with the non-ascii encoding in the second file
that causes the filename read to fail in the c-code in attrib.c??
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
