backu...@kosowsky.org wrote at about 23:48:28 -0400 on Saturday, April 6, 2019:
> backu...@kosowsky.org wrote at about 23:12:32 -0400 on Saturday, April 6,
> 2019:
> > backu...@kosowsky.org wrote at about 14:58:10 -0400 on Sunday, March 31,
> 2019:
> > >
> > > When I run 'BackupPC_migrateV3toV4' (v4.3.0), I get occasional errors
> > > of form:
> > > BackupPC_migrateV3toV4: can't read attribute file
> ./f%2fdata/fmedia/f0/fAndroid/fdata/fcom.ebay.redlaser/fcache/fhttp/attrib
> > >
> > > Plus an associated error:
> > > bpc_attrib_dirRead: got unreasonable file name length 852354
> > >
> > > Similarly if I try to read the attrib file directly using
> BackupPC_attribPrint, I get:
> > > BackupPC_attribPrint: cannot read attrib file
> <path-to-file>/f%2fdata/fmedia/f0/fAndroid/fdata/fcom.ebay.redlaser/fcache/fhttp/attrib
> > >
> > > HOWEVER, if I use the v3.1 version of BackupPC_attribPrint, the attrib
> file prints out just fine!!!
> > > SO, the issue seems to be with how v4.x reads attrib files
> > >
> > > And none of the filenames are longer than 255 characters...
> > > For example the longest is:
> > >
> fcache_http%253A%252F%252Fwww.swap.com%252Fi%252FM%252FAMIfv95PqwhrAhpZvHMI9wTeAMmp74SepSA1FcgE-HwOBvARxlhxJ2MNyBqHSNkxbLWiKa50yAM6dEAP7CA_DpByvhW4MB4wp6KPcmAzKpcUSgAliHCnZ4OTQ8NbA3xOUfehqKnpdFXq-xHdSJ3aU09RXP-0pfnFVDH9YnkSDLTv8IeTU3tbAvE-1000%254080.jpg
> > >
> > > In other cases, I get the same error even though the longest filename
> is less than 100 chars. So that is not the issue...
> > >
> > > NOTE: that the error is rare -- I migrated about 2 million pool files
> repeated across 100 backups... yet I only encountered about a dozen
> "unreadable" attrib files
> > > Also, the "got unreasonable file name length" error is always
> associated with the "can't read attribute file" error (and vice-versa)
> > >
> > > I tried to troubleshoot, but the problem seems to be in the
> > > BackupPC::XS:Attrib read C-code...
> > >
> >
> > OK... I dug into the C-code a bit...
> >
> > The problem occurs in bpc_attrib.c in the lines:
> > } else if ( magic == BPC_ATTRIB_TYPE_UNIX ) {
> > while ( bufP < buf + nRead ) {
> > uint fileNameLen;
> > char *fileName;
> > bpc_attrib_file *file;
> > int64 sizeDiv4GB;
> > uint type;
> >
> > if ( nRead == sizeof(buf) && bufP > buf + nRead - 2 *
> BPC_MAXPATHLEN
> > && read_more_data(&fd, buf, sizeof(buf), &nRead,
> &bufP, attribPath) ) {
> > bpc_fileZIO_close(&fd);
> > printf("JJK: magic 2\n");
> > return -1;
> > }
> >
> > fileNameLen = getVarInt(&bufP, buf + nRead);
> > if ( fileNameLen > 2 * BPC_MAXPATHLEN - 16 ) {
> > bpc_logErrf("bpc_attrib_dirRead: got unreasonable file
> name length %d\n", fileNameLen);
> > bpc_fileZIO_close(&fd);
> > return -1;
> > }
> >
> >
> > For example, fileNameLen comes back as 852354 which is obviously a
> ridiculous number.
> >
> > I also played with the attrib file itself and found the following:
> > Specifically, when I break the attrib file into parts:
> > 1. Every file is individually readable when broken into separate attrib
> files with just one entry
> > 2. Only certain combinations of files reconstructed into an attrib file
> are not readable. For example, in one case, files 7-118 are not readable but
> files 6-117 or 8-118 are readable.
> > 2. In v3.x is always readable.
> >
> > I suspect that Perl is storing longer hashes in a way that is not
> readable by the C-code in v4.x.
> >
> > Happy to share the above examples if anyone is willing to help me
> troubleshoot...
> >
> > Jeff
>
> Here is a test attrib hash that is readable in 3.x but not 4.x
>
> $VAR1 = {
> 'temp' => {
> 'uid' => 1023,
> 'mtime' => 1386359561,
> 'sizeDiv4GB' => 0,
> 'size' => 24090,
> 'sizeMod4GB' => 24090,
> 'mode' => 33204,
> 'gid' => 1023,
> 'type' => 0
> },
>
> 'c.shld.net%2Frpx%2Fi%2Fs%2Fpi%2Fmp%2F3962%2F7417620111p%3Fsrc%3Dhttp%253A%252F%252Fwww.petraimages.com%252F500x500%252FBSH260224.jpg'
> => {
> 'mode' => 33204,
> 'gid' => 1023,
> 'type' => 0,
> 'mtime' => 1385712394,
> 'uid' => 1023,
> 'sizeDiv4GB' => 0,
> 'size' => 0,
> 'sizeMod4GB' => 3330
> }
> };
>
>
> Perhaps the problem is with the non-ascii encoding in the second file
> that causes the filename read to fail in the c-code in attrib.c??
>
I believe the problem is that getVarInt is used instead of
getVarInt_v3 in bpc_attrib_dirRead in places where we are reading a v3
rather than v4 file.
Specifically:
} else if ( magic == BPC_ATTRIB_TYPE_UNIX ) {
while ( bufP < buf + nRead ) {
uint fileNameLen;
char *fileName;
bpc_attrib_file *file;
int64 sizeDiv4GB;
uint type;
if ( nRead == sizeof(buf) && bufP > buf + nRead - 2 * BPC_MAXPATHLEN
&& read_more_data(&fd, buf, sizeof(buf), &nRead, &bufP,
attribPath) ) {
bpc_fileZIO_close(&fd);
return -1;
}
fileNameLen = getVarInt(&bufP, buf + nRead);
if ( fileNameLen > 2 * BPC_MAXPATHLEN - 16 ) {
bpc_logErrf("bpc_attrib_dirRead: got unreasonable file name
length %d\n", fileNameLen);
bpc_fileZIO_close(&fd);
return -1;
}
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
