So, the problem seems to be that backuppc treats mtime as an unsigned
integer -- meaning that any file created before Jan 1, 1970 is not
representable.
v3 represents treats the number as an unsigned integer and basically
shows it's 2's complement (in 32 bits) - i.e. it's packed as an
unsigned 32 bit integer ('N' type in Perl)
v4 assumes a negative number is an error, sets it to zero, then
defeating the purpose displays it as the 2's complement of 0! v4
stores the number as a variable length unsigned integer.
According to Wikipedia,
The Unix time_t data type that represents a point in time is, on many
platforms, a signed integer, traditionally of 32 bits (but see below),
directly encoding the Unix time number as described in the preceding
section. Being 32 bits means that it covers a range of about 136 years
in total. The minimum representable date is Friday 1901-12-13, and the
maximum representable date is Tuesday 2038-01-19. One second after
03:14:07 UTC 2038-01-19 this representation will overflow. This
milestone is anticipated with a mixture of amusement and dread—see
year 2038 problem.
In some newer operating systems, time_t has been widened to 64
bits. This expands the times representable by approximately 293
billion years in both directions, which is over twenty times the
present age of the universe per direction.
There was originally some controversy over whether the Unix time_t
should be signed or unsigned. If unsigned, its range in the future
would be doubled, postponing the 32-bit overflow (by 68
years). However, it would then be incapable of representing times
prior to the epoch. The consensus is for time_t to be signed, and this
is the usual practice.
Seems to me that would be best to continue to represent time as a 32 bit
unsigned
integer in v3 attrib files, represented using 2's complements, so that
any mtime > 2^31 (about 68 years) would be considered as a negative
time -- this seems to be the current implied convention.
In v4, time would be written as 64 bit unsigned integer using
SetVarInt where any mtime > 2^63 (about 292 billion years from now)
would be considered a negative number written in 2's complements
Specifically, rather than testing for the sign of 'value' in
setVarInt, it might be better just to cast value to an unsigned 64-bit
integer that is then variable-length encoded as is done currently.
I am rusty on my 'c' but maybe just:
static void setVarInt(uchar **bufPP, uchar *bufEnd, int64 value)
{
+ unsigned int64 uvalue = value;
uchar *bufP = *bufPP;
- if ( value < 0 ) {
- bpc_logErrf("setVarInt botch: got negative argument %ld; setting to
0\n", (long int)value);
- value = 0;
- }
do {
- uchar c = value & 0x7f;
+ uchar c = uvalue & 0x7f;
value >>= 7;
+ uvalue >>= 7;
- if ( value ) c |= 0x80;
+ if ( uvalue ) c |= 0x80;
if ( bufP < bufEnd ) {
*bufP++ = c;
} else {
bufP++;
}
- } while ( value );
+ } while ( uvalue );
*bufPP = bufP;
}
Both v3 and V4 versions of BackupPC_attribPrint then should
additionally be "fixed" to print mtime as a signed integer converting
from a 32 or 64 bit unsigned integer depending on whether the attrib
file was created with v3 or v4.
backu...@kosowsky.org wrote at about 21:56:08 -0400 on Monday, April 8, 2019:
> I should have said that the v3 case is 2's complement for 32 bit
> integers while the v4 case is 2's complement for 64 bit integers.
>
> Also, the internal representation is clearly right in that both v3 and
> v4 are retrieving the right number, just displaying it incorrectly.
>
> - v3 is showing it as an unsigned rather than signed integer.
> - v4 is realizing it is an unsigned integer but rejecting it as negative
> and then erroneously setting it to zero (and topping on the cake
> recording it as the 64 bit 2's complement of zero)
>
> backu...@kosowsky.org wrote at about 21:51:27 -0400 on Monday, April 8, 2019:
> > OK, figured it out... VERY INTERESTING.
> >
> > The original file has an mtime of -2082826800 which corresponds to 5AM
> > Friday, Jan 1, 1904.
> > The the error message in question is actually *right* and not an
> > error:
> > setVarInt botch: got negative argument -2082826800; setting to 0
> >
> > i.e. it truly is a negative mtime.
> >
> > The reason for this odd time is that the file was created on a Mac128
> > (circa 1985) and apparently, on old Macs (pre-PowerPC), time starts with
> midnight Jan
> > 1, 1904 and runs until 2040. Now 5AM GMT is actually 12AM EST (my home
> > timezone), so the file shows a 'default' old Mac start date of
> > midnight Jan 1, 1904.
> >
> > Now for the interesting part.
> > Both the v3 and v4 versions of BackupPC_attribPrint are actually wrong! 2
> (different) bugs!!!!
> >
> > 1. v3 displays a positive mtime of 2212140496 which is is right only
> > in the sense that it is the decimal equivalent of the 2's
> > complement representation of -2082826800
> >
> > 2. v4 properly catches that the mtime is negative, but signals an
> > error rather than writing it. it then proceeds however to record a
> > wrong (and very large) mtime of 18446744071626724816, rather than
> > 0. The number however is 2^64 which is the 2's complement of 0 for
> > 8 byte integers
> >
> >
> > Fascinating. I learned a lot from nailing this one down.
> > Now both the v3 and v4 bugs should presumably be corrected :)
> >
> > "" wrote at about 20:23:18 -0400 on Monday, April 8, 2019:
> > > "" wrote at about 20:13:33 -0400 on Monday, April 8, 2019:
> > >
> > > Once again seems to be a problem with bpc_attrib.c:
> > > BackupPC_attribPrint
> > >
> > > Using v3:
> > > 'MacWrite 5.0Scrap' => {
> > > 'uid' => 501,
> > > 'mode' => 33188,
> > > 'sizeDiv4GB' => 0,
> > > 'size' => 0,
> > > 'type' => 0,
> > > 'gid' => 501,
> > > 'sizeMod4GB' => 0,
> > > 'mtime' => 2212140496
> > > },
> > >
> > > Using v4:
> > >
> > > 'MacWrite 5.0Scrap' => {
> > > 'compress' => 3,
> > > 'digest' => '',
> > > 'gid' => 501,
> > > 'inode' => 0,
> > > 'mode' => 33188,
> > > 'mtime' => '18446744071626724816',
> > > 'name' => 'MacWrite 5.0Scrap',
> > > 'nlinks' => 0,
> > > 'size' => 0,
> > > 'type' => 0,
> > > 'uid' => 501
> > > },
> > >
> > > Clearly, v4 is messing up on reading 'mtime' which presumably is what
> > > is causing the problem, consistent with the verbose output of
> > > BackupPC_migrateV3toV4 (as more fully excertped below)
> > >
> > > > setVarInt botch: got negative argument -2082826800; setting to 0
> > > > Wrote file MacWrite 5.0Scrap: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 0, inode = 11616435, nlinks = 0, digest = 0xd41d8c...,
> bufUsed = 925
> > >
> > >
> > >
> > > >
> > > > Craig Barratt wrote at about 23:39:50 -0700 on Sunday, April 7,
> 2019:
> > > > > Jeff,
> > > > >
> > > > > Please send me an attrib file that triggers that error.
> > > > >
> > > > > Craig
> > > > >
> > > >
> > > > Here is the relevant verbose output from BackupPc_migrateV3toV4
> > > > starting with the attrib file:
> > > >
> > > > bpc_poolRefDeltaUpdate(5a3631521955e47e179ea20d8a84ac5a, 1), count
> now 1
> > > >
> bpc_attrib_dirWrite(/var/lib/backuppc/pc/consult/13.v4/./f%2f/fvar/fopt/fexecutor/fshare/fhome/fSystem
> Folder/attrib): dirPath =
> /var/lib/backuppc/pc/consult/13.v4/./f%2f/fvar/fopt/fexecutor/fshare/fhome/fSystem
> Folder, attribFileName = attrib, baseAttribFileName = attrib
> > > >
> bpc_path_create(/var/lib/backuppc/pc/consult/13.v4/./f%2f/fvar/fopt/fexecutor/fshare/fhome/fSystem
> Folder)
> > > > Wrote file Extensions: type = 5, mode = 040771, uid/gid = 0/1002,
> size = 4096, inode = 11616442, nlinks = 0, digest = 0x000000..., bufUsed = 4
> > > > Wrote file %mac.rsrc: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 6753, inode = 11616437, nlinks = 0, digest = 0xc874b4..., bufUsed = 37
> > > > Wrote file system.ard: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 0, inode = 11616455, nlinks = 0, digest = 0xd41d8c..., bufUsed = 85
> > > > Wrote file %MacWrite 5.0Scrap: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 198, inode = 11616432, nlinks = 0, digest = 0x85e52a...,
> bufUsed = 133
> > > > Wrote file Excel Toolbars: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 267, inode = 11616447, nlinks = 0, digest = 0x619077...,
> bufUsed = 191
> > > > Wrote file %ParamRAM: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 198, inode = 11616457, nlinks = 0, digest = 0x5a3631..., bufUsed = 245
> > > > Wrote file Printer: type = 0, mode = 0100640, uid/gid = 0/1002,
> size = 0, inode = 11616441, nlinks = 0, digest = 0xd41d8c..., bufUsed = 293
> > > > Wrote file %system.ard: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 798, inode = 11616445, nlinks = 0, digest = 0xdcd6d0..., bufUsed = 338
> > > > Wrote file %System: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 487415, inode = 11616434, nlinks = 0, digest = 0xa9c2a3..., bufUsed =
> 388
> > > > Wrote file System: type = 0, mode = 0100660, uid/gid = 0/1002, size
> = 0, inode = 11616433, nlinks = 0, digest = 0xd41d8c..., bufUsed = 435
> > > > Wrote file Browser: type = 0, mode = 0100640, uid/gid = 0/1002,
> size = 0, inode = 11616450, nlinks = 0, digest = 0xd41d8c..., bufUsed = 479
> > > > Wrote file ParamRAM: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 20, inode = 11616443, nlinks = 0, digest = 0xcef2a5..., bufUsed = 524
> > > > Wrote file %Printer: type = 0, mode = 0100640, uid/gid = 0/1002,
> size = 2279, inode = 11616438, nlinks = 0, digest = 0x07c53e..., bufUsed =
> 570
> > > > Wrote file %Browser: type = 0, mode = 0100640, uid/gid = 0/1002,
> size = 119268, inode = 11616448, nlinks = 0, digest = 0x9fb0e6..., bufUsed =
> 617
> > > > Wrote file %windows.rsrc: type = 0, mode = 0100660, uid/gid =
> 0/1002, size = 15917, inode = 11616451, nlinks = 0, digest = 0xd30945...,
> bufUsed = 665
> > > > Wrote file %Excel Toolbars: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 198, inode = 11616439, nlinks = 0, digest = 0x7c1511...,
> bufUsed = 717
> > > > Wrote file windows.rsrc: type = 0, mode = 0100660, uid/gid =
> 0/1002, size = 0, inode = 11616453, nlinks = 0, digest = 0xd41d8c...,
> bufUsed = 772
> > > > Wrote file mac.rsrc: type = 0, mode = 0100660, uid/gid = 0/1002,
> size = 0, inode = 11616440, nlinks = 0, digest = 0xd41d8c..., bufUsed = 822
> > > > Wrote file Excel Settings (4): type = 0, mode = 0100644, uid/gid =
> 501/501, size = 0, inode = 11616436, nlinks = 0, digest = 0xd41d8c...,
> bufUsed = 868
> > > > setVarInt botch: got negative argument -2082826800; setting to 0
> > > > Wrote file MacWrite 5.0Scrap: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 0, inode = 11616435, nlinks = 0, digest = 0xd41d8c...,
> bufUsed = 925
> > > > Wrote file %godata.sav: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 198, inode = 11616452, nlinks = 0, digest = 0x39c291...,
> bufUsed = 977
> > > > Wrote file %Preferences: type = 0, mode = 0100660, uid/gid =
> 0/1002, size = 198, inode = 11616454, nlinks = 0, digest = 0x087760...,
> bufUsed = 1028
> > > > Wrote file godata.sav: type = 0, mode = 0100644, uid/gid = 501/501,
> size = 1100, inode = 11616456, nlinks = 0, digest = 0xb247c3..., bufUsed =
> 1079
> > > > Wrote file MacWrite 5.0Undo: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 32, inode = 11616446, nlinks = 0, digest = 0x70bc8f...,
> bufUsed = 1129
> > > > Wrote file %MacWrite 5.0Undo: type = 0, mode = 0100644, uid/gid =
> 501/501, size = 198, inode = 11616449, nlinks = 0, digest = 0xb5c763...,
> bufUsed = 1184
> > > > Wrote file Preferences: type = 5, mode = 040771, uid/gid = 0/1002,
> size = 4096, inode = 11616431, nlinks = 0, digest = 0x000000..., bufUsed =
> 1241
> > > >
> > > >
> > > > I am also attaching the relevant attrib file
> > > > [DELETED ATTACHMENT attrib, Untyped binary data]
> >
> >
> > _______________________________________________
> > BackupPC-users mailing list
> > BackupPC-users@lists.sourceforge.net
> > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> > Wiki: http://backuppc.wiki.sourceforge.net
> > Project: http://backuppc.sourceforge.net/
>
>
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
