Bug#852318: lightning: Fake mtime in packaged files

[Peter Lebbing]

On 28/01/17 20:42, Carsten Schoenert wrote:
> then I don't understand your first email.

Do I need to clarify more or did my follow-up mail already do that?

> So I have currently no clue where the timestamps with *.0 are
> coming from.

Maybe I'm nitpicking, maybe I'm pointing out something relevant. I think
all files in a .deb binary package have a 0 fractional seconds part. I
think they are stored with one second resolution. But the problematic
files have a fake mtime of 2010-01-01 0:00 UTC that is the same over
different .deb binary packages.

> But this looks like something that could be come from the
> tools the reproducibly team is using and creating.

That does sound like a clue!

> Next we need to clarify with which package version such timestamps are
> shipped for the first time.

Done!

Using my backups, I could trivially deduce which stable package
introduced the change. In 38.8.0-1~deb8u1, the times are still recent.
In 1:45.1.0-1~deb8u1, the 2010-01-01 times show up.

Next using snapshot.debian.org[1], using a manual binary search through
the .deb files, I could pinpoint that

41.0~b2-1

is the first version where the 2010-01-01 times show up. The one before
that, 40.0~b1-1, still has recent mtimes.

HTH,

Peter.

[1] http://snapshot.debian.org/package/icedove/

-- 
I use the GNU Privacy Guard (GnuPG) in combination with Enigmail.
You can send me encrypted mail if you want some privacy.
My key is available at 

Bug#852318: lightning: Fake mtime in packaged files

[Carsten Schoenert]

Hi Peter,

On Sat, Jan 28, 2017 at 05:45:30PM +0100, Peter Lebbing wrote:
> Hello Carsten,
> 
> On 28/01/17 17:15, Carsten Schoenert wrote:
> > we didn't ever have done some patching here. I did a quick search on the
> > upstream sources and found mozilla bug 1277976 which introduces this
> > change.
> 
> While this sure is the change that made the mtime problem manifest with
> user-visible changes, this change is not the problem.
> 
> > I suggest to get in contact with upstream by simply answering in the bug
> > report with your analyze. We can patch the source once the position of
> > usptream is clear.
> 
> I wonder if this is appropriate. IMO, the bug is about something else
> entirely. It's just that the fix effected a different problem.
> 
> Mozilla bug #1277976 is about what users expect a certain keyboard
> shortcut does, in a nutshell.
> 
> This Debian bug report I opened is about (many) files having fake
> metadata, confusing backup tools.
> 
> While the one led to the other in this specific instance, they are in
> essence unrelated.

then I don't understand your first email.
The only thing related to set up some "faketime" is while creating the
package. We take the timestamp from the changelog entry and this only
for reproducibility.
The timestamps are partially valid for 1:45.6.0-3

---%<---
icedove (1:45.6.0-3) experimental; urgency=medium

...

 -- Christoph Goehre   Tue, 17 Jan 2017 18:26:06 -0500

--->%---

So I have currently no clue where the timestamps with *.0 are
coming from. But this looks like something that could be come from the
tools the reproducibly team is using and creating. Then we need to bring
this to the people there to getting deeper in the reasons why the
timestamps are modified in this way.
But ... the next upload would have a new time from the changelog. I'm
confused.

> If you really suggest I bring it up in that bug
> report, I'll do so. But it feels like conflating issues in a single bug
> report.
> 
> I could also open a new upstream bug report. I'm always a bit uncertain
> whether to report upstream or to Debian. I'm using Debian, it feels like
> I should report it there. But I'm also using upstream, right...
> 
> I think in any case Debian should be aware of this specific issue, and
> decide whether to fix this in stable as well. For users of a whole class
> of backup solutions, it's a potential, but relatively benign data-loss
> issue. Only filesystem-specific tools and backup tools that always
> compare or checksum every byte of every file in the filesystem will not
> be fooled by the fake metadata.

If the issue is completely unrelated to that change then of course there
is no need to bring this up to the Bugzilla of Mozilla. We need to dig
deeper into the real issue.

Next we need to clarify with which package version such timestamps are
shipped for the first time.

Regards
Carsten

Bug#852318: lightning: Fake mtime in packaged files

[Peter Lebbing]

Hello Carsten,

On 28/01/17 17:15, Carsten Schoenert wrote:
> we didn't ever have done some patching here. I did a quick search on the
> upstream sources and found mozilla bug 1277976 which introduces this
> change.

While this sure is the change that made the mtime problem manifest with
user-visible changes, this change is not the problem.

> I suggest to get in contact with upstream by simply answering in the bug
> report with your analyze. We can patch the source once the position of
> usptream is clear.

I wonder if this is appropriate. IMO, the bug is about something else
entirely. It's just that the fix effected a different problem.

Mozilla bug #1277976 is about what users expect a certain keyboard
shortcut does, in a nutshell.

This Debian bug report I opened is about (many) files having fake
metadata, confusing backup tools.

While the one led to the other in this specific instance, they are in
essence unrelated. If you really suggest I bring it up in that bug
report, I'll do so. But it feels like conflating issues in a single bug
report.

I could also open a new upstream bug report. I'm always a bit uncertain
whether to report upstream or to Debian. I'm using Debian, it feels like
I should report it there. But I'm also using upstream, right...

I think in any case Debian should be aware of this specific issue, and
decide whether to fix this in stable as well. For users of a whole class
of backup solutions, it's a potential, but relatively benign data-loss
issue. Only filesystem-specific tools and backup tools that always
compare or checksum every byte of every file in the filesystem will not
be fooled by the fake metadata.

Thanks,

Peter.

-- 
I use the GNU Privacy Guard (GnuPG) in combination with Enigmail.
You can send me encrypted mail if you want some privacy.
My key is available at 

Bug#852318: lightning: Fake mtime in packaged files

[Carsten Schoenert]

Helo Peter,

On Mon, Jan 23, 2017 at 03:52:14PM +0100, Peter Lebbing wrote:
... 
> I suggest these synthetic, fake mtimes should be fixed to reflect actual
> changes. I can't believe that someone wrote the two versions of
> menuOverlay.dtd with two different access keys in the same, very first
> second of 2010, so why does the mtime reflect that this is what
> happened? It confuses backup tools because we don't have the Archive
> attribute that MS-DOS could use for these purposes. Instead they rely on
> truthful metadata, where the mtime is the last time the file content was
> changed, not some fake piece of non-information.

we didn't ever have done some patching here. I did a quick search on the
upstream sources and found mozilla bug 1277976 which introduces this
change.

https://bugzilla.mozilla.org/show_bug.cgi?id=1277976

>From the bugreport there this commit is visible.

https://hg.mozilla.org/comm-central/rev/d273f175a570d3bfca46be43a865de98234a1cb4

I suggest to get in contact with upstream by simply answering in the bug
report with your analyze. We can patch the source once the position of
usptream is clear.

Regards
Carsten

Bug#852318: lightning: Fake mtime in packaged files

[Peter Lebbing]

Package: lightning
Version: 1:45.6.0-3
Severity: normal

Dear maintainer,

Some packaged files have fake, constant modification times, as can for
instance be seen here:

ll --full-time usr/lib/lightning/
total 40
-rw-r--r-- 1 peter peter  563 2010-01-01 01:00:00.0 +0100 app.ini
drwxr-xr-x 2 peter peter 4096 2017-01-18 00:26:06.0 +0100 calendar-js
drwxr-xr-x 6 peter peter 4096 2017-01-23 15:18:24.237512811 +0100 chrome
-rw-r--r-- 1 peter peter 5953 2010-01-01 01:00:00.0 +0100 
chrome.manifest
drwxr-xr-x 2 peter peter 4096 2017-01-18 00:26:06.0 +0100 components
drwxr-xr-x 3 peter peter 4096 2017-01-18 00:26:06.0 +0100 defaults
-rw-r--r-- 1 peter peter 1772 2010-01-01 01:00:00.0 +0100 install.rdf
drwxr-xr-x 2 peter peter 4096 2017-01-18 00:26:06.0 +0100 modules
drwxr-xr-x 2 peter peter 4096 2017-01-18 00:26:06.0 +0100 timezones

Why is this a problem? It's a problem when people use certain backup
solutions. For instance, I use dirvish, which underneath uses rsync with
a --link-dest argument set to the previous backup. Rsync by default
inspects file metadata only to see if it needs to transfer anything. If
mode, mtime and *size* all stay the same, rsync concludes that the file
is unchanged and hardlinks to the file present in the previous backup.

Periodically, I let rsync run with the --checksum argument to check the
integrity of my backups. That's when I discovered these files had
changed only in data, but not in mtime or size:

usr/lib/iceowl-extension/chrome/calendar-en-US/locale/en-US/calendar/menuOverlay.dtd
usr/lib/iceowl-extension/chrome/calendar/content/calendar/calendar-dnd-listener.js
usr/lib/iceowl-extension/chrome/calendar/content/calendar/calendar-migration-dialog.js
usr/share/firefox-esr/browser/defaults/preferences/devtools.js
usr/share/firefox-esr/browser/defaults/preferences/firefox.js
usr/share/firefox-esr/browser/defaults/preferences/webide-prefs.js

This is on Debian stable. All of these files differed just in comments,
/except/ for 
usr/lib/iceowl-extension/chrome/calendar-en-US/locale/en-US/calendar/menuOverlay.dtd:

diff -ur 
20170121-0302/tree/usr/lib/iceowl-extension/chrome/calendar-en-US/locale/en-US/calendar/menuOverlay.dtd
 
20170122-0307/tree/usr/lib/iceowl-extension/chrome/calendar-en-US/locale/en-US/calendar/menuOverlay.dtd
--- 
20170121-0302/tree/usr/lib/iceowl-extension/chrome/calendar-en-US/locale/en-US/calendar/menuOverlay.dtd
 2010-01-01 01:00:00.0 +0100
+++ 
20170122-0307/tree/usr/lib/iceowl-extension/chrome/calendar-en-US/locale/en-US/calendar/menuOverlay.dtd
 2010-01-01 01:00:00.0 +0100
@@ -47,4 +47,4 @@
 
 
 
-
+



Apparently at some time in the past, an updated iceowl-extension package
shipped a new file where this "accesskey" changed from a C to an a. At
that moment, my backups no longer reflected the files on the system. If
I were to restore from backup, the old file with the old C access key
would be restored, and my system's behaviour would have changed because
of it.

I suggest these synthetic, fake mtimes should be fixed to reflect actual
changes. I can't believe that someone wrote the two versions of
menuOverlay.dtd with two different access keys in the same, very first
second of 2010, so why does the mtime reflect that this is what
happened? It confuses backup tools because we don't have the Archive
attribute that MS-DOS could use for these purposes. Instead they rely on
truthful metadata, where the mtime is the last time the file content was
changed, not some fake piece of non-information.

Thanks for your packaging efforts,

Peter.

-- System Information:
Debian Release: 8.7
  APT prefers stable-updates
  APT policy: (990, 'stable-updates'), (990, 'stable'), (610, 'testing'), (600, 
'unstable'), (500, 'testing-updates'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.8.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)