Bug#366555: Caveat! Re: Bug#366555: dpkg-source: Timestamps on documentation advance artificially
Argh, human error, please ignore that laste one if you like -- that was from the draft folder, or rather I was using the 'queue' folder like it was a draft folder again, and hit the wrong Sylpheed 'Send' button. Not that there was anything inflamatory there, but it's been a busy week, so I was still thinking about that stuff, and wanted a shorter and better message. Sorry... -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#366555: dpkg-source: Timestamps on documentation advance artificially
Pending already. Maybe this reply is better late than never, though
it's relevant to the earlier 'freeguide' bug log, particularly
regarding homogenized upstream dates. Sometimes one bug is really
several.
On Thu, 20 Mar 2008 08:22:44 +0100
Raphael Hertzog [EMAIL PROTECTED] wrote:
To be clear: this bug #366555 is assigned to dpkg-dev and dpkg-dev
doesn't control the timestamp of generated documentation. If you want
the generated documentation to have a timestamp that matches the
source, you should just write a tool that makes it easy to do so and
hope that some maintainers will use it to fix timestamp of the
generated documentation (but I doubt it, honestly and the maintainer
of debhelper is very much a reference concerning what is a reasonable
packaging helper tool).
I like your idea, it'd be a workable kludge pending some eventual
systemic fix or '.deb' revision. It might work at install time, a user
tool, rather like 'apt-listchanges' or 'localepurge'. (See below for
details.)
But if 'dpkg-dev' isn't relevent, (IIRC Shaun Jackman reassigned it from
'freeguide' to 'dpkg-dev'), then where to assign this other (correcting
upstream dates) bug?
...Up to now all files from the debian sub-directory are installed by the
diff.gz and thus get a timestamp reset by dpkg-source. In the future,
when we switch over to the new source format the files in the debian
directory will be installed by a debian.tar.gz file and thus we will
preserve timestamps. However upstream patched files will continue to
have their timestamp reset at unpack time.
So if I understand things correctly, the 'upstream' part of a package
is inviolate, never changing. At present there's no preservation of
patch timestamps, (always the latest patch/unpack time), and no
provision for correcting upstream metadata -- the current format
and tool chain are not designed for (you and JH argue) any such
proposed metadata functions. For my purposes the whole source
'.deb' is virtually taboo.
Would a partial fix that only affected '/usr/share/doc/*' be safe
from tricky side-effects?
If the files are simply copied over, their timestamp is already
preserved. And it's unlikely that we patch upstream documentation.
So no, I'm not in favor at all to add any complexity to the
timestamp-resetting rule.
The user changing doc dates after or during the install would be OK?
Assume the source '.deb' is untouched. Would install-time date changes
be any worse for a system than say, 'localepurge'?
Some notes on date advancement (date creep?) in Debian...
Here's some ad hoc coding to see how often repeat dates occur in
'/usr/share/doc', i.e. many doc files per package with the same date.
On my system's 2920 packages, five or more doc date dups occur in
about 1200 packages.
# Top 10:
# usage 'pick 7 6' prints fields $7 $6 (1 to 9 only)
% pick() { z=$* ; while read x ; do set -- $x ; for f in $z ; do eval echo -n
\$$f\ ; done ; echo ; done ; }
% for t in `cd /usr/share/doc/ ; find -maxdepth 1 -type d | sed -e 's#^\./##'
-e '/\./d'` ; do find /usr/share/doc/$t -type f -printf %C@ \n | sort -g |
uniq -c | sort -rg | xargs echo $t | pick 2 1 3 | xargs printf %4i %40s %s\n
; done 2 /dev/null | sort -g /tmp/dupdate.txt
# 1) # of repeat dates
# 2) package name
# 3) the repeated date itself, in seconds since 1970
% tail /tmp/dupdate.txt
199 libcommons-lang-java 1200704724.00
262gnome-dev-doc 1188363464.00
271 kde 1205914766.00
289 debian-guide 1073455729.00
302 openssl 1203115202.00
391HOWTO 1197753227.00
431 lg 1113243150.00
553 lilypond 1205128585.00
641 RFC 1203416145.00
648 mplayer-doc 1205119451.00
I guess some of these might be affected by the pending #366555? For
those that aren't...
Idea about a 'Debian day 1' date for upstream packages like the above.
Earlier I suggested saving the dates the first time it's packaged.
Hess objected to polluting the source '.deb' with an added data
structure and handling code. On 2nd thought: no additional data
structure is needed. An older or installed package version already
contains the desired metadata.
Supposing version numbers are whole numbers:
1) Day 1, version 1, relative to Debian or the users system,
installed package has upstream dates. No change from how
it's done now.
2) Version 2, relative to Debian. Before unpacking, go through
'doc' dir, compare checksums to version 1, if they match,
use the old dates.
3) Version 'n' compare to version 'n-1'.
In theory it's easy to apply the same
Bug#366555: dpkg-source: Timestamps on documentation advance artificially
Hi, On Fri, 28 Mar 2008, A. Costa wrote: To be clear: this bug #366555 is assigned to dpkg-dev and dpkg-dev doesn't control the timestamp of generated documentation. If you want the generated documentation to have a timestamp that matches the source, you should just write a tool that makes it easy to do so and hope that some maintainers will use it to fix timestamp of the generated documentation (but I doubt it, honestly and the maintainer of debhelper is very much a reference concerning what is a reasonable packaging helper tool). I like your idea, it'd be a workable kludge pending some eventual systemic fix or '.deb' revision. It might work at install time, a user tool, rather like 'apt-listchanges' or 'localepurge'. (See below for details.) I rather meant a build-time tool like: sync-timestamp reference destination Where reference is the original file (say in XML format) and the destination is either a file or a directory with multiple files. But if 'dpkg-dev' isn't relevent, (IIRC Shaun Jackman reassigned it from 'freeguide' to 'dpkg-dev'), then where to assign this other (correcting upstream dates) bug? Nowhere. If the idea is to write a new tool, there's no pre-existing package where to reassign it. ...Up to now all files from the debian sub-directory are installed by the diff.gz and thus get a timestamp reset by dpkg-source. In the future, when we switch over to the new source format the files in the debian directory will be installed by a debian.tar.gz file and thus we will preserve timestamps. However upstream patched files will continue to have their timestamp reset at unpack time. So if I understand things correctly, the 'upstream' part of a package is inviolate, never changing. At present there's no preservation of patch timestamps, (always the latest patch/unpack time), and no provision for correcting upstream metadata -- the current format and tool chain are not designed for (you and JH argue) any such proposed metadata functions. Yes. The user changing doc dates after or during the install would be OK? Assume the source '.deb' is untouched. Would install-time date changes be any worse for a system than say, 'localepurge'? Well, I have no problem with the user changing timestamps of installed files if that's something desirable to the user. But I really don't see how you would expect this to work. Idea about a 'Debian day 1' date for upstream packages like the above. Earlier I suggested saving the dates the first time it's packaged. Hess objected to polluting the source '.deb' with an added data structure and handling code. On 2nd thought: no additional data structure is needed. An older or installed package version already contains the desired metadata. Supposing version numbers are whole numbers: 1) Day 1, version 1, relative to Debian or the users system, installed package has upstream dates. No change from how it's done now. 2) Version 2, relative to Debian. Before unpacking, go through 'doc' dir, compare checksums to version 1, if they match, use the old dates. 3) Version 'n' compare to version 'n-1'. Ouch. What a lot of work for simple timestamps. And it has problems: a compressed documentation file will have a differing checksum even if the content does match as the gzip header includes some timestamp as well. So yo should compare uncompressed content. Many auto-generated documentation will also include some sort of timestamp in the generated page as well. So, a util that runs on a package upgrade, before it's unpacked**. Save prior version's doc date checksums to temp file. After unpacking, compare, then backdate unchanged doc files as needed. (**is there a Debian Way to run on upgrade before unpacking? I know how to code the rest, assuming I've not forgotten some other necessary difficulty.) Well, right now there's no such hook defined in dpkg. Thus this feature can probably only be developed as a dpkg patch. But I can tell you in advance, that any such wishlist bug against dpkg will be tagged wontfix as it's way too complicated for too little gain. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#366555: dpkg-source: Timestamps on documentation advance artificially
On Thu, 20 Mar 2008, A. Costa wrote: Thanks for the clarification. Unfortunately, it therefore follows I'd be favoring time wasting over-engineering --- provided of course, there were only two possible ways of dealing with the bug, (1. the wrong way, or 2. no fix at all). But as was suggested to the emphatic JH back in 4/2006, there might be more than one way to do it: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=366555#36 To be clear: this bug #366555 is assigned to dpkg-dev and dpkg-dev doesn't control the timestamp of generated documentation. If you want the generated documentation to have a timestamp that matches the source, you should just write a tool that makes it easy to do so and hope that some maintainers will use it to fix timestamp of the generated documentation (but I doubt it, honestly and the maintainer of debhelper is very much a reference concerning what is a reasonable packaging helper tool). dpkg-dev comes into play only at one place: the timestamp of files patched by the diff.gz (or by debian/patches/* with a newer source package format). Up to now all files from the debian sub-directory are installed by the diff.gz and thus get a timestamp reset by dpkg-source. In the future, when we switch over to the new source format the files in the debian directory will be installed by a debian.tar.gz file and thus we will preserve timestamps. However upstream patched files will continue to have their timestamp reset at unpack time. (Your replies come in fast, but we've been spending a lot of email time backtracking. Not sure what you mean, I've read the whole bug log, I do have my opinion and it probably doesn't match your expectation... IANADD, could you or somebody give a less abstract example? Suppose '/usr/share/doc/freeguide/FAQ.html' was dated a week earlier than '/usr/share/doc/freeguide/TODO'; what bad thing would happen? Nothing of course... Would a partial fix that only affected '/usr/share/doc/*' be safe from tricky side-effects? If the files are simply copied over, their timestamp is already preserved. And it's unlikely that we patch upstream documentation. So no, I'm not in favor at all to add any complexity to the timestamp-resetting rule. ...But it's not the same when we speak of configure and configure.ac and dpkg-source must not a different behaviour ...must not a...; sorry, is there a missing verb? have depending on the filename that it patches, it's just too error-prone. Alas IANADD, and don't know much about 'autoconf', so the meaning of the above is Greek to me. Maybe you're saying the problem owes more to the 'autoconf' package than to 'dpkg-source'. No. Is this one of those political bugs, that belongs in part to several packages, but is practically insoluble because the maintainers fight? No. It's the normal and expected behaviour of the autoconf tools. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#366555: dpkg-source: Timestamps on documentation advance artificially
On Tue, 18 Mar 2008 08:11:35 +0100 Raphael Hertzog [EMAIL PROTECTED] wrote: On Tue, 18 Mar 2008, A. Costa wrote: The long bug log clearly says that there's no point to try to conserve timestamps for generated documentation. And I agree with that. Time permitting, would you kindly tell me where it says that? I reread the whole BTS log for #366555 yesterday, but might have missed or even misread something. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=366555#31 I totally agree with Joey's comment. Adding supplementary infrastructure to conserve timestamp on generated files is just a waste of time. and over-engineering for very little value. Thanks for the clarification. Unfortunately, it therefore follows I'd be favoring time wasting over-engineering --- provided of course, there were only two possible ways of dealing with the bug, (1. the wrong way, or 2. no fix at all). But as was suggested to the emphatic JH back in 4/2006, there might be more than one way to do it: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=366555#36 (Your replies come in fast, but we've been spending a lot of email time backtracking. Perhaps lately you're too busy, rushing about with the DPL thing, for my discursive style of email? And good luck with that election, here's hoping your ideas on transparency and communication will bear fruit!) Re: very little value, that's harsh. No objection if you meant relatively little value, which allows for different priorities and preferences. - but files patched by one or more of the patches in debian/patches/ will always have a timestamp that advances artificially at each unpack. This is required because if we don't ajust their timestamp to a single value, the timestamp difference means that we can have tricky side-effects like regeneration of some files due to timestamp skew (e.g. when you patch *.ac or *.in files from autoconf/automake) IANADD, could you or somebody give a less abstract example? Suppose '/usr/share/doc/freeguide/FAQ.html' was dated a week earlier than '/usr/share/doc/freeguide/TODO'; what bad thing would happen? Nothing of course... Would a partial fix that only affected '/usr/share/doc/*' be safe from tricky side-effects? ...But it's not the same when we speak of configure and configure.ac and dpkg-source must not a different behaviour ...must not a...; sorry, is there a missing verb? depending on the filename that it patches, it's just too error-prone. If you have a Makefile rule: configure: configure.ac #regenerate configure Then the fact that configure.ac is newer than configure will imply execution of the rule when it's not wanted. Alas IANADD, and don't know much about 'autoconf', so the meaning of the above is Greek to me. Maybe you're saying the problem owes more to the 'autoconf' package than to 'dpkg-source'. Is this one of those political bugs, that belongs in part to several packages, but is practically insoluble because the maintainers fight? Summing up: back to 4/2006. More than one way. RH '08. Level of complexity controversial. Some methods deprecated. Where is the bug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#366555: dpkg-source: Timestamps on documentation advance artificially
On Sun, 16 Mar 2008 11:11:22 +0100 Raphael Hertzog [EMAIL PROTECTED] wrote: The long bug log clearly says that there's no point to try to conserve timestamps for generated documentation. And I agree with that. Time permitting, would you kindly tell me where it says that? I reread the whole BTS log for #366555 yesterday, but might have missed or even misread something. However with the new source format we have several changes: - files in the debian directory are stored in a .tar.gz and thus we will conserve their timestamp Sounds good. - but files patched by one or more of the patches in debian/patches/ will always have a timestamp that advances artificially at each unpack. This is required because if we don't ajust their timestamp to a single value, the timestamp difference means that we can have tricky side-effects like regeneration of some files due to timestamp skew (e.g. when you patch *.ac or *.in files from autoconf/automake) IANADD, could you or somebody give a less abstract example? Suppose '/usr/share/doc/freeguide/FAQ.html' was dated a week earlier than '/usr/share/doc/freeguide/TODO'; what bad thing would happen? (If it matters, I'm mainly curious about 'doc' dates, not so much binaries.) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#366555: dpkg-source: Timestamps on documentation advance artificially
On Tue, 18 Mar 2008, A. Costa wrote: The long bug log clearly says that there's no point to try to conserve timestamps for generated documentation. And I agree with that. Time permitting, would you kindly tell me where it says that? I reread the whole BTS log for #366555 yesterday, but might have missed or even misread something. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=366555#31 I totally agree with Joey's comment. Adding supplementary infrastructure to conserve timestamp on generated files is just a waste of time. and over-engineering for very little value. - but files patched by one or more of the patches in debian/patches/ will always have a timestamp that advances artificially at each unpack. This is required because if we don't ajust their timestamp to a single value, the timestamp difference means that we can have tricky side-effects like regeneration of some files due to timestamp skew (e.g. when you patch *.ac or *.in files from autoconf/automake) IANADD, could you or somebody give a less abstract example? Suppose '/usr/share/doc/freeguide/FAQ.html' was dated a week earlier than '/usr/share/doc/freeguide/TODO'; what bad thing would happen? Nothing of course. But it's not the same when we speak of configure and configure.ac and dpkg-source must not a different behaviour depending on the filename that it patches, it's just too error-prone. If you have a Makefile rule: configure: configure.ac #regenerate configure Then the fact that configure.ac is newer than configure will imply execution of the rule when it's not wanted. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
