Re: d/changelog and experimental

[Ian Jackson]

Overall I think it is usually best to include the changelog entries
for experimental versions in the appropriate place in the changelog
for the sid upload, unless they really contain only stuff that was
later undone.  That is helpful for humans and also for computers.

For example, consider what ought to be shown by apt-listchanges
(which shows you the top part of the changelog since the version being
upgraded from).


Simon Richter writes ("Re: d/changelog and experimental"):
> Bugs are closed based on the changes file, which is generated from the
> topmost entry, always.

This is not strictly speaking correct.

If the most recent upload to sid (say) is not the second-topmost
entry in the changes file, then the .changes for the sid upload
should contain *all* the entries since the last upload to sid.

This can arise in situations other than where the intervening versions
were upload to experimental.  Perhaps the intervening versions were
not uploaded at all, or were REJECTed, or were uploaded to another
distro. [1]  In those situations the intervening bug close entries
need to be processed.

You can get this right by passing a -v option to dpkg-genchanges via
the appropriate mechanism for your usual build and upload tools.

Or you can just use dgit which gets this right automatically :-).

Ian.

[1] A workflow is possible where you use one git branch and just add a
new changelog entry for each upload.  Although our data formats would
support uploads to a multiple distros with a single changelog entry
and even a single signed .changes, unfortunately our various tools and
services don't.  But writing an additional changelog is easy.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: d/changelog and experimental

[Ben Hutchings]

On Sun, 2019-12-08 at 23:24 +0100, Simon Richter wrote:
[...]
> The rest of the changelog only exists to preserve history. When you make
> an upload to experimental closing a bug, and you later upload the
> package to unstable, you have to close the bugs again in the changelog
> entry for unstable.
[...]

So long as the experimental versions are listed in the changelog, so
that the BTS recognises the new version as based on them, you don't
need to close the bugs again.

Ben.

-- 
Ben Hutchings
Humour is the best antidote to reality.




signature.asc
Description: This is a digitally signed message part

Re: d/changelog and experimental

[Simon Richter]

Hi,

On 08.12.19 22:29, Guillem Jover wrote:

> I think there are two important properties that need to be preserved
> so that debian/changelog entries keep making sense both for humans
> and machines alike. The first is that the parseable format entries
> should be sorted by version, otherwise things that parse the file
> might get confused when doing range checks or filtering things. The
> second is to preserve the timeline of the changes, so that when a
> human (or even a parser that extracts semantic meaning like with bug
> closures) reads them they should still makes sense.

For backports, we already have cases where entries aren't sorted,
because the changelog still lists the version from sid, and the entry
for the backports version is then prepended with a lower version number.

Bugs are closed based on the changes file, which is generated from the
topmost entry, always.

The rest of the changelog only exists to preserve history. When you make
an upload to experimental closing a bug, and you later upload the
package to unstable, you have to close the bugs again in the changelog
entry for unstable. At this point, it makes sense to condense the
changelog to things that have actually changed.

The target audience of the Debian changelog is a skilled system
administrator who wants to know what changes in behavior to expect from
the new version, especially deviations from what is described in
upstream documentation (because Debian applied a patch).

The format is too terse to serve as a full history of the package
itself, it can only provide pointers to more documentation, ideally
inside the BTS so it is archived within Debian and crosslinked.

The example you give for a merged changelog is confusing, and there is
no way to make it less so save for a dedicated tool to visualize it --
but such a tool could also access the git history of the package instead.

   Simon



signature.asc
Description: OpenPGP digital signature

Re: d/changelog and experimental

[Bernd Zeimetz]

On 12/3/19 8:21 AM, Russ Allbery wrote:
> Paolo Greppi  writes:
> 
>> What is the best approach for d/changelog when releasing a package to
>> unstable after it has been through a few iterations to experimental ?
> 
>> It would seem that the right thing to do is to keep all experimental
>> changelog entries, and add a new one for unstable.
> 
> This is the typical practice, including all the intermediate experiments
> or false starts in experimental.
> 
> [.]

I'm all in favour of keeping the upload history, but recently I had to
do a major rework of the packaging in experimental while updating
unstable at the same time. So over several weeks, unstable got various
cherry-picks from experimental and at some point a big merge happened,
which made it more or less impossible to keep the changelog from
experimental as lots of changes would be documented twice and, even
worse, there would be a mix of version X and X+1 entries in the
changelog as the uploads were not linear.


-- 
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F

Re: d/changelog and experimental

[Guillem Jover]

Hi!

On Tue, 2019-12-03 at 08:15:19 +0100, Paolo Greppi wrote:
> What is the best approach for d/changelog when releasing a package
> to unstable after it has been through a few iterations to experimental ?
>
> It would seem that the right thing to do is to keep all experimental
> changelog entries, and add a new one for unstable.

That really depends on how one manages these branches and uploads.

I think there are two important properties that need to be preserved
so that debian/changelog entries keep making sense both for humans
and machines alike. The first is that the parseable format entries
should be sorted by version, otherwise things that parse the file
might get confused when doing range checks or filtering things. The
second is to preserve the timeline of the changes, so that when a
human (or even a parser that extracts semantic meaning like with bug
closures) reads them they should still makes sense.

> But sometimes experimental uploads are, well ... experimental, there
> may be changes that cancel out (I tried this but it did not work, then
> I tried that etc.).
> So another way is to consolidate all experimental changelog entries
> into the unstable one, tidying it up.
> Also as most people never see the experimental releases, they only
> care for what's new since the last release to unstable / stable.

I think the two main ways to handle these are either based on
something ressembling the shape of «git cherry-pick» or «git merge».

* For a «git cherry-pick» kind of approach, an upload to experimental
  does not imply that the changes there should or might end up all in
  unstable, it might end up being, say, a dead-end diverging VCS branch,
  and you could simply backport the various atomic changes in atomic
  squashed logical units into unstable to drop any intermediate
  changes/commits.

  Or another similar approach might keep them in sync, by doing the same
  packaging changes to both the unstable and experimental branches, and
  upload both at around the same time. Then you could eventually just
  upload the package from experimental into unstable. You'd lose upload
  information, but not change information. The former can easily happen
  within stable vs testing/unstable anyway, or with not ACKed NMUs for
  example, so I don't think it's a big deal, and the upload information
  is anyway tracked in DAK or tracker.d.o.

* For a «git merge» kind of approach, then what can happen is that
  you've got changes reverted in one place and reintroduced in another,
  so how you sort the entries in debian/changelog is even more important
  to avoid confusing people reading the entries. In this case, if using
  version-sorted entries, it can make them stop making sense, and
  date-sorting would confuse automated parsing, neither of these would
  really represent the actual timeline of the changes, so I'd recommend
  injecting the entries as sub entries. So as an example, it could look
  something like this:

  ,---
  pkg (1.0.0-11) unstable; urgency=medium

* Enable feature foo.

   -- Name   Date N + 6

  pkg (1.0.0-10) unstable; urgency=medium

* Merge from experimental:

pkg (2.0.0-5) experimental; urgency=medium

  * Disable feature foo.
  * Some other change Z.

 -- Name   Date N + 3

pkg (2.0.0-4) experimental; urgency=medium

  * Enable feature foo.
  * Some other change Y.

 -- Name   Date N + 1

* Some other change B.

   -- Name   Date N + 5

  pkg (1.0.0-9) unstable; urgency=medium

* Some other change A.

   -- Name   Date N + 2

  pkg (1.0.0-8) unstable; urgency=medium

* Disable feature foo.

   -- Name   Date N + 0
  `---

  Of course if “Enable feature foo” would have happened already in
  1.0.0-9, then that'd be reather confusing, but would imply a mix
  of different merge strategies. :)

Thanks,
Guillem

Re: d/changelog and experimental

[Colin Watson]

On Tue, Dec 03, 2019 at 09:53:43AM -0800, Russ Allbery wrote:
> Mike Hommey  writes:
> > Well... I don't think there's anything good that can be done about the
> > entries about the unstable uploads that have happened in between...
> 
> Yeah, that's true.  Upload history is a graph and I have no idea how to
> linearize that.  In that scenario, I'm not sure I have any
> recommendations.

I normally just sort the merged portion of the changelog by version, or
occasionally chronologically if that seems to make more sense (but IME
version-sorting usually feels better to me).  I don't think it matters
too much as long as it remains topologically-sorted, which both of these
approaches satisfy.

-- 
Colin Watson   [cjwat...@debian.org]

Re: d/changelog and experimental

[Paul Gevers]

Hi,

On 03-12-2019 08:21, Russ Allbery wrote:
> Paolo Greppi  writes:
> 
>> What is the best approach for d/changelog when releasing a package to
>> unstable after it has been through a few iterations to experimental ?
> 
>> It would seem that the right thing to do is to keep all experimental
>> changelog entries, and add a new one for unstable.
> 
> This is the typical practice, including all the intermediate experiments
> or false starts in experimental.
> 
> I sympathize with wanting to clean up some of that, but as a project we
> generally have decided to live with that.  Most users don't read the full
> changelog anyway (user-visible notes should be in NEWS instead, which are
> shown to more people via apt-listchanges), and removing versions from the
> history has bad effects on the bug-tracking system, historical archives,
> etc.

The bts parses the changelogs to determine the history of packages. So
it's really the best to always keep the changelog entries that are part
of the history of the version of the package. So, if you upload a
descendant of the package in experimental, the changelog of the
experimental changes really should stay.

Paul



signature.asc
Description: OpenPGP digital signature

Re: d/changelog and experimental

[Russ Allbery]

Mike Hommey  writes:
> On Mon, Dec 02, 2019 at 11:21:12PM -0800, Russ Allbery wrote:

>> This is the typical practice, including all the intermediate
>> experiments or false starts in experimental.

>> I sympathize with wanting to clean up some of that, but as a project we
>> generally have decided to live with that.  Most users don't read the
>> full changelog anyway (user-visible notes should be in NEWS instead,
>> which are shown to more people via apt-listchanges), and removing
>> versions from the history has bad effects on the bug-tracking system,
>> historical archives, etc.

> Well... I don't think there's anything good that can be done about the
> entries about the unstable uploads that have happened in between...

Yeah, that's true.  Upload history is a graph and I have no idea how to
linearize that.  In that scenario, I'm not sure I have any
recommendations.

-- 
Russ Allbery (r...@debian.org)  

Re: d/changelog and experimental

[Mike Hommey]

On Mon, Dec 02, 2019 at 11:21:12PM -0800, Russ Allbery wrote:
> Paolo Greppi  writes:
> 
> > What is the best approach for d/changelog when releasing a package to
> > unstable after it has been through a few iterations to experimental ?
> 
> > It would seem that the right thing to do is to keep all experimental
> > changelog entries, and add a new one for unstable.
> 
> This is the typical practice, including all the intermediate experiments
> or false starts in experimental.
> 
> I sympathize with wanting to clean up some of that, but as a project we
> generally have decided to live with that.  Most users don't read the full
> changelog anyway (user-visible notes should be in NEWS instead, which are
> shown to more people via apt-listchanges), and removing versions from the
> history has bad effects on the bug-tracking system, historical archives,
> etc.

Well... I don't think there's anything good that can be done about the
entries about the unstable uploads that have happened in between...

Mike

Re: d/changelog and experimental

[Xavier]

Hi,

I think that every published versions must have an entry in d/changelog

If unstable version follows some experimental ones, then I prefer to have the 
full entries.

This can also help to debug when an other package has a ">= exp-version"


Le 3 décembre 2019 08:21:12 GMT+01:00, Russ Allbery  a écrit :
>Paolo Greppi  writes:
>
>> What is the best approach for d/changelog when releasing a package to
>> unstable after it has been through a few iterations to experimental ?
>
>> It would seem that the right thing to do is to keep all experimental
>> changelog entries, and add a new one for unstable.
>
>This is the typical practice, including all the intermediate
>experiments
>or false starts in experimental.
>
>I sympathize with wanting to clean up some of that, but as a project we
>generally have decided to live with that.  Most users don't read the
>full
>changelog anyway (user-visible notes should be in NEWS instead, which
>are
>shown to more people via apt-listchanges), and removing versions from
>the
>history has bad effects on the bug-tracking system, historical
>archives,
>etc.
>
>-- 
>Russ Allbery (r...@debian.org) 
>

-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma 
brièveté.

Re: d/changelog and experimental

[Russ Allbery]

Paolo Greppi  writes:

> What is the best approach for d/changelog when releasing a package to
> unstable after it has been through a few iterations to experimental ?

> It would seem that the right thing to do is to keep all experimental
> changelog entries, and add a new one for unstable.

This is the typical practice, including all the intermediate experiments
or false starts in experimental.

I sympathize with wanting to clean up some of that, but as a project we
generally have decided to live with that.  Most users don't read the full
changelog anyway (user-visible notes should be in NEWS instead, which are
shown to more people via apt-listchanges), and removing versions from the
history has bad effects on the bug-tracking system, historical archives,
etc.

-- 
Russ Allbery (r...@debian.org)  

d/changelog and experimental

[Paolo Greppi]

What is the best approach for d/changelog when releasing a package to unstable 
after it has been through a few iterations to experimental ?

It would seem that the right thing to do is to keep all experimental changelog 
entries, and add a new one for unstable.

But sometimes experimental uploads are, well ... experimental, there may be 
changes that cancel out (I tried this but it did not work, then I tried that 
etc.).
So another way is to consolidate all experimental changelog entries into the 
unstable one, tidying it up.
Also as most people never see the experimental releases, they only care for 
what's new since the last release to unstable / stable.

On this topic I found nothing definitive in policy:
https://www.debian.org/doc/debian-policy/ch-source.html#s-dpkgchangelog
except footnote 3 which does not really discuss experimental.

On debian-devel I found this old thread:
https://lists.debian.org/debian-devel/2005/01/msg00791.html
where different opinions are voiced.

What the best approach ?

Paolo

(originally posted as: 
https://alioth-lists.debian.net/pipermail/pkg-javascript-devel/2019-November/037083.html)