Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Henrique de Moraes Holschuh]

On Tue, Feb 14, 2017, at 19:42, Ian Jackson wrote:
> Simon McVittie writes ("Re: changelog practice, unfinalised vs UNRELEASED
> vs ~version"):
> > On Mon, 13 Feb 2017 at 09:42:32 +1100, Ben Finney wrote:
> > > I don't see how this complaint is any different from the need to merge,
> > > for example, changes to API documentation and test cases that accompany
> > > a functional change.
> > 
> > It's the difference between "sometimes conflicts" and "always conflicts".
> 
> This is a very real problem for debian/changelog, but mergechangelogs
> helps a lot.  In any case as I say I'm not trying to mandate the

Usable VCSes have pluggable merge drivers, and there are such drivers
for both GNU-style ChangeLog, and debian/changelog[1].  IME, they "just
work".  So, yeah, one *can* have the chagelog-update-along-with-change
workflow sanely most of the time nowadays.  

I would never use such an workflow unless forced, as I believe the
"proper commit log Tao" is better in the long run, and when your commit
logs are proper (and quite verbose), the natural workflow [2] asks for
changelog-updating commits before release (or maybe at the tip of every
ready-for-integration branch, etc), but...

[1] refer to dpkg-mergechangelogs(1) for the git driver for
debian/changelog, for example.

[2] as in "less busywork" while hacking, and no weird (and possibly
maintenance-hell-creating) disparity from commit log contents to in-tree
changelog contents.

-- 
  Henrique de Moraes Holschuh <h...@debian.org>

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Guillem Jover]

Hi!

On Thu, 2017-02-23 at 17:57:33 +, Ian Jackson wrote:
> Simon McVittie writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
> ~version"):
> > On Thu, 23 Feb 2017 at 12:53:40 +, Ian Jackson wrote:
> > > We could decorate the suite instead, but that does not have any
> > > effect on generated binaries.
> > 
> > It does have an effect on the changelog inside the source and binary
> > packages,
> 
> True.  But I still think it would be a good idea for unreleased
> binaries to show up as such in the dpkg database.

I can agree with that.

> > and on the changes file (which is precisely the signed
> > instruction to the archive,
> 
> (Of course such things should not end up signed, but, this point is
> well-made.)
> 
> How about the following, which provides more flexibility for what
> individual users want:
> 
>  * Unfinalised changeslogs should contain UNRELEASED in either
>the suite name or the version number.  (Currently there is
>no consensus about this.)

Please, not the version number. Globally overloading the version to
convey this kind of information would be unfortunate IMO. This, not
only affects us but derivatives too, and depending on whether the
package is native or not, and depending ono where the marking is
placed (upstream or revision part) it also affects the versioning
of upstream projects.

I was very happy when proposing the binary-only=yes keyword and it
being accepted, so that I will be able to eventually remove the
hardcoded parsing of +bN from dpkg-dev. Unfortunately we still have
hardcoded parsing for ~bpo and ~vola to use magic behavior based on
that. I should propose a new backport=yes or similar to deprecate
that too.

>  * Where the suite contains UNRELEASED, it SHOULD NOT be simply
>`UNRELEASED'.   It should be `unstable-UNRELEASED' or
>`experimental-UNRELEASED' or such.

The suite should be "invalid" to avoid accidental or unwanted uploads
by third parties in case one needs to sign an intermediate build.

I think using a magic value in the suite name is better. We then could
either use:

  source (1.0) UNRELEASED; urgency=medium, suite=unstable

or

  source (1.0) UNRELEASED; urgency=medium, unreleased-for=unstable

or

  source (1.0) unstable-UNRELEASED; urgency=medium

The advantage of the ones using the metadata key/values is that those
can very easily end up in the generated binaries as fields. So in that
case something like the "Unreleased-For: unstable" would be a pretty
obvious marker?

>  * Tools which generates .changes files (specifically,
>dpkg-genchanges) should, if they find that the version contains
>UNRELEASED and the suite does not, append UNRELEASED to the suite.
>(This is to help save downstreams etc.)
>
>  * The Debian archive will be taught to reject anything with a version
>containing UNRELEASED (as I proposed before).

For these two points, see above. Also this implies having to hunt down
any archive software and fix it, even local scripts etc. It seems too
unsafe to me. :/

> > As I noted on #542747, this property of changes files also means that if
> > a Debian derivative (or an addon repository like apt.postgresql.org) has
> > DD contributors, then that derivative should avoid using a Debian suite
> > name as its suite name. This is because anyone obtaining a DD-signed
> > .changes file for that addon repository could upload it to the main
> > Debian repository, and it would normally be accepted into the suite
> > of the same name. apt.postgresql.org seems to be one of the few addon
> > repositories that does this (IMO) correctly, by using a distinct suite
> > name (in their case jessie-pgdg).
> 
> Yes.

Right, that's why for exampel we initially chose "unreleased" as suite
name for the Debian ports archive when they were hosted at gnuab.org!

> > I can't help wondering whether changes files ought to include some
> > globally unique indication of the project in which the change is
> > requested
> 
> Yes, they should.

We are currently somewhat lacking both, where the changes were built
in, and what they were targetting.

I've had a draft [O] for this for years (as a result of an initial
proposal for more generalized bug closure support [C]), which was the
base for the dpkg-vendor, but it seems I never ended up discussing it
more widely within the project for full support. :/ At least from the
initial discussion years ago of [C] in [T], there didn't seem to be
much consensus or wide interest.

  [C] <http://www.hadrons.org/~guillem/debian/docs/closes.proposal>
  [O] <http://www.hadrons.org/~guillem/debian/docs/origin.proposal>
  [T] <https://lists.debian.org/debian-project/2006/11/msg00100.html

We are now already recording the Origin 

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Simon McVittie writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
~version"):
> On Thu, 23 Feb 2017 at 12:53:40 +, Ian Jackson wrote:
> > We could decorate the suite instead, but that does not have any
> > effect on generated binaries.
> 
> It does have an effect on the changelog inside the source and binary
> packages,

True.  But I still think it would be a good idea for unreleased
binaries to show up as such in the dpkg database.

> and on the changes file (which is precisely the signed
> instruction to the archive,

(Of course such things should not end up signed, but, this point is
well-made.)

How about the following, which provides more flexibility for what
individual users want:

 * Unfinalised changeslogs should contain UNRELEASED in either
   the suite name or the version number.  (Currently there is
   no consensus about this.)

 * Where the suite contains UNRELEASED, it SHOULD NOT be simply
   `UNRELEASED'.   It should be `unstable-UNRELEASED' or
   `experimental-UNRELEASED' or such.

 * Tools which generates .changes files (specifically,
   dpkg-genchanges) should, if they find that the version contains
   UNRELEASED and the suite does not, append UNRELEASED to the suite.
   (This is to help save downstreams etc.)

 * The Debian archive will be taught to reject anything with a version
   containing UNRELEASED (as I proposed before).

> As I noted on #542747, this property of changes files also means that if
> a Debian derivative (or an addon repository like apt.postgresql.org) has
> DD contributors, then that derivative should avoid using a Debian suite
> name as its suite name. This is because anyone obtaining a DD-signed
> .changes file for that addon repository could upload it to the main
> Debian repository, and it would normally be accepted into the suite
> of the same name. apt.postgresql.org seems to be one of the few addon
> repositories that does this (IMO) correctly, by using a distinct suite
> name (in their case jessie-pgdg).

Yes.

> I can't help wondering whether changes files ought to include some
> globally unique indication of the project in which the change is
> requested

Yes, they should.

>  (for instance debian.org, ubuntu.com or [apt.]postgresql.org);
> but at the moment they don't, and I don't see an obvious place in the
> workflow to declare "this is really for Debian" vs. "this is for
> apt.postgresql.org". Maybe at debsign time?

(`dgit push' already (thinks it) knows the answer to this question and
could insert the right information.)

I don't think debsign is the right place.

If the suite name is not globally unique, then the full qualification
should be in the changelog.  Ie, the changelog should contain the
intended target archive.

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Wookey]

On 2017-02-14 23:01 +, Ian Jackson wrote:
> Wookey writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
> ~version"):
> > I'd be happier if I didn't have to deal with 'UNRELEASED' at all
> > unless I actually wanted to, so I'm not at all keen on your suggestion
> > of making it mandatory.
> 
> Well, tools can be configured, too.  If my UNRELEASED version proposal
> is adopted, we could easily let you have a DEB_ENABLE_FOOTGUN variable
> that will let you become confused about what binaries and sources are
> what, by reusing version numbers from ad-hoc test builds.

I have a 'current' version, which is the one that will be uploaded
when it's deemed good enough to upload. I don't find that confusing.
 
> > 1. I really dislike dch's enthusiasm for putting in 'UNRELEASED'. It
> > gives me nothing I wanted, and just provides the opportunity to really
> > do a final, clean, tested, build, only to find on upload that it's
> > still marked 'UNRELASED', and I have to do the build, test, upload
> > step again - for big packages that is a huge pain and happens way too
> > often. I really resent that there is no way to do dch -i;dch -r in one
> > go - the separation of these is just makework for me. 
> 
> Does that mean that if you tested ~UNRELEASED, and it passed all your
> tests, you would be unhappy to strip the ~UNRELEASED from the version
> and upload it ?

Well. If I changed the version I'd still have to do a rebuild to get a
matching .changes and .dsc. (perhaps there is a shortcut to this?). So
it's just the same problem as having UNRELEASED in the changelog,
except that it's much easier to notice, becuase it's right there in
the version in the filename.
 
> (Of course in principle there might be situations where version
> X~UNRELEASED would be treated differently to X.  If that's likely to
> be going on then I would have to, regrettably, use the actual
> for-release version number for your ad-hoc tests.)

Indeed. I like to test exactly the thing I'm going to upload, not
something with a different suite or version number.

> > > Proposal:
> > > 
> > >  * Tools which add/edit changelog change items should insist that the
> > >changelog is unfinalised and contains ~UNRELEASED in its version.
> > 
> > I really don't want this to be _required_. Dicking with the version is
> > better than dicking with the suite, but I really would prefer it if
> > the tools did neither, or could simply be asked to.
> > 
> > I realise that I'm pushing uphill somewhat here and no-one much cares
> > about people who still don't use git if they don't have to. But the
> > UNRELEASED/'dch -r' thing pisses me off on a daily basis, and this
> > seemed like the time to point out that some of us don't find it all
> > helpful. From that POV, moving it from suite to version would
> > definitely be less annoying.
> 
> Well, I guess you'd be satisfied with an option to disable this
> behaviour.

I would. And I'm very grateful to whoever it was in this thread that
pointed out that one currently exists.
 
> But: it seems that intend to make commits whose debian/changelog has a
> trailer line, a real suite, and a real version, but where the commit
> does not contain all the changes ? 

You are still talking about commits, which implies a VCS. I have a
plain directory containing 'the package' (quilt format so my changes
are reasonably clear). When it's finished I upload it. Debian's
archive stores the definitive 'tagged' version for me.

> I think that's poor practice.
> These commits are a liability.  If you push such a commit, and then
> later make more changes and do the upload, but forget to to push, a
> subsequent upload made from the vcs might lack your changes.

All N/A so far as I can see. The version in front of me is the version
I'm working on. It has a version number. Changes are added to the
changelog until all the changes are done (as a changelog is intended
to be used). Then it is uploaded. I am a very simple individual :-)

> I don't know how you avoid mistakes.  Perhaps you're just very
> careful.

I avoid making mistakes by not using a vcs I don't really
understand. :-) Each version of the package lives in a directory. I
only work on 'latest'. uscan and uupdate manage the creation of new
directories containing new versions, from new upstream releases. dch
manages changelog editing without screwing up the format. quilt
manages the creation of discrete changes. I take some care not to
confuse quilt (which is very easily done, but I understand it so can
fix things when I mess it up).

None of this is helped by the 'UNRELEASED' mechanism. 

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Simon McVittie]

On Thu, 23 Feb 2017 at 12:53:40 +, Ian Jackson wrote:
> We could
> decorate the suite instead, but that does not have any effect on
> generated binaries.

It does have an effect on the changelog inside the source and binary
packages, and on the changes file (which is precisely the signed
instruction to the archive, "please add the sources and/or
binaries referenced in Files/Checksums-* to the suite referenced in
Distribution"). One of the advantages of decorating the suite is that
tools like dak and reprepro will already reject a changes file with a
wrong suite, without even needing explicit code for it, because their
configuration presumably doesn't include a suite named UNRELEASED or
jessie-backports-UNRELEASED.

Unfortunately, if you build your release binaries with sbuild, a
common configuration will overwrite the Distribution in the changes file
with an unintended one. I wrote a patch for Lintian to detect this
back in 2010 (#542747), but it hasn't landed yet.

Because changes files are exactly an instruction to dak (or similar)
to include the built sources/binaries in the designated suite, I think
it's probably a bad idea to encourage DDs to produce changes files
that will say "Distribution: unstable" (or some other valid suite)
for unfinished packages.

As I noted on #542747, this property of changes files also means that if
a Debian derivative (or an addon repository like apt.postgresql.org) has
DD contributors, then that derivative should avoid using a Debian suite
name as its suite name. This is because anyone obtaining a DD-signed
.changes file for that addon repository could upload it to the main
Debian repository, and it would normally be accepted into the suite
of the same name. apt.postgresql.org seems to be one of the few addon
repositories that does this (IMO) correctly, by using a distinct suite
name (in their case jessie-pgdg).

I can't help wondering whether changes files ought to include some
globally unique indication of the project in which the change is
requested (for instance debian.org, ubuntu.com or [apt.]postgresql.org);
but at the moment they don't, and I don't see an obvious place in the
workflow to declare "this is really for Debian" vs. "this is for
apt.postgresql.org". Maybe at debsign time?

S

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Guido Günther writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
~version"):
> On Mon, Feb 20, 2017 at 08:05:02PM +, Ian Jackson wrote:
> > I think the asnwer to this is the same as that I gave to Wookey who
> > mentioned a workflow involving vcs tags.
> > 
> >   I think we should agree, as a project on some conventions about
> >   what debian/changelog would mean if you find it in some vcs branch
> >   (or, for that matter, a tarball or whatever someone sends you).  I
> >   definitely don't think vcs tags are the right answer.  They are not
> >   always transported with the revision and vcs-unaware tools cannot
> >   see them at all.
> > 
> > This is (almost) as true for branches as it is for tags.
> 
> It's fine to send tarballs around but it should (hopefully) rather be
> the exception to "git format-patch" or a public VCS repo. So it's IMHO
> rather the later case we should optimize for.

Even if a VCS is used, the branch name may fail to indicate the suite.
"refs/heads/tmp.for-guido" doesn't say "I mean this for experimental".
(And of course not all VCSs do branches the same way.)

It is the very contents of the tree (ie, its actual source code
contents) that mean it is suitable for a particular suite.  The
metadata that indicates this to the tools should be in-tree too.

> > > > Q3. Should the version number be the intended next version number,
> > > > or should it be decorated with ~something ?  If it should
> > > > be decorated, what should it be decorated with ?
> > > 
> > > gbp dch adds a ~.gbp by default. [...]
> > 
> > This is all fine.  What it lacks is a way to stop you accidentally
> > uploading an unfinished ~gbp snapshot, based on the version number.
> > I was proposing ~UNRELEASED.  Obviously that could be
> > ~UNRELEASED. (with N and M from your notation).
> 
> But if the suite is UNRELEASED you don't have that issue or am I missing
> something?

See above for why the suite needs to name the real suite.  We could
decorate the suite instead, but that does not have any effect on
generated binaries.  As I say, the unfinalised version of the package
should generate binaries with non-release versions.

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Guido Günther]

On Mon, Feb 20, 2017 at 08:05:02PM +, Ian Jackson wrote:
> Guido Günther writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
> ~version"):
> > On Sun, Feb 12, 2017 at 12:48:35PM +, Ian Jackson wrote:
> > > We do not seem to have a coherent approach to how to handle
> > > debian/changelog in trees (eg, vcs branches) which are not yet ready
> > > for upload.
> > > 
> > > See below two examples of things done differently.
> > > The questions are:
> > > 
> > > Q1. Should the suite in the changelog entry be UNRELEASED,
> > > or the suite at which the vcs branch is targeted ?
> > 
> > DEP14 allows for branching schemes that have the desired suite included
> > like debian/sid, debian/experimental, debian/stretch so the information
> > is included in the VCS and UNRELEASED only indicates the "not finished
> > yet" part.
> 
> I think the asnwer to this is the same as that I gave to Wookey who
> mentioned a workflow involving vcs tags.
> 
>   I think we should agree, as a project on some conventions about
>   what debian/changelog would mean if you find it in some vcs branch
>   (or, for that matter, a tarball or whatever someone sends you).  I
>   definitely don't think vcs tags are the right answer.  They are not
>   always transported with the revision and vcs-unaware tools cannot
>   see them at all.
> 
> This is (almost) as true for branches as it is for tags.

It's fine to send tarballs around but it should (hopefully) rather be
the exception to "git format-patch" or a public VCS repo. So it's IMHO
rather the later case we should optimize for.

> > > Q3. Should the version number be the intended next version number,
> > > or should it be decorated with ~something ?  If it should
> > > be decorated, what should it be decorated with ?
> > 
> > gbp dch adds a ~.gbp by default. N being the snapshot number (the
> > nth time you ran dbp dch since the last release) and  being the first
> > digits of the sha1. This allows for
> > 
> >* easy identification of snapshot builds
> >* increasing version numbers useful for e.g. CI builds.
> >* a version number that is smaller than the final release number
> >  so systems having snapshot builds can be easily upgraded to
> >  release versions
> >* a mapping back to the git commit until when the changelog was
> >  filled.
> 
> This is all fine.  What it lacks is a way to stop you accidentally
> uploading an unfinished ~gbp snapshot, based on the version number.
> I was proposing ~UNRELEASED.  Obviously that could be
> ~UNRELEASED. (with N and M from your notation).

But if the suite is UNRELEASED you don't have that issue or am I missing
something?
Cheers,
 -- Guido

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Guido Günther writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
~version"):
> On Sun, Feb 12, 2017 at 12:48:35PM +, Ian Jackson wrote:
> > We do not seem to have a coherent approach to how to handle
> > debian/changelog in trees (eg, vcs branches) which are not yet ready
> > for upload.
> > 
> > See below two examples of things done differently.
> > The questions are:
> > 
> > Q1. Should the suite in the changelog entry be UNRELEASED,
> > or the suite at which the vcs branch is targeted ?
> 
> DEP14 allows for branching schemes that have the desired suite included
> like debian/sid, debian/experimental, debian/stretch so the information
> is included in the VCS and UNRELEASED only indicates the "not finished
> yet" part.

I think the asnwer to this is the same as that I gave to Wookey who
mentioned a workflow involving vcs tags.

  I think we should agree, as a project on some conventions about
  what debian/changelog would mean if you find it in some vcs branch
  (or, for that matter, a tarball or whatever someone sends you).  I
  definitely don't think vcs tags are the right answer.  They are not
  always transported with the revision and vcs-unaware tools cannot
  see them at all.

This is (almost) as true for branches as it is for tags.

> > Q3. Should the version number be the intended next version number,
> > or should it be decorated with ~something ?  If it should
> > be decorated, what should it be decorated with ?
> 
> gbp dch adds a ~.gbp by default. N being the snapshot number (the
> nth time you ran dbp dch since the last release) and  being the first
> digits of the sha1. This allows for
> 
>* easy identification of snapshot builds
>* increasing version numbers useful for e.g. CI builds.
>* a version number that is smaller than the final release number
>  so systems having snapshot builds can be easily upgraded to
>  release versions
>* a mapping back to the git commit until when the changelog was
>  filled.

This is all fine.  What it lacks is a way to stop you accidentally
uploading an unfinished ~gbp snapshot, based on the version number.
I was proposing ~UNRELEASED.  Obviously that could be
~UNRELEASED. (with N and M from your notation).

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Guido Günther]

Hi Ian,
On Sun, Feb 12, 2017 at 12:48:35PM +, Ian Jackson wrote:
> We do not seem to have a coherent approach to how to handle
> debian/changelog in trees (eg, vcs branches) which are not yet ready
> for upload.
> 
> See below two examples of things done differently.
> The questions are:
> 
> Q1. Should the suite in the changelog entry be UNRELEASED,
> or the suite at which the vcs branch is targeted ?

DEP14 allows for branching schemes that have the desired suite included
like debian/sid, debian/experimental, debian/stretch so the information
is included in the VCS and UNRELEASED only indicates the "not finished
yet" part.

> Q2. Should the changelog entry be finalised ?  That is, should it
> have an uploader name and date ?
> 
> Q3. Should the version number be the intended next version number,
> or should it be decorated with ~something ?  If it should
> be decorated, what should it be decorated with ?

gbp dch adds a ~.gbp by default. N being the snapshot number (the
nth time you ran dbp dch since the last release) and  being the first
digits of the sha1. This allows for

   * easy identification of snapshot builds
   * increasing version numbers useful for e.g. CI builds.
   * a version number that is smaller than the final release number
 so systems having snapshot builds can be easily upgraded to
 release versions
   * a mapping back to the git commit until when the changelog was
 filled.

Cheers,
 -- Guido

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Guido Günther]

On Sun, Feb 12, 2017 at 02:11:12PM +, Simon McVittie wrote:
[..snip..]
> with a middle ground that is less theoretically defensible than
> either, but pragmatically quite useful:
> 
> * Mostly write the changelog later, as in the second model.
>   Periodically summarize the changelog so far (since the last
>   commit to the changelog) and commit it as an UNRELEASED
>   changelog entry, providing a reference point.
>   `gbp dch --since=1234567` is useful for this model.

gbp dch also checks when the changelog was last touched so this mixed
mode should work without even using --since. This should help people
using a mixed model.
Cheers,
 -- Guido

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Wookey writes ("Re: changelog practice, unfinalised vs UNRELEASED vs ~version"):
> I'd be happier if I didn't have to deal with 'UNRELEASED' at all
> unless I actually wanted to, so I'm not at all keen on your suggestion
> of making it mandatory.

Well, tools can be configured, too.  If my UNRELEASED version proposal
is adopted, we could easily let you have a DEB_ENABLE_FOOTGUN variable
that will let you become confused about what binaries and sources are
what, by reusing version numbers from ad-hoc test builds.

> 1. I really dislike dch's enthusiasm for putting in 'UNRELEASED'. It
> gives me nothing I wanted, and just provides the opportunity to really
> do a final, clean, tested, build, only to find on upload that it's
> still marked 'UNRELASED', and I have to do the build, test, upload
> step again - for big packages that is a huge pain and happens way too
> often. I really resent that there is no way to do dch -i;dch -r in one
> go - the separation of these is just makework for me. 

Does that mean that if you tested ~UNRELEASED, and it passed all your
tests, you would be unhappy to strip the ~UNRELEASED from the version
and upload it ?

I often do that.  My tools make it very hard for me to mess this up by
uploading anything whose source tree (besides the changelog and
therefore besides the version number) differs to what I tested.

(Of course in principle there might be situations where version
X~UNRELEASED would be treated differently to X.  If that's likely to
be going on then I would have to, regrettably, use the actual
for-release version number for your ad-hoc tests.)

> > Proposal:
> > 
> >  * Tools which add/edit changelog change items should insist that the
> >changelog is unfinalised and contains ~UNRELEASED in its version.
> 
> I really don't want this to be _required_. Dicking with the version is
> better than dicking with the suite, but I really would prefer it if
> the tools did neither, or could simply be asked to.
> 
> I realise that I'm pushing uphill somewhat here and no-one much cares
> about people who still don't use git if they don't have to. But the
> UNRELEASED/'dch -r' thing pisses me off on a daily basis, and this
> seemed like the time to point out that some of us don't find it all
> helpful. From that POV, moving it from suite to version would
> definitely be less annoying.

Well, I guess you'd be satisfied with an option to disable this
behaviour.

But: it seems that intend to make commits whose debian/changelog has a
trailer line, a real suite, and a real version, but where the commit
does not contain all the changes ?  I think that's poor practice.
These commits are a liability.  If you push such a commit, and then
later make more changes and do the upload, but forget to to push, a
subsequent upload made from the vcs might lack your changes.

Of course if you really insist you can do this, and probably if the
commits are buried in the history so that no-one every sees them
except when doing archaeology, then there is little chance of your
metadata misleading anyone but you into a mistake.

I don't know how you avoid mistakes.  Perhaps you're just very
careful.

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Sean Whitton writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
~version"):
> On Sun, Feb 12, 2017 at 12:48:35PM +, Ian Jackson wrote:
> > Q1. Should the suite in the changelog entry be UNRELEASED,
> > or the suite at which the vcs branch is targeted ?
> > [...]
> > Q1: Replacing the suite with "UNRELEASED" means that it is not
> > possible to distinguish a branch intended for experimental from one
> > intended for sid, other than by branch name (and of course branch
> > names are often used for other purposes).  I think this is very
> > undesirable.  AFAICT only reason to use UNRELEASED is to prevent
> > accidental upload, but maybe we can do that some other way.
> 
> Why do you think this is "very" undesirable?  At worst, you have to ask
> the person who committed to the branch what their intention was, and
> you're probably talking to them anyway.

No, at worst you upload to unstable something that ought to have gone
to experimental, because the tooling failed to notice your mistake.
If you had been able to note somehow in the relevant vcs branch that
it was intended for experimental, you have a much better chance of
having done so - and the tools would then be able to catch it.

> > Q2. Should the changelog entry be finalised ?  That is, should it
> > have an uploader name and date ?
> 
> Just a terminological point:
> 
> I use "finalised" to mean "suite is not UNRELEASED and timestamp is not
> behind the last change" -- i.e. `dch -r` has been run and the result
> committed.
> 
> I suspect I'm not alone in using "finalised" to refer to this, given the
> behaviour of `dch`.  It is only the Emacs changelog mode which leaves no
> name and e-mail.

There's two meanings of "finalised" then.

One is "contains name, email address, and date of purported decision
to upload".  (Let's call that "with trailer" from now on.)

The other meaning is "has been updated to reflect decision to upload,
including recording name and email of the person making that decison
and the time they did so".  (Let's call that "upload decided".)

I think that recording a false or dummy set of information about a
purported upload decision is a silly way of recording the fact that
the upload decision has not, in fact, been taken.

I think tools and people only do it this way because some other tools
have been grumpy about changelogs without trailers.  This is also
silly, since they're all our tools and we could just fix them.

> > Q3. Should the version number be the intended next version number,
> > or should it be decorated with ~something ?  If it should
> > be decorated, what should it be decorated with ?
> > [...]
> > Q3. If the version number is not decorated, then binaries (or even
> > source packages!) built for testing (or for other reasons other than
> > upload) are not always distinguishable from intended-for-release
> > builds.  IMO this is very undesirable.
> 
> If we use dgit for all our uploads then this doesn't really matter :)

That's not true.  I would like to know, for example, whether the
version of sysvinit I have running on my laptop is the version I
finally uploaded to sid, or some pre-release which might not be
identical to sid's.

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Simon McVittie writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
~version"):
> On Mon, 13 Feb 2017 at 09:42:32 +1100, Ben Finney wrote:
> > I don't see how this complaint is any different from the need to merge,
> > for example, changes to API documentation and test cases that accompany
> > a functional change.
> 
> It's the difference between "sometimes conflicts" and "always conflicts".

This is a very real problem for debian/changelog, but mergechangelogs
helps a lot.  In any case as I say I'm not trying to mandate the
workflow where the changelog is updated as-we-go.

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ian Jackson]

Simon McVittie writes ("Re: changelog practice, unfinalised vs UNRELEASED vs 
~version"):
> If I'm understanding correctly, your motivation to do so is that you
> have a strong belief that building a Debian source package with `debuild`
> or similar, directly from a VCS checkout, should always work and should
> always produce results that could be considered correct (in terms of not
> having the version number of a prior version, not having the version
> number of a future version either, not claiming to have been released
> by a person who did not in fact release it, and so on).

I wouldn't say I have a "strong belief" in these notions.  I mean, I
do broadly agree with them.

I think it is much more tolerable to have build reuse the version
number of an actually-uploaded previous version.  This is because in
that case the uploaded version is still available via the archive.  So
confusion is less likely.  But I still don't think it's very
desirable.

> Broadly, the two extremes of workflows for Debian packages' changelogs
> maintained in git seem to be:
> 
> * Write the changelog as you go along: [...]
> * Write the changelog later: each commit just has a commit message
>   in a normal git way, and its debian/changelog is out of date. [...]
> * Mostly write the changelog later, as in the second model.
>   Periodically summarize the changelog so far [...]

Contrary to what Ben seems to have read into my message, I don't have
an opinion about this.  I use all three of these workflows on
different occasions.  I don't understand why you and Ben see a
connection between my suggestions and the choice between these
workflows.

> While as an abstract model I agree that the uploader name and date
> are not meaningful in an unreleased version, I can't help thinking
> that this is a "boil the ocean" sort of change - a lot of tools follow
> and require Policy syntax, in which the uploader name and date are
> non-optional.

Policy can be changed.  Obviously what I am proposing should be
written in policy.

> Allowing and ignoring an arbitrary and non-meaningful uploader and date,
> or possibly establishing a convention like Unreleased <nobody@localhost>
> for the unreleased uploader, seems more pragmatic.

I don't understand why it is better to have dummy (or wrong) data than
no data.  Of course there will be a bit of work to update our tools
but there is a fairly limited set of them and the changes are not
difficult.

> Our experience with this script and workflow has mostly been very positive,
> and I intend to integrate similar functionality into Vectis[2] soon. The
> one thing I'm regretting is making it switch behaviour based on whether
> the current version is tagged; I think that's "too much magic", and it
> forces you to tag a version before you have built and tested it.

I confess I don't understand all of this, but I think we should agree,
as a project on some conventions about what debian/changelog would
mean if you find it in some vcs branch (or, for that matter, a tarball
or whatever someone sends you).  I definitely don't think vcs tags are
the right answer.  They are not always transported with the revision
and vcs-unaware tools cannot see them at all.

> Clearly, the down side of this approach is that it doesn't work as-is
> with debuild or similar, violating what I believe to be the axiom you're
> working from - the developer has to run a special script instead.

I think we should be aiming to reduce the need for special scripts.

I think our current practices (and tooling) are mostly a mess, and
when they are not a mess they are quite suboptimal.  If we want to
improve this, it can't be done by telling everyone to "just run this
script".

> > Q1. Should the suite in the changelog entry be UNRELEASED,
> > or the suite at which the vcs branch is targeted ?
> 
> If you're trying to change common practice (being prescriptive rather than
> descriptive) *anyway*, maybe something like experimental-UNRELEASED that
> contains both, with UNRELEASED being shorthand for unstable-UNRELEASED
> (or possibly ${current development suite}-UNRELEASED in Ubuntu
> and other derivatives)?

This would be another approach.  But it is not as good as decorating
the version, because it's not usually desirable to generate binaries
with version numbers that will be reused.

Ian.

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Scott Kitterman]

On February 12, 2017 3:37:59 PM EST, Luca Capello  wrote:
>Hi there,
>
>On Sun, 12 Feb 2017 14:11:12 +, Simon McVittie wrote:
>> On Sun, 12 Feb 2017 at 12:48:35 +, Ian Jackson wrote:
>> > What do people think ?
...
>
>> I don't think Debian would be as
>> large or successful a project as it is if we restricted our
>contributor
>> base to people who know the corner cases as well as Ian Jackson does
>:-)
>
>IMHO Debian is so successful exactly because we have not imposed any
>constraint, as Holger already pointed out:
>
>
>
>Thx, bye,
>Gismo / Luca

+1. So far I don't see any value in adding more constraints on how developers 
do their work.

Scott K

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Gunnar Wolf]

Sean Whitton dijo [Mon, Feb 13, 2017 at 04:33:15PM -0700]:
> > So, my idea was, in short: Thinking in a post-Buster world, do we even
> > need the finalized line? I mean, take a look at debian/changes. The
> > archive handling tools do get both «Date» and «Changed-By» fields,
> > which reflect when the package was last *built* (which has a much
> > clearer meaning than a nondescript finalization line). Debian tools
> > can act from there. We could then just remove this dissonant bit :-)
> 
> The final line of a finalised changelog indicates who signed off on the
> package: the person who said "it's time to upload this".

At least according to some readings. Up to now, I never gave any
thought to this line; usually dch updates the date for me, but I often
upload packages "signed by" others, or the opposite.

> I think that we should continue to record the person who made that
> judgement.  Someone who made a small change and whose name appears [in
> square brackets ] shouldn't automatically have responsibility for the
> whole upload -- but *someone* should have that overarching
> responsibility.

I see, and it is a valuable reading. I wonder if I'm alone in not
considering it important so far (after all, I've only been a DD for 14
years...)


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Gunnar Wolf]

Sean Whitton dijo [Mon, Feb 13, 2017 at 04:35:19PM -0700]:
> > (Of course, the signoff line in the changelog is redundant with
> > the GPG signature, which is what actually matters but isn't at all
> > user-visible...)
> 
> It's not redundant for sponsored uploads where the sponsor is not a
> member of the relevant team (which are quite common on
> d-mentors@lists.d.o).  The sponsee needs to sign off on the whole
> upload, in addition to the sponsor.

Right, I didn't think about that. Maybe it's time I go back to
sponsoring uploads to understanda gain that side of Debian ;-)


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Sean Whitton]

Hello Simon,

On Mon, Feb 13, 2017 at 05:14:17PM +, Simon McVittie wrote:
> (Of course, the signoff line in the changelog is redundant with
> the GPG signature, which is what actually matters but isn't at all
> user-visible...)

It's not redundant for sponsored uploads where the sponsor is not a
member of the relevant team (which are quite common on
d-mentors@lists.d.o).  The sponsee needs to sign off on the whole
upload, in addition to the sponsor.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Sean Whitton]

On Mon, Feb 13, 2017 at 10:53:18AM -0600, Gunnar Wolf wrote:
> So, my idea was, in short: Thinking in a post-Buster world, do we even
> need the finalized line? I mean, take a look at debian/changes. The
> archive handling tools do get both «Date» and «Changed-By» fields,
> which reflect when the package was last *built* (which has a much
> clearer meaning than a nondescript finalization line). Debian tools
> can act from there. We could then just remove this dissonant bit :-)

The final line of a finalised changelog indicates who signed off on the
package: the person who said "it's time to upload this".

I think that we should continue to record the person who made that
judgement.  Someone who made a small change and whose name appears [in
square brackets ] shouldn't automatically have responsibility for the
whole upload -- but *someone* should have that overarching
responsibility.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Holger Levsen]

On Mon, Feb 13, 2017 at 10:53:18AM -0600, Gunnar Wolf wrote:
> So, my idea was, in short: Thinking in a post-Buster world, do we even
> need the finalized line? I mean, take a look at debian/changes. The
 
the changes file(s) is/are not part of the source packages and we are using
the last date in debian/changelog to set SOURCE_DATE_EPOCH during (re)builds.

https://reproducible-builds.org/specs/source-date-epoch/

I bet there are lot of other consumers of that field as well.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Simon McVittie]

On Mon, 13 Feb 2017 at 10:53:18 -0600, Gunnar Wolf wrote:
> Interesting discussion. This (and not particularly your message, but
> this whole thread even leads me to questioning: Does our "finalized"
> changelog lines make *any* sense nowadays?

In the actual upload to the archive (as opposed to the version
under preparation in a VCS) it has value: it's the maintainer
responsible for deciding this unfinished change is ready to be
released to actual users. They might not even have contributed
any code: I'm pretty sure I've done team uploads where my only
input was to decide that my co-maintainers' actions added up to
a useful package update to send to the archive, and take
responsibility for that.

(Of course, the signoff line in the changelog is redundant with
the GPG signature, which is what actually matters but isn't at all
user-visible...)

> > * Write the changelog later: each commit just has a commit message
> >   in a normal git way, and its debian/changelog is out of date.
> >   At release time, write a cumulative debian/changelog entry for
> >   everything that happened since the last release, finalize it and
> >   commit it. The `gbp dch` command assumes this model (and is very
> >   useful when following it).
> 
> In the specific case of this team, we could most probably compose
> debian/changelog by reading git log since the last tag. But... I am
> not convinced I want to be constrained by this!

I'm not saying that should be *required*, only that it should be
*allowed*. In the work project where I'm closest to consistently using
this model, I often summarize the changelog a bit when making a release
rather than using `gbp dch` as-is, merging multiple commits' changelog
entries into a brief description of the feature that they add up to -
but the time of the release is a good time to do that, because then,
I have an overview of everything that happened since the last release.

S

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Gunnar Wolf]

Simon McVittie dijo [Sun, Feb 12, 2017 at 02:11:12PM +]:
> On Sun, 12 Feb 2017 at 12:48:35 +, Ian Jackson wrote:
> > What do people think ?
> 
> I think you're the only person I've ever seen using unfinalized
> changelog entries for Debian packages.
> 
> If I'm understanding correctly, your motivation to do so is that you
> have a strong belief that building a Debian source package with `debuild`
> or similar, directly from a VCS checkout, should always work and should
> always produce results that could be considered correct (in terms of not
> having the version number of a prior version, not having the version
> number of a future version either, not claiming to have been released
> by a person who did not in fact release it, and so on).
> 
> These might be valid axioms for your particular workflow, but they do
> not fit all workflows, and I don't think they are necessarily the
> axioms that lead to the best practical results.

Interesting discussion. This (and not particularly your message, but
this whole thread even leads me to questioning: Does our "finalized"
changelog lines make *any* sense nowadays?

Let me explain. I think this line has clear signs of days long past:

 -- Gunnar Wolf   Mon, 13 Feb 2017 10:37:57 +0600

Yes, in some way it summarizes who did the last (or first? or n-th?)
modification to the changelog entry in case. But, given we see
team-maintained workflows as preferable, it is very common to also see
the following in the lines behind it:

  [ Gunnar Wolf ]
  * Frobbed the foobarnicators
  * Oiled up the grease

  [ Other Maintainer ]
  * Replaced quux with blah (Closes: #876543)

A text line documenting who (something)ed (first|last) with the
changelog is not really relevant. The date is even treacherous; it
could have been introduced by me when frobbing up the
foorbarnicators. There is no indication as to whether Other did his
work before I oiled up the grease — at least in debian-keyring we have
the habit of grouping maintainer messages (by using dch
--multimaint-merge) instead of keeping time-order. Maybe this would be
the real sequence of events in my example changelog:

  [ Gunnar Wolf ]
  * Frobbed the foobarnicators

  [ Other Maintainer ]
  * Replaced quux with blah (Closes: #876543)

  [ Gunnar Wolf ]
  * Oiled up the grease

But it creates too much unnecessary and (at least in some aspects)
redundant noise.

But... Yes, even though in our case (debian-keyring) the changelog
closely follows the Git commit messages (the first line matches for
all "routine" changelog entries), debian/changelog and git log have
somewhat different and general meanings.

> * Write the changelog later: each commit just has a commit message
>   in a normal git way, and its debian/changelog is out of date.
>   At release time, write a cumulative debian/changelog entry for
>   everything that happened since the last release, finalize it and
>   commit it. The `gbp dch` command assumes this model (and is very
>   useful when following it).

In the specific case of this team, we could most probably compose
debian/changelog by reading git log since the last tag. But... I am
not convinced I want to be constrained by this!

Anyway, I'm steering quite a bit off the track

> > Q2. Should the changelog entry be finalised ?  That is, should it
> > have an uploader name and date ?
> 
> While as an abstract model I agree that the uploader name and date
> are not meaningful in an unreleased version, I can't help thinking
> that this is a "boil the ocean" sort of change - a lot of tools follow
> and require Policy syntax, in which the uploader name and date are
> non-optional. Obviously, Policy only really applies to finished packages,
> and unfinished packages often violate the semantics of Policy (for
> instance by using UNRELEASED as a suite name); but it seems reasonable
> for a tool author to oppose changes that, as well as violating Policy
> semantics, also violate Poliy syntax.

So, my idea was, in short: Thinking in a post-Buster world, do we even
need the finalized line? I mean, take a look at debian/changes. The
archive handling tools do get both «Date» and «Changed-By» fields,
which reflect when the package was last *built* (which has a much
clearer meaning than a nondescript finalization line). Debian tools
can act from there. We could then just remove this dissonant bit :-)



signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Russ Allbery]

Ben Finney  writes:
> Russ Allbery  writes:

>> I really want something that will pass Lintian completely but that
>> dput will refuse to upload, which is what UNRELEASED currently
>> accomplishes.

> Wookey  writes:

>> 1. I really dislike dch's enthusiasm for putting in 'UNRELEASED'. It
>> gives me nothing I wanted, and just provides the opportunity to really
>> do a final, clean, tested, build, only to find on upload that it's
>> still marked 'UNRELASED', and I have to do the build, test, upload
>> step again - for big packages that is a huge pain and happens way too
>> often.

> Those two positions seem incompatible as described.

> Can the two of you discuss further what it would take to reconcile
> what each of you wants from the changelog-adjacent tools?

As previously mentioned, it's configurable behavior of dch (which I don't
use anyway), so we can both be happy.  (I don't use dch anyway.)

Having to repeat the build and test cycle one last time when I'm ready to
upload is a *feature*, as far as I'm concerned, not a bug.  I use that
intentionally.

I'm pretty happy with the current workflow, so as long as no one breaks
it, I don't really have any worries.  :)  I have a mild preference against
messing about with it, since it already works for me, but if that happens,
I suppose I'll figure it out.

-- 
Russ Allbery (r...@debian.org)   

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ben Finney]

Simon McVittie  writes:

> On Mon, 13 Feb 2017 at 09:42:32 +1100, Ben Finney wrote:
> > I'm in agreement with Ian that the “write the documentation
> > (including the changelog) along with the change it describes”
> > workflow should have full support from our tools.
>
> Are you making the stronger assertion, as Ian seems to be, that this
> workflow should be the *only* thing that has full support from our
> tools?

On that, I'm currently undecided. Thanks for distinguishing those two
positions.

-- 
 \  “Writing a book is like washing an elephant: there no good |
  `\place to begin or end, and it's hard to keep track of what |
_o__)  you've already covered.” —anonymous |
Ben Finney

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Simon McVittie]

On Mon, 13 Feb 2017 at 09:42:32 +1100, Ben Finney wrote:
> Simon McVittie  writes:
> >   This works fine if every commit is final and immutable and is sent
> >   directly to the master VCS immediately, but works very poorly if you
> >   are proposing commits for someone else to merge at a later date -
> 
> I don't see how this complaint is any different from the need to merge,
> for example, changes to API documentation and test cases that accompany
> a functional change.

It's the difference between "sometimes conflicts" and "always conflicts".

If Alice and Bob are concurrently updating API documentation and test
cases, and they aren't particularly unlucky or working on particularly
adjacent bits of code, Alice's changes will often not touch exactly the
same parts of the same files as Bob's changes. (If they are working on
adjacent code then they probably need to be talking to each other
anyway.)

If they are concurrently updating debian/changelog or a GNU-style
ChangeLog at the same as developing and documenting, the right place
for Alice to document recent changes (appending to the non-finalized
debian/changelog entry, or prepending to a GNU ChangeLog) is exactly the
same as the right place for Bob to document recent changes, guaranteeing
a collision.

> I'm in agreement with Ian that the “write the documentation (including
> the changelog) along with the change it describes” workflow should have
> full support from our tools.

Are you making the stronger assertion, as Ian seems to be, that this
workflow should be the *only* thing that has full support from our tools?

I am not against tools supporting writing the changelog in advance;
that's what I mostly still do in my own Debian packages, although I am
increasingly unconvinced that it's the best model in general. I *am*
against tools that aim to be universally used, but *only* support that
model of operation.

S

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ben Finney]

Russ Allbery  writes:

> I really want something that will pass Lintian completely but that
> dput will refuse to upload, which is what UNRELEASED currently
> accomplishes.

Wookey  writes:

> 1. I really dislike dch's enthusiasm for putting in 'UNRELEASED'. It
> gives me nothing I wanted, and just provides the opportunity to really
> do a final, clean, tested, build, only to find on upload that it's
> still marked 'UNRELASED', and I have to do the build, test, upload
> step again - for big packages that is a huge pain and happens way too
> often.

Those two positions seem incompatible as described.

Can the two of you discuss further what it would take to reconcile
what each of you wants from the changelog-adjacent tools?

-- 
 \  “Often, the surest way to convey misinformation is to tell the |
  `\   strict truth.” —Mark Twain, _Following the Equator_ |
_o__)  |
Ben Finney

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Ben Finney]

Simon McVittie  writes:

> I think you're the only person I've ever seen using unfinalized
> changelog entries for Debian packages.

I think the fact you don't know we're using this workflow isn't very
informative :-)

> Broadly, the two extremes of workflows for Debian packages' changelogs
> maintained in git seem to be:
>
> * Write the changelog as you go along: each commit includes its own
>   Debian changelog entry.

Yes. The Debian changelog is documentation, and describes a particular
state of the package. When that state changes, it's a fine practice to
immediately update the documentation describing the package.

>   This works fine if every commit is final and immutable and is sent
>   directly to the master VCS immediately, but works very poorly if you
>   are proposing commits for someone else to merge at a later date -

I don't see how this complaint is any different from the need to merge,
for example, changes to API documentation and test cases that accompany
a functional change.

> * Write the changelog later: each commit just has a commit message
>   in a normal git way, and its debian/changelog is out of date.

For the record, I don't have anything against that workflow either. I
disagree that it should be preferred.

> I'm concerned that the first model is optimized for people who know
> Debian as well as you do, and do not need pre-commit review because
> they get everything right first time.

That concern is, I'd think, addressed by realising changes are not
immutable; a new change is merely an additional commit away. Resolving a
conflict in documentation isn't especially onerous compared with the
general resolution of conflicts in a collaborative code base.

I'm in agreement with Ian that the “write the documentation (including
the changelog) along with the change it describes” workflow should have
full support from our tools.

-- 
 \  “We tend to scoff at the beliefs of the ancients. But we can't |
  `\scoff at them personally, to their faces, and this is what |
_o__) annoys me.” —Jack Handey |
Ben Finney

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Luca Capello]

Hi there,

On Sun, 12 Feb 2017 14:11:12 +, Simon McVittie wrote:
> On Sun, 12 Feb 2017 at 12:48:35 +, Ian Jackson wrote:
> > What do people think ?
> 
> I think you're the only person I've ever seen using unfinalized
> changelog entries for Debian packages.

FWIW, I am another user of this workflow...

  


> If I'm understanding correctly, your motivation to do so is that you
> have a strong belief that building a Debian source package with `debuild`
> or similar, directly from a VCS checkout, should always work and should
> always produce results that could be considered correct (in terms of not
> having the version number of a prior version, not having the version
> number of a future version either, not claiming to have been released
> by a person who did not in fact release it, and so on).

...and agree as well on the above statements, adding that IMHO releasing
something should always be a manual process, i.e. a real person should
be able to be blamed for ;-)

I have discussed a lot about my workflow with other fellow Debian
members, always because it disrupts automatic building from VCS.  Well,
for this situation IMHO the automatic builder should first calls `dch
-r` with appropriate uploader name/email (e.g. "VCS autobuilder
") and, whenever possible, guess the
correct distribution.

> I'm concerned that the first model is optimized for people who know
> Debian as well as you do, and do not need pre-commit review because
> they get everything right first time.

I also prefer the first model, i.e. writing debian/changelog entries as
you go along, and I do not see any problem in fixing a previous commit
without changing the corresponding debian/changelog entry.

The debian/changelog is the history of changes for the Debian source
package (.orig.tar.$COMPRESSION, .debian.tar.$COMPRESSION and .dsc). The
VCS changelog is the history of changes to achieve the former (which can
be one or more commits or even a complete branch).  Inflating these 2
histories is IMHO wrong.

Which is also the reason why I do not like debian/changelog with Git
commits entries.

> I don't think Debian would be as
> large or successful a project as it is if we restricted our contributor
> base to people who know the corner cases as well as Ian Jackson does :-)

IMHO Debian is so successful exactly because we have not imposed any
constraint, as Holger already pointed out:

  

Thx, bye,
Gismo / Luca


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[James McCoy]

On Sun, Feb 12, 2017 at 06:34:26PM +, Wookey wrote:
> But the
> UNRELEASED/'dch -r' thing pisses me off on a daily basis, and this
> seemed like the time to point out that some of us don't find it all
> helpful. From that POV, moving it from suite to version would
> definitely be less annoying.
> 
> I suppose I should file a wishlist bug about dch's annoying
> 'UNRELEASED' behaviour, and lack of a workaround.

$ echo 'DEBCHANGE_RELEASE_HEURISTIC=log' >> ~/.devscripts

Now dch won't use UNRELEASED and will simply look for a log of an upload
in the parent directory to determine whether the version is released or
not.  The suite will be set to whatever is deemed appropriate based on
the previous changelog entry or the -D switch, instead of using
UNRELEASED.

Cheers,
-- 
James
GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7  2D23 DFE6 91AE 331B A3DB

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Wookey]

On 2017-02-12 12:48 +, Ian Jackson wrote:
> We do not seem to have a coherent approach to how to handle
> debian/changelog in trees (eg, vcs branches) which are not yet ready
> for upload.
> 
> See below two examples of things done differently.
> The questions are:
> 
> Q1. Should the suite in the changelog entry be UNRELEASED,
> or the suite at which the vcs branch is targeted ?
> 
> Q2. Should the changelog entry be finalised ?  That is, should it
> have an uploader name and date ?
> 
> Q3. Should the version number be the intended next version number,
> or should it be decorated with ~something ?  If it should
> be decorated, what should it be decorated with ?

I don't have strong opinions on this, but as someone who still largely
uses tarball-based processes, and git-based ones only when working
with other people, there are things I feel it's vital to preserve, and
things I'm very happy to see go.

1. I really dislike dch's enthusiasm for putting in 'UNRELEASED'. It
gives me nothing I wanted, and just provides the opportunity to really
do a final, clean, tested, build, only to find on upload that it's
still marked 'UNRELASED', and I have to do the build, test, upload
step again - for big packages that is a huge pain and happens way too
often. I really resent that there is no way to do dch -i;dch -r in one
go - the separation of these is just makework for me. 

I _always_ want to bump my version (dch -i) before I change _anything_
because otherwise I'll accidentally overwrite the old version on
build, so can't easily make patches/check diffs. I _don't_ want to
change the targetted suite from unstable very often at all.  Your
suggestion of putting UNRELEASED in the version rather than the suite
would be a major improvement (because it's much more obvious when
building/testing). So overall your proposal is an improvement on the
status quo, but I really don't like the status quo much.

I'd be happier if I didn't have to deal with 'UNRELEASED' at all
unless I actually wanted to, so I'm not at all keen on your suggestion
of making it mandatory.

I wasn't aware that unfinalised changelog entries were a thing at
all. In my experience if they aren't perfectly formatted (which is why
I took to using dch in the first place) something will complain and
you can can't build the package. They make sense to me as a concept,
although I'm currently perfectly happy that the date in there tells me
when I started work on this new release; that seems fair enough, but I
do agree that removing them to make patch (or git) merging easier
makes sense. Gratuitous changelog conflicts are annoying.

> Proposal:
> 
>  * Tools which add/edit changelog change items should insist that the
>changelog is unfinalised and contains ~UNRELEASED in its version.

I really don't want this to be _required_. Dicking with the version is
better than dicking with the suite, but I really would prefer it if
the tools did neither, or could simply be asked to.

I realise that I'm pushing uphill somewhat here and no-one much cares
about people who still don't use git if they don't have to. But the
UNRELEASED/'dch -r' thing pisses me off on a daily basis, and this
seemed like the time to point out that some of us don't find it all
helpful. From that POV, moving it from suite to version would
definitely be less annoying.

I suppose I should file a wishlist bug about dch's annoying
'UNRELEASED' behaviour, and lack of a workaround.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Russ Allbery]

Ian Jackson  writes:

> Q2: The information in the trailer line of the changelog entry in a new
> branch is not meaningful.  Depending on the vcs etc. in use, it can
> cause pointless merge conflicts.  Often the information is quite wrong:
> for example, a date which is earlier than the most recent commit.  The
> Emacs changelog mode doesn't like editing finalised changelog entries.
> IME several of our tools deal badly with unfinalised changelog entries.
> For example, mergechangelogs mishandles them, and some parts of the
> package building toolchain sometimes complain.  However, such bugs can
> be fixed and the fixes deployed fairly easily (via backports if
> necessary).

It also throws a bunch of Lintian errors, although I suppose that's
fixable as well.  I really want something that will pass Lintian
completely but that dput will refuse to upload, which is what UNRELEASED
currently accomplishes.

-- 
Russ Allbery (r...@debian.org)   

Pocket/suite terminology (was Re: changelog practice, unfinalised vs UNRELEASED vs ~version)

[Colin Watson]

On Sun, Feb 12, 2017 at 02:11:12PM +, Simon McVittie wrote:
> On Sun, 12 Feb 2017 at 12:48:35 +, Ian Jackson wrote:
> > Q1. Should the suite in the changelog entry be UNRELEASED,
> > or the suite at which the vcs branch is targeted ?
> 
> If you're trying to change common practice (being prescriptive rather than
> descriptive) *anyway*, maybe something like experimental-UNRELEASED that
> contains both, with UNRELEASED being shorthand for unstable-UNRELEASED
> (or possibly ${current development suite}-UNRELEASED in Ubuntu
> and other derivatives)?

I like this.

> That makes these special pseudo-suites look a lot like a partial suite
> (or a pocket in Ubuntu terminology, for example jessie-backports or
> stretch-proposed-updates), which I think they sort of are, if you
> look at them from the right angle.

Slight quibble: in Ubuntu's terminology, "pocket" actually refers to the
part after the "-", i.e. the axis within the space of suites that
roughly indicates the expected stability level of packages in a given
suite rather than the release (or "series") that they're for.
"zesty-proposed" (for example) is still correctly referred to as a
suite, which can be decomposed into series and pocket.

(IMO the "pocket" terminology is terrible and it's no wonder people are
confused by it, but it's unfortunately entrenched.  It *is* useful to
have a distinct term for that segment of the suite for tools such as
Launchpad that need to apply policy to the same pocket in multiple
series, such as "can't change -updates when the series hasn't been
released yet" or "-backports is published with NotAutomatic".  If we
were going to go to the effort of renaming it nowadays then I suspect
we'd probably pick something like "channel".)

-- 
Colin Watson   [cjwat...@debian.org]

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Sean Whitton]

Hello Ian,

On Sun, Feb 12, 2017 at 12:48:35PM +, Ian Jackson wrote:
> Q1. Should the suite in the changelog entry be UNRELEASED,
> or the suite at which the vcs branch is targeted ?
> [...]
> Q1: Replacing the suite with "UNRELEASED" means that it is not
> possible to distinguish a branch intended for experimental from one
> intended for sid, other than by branch name (and of course branch
> names are often used for other purposes).  I think this is very
> undesirable.  AFAICT only reason to use UNRELEASED is to prevent
> accidental upload, but maybe we can do that some other way.

Why do you think this is "very" undesirable?  At worst, you have to ask
the person who committed to the branch what their intention was, and
you're probably talking to them anyway.

(I don't disagree that it could be useful to have the intended suite
indicated.)

> Q2. Should the changelog entry be finalised ?  That is, should it
> have an uploader name and date ?

Just a terminological point:

I use "finalised" to mean "suite is not UNRELEASED and timestamp is not
behind the last change" -- i.e. `dch -r` has been run and the result
committed.

I suspect I'm not alone in using "finalised" to refer to this, given the
behaviour of `dch`.  It is only the Emacs changelog mode which leaves no
name and e-mail.

> Q3. Should the version number be the intended next version number,
> or should it be decorated with ~something ?  If it should
> be decorated, what should it be decorated with ?
> [...]
> Q3. If the version number is not decorated, then binaries (or even
> source packages!) built for testing (or for other reasons other than
> upload) are not always distinguishable from intended-for-release
> builds.  IMO this is very undesirable.

If we use dgit for all our uploads then this doesn't really matter :)

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Holger Levsen]

On Sun, Feb 12, 2017 at 12:48:35PM +, Ian Jackson wrote:
> We do not seem to have a coherent approach to how to handle
> debian/changelog in trees (eg, vcs branches) which are not yet ready
> for upload.
 
True. We also don't have one recommended way how to write debian/rules, how to
build packages, nor if and which SCM to use and we neither agree on a 
recommended
debian/source/format, to name a few of the things were Debian is technically
very diverse. Maybe too diverse… OTOH this is certainly also a strength.

I'd just like to see more commonly agreed and recommended best practices in
our world. And while we were doing this that we wouldn't have to be ignorant
about alternative realities^wapproaches.

:)


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: changelog practice, unfinalised vs UNRELEASED vs ~version

[Simon McVittie]

On Sun, 12 Feb 2017 at 12:48:35 +, Ian Jackson wrote:
> What do people think ?

I think you're the only person I've ever seen using unfinalized
changelog entries for Debian packages.

If I'm understanding correctly, your motivation to do so is that you
have a strong belief that building a Debian source package with `debuild`
or similar, directly from a VCS checkout, should always work and should
always produce results that could be considered correct (in terms of not
having the version number of a prior version, not having the version
number of a future version either, not claiming to have been released
by a person who did not in fact release it, and so on).

These might be valid axioms for your particular workflow, but they do
not fit all workflows, and I don't think they are necessarily the
axioms that lead to the best practical results.

Broadly, the two extremes of workflows for Debian packages' changelogs
maintained in git seem to be:

* Write the changelog as you go along: each commit includes its own
  Debian changelog entry. The `debcommit` command assumes this model.
  This works fine if every commit is final and immutable and is sent
  directly to the master VCS immediately, but works very poorly if you
  are proposing commits for someone else to merge at a later date -
  a lot like how the GNU model of writing ChangeLog as you go along is
  widely considered deprecated, because it guarantees that every git
  cherry-pick or merge will have conflicts in ChangeLog.

* Write the changelog later: each commit just has a commit message
  in a normal git way, and its debian/changelog is out of date.
  At release time, write a cumulative debian/changelog entry for
  everything that happened since the last release, finalize it and
  commit it. The `gbp dch` command assumes this model (and is very
  useful when following it).

with a middle ground that is less theoretically defensible than
either, but pragmatically quite useful:

* Mostly write the changelog later, as in the second model.
  Periodically summarize the changelog so far (since the last
  commit to the changelog) and commit it as an UNRELEASED
  changelog entry, providing a reference point.
  `gbp dch --since=1234567` is useful for this model.

I'm concerned that the first model is optimized for people who know
Debian as well as you do, and do not need pre-commit review because
they get everything right first time. I don't think Debian would be as
large or successful a project as it is if we restricted our contributor
base to people who know the corner cases as well as Ian Jackson does :-)

> Q2. Should the changelog entry be finalised ?  That is, should it
> have an uploader name and date ?

While as an abstract model I agree that the uploader name and date
are not meaningful in an unreleased version, I can't help thinking
that this is a "boil the ocean" sort of change - a lot of tools follow
and require Policy syntax, in which the uploader name and date are
non-optional. Obviously, Policy only really applies to finished packages,
and unfinished packages often violate the semantics of Policy (for
instance by using UNRELEASED as a suite name); but it seems reasonable
for a tool author to oppose changes that, as well as violating Policy
semantics, also violate Poliy syntax.

Allowing and ignoring an arbitrary and non-meaningful uploader and date,
or possibly establishing a convention like Unreleased 
for the unreleased uploader, seems more pragmatic.

I should point out here that `dch`, which is commonly used for the "write
the changelog as you go" model, preserves the uploader and date of the
first commit after the previous release until the next `dch -r`. This is
not theoretically defensible at all - using the most recent committer
and date would be more correct - but in practice it works pretty well
as a way to minimize conflicts, reducing them to an acceptable level
(although still more than if you write the changelog at release time).

> Q3. Should the version number be the intended next version number,
> or should it be decorated with ~something ?  If it should
> be decorated, what should it be decorated with ?

The solution to this I used in build-snapshot[1], a continuous integration
script for the Debian derivative Apertis, was to see whether the current
version is tagged, and if not, use dch(1) to synthesize a changelog
entry for a version number based on `git describe` (1.2-3+45+g1234567
for commit 1234567, 45 commits later than 1.2-3) with a dummy uploader
named Snapshot . In theory this pseudo-release
could itself be committed to git, but in practice we don't either need
or want to do so (and it would be easy to create an infinite loop
where the new commit is subjected to continuous integration, generating
a new pseudo-release and so on).

In the project where this script originated, we mostly follow the
"write the changelog later" workflow, because that's what is familiar
to