Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello Bill,

On Thu, Apr 05 2018, Bill Allombert wrote:

> The standard way is to have a transition period where policy allows
> for both behaviour. This way, debhelper can be updated without
> breaking policy and developers would have a reference for the new
> behaviour.
>
> Then the old behaviour is deprecated.

Thank you for this, Bill.

This would be a good way to proceed if we are indeed able to get
consensus on what the new behaviour should be.  I've been pushing the
debhelper route because I suspect that consensus will not be
forthcoming, given what's happened when people have tried to resolve
this bug before, but I'd be very glad to be proved wrong!

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[Bill Allombert]

On Wed, Apr 04, 2018 at 12:00:29PM -0700, Sean Whitton wrote:
> Hello Adrian,
> 
> Thank you for your continued effort to get this bug resolved.
> 
> On Sat, Mar 10 2018, Adrian Bunk wrote:
> 
> >> Please expand on why you think this is the way we have to proceed.
> >
> > you skipped the part of my email with the explanation:
> >
> >   with such a piecemeal approach
> >   we risk fragmentation based on debhelper compat level used, with every
> >   new compat level installing different files in different locations.
> 
> This is not inevitable.  What I am envisaging is:
> 
> - we hash out our preferred solution either in this bug or in the
>   debhelper bug, with the debhelper maintainer having the final say on
>   what gets implemented
> - debhelper implements all of that solution at exactly one compat level
> - the archive starts to use that compat level
> - Policy is updated.
> 
> This is the standard way to make changes to Policy.  The alternative is
> releases of Policy rendering many packages buggy, and that is undesirable.

The standard way is to have a transition period where policy allows for both
behaviour. This way, debhelper can be updated without breaking policy
and developers would have a reference for the new behaviour.

Then the old behaviour is deprecated.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello,

On Wed, Apr 04 2018, Adrian Bunk wrote:

> This ensures that policy will always be horribly outdated.

Policy is meant to lag behind practice.  One of the reasons for this is
that it ensures that no-one feels they have to update Policy before
innovating.

The extent to which it lags is a function of how many people are willing
to produce patches to the text of Policy (if you look at our bug lists,
this is the main area in which we are lacking manpower).

>> This is the standard way to make changes to Policy. The alternative is
>> releases of Policy rendering many packages buggy, and that is undesirable.
>>...
>
> A new debhelper compat level making packages buggy according to policy
> is what is desirable?

Yes.  It's okay for packages to break Policy that is thought to be
outdated, when the break that they introduce is an attempt to move
forward.  It's not okay for Policy to change first, except in those
exceptional cases where the only way to make sense of the change is to
change Policy first.

> Let me repeat what I already wrote earlier in this bug:
>
> IMHO the way forward is to find a consensus text for a policy change,
> Cc debian-devel on the proposal, and if no objections are raised release
> a new policy with these changes.

I would like to suggest that you do exactly that, but insert the
debhelper change before your final step of releasing Policy.

> The Policy Editors are a delegation, and the debhelper maintainer is
> just a random person who happens to maintain this specific package.

I don't see how that's relevant.

> Precedent is also that the Policy Editors do use their powers to
> push through even a change that makes thousands of packages buggy
> after an objection has been raised:
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844431#334

Thank you for bringing up that example.

We would not have made that change had, say, less than 70% of the
archive already been reproducible.  Right now, without an implementation
in debhelper, this change would be making far, far more of the archive
buggy, and it would stay buggy for longer.

Further, we do not have a consensus on this bug in the way that we did
in the case of reproducible builds.

> I begin to wonder whether I should take this personally.

You absolutely should not.  As I said, I'm grateful for your interest in
working on this.  Moreover, I want to see this bug closed as much as you
do.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[Adrian Bunk]

On Wed, Apr 04, 2018 at 12:00:29PM -0700, Sean Whitton wrote:
> Hello Adrian,
> 
> Thank you for your continued effort to get this bug resolved.
> 
> On Sat, Mar 10 2018, Adrian Bunk wrote:
> 
> >> Please expand on why you think this is the way we have to proceed.
> >
> > you skipped the part of my email with the explanation:
> >
> >   with such a piecemeal approach
> >   we risk fragmentation based on debhelper compat level used, with every
> >   new compat level installing different files in different locations.
> 
> This is not inevitable.  What I am envisaging is:
> 
> - we hash out our preferred solution either in this bug or in the
>   debhelper bug, with the debhelper maintainer having the final say on
>   what gets implemented
> - debhelper implements all of that solution at exactly one compat level
> - the archive starts to use that compat level
> - Policy is updated.

This ensures that policy will always be horribly outdated.

> This is the standard way to make changes to Policy. The alternative is
> releases of Policy rendering many packages buggy, and that is undesirable.
>...

A new debhelper compat level making packages buggy according to policy
is what is desirable?

> Our delegation gives the Policy Editors editorial authority over the
> contents of the Policy Manual, but by means of the Policy Changes
> Process we delegate that authority back to the body of DDs.
> 
> That means that changes have to establish consensus.  How do we
> establish consensus on an issue like this, just by talking about it
> where everyone has a view?  That's extremely hard.  There's no way to
> bring the discussion to a close, and so the bug is still open after 10
> years.

Let me repeat what I already wrote earlier in this bug:

IMHO the way forward is to find a consensus text for a policy change,
Cc debian-devel on the proposal, and if no objections are raised release 
a new policy with these changes.

> What we can do in this case is shortcircuit that process of establishing
> consensus by means of debhelper.  The debhelper maintainer has authority
> over what debhelper will implement.  So he implements something.  If
> it's basically a sensible convention, almost everyone will start using
> it, even if it is not their number one choice of solution.  But then we
> have enough of a consensus, so it's no longer problematic to change

The Policy Editors are a delegation, and the debhelper maintainer is 
just a random person who happens to maintain this specific package.


Precedent is also that the Policy Editors do use their powers to 
push through even a change that makes thousands of packages buggy 
after an objection has been raised:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844431#334

I begin to wonder whether I should take this personally.

> Sean Whitton

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello Adrian,

Thank you for your continued effort to get this bug resolved.

On Sat, Mar 10 2018, Adrian Bunk wrote:

>> Please expand on why you think this is the way we have to proceed.
>
> you skipped the part of my email with the explanation:
>
>   with such a piecemeal approach
>   we risk fragmentation based on debhelper compat level used, with every
>   new compat level installing different files in different locations.

This is not inevitable.  What I am envisaging is:

- we hash out our preferred solution either in this bug or in the
  debhelper bug, with the debhelper maintainer having the final say on
  what gets implemented
- debhelper implements all of that solution at exactly one compat level
- the archive starts to use that compat level
- Policy is updated.

This is the standard way to make changes to Policy.  The alternative is
releases of Policy rendering many packages buggy, and that is undesirable.

I don't think you've explained why you think that can't happen for this
bug too.  You're right to note the piecemeal compat level risk, but now
that we're aware of that risk, it seems straightforward to mitigate.
We just avoid implementing anything in debhelper until we can implement
it all in debhelper.

> Your point of view sounds to me like
>   In 15 years policy will be changed to document whatever the
>   debhelper maintainer decides to implement in dh compat 12.
>
> To me this doesn't make sense.
>
> This bug is now over 10 years old, and at some point someone has to
> decide which files should be installed and where.
>
> The obvious way forward would be to try to find a consensus here,
> and then ask on debian-devel whether there are any objections.
>
> If anything ends up being controverial and a decision is required, our
> constitution already makes it clear who is responsible for the decision
> (and this is not the debhelper maintainer).

Our delegation gives the Policy Editors editorial authority over the
contents of the Policy Manual, but by means of the Policy Changes
Process we delegate that authority back to the body of DDs.

That means that changes have to establish consensus.  How do we
establish consensus on an issue like this, just by talking about it
where everyone has a view?  That's extremely hard.  There's no way to
bring the discussion to a close, and so the bug is still open after 10
years.

What we can do in this case is shortcircuit that process of establishing
consensus by means of debhelper.  The debhelper maintainer has authority
over what debhelper will implement.  So he implements something.  If
it's basically a sensible convention, almost everyone will start using
it, even if it is not their number one choice of solution.  But then we
have enough of a consensus, so it's no longer problematic to change
Policy.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[Adrian Bunk]

On Sat, Mar 10, 2018 at 11:03:05PM +0200, Adrian Bunk wrote:
>...
> > > I think
> > > the most valuable starting point would to be to standardize on
> > > /usr/share/doc/package/NEWS.gz for the human-readable summary and
> > > explicitly say to never install that as
> > > /usr/share/doc/package/changelog.gz.  
> 
> On the contents of the proposal I agree with Gregor that I do not 
> consider it a good idea to install a different file to changelog.gz
> than has been in Debian for the past quarter century.
> 
> That's my personal opinion on the contents, others might differ on that.
> 
> Regarding the process, something like that should be decided before 
> installation of the human-readable summary is done by default by 
> debhelper.
> 
> A complete mess would be something like:
> - dh compat >= 12 installs NEWS to /usr/share/doc/package/NEWS.gz
> - dh compat >= 13 no longer installs ChangeLog to 
> /usr/share/doc/package/changelog.gz
> - dh compat >= 14 installs NEWS to /usr/share/doc/package/changelog.gz
>...

Sorry, I messed up and the quoted suggestion above is actually
a different suggestion, where the mess would be like
- dh compat >= 12 installs NEWS to /usr/share/doc/package/NEWS.gz
- dh compat >= 13 no longer installs ChangeLog to 
/usr/share/doc/package/changelog.gz
- dh compat >= 13 or when policy changes, some packages install a 
  ChangeLog that was previously in /usr/share/doc/package/changelog.gz
  as /usr/share/doc/package/NEWS.gz

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#459427: changelog vs. NEWS handling

[Adrian Bunk]

On Tue, Feb 27, 2018 at 10:09:59PM -0700, Sean Whitton wrote:
> Hello,

Hi Sean,

> On Tue, Feb 27 2018, Adrian Bunk wrote:
> 
> > Policy should define the naming before dh_installchangelogs should
> > change.
> 
> Please expand on why you think this is the way we have to proceed.

you skipped the part of my email with the explanation:

  with such a piecemeal approach
  we risk fragmentation based on debhelper compat level used, with every 
  new compat level installing different files in different locations.

This also referred to a proposal quoted in my email
(which happened to come from a policy editor):

> > I think
> > the most valuable starting point would to be to standardize on
> > /usr/share/doc/package/NEWS.gz for the human-readable summary and
> > explicitly say to never install that as
> > /usr/share/doc/package/changelog.gz.  

On the contents of the proposal I agree with Gregor that I do not 
consider it a good idea to install a different file to changelog.gz
than has been in Debian for the past quarter century.

That's my personal opinion on the contents, others might differ on that.

Regarding the process, something like that should be decided before 
installation of the human-readable summary is done by default by 
debhelper.

A complete mess would be something like:
- dh compat >= 12 installs NEWS to /usr/share/doc/package/NEWS.gz
- dh compat >= 13 no longer installs ChangeLog to 
/usr/share/doc/package/changelog.gz
- dh compat >= 14 installs NEWS to /usr/share/doc/package/changelog.gz

> We try to have Policy reflect changes already in the archive, rather
> than the other way around, except in situations where that can't make
> sense.

Your point of view sounds to me like
  In 15 years policy will be changed to document whatever the
  debhelper maintainer decides to implement in dh compat 12.

To me this doesn't make sense.

This bug is now over 10 years old, and at some point someone has to 
decide which files should be installed and where.

The obvious way forward would be to try to find a consensus here,
and then ask on debian-devel whether there are any objections.

If anything ends up being controverial and a decision is required, our 
constitution already makes it clear who is responsible for the decision 
(and this is not the debhelper maintainer).

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello,

On Tue, Feb 27 2018, Adrian Bunk wrote:

> Policy should define the naming before dh_installchangelogs should
> change.

Please expand on why you think this is the way we have to proceed.

We try to have Policy reflect changes already in the archive, rather
than the other way around, except in situations where that can't make
sense.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[Adrian Bunk]

On Sun, Dec 03, 2017 at 05:24:09PM +0100, gregor herrmann wrote:
> On Thu, 30 Nov 2017 20:45:51 -0800, Russ Allbery wrote:
> 
> > I think
> > the most valuable starting point would to be to standardize on
> > /usr/share/doc/package/NEWS.gz for the human-readable summary and
> > explicitly say to never install that as
> > /usr/share/doc/package/changelog.gz.  
> 
> That would mean that we have to manually add something to d/rules for
> thousands of perl packages in order to make ChangeLog, Changes etc.
> getting installed as /usr/share/doc/package/NEWS.gz.
> (Unless dh_installchangelogs gets some logic to install ChangeLog,
> Changes etc. as NEWS.gz if there is no NEWS in the upstream tarball.
> But that might be confusing …)
> 
> On Fri, 01 Dec 2017 13:34:01 -0700, Sean Whitton wrote:
> 
> > We can at least make dh_installchangelogs capable of installing both
> > changelog.gz and NEWS.gz before we change Policy.  Otherwise, fixing the
> > minor bugs is significantly more annoying.
> 
> Ack, this would be a good start.

Policy should define the naming before dh_installchangelogs should 
change.

Related is also the question whether any change at all will be
done before debhelper compat 12 - with such a piecemeal approach
we risk fragmentation based on debhelper compat level used, with every 
new compat level installing different files in different locations.

This bug has already celebrated its 10th birthday.

IMHO the way forward is to find a consensus text for a policy change,
Cc debian-devel on the proposal, and if no objections are raised release 
a new policy with these changes.

If additionally installing NEWS turns out to be the only consensus 
change then let's do that in policy now and close the topic.

If there can be consensus on more, then let's do this now.

> > If we are going to add something that says that upstream NEWS/release
> > notes should also be installed, why not standardise on the location?  It
> > seems odd to have a standard location for upstream changelogs but not
> > for upstream NEWS/release notes.
> > 
> > Or perhaps we should drop the standard location for upstream changelogs.
>  
> Having some standardization makes it easier for both users and their
> tools to quickly find what they are looking for. But yeah, I think we
> have at least 2 problems in this area:
> - no single standard on the upstream side (c.f. the GNU style vs. the
>   CPAN style(s))
> - needed changes in our tooling (dh_installchangelogs)
> 
> And we have 2 questions to answer:
> 1) Which files to install?
> 2) Under what name?
> 
> What policy could say is:
> 1) - If there is a human-facing summary, install it.
>- If there is a human-facing summary and a VCS-style log, install
>  the former but not the latter.
>- If there is no human-facing summary but a a VCS-style log,
>  consider installing the latter.
>(Without mentioning file names, or maybe as examples, as they are
>not standardized).
> 2) Install NEWS as /usr/share/doc/package/NEWS.gz and /Change.*/i as
>/usr/share/doc/package/changelog.gz if you install them.
>(That's also something dh_installchangelogs could do easily.)

This sounds like a good suggestion to me and I'd be willing to make a 
policy proposal based on that, with a wording that the maintainer "may" 
install a VCS-style log even when a human-facing summary is present.

If anyone has objections to this proposal, please speak up.

Coordinating with Niels regarding how to synchronize policy and 
debhelper changes is something I'll do after we have agreement
how the end result should look like.

> Cheers,
> gregor

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#459427: changelog vs. NEWS handling

[Bill Allombert]

On Fri, Dec 08, 2017 at 10:29:50AM -0500, Peter Eisentraut wrote:
> On 12/1/17 11:19, Ian Jackson wrote:
> > Is there some reason why exacdt standardisation of the filenames is
> > necessary here ?  For most of the uses I can think of, it is OK to
> > look in a handful of files to see which one might answer the question.
> 
> I wrote the bug originally.
> 
> My goal was simply to get more packages to ship release notes (NEWS),
> and optionally, to get fewer packages to ship useless source-level
> changelogs.
> 
> A policy adjustment can help that a bit because that also influences
> things like debhelper and a lot of packages just use that without much
> further thought.

The issue is that dh_installchangelogs cannot easily detect if a
changelog is a VCS changelog and whether a NEWS is actually a changelog.

The only reasonable thing dh_installchangelogs can do is to install
everything that looks like a changelog or a NEWS file.

However, I am concerned that having dh_installchangelogs removing files will
lead to less predictability.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#459427: changelog vs. NEWS handling

[Ian Jackson]

Peter Eisentraut writes ("Re: Bug#459427: changelog vs. NEWS handling"):
> On 12/1/17 11:19, Ian Jackson wrote:
> > Is there some reason why exacdt standardisation of the filenames is
> > necessary here ?  For most of the uses I can think of, it is OK to
> > look in a handful of files to see which one might answer the question.
> 
> I wrote the bug originally.
> 
> My goal was simply to get more packages to ship release notes (NEWS),
> and optionally, to get fewer packages to ship useless source-level
> changelogs.

Those seem good goals.

> A policy adjustment can help that a bit because that also influences
> things like debhelper and a lot of packages just use that without much
> further thought.

Indeed.

I think that means you are answering "no" to the question I posed
above ?  I think we are moving towards a form of words that would
achieve your goals without requiring a lot of work to 100% standardise
on the filenames.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#459427: changelog vs. NEWS handling

[Peter Eisentraut]

On 12/1/17 11:19, Ian Jackson wrote:
> Is there some reason why exacdt standardisation of the filenames is
> necessary here ?  For most of the uses I can think of, it is OK to
> look in a handful of files to see which one might answer the question.

I wrote the bug originally.

My goal was simply to get more packages to ship release notes (NEWS),
and optionally, to get fewer packages to ship useless source-level
changelogs.

A policy adjustment can help that a bit because that also influences
things like debhelper and a lot of packages just use that without much
further thought.

Bug#459427: changelog vs. NEWS handling

[gregor herrmann]

On Thu, 30 Nov 2017 20:45:51 -0800, Russ Allbery wrote:

> I think
> the most valuable starting point would to be to standardize on
> /usr/share/doc/package/NEWS.gz for the human-readable summary and
> explicitly say to never install that as
> /usr/share/doc/package/changelog.gz.  

That would mean that we have to manually add something to d/rules for
thousands of perl packages in order to make ChangeLog, Changes etc.
getting installed as /usr/share/doc/package/NEWS.gz.
(Unless dh_installchangelogs gets some logic to install ChangeLog,
Changes etc. as NEWS.gz if there is no NEWS in the upstream tarball.
But that might be confusing …)


On Fri, 01 Dec 2017 13:34:01 -0700, Sean Whitton wrote:

> We can at least make dh_installchangelogs capable of installing both
> changelog.gz and NEWS.gz before we change Policy.  Otherwise, fixing the
> minor bugs is significantly more annoying.

Ack, this would be a good start.
 
> If we are going to add something that says that upstream NEWS/release
> notes should also be installed, why not standardise on the location?  It
> seems odd to have a standard location for upstream changelogs but not
> for upstream NEWS/release notes.
> 
> Or perhaps we should drop the standard location for upstream changelogs.
 
Having some standardization makes it easier for both users and their
tools to quickly find what they are looking for. But yeah, I think we
have at least 2 problems in this area:
- no single standard on the upstream side (c.f. the GNU style vs. the
  CPAN style(s))
- needed changes in our tooling (dh_installchangelogs)

And we have 2 questions to answer:
1) Which files to install?
2) Under what name?

What policy could say is:
1) - If there is a human-facing summary, install it.
   - If there is a human-facing summary and a VCS-style log, install
 the former but not the latter.
   - If there is no human-facing summary but a a VCS-style log,
 consider installing the latter.
   (Without mentioning file names, or maybe as examples, as they are
   not standardized).
2) Install NEWS as /usr/share/doc/package/NEWS.gz and /Change.*/i as
   /usr/share/doc/package/changelog.gz if you install them.
   (That's also something dh_installchangelogs could do easily.)


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Jim Croce: Time In A Bottle


signature.asc
Description: Digital Signature

Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello,

On Thu, Nov 30 2017, Russ Allbery wrote:

> Unfortunately, there's no good way to do this transition without
> making a whole ton of packages buggy, since we're horribly
> inconsistent about how we handle this now.  I think that's just
> something we should tackle, and make it very clear that this is a
> *minor* bug and people shouldn't harass maintainers about it, but we'd
> like to sort out this historic mess and switch to consistent usage of
> these two files.

We can at least make dh_installchangelogs capable of installing both
changelog.gz and NEWS.gz before we change Policy.  Otherwise, fixing the
minor bugs is significantly more annoying.

On Fri, Dec 01 2017, Ian Jackson wrote:

> Is there some reason why exacdt standardisation of the filenames is
> necessary here ?  For most of the uses I can think of, it is OK to
> look in a handful of files to see which one might answer the question.

Policy does not currently refer to the upstream NEWS/release notes.  I
think it should at least say that such a thing should be installed if it
exists.

Currently, of upstream changelogs, Policy says

If an upstream changelog is available, it should be accessible as
/usr/share/doc/package/changelog.gz in plain text.  If the upstream
changelog is distributed in HTML ...

If we are going to add something that says that upstream NEWS/release
notes should also be installed, why not standardise on the location?  It
seems odd to have a standard location for upstream changelogs but not
for upstream NEWS/release notes.

Or perhaps we should drop the standard location for upstream changelogs.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[Bill Allombert]

On Thu, Nov 30, 2017 at 08:45:51PM -0800, Russ Allbery wrote:
> Bill Allombert  writes:
> 
> > git log might be more useful in some situation and extremly inconvenient
> > in some others (to start with it require network access and cloning the
> > full project history).
> 
> A complete changelog is often an appreciable percentage of the size of
> that full project history.  I'm dubious that, to avoid requiring users
> clone the full project history, we should ship the full project history in
> Debian packages.

> I've stopped including VCS-level changelogs in any of my packages even if
> upstream ships them.  The compressed changelog was sometimes over 80% of
> the size of the entire package, which was a complete waste.

I agree with your point about VCS-level changelog, however projects that
ship a VCS-changelog masquaraded as a true changelog are a minority.
A true changelog is much smaller than the VCS changelog.

Also some project split the changelog by major version, so one can only
include the changelog of the two last major version if space is a
concern.

> That said, I'm happy to leave this to the Debian package maintainer
> discretion in cases where it really is useful for some reason.  I think
> the most valuable starting point would to be to standardize on
> /usr/share/doc/package/NEWS.gz for the human-readable summary and
> explicitly say to never install that as
> /usr/share/doc/package/changelog.gz.  Then, we can define changelog.gz as
> always the detailed change-by-change upstream log (if such a thing
> exists), and clearly indicate that it's optional and maintainers should
> use their discretion in deciding whether it's useful to include, but
> should always include the human-readable release notes and change summary
> as NEWS.

Some package include a changelog file that you would call a NEWS file
while also including a NEWS file.

Mayabe policy could say: "do not include the changelog if it is actually
a VCS dump, in that case ship the NEWS file as changelog".

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#459427: changelog vs. NEWS handling

[Ian Jackson]

Russ Allbery writes ("Bug#459427: changelog vs. NEWS handling"):
> Unfortunately, there's no good way to do this transition without making a
> whole ton of packages buggy, since we're horribly inconsistent about how
> we handle this now.  I think that's just something we should tackle, and
> make it very clear that this is a *minor* bug and people shouldn't harass
> maintainers about it, but we'd like to sort out this historic mess and
> switch to consistent usage of these two files.

We could avoid making all packages buggy, by giving more flexible
advice.

The kind of stuff found in an upstream NEWS clearly ought to be in the
.deb somewhere.  The kind of stuff found in a "git log" is useless.

The policy was written when lots of packges had a GNU-style ChangeLog,
and those can be useful.  I have sometimes looked in them (in their
guise as /usr/share/doc/P/changelog) to find out why something is
broken.

IMO it all depends on what upstream does.

Is there some reason why exacdt standardisation of the filenames is
necessary here ?  For most of the uses I can think of, it is OK to
look in a handful of files to see which one might answer the question.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#459427: changelog vs. NEWS handling

[Josh Triplett]

On Thu, Nov 30, 2017 at 09:37:32PM -0800, Russ Allbery wrote:
> This is okay 80% of the time and badly needs manual editing the remaining
> 20% of the time.  I personally would never be willing to forgo good
> changelogs in that remaining 20% of the time that can't really be handled
> with commit messages, and the systems I've seen for embedding metadata in
> Git just seem awkward.

I've seen some systems used very well, which can reliably generate NEWS
files with zero interaction. Those systems ought to be able to generate
debian/changelog similarly.

> That said, one reason why I hold this position is that I've personally
> found the concerns with changelogs checked into Git to be vastly
> overblown, and they've never caused me significant problems in practice.
> Those who have had other experiences might reach different cost/benefit
> tradeoffs.

I've had quite a few negative experiences with that, including merge
conflicts and having to write messages twice.

Bug#459427: changelog vs. NEWS handling

[Russ Allbery]

Josh Triplett  writes:

> I *do* use apt-listchanges to reach changelogs, and I'm not advocating
> that they not exist; I'm simply arguing that they make it a pain to keep
> a Debian package in git, and that we ought to autogenerate them from git
> log and some care taken in commit messages.

This is okay 80% of the time and badly needs manual editing the remaining
20% of the time.  I personally would never be willing to forgo good
changelogs in that remaining 20% of the time that can't really be handled
with commit messages, and the systems I've seen for embedding metadata in
Git just seem awkward.

That said, one reason why I hold this position is that I've personally
found the concerns with changelogs checked into Git to be vastly
overblown, and they've never caused me significant problems in practice.
Those who have had other experiences might reach different cost/benefit
tradeoffs.

-- 
Russ Allbery (r...@debian.org)   

Bug#459427: changelog vs. NEWS handling

[Josh Triplett]

On Thu, Nov 30, 2017 at 08:50:11PM -0800, Russ Allbery wrote:
> Jonathan Nieder  writes:
> >> I would go so far as to say that I hope we one day stop shipping a
> >> non-generated debian/changelog in source packages, because it incurs
> >> all the same pain.
> 
> > I've been trying to make debian/changelog in packages I work on
> > user-focused, and no one has complained yet.
> 
> Yeah, I don't agree on debian/changelog.  I write debian/changelog by
> hand, and will always continue to do so, and I think it's extremely
> valuable.  I also read all debian/changelog files for all packages I
> install on my system, and find that incredibly useful when maintainers put
> real information in them.

I *do* use apt-listchanges to reach changelogs, and I'm not advocating
that they not exist; I'm simply arguing that they make it a pain to keep
a Debian package in git, and that we ought to autogenerate them from git
log and some care taken in commit messages.

- Josh Triplett

Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello Jonathan,

On Thu, Nov 30 2017, Jonathan Nieder wrote:

> I've been trying to make debian/changelog in packages I work on
> user-focused, and no one has complained yet.
>
> I also use NEWS.Debian for notes about incompatibilities that will
> affect sysadmins upgrading.

Yes.  If you read the Developer's Reference, d/changelog is meant to be
"user visible" changes, and NEWS.Debian really urgent user visible
changes.

Which is different from the distinction between the upstream CHANGELOG
and NEWS distinction we've been working with in recent e-mails to this
thread...

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[Russ Allbery]

Jonathan Nieder  writes:

> How do you feel about generated changelogs in release tarballs that
> are generated by tools like "git log"?

I think they're a waste of space and effort.  The circumstances in which
those are useful are so obscure that I think more harm than good is being
done by making people download the additional bytes in release tarballs.
Essentially everyone who would read that file is much better off just git
cloning the repository.

>> I would go so far as to say that I hope we one day stop shipping a
>> non-generated debian/changelog in source packages, because it incurs
>> all the same pain.

> I've been trying to make debian/changelog in packages I work on
> user-focused, and no one has complained yet.

Yeah, I don't agree on debian/changelog.  I write debian/changelog by
hand, and will always continue to do so, and I think it's extremely
valuable.  I also read all debian/changelog files for all packages I
install on my system, and find that incredibly useful when maintainers put
real information in them.

-- 
Russ Allbery (r...@debian.org)   

Bug#459427: changelog vs. NEWS handling

[Russ Allbery]

Bill Allombert  writes:

> git log might be more useful in some situation and extremly inconvenient
> in some others (to start with it require network access and cloning the
> full project history).

A complete changelog is often an appreciable percentage of the size of
that full project history.  I'm dubious that, to avoid requiring users
clone the full project history, we should ship the full project history in
Debian packages.  Debian does not need to provide every piece of possibly
useful data provided by upstream; at some point, people really should go
directly to upstream for the obscure and marginally useful trivia.

I've stopped including VCS-level changelogs in any of my packages even if
upstream ships them.  The compressed changelog was sometimes over 80% of
the size of the entire package, which was a complete waste.

That said, I'm happy to leave this to the Debian package maintainer
discretion in cases where it really is useful for some reason.  I think
the most valuable starting point would to be to standardize on
/usr/share/doc/package/NEWS.gz for the human-readable summary and
explicitly say to never install that as
/usr/share/doc/package/changelog.gz.  Then, we can define changelog.gz as
always the detailed change-by-change upstream log (if such a thing
exists), and clearly indicate that it's optional and maintainers should
use their discretion in deciding whether it's useful to include, but
should always include the human-readable release notes and change summary
as NEWS.

Unfortunately, there's no good way to do this transition without making a
whole ton of packages buggy, since we're horribly inconsistent about how
we handle this now.  I think that's just something we should tackle, and
make it very clear that this is a *minor* bug and people shouldn't harass
maintainers about it, but we'd like to sort out this historic mess and
switch to consistent usage of these two files.

-- 
Russ Allbery (r...@debian.org)   

Bug#459427: changelog vs. NEWS handling

[Josh Triplett]

On Thu, Nov 30, 2017 at 06:56:53PM -0800, Jonathan Nieder wrote:
> Josh Triplett wrote:
> > On Fri, 1 Dec 2017 00:04:20 +0100 Bill Allombert  
> > wrote:
> 
> >> The fact that some upstream do not bother to ship useful changelog does
> >> not mean that all changelog are useless, and by removing them we
> >> discourage upstream of producing useful changelog.
> >
> > I sincerely hope so.  Convincing upstreams to "git rm ChangeLog" becomes
> > much easier the more widespread existing practice we can point to.
> > Having a ChangeLog file in version control is actively detrimental.
> 
> I agree with you about GNU-style changelogs.  But I don't think I've
> seen those much outside of GNU packages.

I still see them in some other packages that started out pre-git, where
nobody quite felt empowered to delete the changelog.

> How do you feel about generated changelogs in release tarballs that
> are generated by tools like "git log"?

As long as they're completely autogenerated, they don't do any harm. I'd
sooner not ship them at all and just say "see the git repository", but
if you have to have one then autogenerate it.

> I also use NEWS.Debian for notes about incompatibilities that will
> affect sysadmins upgrading.

NEWS.Debian is *absolutely* valuable.

Bug#459427: changelog vs. NEWS handling

[Jonathan Nieder]

Josh Triplett wrote:
> On Fri, 1 Dec 2017 00:04:20 +0100 Bill Allombert  wrote:

>> The fact that some upstream do not bother to ship useful changelog does
>> not mean that all changelog are useless, and by removing them we
>> discourage upstream of producing useful changelog.
>
> I sincerely hope so.  Convincing upstreams to "git rm ChangeLog" becomes
> much easier the more widespread existing practice we can point to.
> Having a ChangeLog file in version control is actively detrimental.

I agree with you about GNU-style changelogs.  But I don't think I've
seen those much outside of GNU packages.

How do you feel about generated changelogs in release tarballs that
are generated by tools like "git log"?

> I would go so far as to say that I hope we one day stop shipping
> a non-generated debian/changelog in source packages, because it incurs
> all the same pain.

I've been trying to make debian/changelog in packages I work on
user-focused, and no one has complained yet.

I also use NEWS.Debian for notes about incompatibilities that will
affect sysadmins upgrading.

Thanks,
Jonathan

Bug#459427: changelog vs. NEWS handling

[Josh Triplett]

On Fri, 1 Dec 2017 00:04:20 +0100 Bill Allombert  wrote:
> Both the content and the name of the upstream changelogs is an upstream
> issue. The fact that a file is named by upstream Changelog instead of
> NEWS does not imply anything on its usefulness. It might even happen
> that NEWS is the real changelog.

That's certainly possible, though not a particularly good or
conventional practice.  But using NEWS for user-visible release notes
rather than a full source-code changelog seems a sufficiently common
practice to make it the default assumption, which a source package could
override if necessary.

> The fact that some upstream do not bother to ship useful changelog does
> not mean that all changelog are useless, and by removing them we
> discourage upstream of producing useful changelog.

I sincerely hope so.  Convincing upstreams to "git rm ChangeLog" becomes
much easier the more widespread existing practice we can point to.
Having a ChangeLog file in version control is actively detrimental.

I would go so far as to say that I hope we one day stop shipping
a non-generated debian/changelog in source packages, because it incurs
all the same pain.

Now, a NEWS file or similar, containing user-visible *release notes* and
similar highlights, would most certainly be useful.  I would love to see
*more* upstreams providing files like those.  But those certainly aren't
changelogs.

- Josh Triplett

Bug#459427: changelog vs. NEWS handling

[Bill Allombert]

On Tue, Nov 28, 2017 at 11:01:08PM -0500, Jeremy Bicha wrote:
> As others have said, running 'git log' is far more useful than a
> complete changelog and in my experience, most projects these days
> outside of GNU don't bother shipping changelogs.
> 
> Most of my Debian and Ubuntu work involves GNOME packaging. For the
> most part, GNOME components ships NEWS files which are much more
> interesting for users or developers to read for highlights of what
> changed when.
> 
> Ubuntu took the position 7 years ago that shipping full upstream
> changelogs is a waste of space. [1] This whole situation introduces a
> bad problem: for the hundreds of packages that use
> 'dh_installchangelogs NEWS', [2] Ubuntu silently drops the NEWS file
> (renamed as changelog.gz)!
> 
> I believe Policy's advice to install upstream changelogs should be
> dropped. In its place, I think a recommendation to ship NEWS files in
> /usr/share/doc/ would be useful. Notably, debhelper does not currently
> install NEWS files unless explicitly told to.[3]

I disagree with removing this requirement.

Both the content and the name of the upstream changelogs is an upstream
issue. The fact that a file is named by upstream Changelog instead of
NEWS does not imply anything on its usefulness. It might even happen
that NEWS is the real changelog.

The fact that some upstream do not bother to ship useful changelog does
not mean that all changelog are useless, and by removing them we
discourage upstream of producing useful changelog.

git log might be more useful in some situation and extremly inconvenient
in some others (to start with it require network access and cloning
the full project history).

The ability to extract upstream changelog from .deb is useful.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#459427: changelog vs. NEWS handling

[Jonathan Nieder]

Hi,

gregor herrmann wrote:

> From the Perl world, looking at roughly ~3400 packages I have locally
> cloned:
>
> 28 have a NEWS file (most of them with a Gnome/GTK background), 1
> News, 1 news.
>
> 3368 have a Changes, CHANGES, Changelog, ChangeLog, (and some other
> variations like Change{s,Log}.{pod,ini,1,txt}); and those files are the
> manually created user-facing summaries of relevant changes of the
> release (in almost all cases).
>
> For 10 packages we have `dh_installchangelogs NEWS' in debian/rules.
>
>
> I'm all for installing NEWS if it's a summary in the GNU style; but
> assuming that ChangeLog etc. are detailed/auto-generated/boring
> doesn't reflect reality in the Perl universe.

Yes.

Some more questions from a policy pov:

 1. What to do with packages that make multiple per-release release
note files?  Should the packager concatenate them or is shipping
them as multiple files ok?  (E.g. see /usr/share/doc/git/RelNotes.)

 2. What about packages that prune old release notes from their source
tarball?  (E.g. Samba's WHATSNEW.TXT doesn't go back very far,
sometimes not even as far as the previous stable Debian release.)

Should packagers copy back in such archived release notes to make
the changelog usable to users?

 3. Any advice for packagers to make complying with the GPL section 5+6
straightforward?

  a) The work must carry prominent notices stating that you modified
  it, and giving a relevant date.

What about when Debian's upstream is downstream from someone else?

Thanks,
Jonathan

Bug#459427: changelog vs. NEWS handling

[Sean Whitton]

Hello Simon,

On Wed, Nov 29 2017, Simon McVittie wrote:

> How about something like this?
> [...]

Thank you for taking the time to write up patch text for this bug.  FWIW
I would second it, however:

I think that debhelper's behaviour needs to be sorted out before we can
change what Policy says.  Otherwise, we make hundreds of packages buggy,
and Policy changes aren't meant to do that.

Let's get debhelper doing what most makes sense in 2017, and then update
Policy.  I would suggest that you/Jeremy write to #666056, perhaps
retitling that bug, and saying something like "please make debhelper
install NEWS as NEWS.gz not as changelog.gz."

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#459427: changelog vs. NEWS handling

[gregor herrmann]

On Wed, 29 Nov 2017 11:34:21 +, Simon McVittie wrote:

> > Most of my Debian and Ubuntu work involves GNOME packaging. For the
> > most part, GNOME components ships NEWS files which are much more
> > interesting for users or developers to read for highlights of what
> > changed when.
> This is in line with the roles of NEWS and ChangeLog in the GNU Coding
> Standards, which is perhaps a more canonical reference than "what
> GNOME does".

From the Perl world, looking at roughly ~3400 packages I have locally
cloned:

28 have a NEWS file (most of them with a Gnome/GTK background), 1
News, 1 news.

3368 have a Changes, CHANGES, Changelog, ChangeLog, (and some other
variations like Change{s,Log}.{pod,ini,1,txt}); and those files are the
manually created user-facing summaries of relevant changes of the
release (in almost all cases).

For 10 packages we have `dh_installchangelogs NEWS' in debian/rules.


I'm all for installing NEWS if it's a summary in the GNU style; but
assuming that ChangeLog etc. are detailed/auto-generated/boring
doesn't reflect reality in the Perl universe.


> > I think the idea of renaming NEWS to changelog (as is done by
> > dh_installchangelogs NEWS) is wrong, because it goes against people's
> > understanding of what a changelog generally looks like.
> At the risk of making more packages immediately buggy, I think I agree
> with this, and hence prefer option 2 over option 3. 

I think that depends on where the people are coming from; if they've
used Debian for long enough they might be familiar with the practice
that /usr/share/doc//changelog.gz contains user-focused
summaries. -- But I'm fine with installing NEWS as
/usr/share/doc//NEWS.gz instead of renaming it when it
exists.


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Leonard Cohen: In My Secret Life


signature.asc
Description: Digital Signature

Bug#459427: changelog vs. NEWS handling

[Simon McVittie]

On Tue, 28 Nov 2017 at 23:01:08 -0500, Jeremy Bicha wrote:
> As others have said, running 'git log' is far more useful than a
> complete changelog and in my experience, most projects these days
> outside of GNU don't bother shipping changelogs.

Many of those projects that do ship a ChangeLog generate it from
`git log`, so it can't possibly be more useful than `git log` (except
for the property that it's already on your system; but that's a
double-edged sword, because the detailed changelog of the packages that
you're currently investigating are maybe useful, the detailed changelogs
of all other packages are a waste of space, and those roles can change
at any time).

> Most of my Debian and Ubuntu work involves GNOME packaging. For the
> most part, GNOME components ships NEWS files which are much more
> interesting for users or developers to read for highlights of what
> changed when.

This is in line with the roles of NEWS and ChangeLog in the GNU Coding
Standards, which is perhaps a more canonical reference than "what
GNOME does".

> I believe Policy's advice to install upstream changelogs should be
> dropped. In its place, I think a recommendation to ship NEWS files in
> /usr/share/doc/ would be useful. Notably, debhelper does not currently
> install NEWS files unless explicitly told to.[3]

How about something like this?

"""
If the upstream project provides a user-facing summary of changes
or release notes (often named NEWS, for example in projects that
follow the GNU Coding Standards), then it should be made accessible
as /usr/share/doc//NEWS.gz in gzip-compressed plain text. If
the summary of changes is distributed in HTML, it should be made
available in that form as /usr/share/doc//NEWS.html.gz, and
a plain text NEWS.gz should be generated from it using, for example,
lynx -dump -nolist.

If the upstream project only provides a detailed change log
(often named ChangeLog, for example in projects that follow the GNU
Coding Standards), then it [TODO: should? may?] be made accessible as
/usr/share/doc//changelog.gz in gzip-compressed plain text.
If the detailed change log is distributed in HTML, it [should? may?]
be made available in that form as
/usr/share/doc//changelog.html.gz; if so, a plain text
changelog.gz should be generated from it.

If the upstream project provides both a user-facing summary of changes
and a detailed change log, then latter should not usually be provided
in binary packages.

If the upstream changelog files do not already conform to this naming
convention, then this may be achieved either by renaming the files,
or by adding a symbolic link, at the maintainer’s discretion.
"""

I suspect the bits about .html.gz should also say .html, because typical
web browsers don't display file:///some/path/foo.html.gz the way we'd
want, and shipping NEWS instead of ChangeLog limits the space wasted by
lack of compression.

Looking further back in the history of this bug, I think this agrees
with option 2 from the mail that opened the bug, except that a few
people wanted to install the upstream NEWS as changelog.gz instead of
NEWS.gz (option 3).

I'm not sure what Policy should recommend for the case where an upstream
distributes a detailed ChangeLog but no user-facing NEWS. One point of
view is that the detailed ChangeLog is user-hostile but better than
nothing; another is that it's probably a waste of space, because anyone
doing sufficient archaeology to be willing to read a detailed ChangeLog
should be going straight to the project's VCS repository.

> I think the idea of renaming NEWS to changelog (as is done by
> dh_installchangelogs NEWS) is wrong, because it goes against people's
> understanding of what a changelog generally looks like.

At the risk of making more packages immediately buggy, I think I agree
with this, and hence prefer option 2 over option 3. If packages continue
to have both files (perhaps because this particular upstream writes a
sufficiently abbreviated summary in NEWS that referring to the detailed
changelog is more frequently necessary), then they should keep their
current/historical names NEWS.gz and changelog.gz; and if only one is
installed, then it should have the same name as if both were, to create
the right expectation about their contents.

This nicely mirrors NEWS.Debian.gz and changelog.Debian.gz, too.

smcv

Bug#459427: changelog vs. NEWS handling

[Jeremy Bicha]

As others have said, running 'git log' is far more useful than a
complete changelog and in my experience, most projects these days
outside of GNU don't bother shipping changelogs.

Most of my Debian and Ubuntu work involves GNOME packaging. For the
most part, GNOME components ships NEWS files which are much more
interesting for users or developers to read for highlights of what
changed when.

Ubuntu took the position 7 years ago that shipping full upstream
changelogs is a waste of space. [1] This whole situation introduces a
bad problem: for the hundreds of packages that use
'dh_installchangelogs NEWS', [2] Ubuntu silently drops the NEWS file
(renamed as changelog.gz)!

I believe Policy's advice to install upstream changelogs should be
dropped. In its place, I think a recommendation to ship NEWS files in
/usr/share/doc/ would be useful. Notably, debhelper does not currently
install NEWS files unless explicitly told to.[3]

I think the new recommendation would do some good, because debhelper
apparently isn't installing NEWS by default because of this unresolved
bug and because Policy doesn't say it should. [4] But it seems like it
would be less boilerplate and less mistakes if the NEWS files would
just be installed automatically (maybe in debhelper compat 12).

I think the idea of renaming NEWS to changelog (as is done by
dh_installchangelogs NEWS) is wrong, because it goes against people's
understanding of what a changelog generally looks like. I think the
idea of a NEWS file is fairly well understood in open source
development.

[1] https://launchpad.net/ubuntu/+source/debhelper/7.4.20ubuntu2
[2] https://codesearch.debian.net/search?q=dh_installchangelogs.*NEWS
[3] Many more packages use dh_Installdocs for this than dh_installchangelogs:
https://codesearch.debian.net/search?q=path%3Adebian%2F.*docs+NEWS
https://codesearch.debian.net/search?q=dh_installdocs.*NEWS
By the way, CDBS does install NEWS by default.
[4] https://bugs.debian.org/666056

Thanks,
Jeremy Bicha

Bug#459427: changelog vs. NEWS handling

[Russ Allbery]

Yuri D'Elia  writes:
> On Mon, Jan 02 2017, Russ Allbery wrote:

>> No matter what we do here, we're going to make a bunch of packages
>> buggy, because the archive is very divided on current best practice.
>> :(

> Buggy in the sense that existing packages wouldn't comply with the new
> rules?

Right, we'd be saying that you're supposed to do something different than
what a bunch of packages are currently doing.  Although it would just be a
regular bug (and really no one should bother reporting it for right now),
we usually try not to do that.  But I don't think that's a huge problem
here.

> I don't see this as "buggyness", but more of a simple transition like
> any other. By this metric, the current state is equally buggy. I see the
> current mixture of changelog and release logs as proof that maintainers
> would like to do the best thing for the users, but get tangled by the
> policy.

Yup, I agree.

-- 
Russ Allbery (r...@debian.org)   

Bug#459427: changelog vs. NEWS handling

[Yuri D'Elia]

On Mon, Jan 02 2017, Russ Allbery wrote:
> No matter what we do here, we're going to make a bunch of packages buggy,
> because the archive is very divided on current best practice.  :(

Buggy in the sense that existing packages wouldn't comply with the new
rules?

I don't see this as "buggyness", but more of a simple transition like
any other. By this metric, the current state is equally buggy. I see the
current mixture of changelog and release logs as proof that maintainers
would like to do the best thing for the users, but get tangled by the
policy.

I had the same dilemma when packaging. I *also* abuse "changelog" to
ship the release notes. It feels doubly weird that I have to override
the default dh_installchangelogs rule to do this.

Bug#459427: changelog vs. NEWS handling

[Russ Allbery]

Yuri D'Elia  writes:

> In fact, I'd rather have a consistent NEWS location, and shift the focus
> to this release summary instead, while not changing the existing
> changelog rules. It's way more consistent with the best practices
> already seen everywhere in source tarballs.

Yeah, this is basically my opinion too.  The changelog (if used in the
original sense as a file like the GNU ChangeLog file) is basically never
useful to me.  If I cared that much, I'd probably clone the upstream
repository and start looking through commits with better tools than
parsing a text file.  What I actually care about is the NEWS file.

My inclination is to standardize /usr/share/doc//NEWS.gz (which
is already pretty widely used) and relax the urging to install an upstream
changelog.gz file.  I think that's the least disruptive change.  Although
an argument could be made for eliminating the suggestion to install an
upstream changelog file entirely and just recommending that NEWS be
installed as changelog.gz, on the grounds that the upstream detailed
changelog is mostly a waste of disk space (and is often huge) and, in the
few times when it is useful, one can just grab the source package and look
at it there.

No matter what we do here, we're going to make a bunch of packages buggy,
because the archive is very divided on current best practice.  :(

-- 
Russ Allbery (r...@debian.org)   

Bug#459427: changelog vs. NEWS handling

[Yuri D'Elia]

On Sun, Jan 01 2017, Andreas Henriksson wrote:
> My personal opinion is that a changelog is something where every change
> is guaranteed to be logged. (And that guarantee is basically just saying
> that if it's ever noticed that a change was not logged, that's indeed a
> bug.)
> A NEWS file is something different to me in that it's a subset of the
> changelog. Sometimes that subset might be equal to the entire changelog
> set, but there's no guarantee that NEWS logs every change - rather the
> opposite.

Fully agree here.

> Additionally possibly clarify that when something is not available from
> upstream people should *not* go out of their way to find something to
> stuff in there. Many times I've heard that "policy says I have to"
> when asking people why they're insisting on showing a bad lie down my
> throut when they could have just left it out (because all
> policy really said was just "...should ... if ...").
>
> In other words, I support suggestion number 1 (with the extension of
> standardizing the NEWS name along side changelog).

I would also like this approach. Reading a release summary in a
changelog is a "quirk" I've sometimes came to expect from packages.

There is a push to ship a changelog, which is why many packages go to
great lengths to put something there. But changelogs are rarely useful
for users. I'm a developer, and I never read changelogs, as they're
almost never useful unless you also have the source at hand.

If I need to, I go to the source repository and get the history from
there. I /could/ want the source package to include it (as it generally
strips the repository metadata), but not the binary.

In fact, I'd rather have a consistent NEWS location, and shift the focus
to this release summary instead, while not changing the existing
changelog rules. It's way more consistent with the best practices
already seen everywhere in source tarballs.

As an user, I really want the release summary.

Bug#459427: changelog vs. NEWS handling

[Andreas Henriksson]

Hello,

This is one issue I think would be nice to have resolved so will
add my personal opinion below

On Sun, Jan 06, 2008 at 02:55:00PM +0100, Peter Eisentraut wrote:
> Package: debian-policy
> Version: 3.7.3.0
> Severity: normal
> 
> There is some lack of clarity in the policy or perhaps some confusion among
> packagers and thence inconsistencies among packages regarding the handling of
> upstream changelog files. 

This ever growing inconsistency and confusion among packagers causes an
even bigger confusion among users. What can I as a user expect from
/usr/share/doc/package/changelog.gz ? Is it just random mumblings
because policy said the file had to be provided?

If that file is just going to be untrustworthy and potentially misleading
I better avoid opening it at all.

This is why I think /any/ resolution to this issue is important.

Ofcourse I have a personal favourite resolution, but as said /any/
resolution is better than prolonging the ongoing confusion.

> Policy says that upstream changelogs should be installed as
> /usr/share/doc/package/changelog.gz.  Many packages, however, come
> with two kinds of changelogs: a source-level list of changes directed at
> developers, often called ChangeLog in a GNU-type package, and a user-level 
> list
> of changes, sometimes called release notes, often in a file called NEWS in a
> GNU-type package.
> 
> Debian packages appear to handle this in different ways: Some take the policy
> literally and install the ChangeLog as /usr/share/doc/package/changelog.gz and
> then install NEWS as additional documentation in 
> /usr/share/doc/package/NEWS.gz
> or whatever the file is called in the particular case.  Sometimes the source
> package doesn't come with a useful changelog, so they install NEWS or the
> release notes as /usr/share/doc/package/changelog.gz; others would then not
> install a "changelog" and install /usr/share/doc/package/NEWS.gz or some other
> name instead.
> 
> This has two major problems: I think that installing a source-level change 
> list
> is hardly ever useful for a binary package.  Most users would probably rather
> read the release notes, but these are currently not be found at a uniform
> location.  The intent behind all this was probably to give the package user a
> list of user-level changes.  So in that sense most packages do a less than
> ideal job at the moment.

My personal opinion is that a changelog is something where every change
is guaranteed to be logged. (And that guarantee is basically just saying
that if it's ever noticed that a change was not logged, that's indeed a
bug.)
A NEWS file is something different to me in that it's a subset of the
changelog. Sometimes that subset might be equal to the entire changelog
set, but there's no guarantee that NEWS logs every change - rather the
opposite.

If someone hands me something and tells me it's a changelog then I want
to be able to trust the guarantee that nothing is hidden. The reason I
would be reading a changelog is often to try to figure out where a bug
might have sneaked in and that's often in the "uninteresting" changes.
If someone lied to me and gave me a censored changelog they're actively
wasting my time on misleading me.

I thus think a ChangeLog (or "source level changelog") should be
installed as changelog when available (and not when noone is available
or easily generatable).
When a NEWS (or "user level changelog") or similar is available that
should be installed as NEWS.

Additionally possibly clarify that when something is not available from
upstream people should *not* go out of their way to find something to
stuff in there. Many times I've heard that "policy says I have to"
when asking people why they're insisting on showing a bad lie down my
throut when they could have just left it out (because all
policy really said was just "...should ... if ...").

In other words, I support suggestion number 1 (with the extension of
standardizing the NEWS name along side changelog).

> 
> I can think of three possibilities to address this:
> 
> 1. Clarify the policy that a source-level changelog should be installed as
> /usr/share/doc/package/changelog.gz and user-level change lists/release notes
> should be installed as /usr/share/doc/package/NEWS.gz, whichever of these is
> available and deemed useful.  This has the advantage that it is
> backward-compatible with respect to the changelog handling, and it would allow
> users to find the release notes under the familiar name "NEWS".  It would also
> be somewhat consistent with the GNU names for these files and the handling of
> changelog.Debian vs. NEWS.Debian.
> 


My personal preference does thus not go to option number 2 (or 3), but I
would still rather see either of them being used explicitly than leaving
things at the current status quo. If option 2 or 3 is implemented
then atleast I can know that I should personally never care for
/usr/share/doc/package/changelog.gz again.

I also guess that the 

Bug#459427: changelog vs. NEWS handling

[Bill Allombert]

On Thu, Nov 20, 2014 at 09:21:02PM +0100, Jakub Wilk wrote:
 * Peter Eisentraut pete...@gmx.net, 2008-01-06, 14:55:
 I think that installing a source-level change list is hardly ever
 useful for a binary package.
 
 It's normally more useful that no changelog at all. :-)
 
 What I tend to do in my packages is:
 
 if user-level change list exists:
   install it as /usr/share/doc/PACKAGE/changelog
 elsif source-level changelog exists:
   install it as /usr/share/doc/PACKAGE/changelog
 else:
   curl into a ball and cry

Agreed.

Upstream changelog are useful when we need to find which version of a package
has such feature (or such bug). If the user-level changelog does not include 
this
information, but the source-level changelog does, it is nice to ship both.

That being said, splitting between 'user-level changelog' and 'source-level
changelog' is oversimplifying the issue. In practice upstream tarball includes
files which might fit in these categories or not.

In general, we should leave to the package maintainers best judgement to chose
which file to ship as changelog.gz, which files to ship as something else and
which files to omit entirely.

We can provide guideline, but maybe it is more expedient to do that in
devref.

The issue being with use of packaging helper which will pick one file for
changelog.gz without the maintainer having made a conscious choice. 

Cheers,
-- 
Bill. ballo...@debian.org

Imagine a large red swirl here. 


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#459427: changelog vs. NEWS handling

[Andrey Rahmatullin]

On Sun, Jan 06, 2008 at 02:55:00PM +0100, Peter Eisentraut wrote:
 There is some lack of clarity in the policy or perhaps some confusion among
 packagers and thence inconsistencies among packages regarding the handling of
 upstream changelog files.  Policy says that upstream changelogs should be
 installed as /usr/share/doc/package/changelog.gz.  Many packages, however, 
 come
 with two kinds of changelogs: a source-level list of changes directed at
 developers, often called ChangeLog in a GNU-type package, and a user-level 
 list
 of changes, sometimes called release notes, often in a file called NEWS in a
 GNU-type package.
 
 Debian packages appear to handle this in different ways: Some take the policy
 literally and install the ChangeLog as /usr/share/doc/package/changelog.gz and
 then install NEWS as additional documentation in 
 /usr/share/doc/package/NEWS.gz
 or whatever the file is called in the particular case.  Sometimes the source
 package doesn't come with a useful changelog, so they install NEWS or the
 release notes as /usr/share/doc/package/changelog.gz; others would then not
 install a changelog and install /usr/share/doc/package/NEWS.gz or some other
 name instead.
 
 This has two major problems: I think that installing a source-level change 
 list
 is hardly ever useful for a binary package.  Most users would probably rather
 read the release notes, but these are currently not be found at a uniform
 location.  The intent behind all this was probably to give the package user a
 list of user-level changes.  So in that sense most packages do a less than
 ideal job at the moment.
I agree with this rationale.

 I can think of three possibilities to address this:
 
 1. Clarify the policy that a source-level changelog should be installed as
 /usr/share/doc/package/changelog.gz and user-level change lists/release notes
 should be installed as /usr/share/doc/package/NEWS.gz, whichever of these is
 available and deemed useful.  This has the advantage that it is
 backward-compatible with respect to the changelog handling, and it would allow
 users to find the release notes under the familiar name NEWS.  It would also
 be somewhat consistent with the GNU names for these files and the handling of
 changelog.Debian vs. NEWS.Debian.
 
 2. Modify policy to say that source-level changelogs should not be installed
 unless there is some overriding reason.  Also say that user-level release 
 notes
 should be installed as /usr/share/doc/package/changelog.gz.  This has the
 advantage that the currently used name changelog is preserved, but the
 disadvantage would be that it would take on a new meaning for many packages.
 It would also create an inconsistent naming scheme compared to the handling of
 changelog.Debian vs. NEWS.Debian.
 
 3. Modify policy to say that source-level changelogs should not be installed
 unless there is some overriding reason.  If they are installed, they should be
 installed as /usr/share/doc/package/changelog.gz.  Add to policy that
 user-level release notes should be installed as 
 /usr/share/doc/package/NEWS.gz.
 This has the advantage that it would preserve the meaning of the changelog
 file for most packages, but most packages could opt to drop them since they 
 are
 probably useless in most cases.  It would also create a new uniform policy for
 installing upstream release notes, which are currently handled inconsistently,
 and it would use the familiar name NEWS for that file, consistent with
 GNU-type packages and the name NEWS.Debian.
I would prefer not installing the source-level changelogs, then the
logical next step would be installing hand-written changelogs as
changelog.gz. I think at least some of packages where there is no
source-level changelog in the upstream source already install hand-written
changelogs (whatever their upstream name) as changelog.gz.
dh_installchangelogs(1) already installs one of the following as changelog.gz:
{.,doc,docs}/{changelog,changes,changelog.txt,changes.txt,history,history.txt,changelog.md}
At least some of these files are usually hand-written.
So, I want to hear other opinions but my opinion is closer to the option 2
above.

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Bug#459427: changelog vs. NEWS handling

[Jakub Wilk]

* Peter Eisentraut pete...@gmx.net, 2008-01-06, 14:55:
I think that installing a source-level change list is hardly ever 
useful for a binary package.


It's normally more useful that no changelog at all. :-)

What I tend to do in my packages is:

if user-level change list exists:
install it as /usr/share/doc/PACKAGE/changelog
elsif source-level changelog exists:
install it as /usr/share/doc/PACKAGE/changelog
else:
curl into a ball and cry


This sounds a bit like:

2. Modify policy to say that source-level changelogs should not be 
installed unless there is some overriding reason. Also say that 
user-level release notes should be installed as 
/usr/share/doc/package/changelog.gz. This has the advantage that the 
currently used name changelog is preserved, but the disadvantage 
would be that it would take on a new meaning for many packages. It 
would also create an inconsistent naming scheme compared to the 
handling of changelog.Debian vs. NEWS.Debian.


--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#459427: changelog vs. NEWS handling

[Peter Eisentraut]

Package: debian-policy
Version: 3.7.3.0
Severity: normal

There is some lack of clarity in the policy or perhaps some confusion among
packagers and thence inconsistencies among packages regarding the handling of
upstream changelog files.  Policy says that upstream changelogs should be
installed as /usr/share/doc/package/changelog.gz.  Many packages, however, come
with two kinds of changelogs: a source-level list of changes directed at
developers, often called ChangeLog in a GNU-type package, and a user-level list
of changes, sometimes called release notes, often in a file called NEWS in a
GNU-type package.

Debian packages appear to handle this in different ways: Some take the policy
literally and install the ChangeLog as /usr/share/doc/package/changelog.gz and
then install NEWS as additional documentation in /usr/share/doc/package/NEWS.gz
or whatever the file is called in the particular case.  Sometimes the source
package doesn't come with a useful changelog, so they install NEWS or the
release notes as /usr/share/doc/package/changelog.gz; others would then not
install a changelog and install /usr/share/doc/package/NEWS.gz or some other
name instead.

This has two major problems: I think that installing a source-level change list
is hardly ever useful for a binary package.  Most users would probably rather
read the release notes, but these are currently not be found at a uniform
location.  The intent behind all this was probably to give the package user a
list of user-level changes.  So in that sense most packages do a less than
ideal job at the moment.

I can think of three possibilities to address this:

1. Clarify the policy that a source-level changelog should be installed as
/usr/share/doc/package/changelog.gz and user-level change lists/release notes
should be installed as /usr/share/doc/package/NEWS.gz, whichever of these is
available and deemed useful.  This has the advantage that it is
backward-compatible with respect to the changelog handling, and it would allow
users to find the release notes under the familiar name NEWS.  It would also
be somewhat consistent with the GNU names for these files and the handling of
changelog.Debian vs. NEWS.Debian.

2. Modify policy to say that source-level changelogs should not be installed
unless there is some overriding reason.  Also say that user-level release notes
should be installed as /usr/share/doc/package/changelog.gz.  This has the
advantage that the currently used name changelog is preserved, but the
disadvantage would be that it would take on a new meaning for many packages.
It would also create an inconsistent naming scheme compared to the handling of
changelog.Debian vs. NEWS.Debian.

3. Modify policy to say that source-level changelogs should not be installed
unless there is some overriding reason.  If they are installed, they should be
installed as /usr/share/doc/package/changelog.gz.  Add to policy that
user-level release notes should be installed as /usr/share/doc/package/NEWS.gz.
This has the advantage that it would preserve the meaning of the changelog
file for most packages, but most packages could opt to drop them since they are
probably useless in most cases.  It would also create a new uniform policy for
installing upstream release notes, which are currently handled inconsistently,
and it would use the familiar name NEWS for that file, consistent with
GNU-type packages and the name NEWS.Debian.



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]