Bug#821363: debian-policy: Allow line-end comments in all Debian packaging control files

[Guillem Jover]

[ Also redireting to the bug report. ]

On Wed, 2016-04-20 at 11:29:05 +1000, Ben Finney wrote:
> Guillem Jover  writes:
> > In the case of the copyright file, the document specifying it does not
> > list # as valid comment markers.
> 
> When we were drafting that specification, I at least intended this text:
> 
>The syntax of the file is the same as for other Debian control files,
>as specified in the Debian Policy Manual. See its section 5.1 for
>details.
> 
> to mean the same syntax as ‘debian/control’. I wasn't even aware when
> drafting it, that a question of comments in one file but not another
> could arise based on the file name.

See my other mail.

> Certainly my intention in that specification was for comments to be
> allowed in the ‘debian/copyright’ file just as they are in
> ‘debian/control’.

If that was clear to everyone then adding it should be fine, indeed.
And then parsers not supporting comments should just be considered
buggy.

> > For example parsing debian/copyright files (or any other file using
> > deb822-based syntax) via the Dpkg::Control::Hash perl module would
> > make it accept such comments, but I don't know if other parsers will
> > accept those too.
> 
> That's a good criterion. Which parsers would you suggest we verify? The
> Emacs mode, the Dpkg Perl modules, and other tools I have used do not
> complain when comments are used in any control-syntax file.
> 
> What would constitute a sufficient degree of investigation of existing
> parsers for this issue?

Some time ago I created a wiki page intended to track this kind of
issues and support in software handling deb822-style syntax:

  

But didn't have the time to check other implementations and add more
detailed features. But I think that might be a good place to track
support for this too.

The list of additional parsers depends on whether you want to check
for support in deb822 parsers in general or just for debian/copyright.
For the latter I'd add at least:

  * libconfig-model-dpkg-perl
  * libdebian-copyright-perla
  * license-reconcile
  * adequate
  * debmake (?)
  * dpkg-dev-el
  * devscripts (?)

For the former also:

  * apt
  * python-apt
  * python-debian
  * ruby-debian
  * vim
  * grep-dctrl
  * lintian

and probably several of the ones listed in either:

  
  

Thanks,
Guillem

Bug#821363: debian-policy: Allow line-end comments in all Debian packaging control files

[Guillem Jover]

[ Switching from d-p@l.d.o to the bug so that the thread is tracked. ]

On Wed, 2016-04-20 at 11:23:19 +1000, Ben Finney wrote:
> Guillem Jover  writes:
> > On Mon, 2016-04-18 at 14:52:22 +1000, Ben Finney wrote:
> > > What is the rationale for explicitly disallowing line-end comments
> > > in any but that one file?
> >
> > This just predates other files in source packages using deb822 syntax.
> > The reason is that the control file in a binary package does not
> > support comments for example.

> Should the wording at least be changed so that it only prohibits
> line-end comments in binary package control file?

I think an explicit list of files supporting comments instead of one
of those not supporting them would be better.

> I ask this because I'm pretty sure that people who hand-wave a
> specification as “same syntax as ‘debian/control’ file” intend for it to
> have all the allowed syntax of that file, including comments.

Perhaps, but in the end it does not matter, if the file has not been
explicitly specified as supporting them and the current code handling
those other files cannot handle comments anyway.

There are more files with deb822 style syntax that are neither in
source nor binary packages, several of which I'm doubtful might support
comments(?):

  * (In)Release, Packages, Sources, Translation files in repositories.
  * .dsc files
  * .changes files.
  * dpkg origins (or vendor file), specified as supporting comments.
  * dpkg status file (not supported).
  * …

So, as stated I'd list them explicitly, or restrict to only files inside
a source package perhaps. I'll be adding the origins file to the new
deb822(5) man page describing the format in dpkg's git master, though.

Thanks,
Guillem

Bug#821363: debian-policy: Allow line-end comments in all Debian packaging control files

[Guillem Jover]

Hi!

On Mon, 2016-04-18 at 14:52:22 +1000, Ben Finney wrote:
> Package: debian-policy
> Severity: normal
> Control: tags -1 patch

> The specification of Debian control files in Policy §5.1 says:
> 
> Lines starting with # without any preceding whitespace are
> comments lines that are only permitted in source package control
> files (debian/control). These comment lines are ignored, even
> between two continuation lines. They do not end logical lines.
> 
> What is the rationale for explicitly disallowing line-end comments in
> any but that one file?

This just predates other files in source packages using deb822 syntax.
The reason is that the control file in a binary package does not
support comments for example. If you want to extend this list, please
make sure each and every file you want to add supports comments by
the major parsers. Remember that policy tends to be descriptive not
prescriptive.

> The ‘debian/copyright’ file is an example of a file of the described
> syntax, that benefits from line-end comments (e.g. for editor hints,
> or for temporarily disabling some lines, or for explaining an unusual
> value for a field).

The fact that it might benefit from it does not mean that this should
be documented as such w/o getting agreement that this is alright, and
implementing the support in the major parsers. In the case of the
copyright file, the document specifying it does not list # as valid
comment markers.

For example parsing debian/copyright files (or any other file using
deb822-based syntax) via the Dpkg::Control::Hash perl module would
make it accept such comments, but I don't know if other parsers will
accept those too.

> If there is no good rationale to categorically deny all line end
> comments in other files with that syntax, I suggest the attached
> patches.

Yes, they are probably just not supported.

Thanks,
Guillem

Bug#821363: debian-policy: Allow line-end comments in all Debian packaging control files

[Ben Finney]

Package: debian-policy
Severity: normal
Control: tags -1 patch

The specification of Debian control files in Policy §5.1 says:

Lines starting with # without any preceding whitespace are
comments lines that are only permitted in source package control
files (debian/control). These comment lines are ignored, even
between two continuation lines. They do not end logical lines.

What is the rationale for explicitly disallowing line-end comments in
any but that one file?

The ‘debian/copyright’ file is an example of a file of the described
syntax, that benefits from line-end comments (e.g. for editor hints,
or for temporarily disabling some lines, or for explaining an unusual
value for a field).

There are likely other Debian packaging files of the same syntax where
line-end comments are useful.

If there is no good rationale to categorically deny all line end
comments in other files with that syntax, I suggest the attached
patches.

-- 
 \ “What is it that makes a complete stranger dive into an icy |
  `\   river to save a solid gold baby? Maybe we'll never know.” —Jack |
_o__)   Handey |
Ben Finney 
From 04bffa59187df69969d9736dc865514fcd30b840 Mon Sep 17 00:00:00 2001
From: Ben Finney 
Date: Mon, 18 Apr 2016 13:31:14 +1000
Subject: [PATCH 1/2] Clarify which characters constitute syntax of control
 files.

---
 policy.sgml | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/policy.sgml b/policy.sgml
index 404dc737..236bf743 100644
--- a/policy.sgml
+++ b/policy.sgml
@@ -2541,13 +2541,16 @@ endif
 	
 
 	
-	  Each paragraph consists of a series of data fields.  Each field
-	  consists of the field name followed by a colon and then the
-	  data/value associated with that field.  The field name is
-	  composed of US-ASCII characters excluding control characters,
-	  space, and colon (i.e., characters in the ranges 33-57 and
-	  59-126, inclusive).  Field names must not begin with the comment
-	  character, #, nor with the hyphen character, -.
+	  Each paragraph consists of a series of data fields. Each
+	  field consists of the field name followed by a colon and
+	  then the data/value associated with that field. The field
+	  name is composed of US-ASCII characters excluding control
+	  characters, space, and colon (i.e., characters in the ranges
+	  U+0021 “!” through U+0039 “9”, and U+003B
+	  “;” through U+007E “~”, inclusive). Field
+	  names must not begin with the comment character (U+0023
+	  “#”), nor with the hyphen character (U+002D
+	  “-”).
 	
 
 	
@@ -2624,17 +2627,20 @@ Package: libc6
 	
 
 	
-	  Paragraph separators (empty lines) and lines consisting only of
-	  spaces and tabs are not allowed within field values or between
-	  fields.  Empty lines in field values are usually escaped by
-	  representing them by a space followed by a dot.
+	  Paragraph separators (empty lines), and lines consisting
+	  only of U+0020 SPACE and U+0009 TAB, are not allowed within
+	  field values or between fields. Empty lines in field values
+	  are usually escaped by representing them by a U+0020 SPACE
+	  followed by a U+002E “.”.
 	
 
 	
-	  Lines starting with # without any preceding whitespace are comments
-	  lines that are only permitted in source package control files
-	  (debian/control).  These comment lines are ignored, even
-	  between two continuation lines.  They do not end logical lines.
+	  Lines starting with U+0023 “#”, without any
+	  preceding whitespace, are comment lines that are only
+	  permitted in source package control files
+	  (debian/control). These comment lines are
+	  ignored, even between two continuation lines. They do not
+	  end logical lines.
 	
 
 	
-- 
2.8.0.rc3

From 3a00ca0e08f19926a6e64e0e47a8d823eda9846f Mon Sep 17 00:00:00 2001
From: Ben Finney 
Date: Mon, 18 Apr 2016 13:46:44 +1000
Subject: [PATCH 2/2] Remove restriction of which control files may contain
 comments.

---
 policy.sgml | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/policy.sgml b/policy.sgml
index 236bf743..9aca360c 100644
--- a/policy.sgml
+++ b/policy.sgml
@@ -2636,9 +2636,7 @@ Package: libc6
 
 	
 	  Lines starting with U+0023 “#”, without any
-	  preceding whitespace, are comment lines that are only
-	  permitted in source package control files
-	  (debian/control). These comment lines are
+	  preceding whitespace, are comment lines. These comment lines are
 	  ignored, even between two continuation lines. They do not
 	  end logical lines.
 	
-- 
2.8.0.rc3



signature.asc
Description: PGP signature