Bug#970234: consider dropping "No hard links in source packages"

[Russ Allbery]

Guillem Jover  writes:

> I'm not really sure what the footnote really refers to, TBH, as I'm not
> aware of any such check, or what would require a fair amount of
> work.

Yeah, it seems to be a mystery to everyone.  There is an explicit entry in
the debian/changelog of Policy from Ian Jackson about it for version
2.1.1, but all it says is:

  * Hard links are forbidden in source packages (they didn't work anyway,
and can't easily be made to work reliably).

This is from when Ian was maintaining dpkg, so presumably he thought
something was broken at the time, but it seems to have been lost in
history.  They do obviously work now.

> Besides the other potential issues mentioned on the bug, which I agree
> we might not care much about, a case that comes to mind would be that
> patching hard linked source files can end up being very confusing, and
> might not really break the build but can end up not producing what one
> might expect. I've quickly prepared the attached tentative and
> completely untested patch, to add a warning in that case, which I guess
> I might be targeting for dpkg 1.22.1 (once I've added some tests).

Thanks, that does seem like a good idea.

> But given that hard links in source packages do not seem prevalent at
> all, and that the tooling or linters can be improved in that direction I
> suppose it might make sense to lift this specific restriction.

Thank you for the review!  Now applied for the next release of Policy.

-- 
Russ Allbery (r...@debian.org)  

Bug#970234: consider dropping "No hard links in source packages"

[Edward Little]

Please remove the following email address:  e.little...@gmail.com

On Sun, Sep 10, 2023 at 1:45 PM Russ Allbery  wrote:

> This patch is still waiting for one more second.  It was previously
> seconded by Helmut.
>
> Russ Allbery  writes:
>
> > Here is a patch dropping the restriction on hard links in source
> > packages that I think is ready for seconds.  I'm copying Guillem for his
> > review, in case there's some dpkg concern.
>
> > From 12b014c4b930577a728dfb1254b64aac6a5eb1e0 Mon Sep 17 00:00:00 2001
> > From: Russ Allbery 
> > Date: Thu, 22 Sep 2022 19:15:52 -0700
> > Subject: [PATCH] Allow hard links in source packages
>
> > It's not clear why this restriction was in place, and Debian
> > included a package containing hard links without anyone noticing
> > in the last release.
> > ---
> >  policy/ch-source.rst | 11 ++-
> >  1 file changed, 2 insertions(+), 9 deletions(-)
>
> > diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> > index c7415fc..a7df539 100644
> > --- a/policy/ch-source.rst
> > +++ b/policy/ch-source.rst
> > @@ -282,8 +282,8 @@ source files in a package, as far as is reasonably
> possible.  [#]_
> >  Restrictions on objects in source packages
> >  --
> >
> > -The source package must not contain any hard links,  [#]_ device special
> > -files, sockets or setuid or setgid files.. [#]_
> > +The source package must not contain device special files, sockets, or
> > +setuid or setgid files. [#]_
> >
> >  .. _s-debianrules:
> >
> > @@ -918,13 +918,6 @@ must not exist a file ``debian/patches/foo.series``
> for any ``foo``.
> > would be nice if the modification time of the upstream source would
> > be preserved.
> >
> > -.. [#]
> > -   This is not currently detected when building source packages, but
> > -   only when extracting them.
> > -
> > -   Hard links may be permitted at some point in the future, but would
> > -   require a fair amount of work.
> > -
> >  .. [#]
> > Setgid directories are allowed.
>
> --
> Russ Allbery (r...@debian.org)  
>
>

Bug#970234: consider dropping "No hard links in source packages"

[Guillem Jover]

Hi!

On Thu, 2022-09-22 at 19:20:00 -0700, Russ Allbery wrote:
> Russ Allbery  writes:
> > The fact that this has gone unnoticed in a source package in an existing
> > release makes a pretty strong argument that nothing in Debian cares and
> > we should just remove the constraint.
> 
> Here is a patch dropping the restriction on hard links in source packages
> that I think is ready for seconds.  I'm copying Guillem for his review, in
> case there's some dpkg concern.

I'm not really sure what the footnote really refers to, TBH, as I'm
not aware of any such check, or what would require a fair amount of
work. Besides the other potential issues mentioned on the bug, which
I agree we might not care much about, a case that comes to mind would
be that patching hard linked source files can end up being very
confusing, and might not really break the build but can end up not
producing what one might expect. I've quickly prepared the attached
tentative and completely untested patch, to add a warning in that case,
which I guess I might be targeting for dpkg 1.22.1 (once I've added
some tests).

But given that hard links in source packages do not seem prevalent
at all, and that the tooling or linters can be improved in that
direction I suppose it might make sense to lift this specific
restriction.

> >From 12b014c4b930577a728dfb1254b64aac6a5eb1e0 Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Thu, 22 Sep 2022 19:15:52 -0700
> Subject: [PATCH] Allow hard links in source packages
> 
> It's not clear why this restriction was in place, and Debian
> included a package containing hard links without anyone noticing
> in the last release.
> ---
>  policy/ch-source.rst | 11 ++-
>  1 file changed, 2 insertions(+), 9 deletions(-)
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index c7415fc..a7df539 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -282,8 +282,8 @@ source files in a package, as far as is reasonably 
> possible.  [#]_
>  Restrictions on objects in source packages
>  --
>  
> -The source package must not contain any hard links,  [#]_ device special
> -files, sockets or setuid or setgid files.. [#]_
> +The source package must not contain device special files, sockets, or
> +setuid or setgid files. [#]_
>  
>  .. _s-debianrules:
>  
> @@ -918,13 +918,6 @@ must not exist a file ``debian/patches/foo.series`` for 
> any ``foo``.
> would be nice if the modification time of the upstream source would
> be preserved.
>  
> -.. [#]
> -   This is not currently detected when building source packages, but
> -   only when extracting them.
> -
> -   Hard links may be permitted at some point in the future, but would
> -   require a fair amount of work.
> -
>  .. [#]
> Setgid directories are allowed.

Seconded.

Thanks,
Guillem
diff --git i/scripts/Dpkg/Source/Patch.pm w/scripts/Dpkg/Source/Patch.pm
index 0da35b9e5..99020885b 100644
--- i/scripts/Dpkg/Source/Patch.pm
+++ w/scripts/Dpkg/Source/Patch.pm
@@ -513,6 +513,11 @@ sub analyze {
 	error(g_("diff '%s' patches something which is not a plain file"),
 	  $diff);
 	}
+my $nlink = (stat _)[3];
+if ($nlink > 1) {
+warning(g_("diff '%s' patches hard link %s which can have " .
+   "unintended consequences"), $diff, $fn);
+}
 
 	if ($filepatched{$fn}) {
 $filepatched{$fn}++;


signature.asc
Description: PGP signature

Bug#970234: consider dropping "No hard links in source packages"

[Russ Allbery]

This patch is still waiting for one more second.  It was previously
seconded by Helmut.

Russ Allbery  writes:

> Here is a patch dropping the restriction on hard links in source
> packages that I think is ready for seconds.  I'm copying Guillem for his
> review, in case there's some dpkg concern.

> From 12b014c4b930577a728dfb1254b64aac6a5eb1e0 Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Thu, 22 Sep 2022 19:15:52 -0700
> Subject: [PATCH] Allow hard links in source packages

> It's not clear why this restriction was in place, and Debian
> included a package containing hard links without anyone noticing
> in the last release.
> ---
>  policy/ch-source.rst | 11 ++-
>  1 file changed, 2 insertions(+), 9 deletions(-)

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index c7415fc..a7df539 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -282,8 +282,8 @@ source files in a package, as far as is reasonably 
> possible.  [#]_
>  Restrictions on objects in source packages
>  --
>  
> -The source package must not contain any hard links,  [#]_ device special
> -files, sockets or setuid or setgid files.. [#]_
> +The source package must not contain device special files, sockets, or
> +setuid or setgid files. [#]_
>  
>  .. _s-debianrules:
>  
> @@ -918,13 +918,6 @@ must not exist a file ``debian/patches/foo.series`` for 
> any ``foo``.
> would be nice if the modification time of the upstream source would
> be preserved.
>  
> -.. [#]
> -   This is not currently detected when building source packages, but
> -   only when extracting them.
> -
> -   Hard links may be permitted at some point in the future, but would
> -   require a fair amount of work.
> -
>  .. [#]
> Setgid directories are allowed.

-- 
Russ Allbery (r...@debian.org)  

Bug#970234: consider dropping "No hard links in source packages"

[Helmut Grohne]

Hi Russ,

On Thu, Sep 22, 2022 at 07:20:00PM -0700, Russ Allbery wrote:
> From 12b014c4b930577a728dfb1254b64aac6a5eb1e0 Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Thu, 22 Sep 2022 19:15:52 -0700
> Subject: [PATCH] Allow hard links in source packages
> 
> It's not clear why this restriction was in place, and Debian
> included a package containing hard links without anyone noticing
> in the last release.
> ---
>  policy/ch-source.rst | 11 ++-
>  1 file changed, 2 insertions(+), 9 deletions(-)
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index c7415fc..a7df539 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -282,8 +282,8 @@ source files in a package, as far as is reasonably 
> possible.  [#]_
>  Restrictions on objects in source packages
>  --
>  
> -The source package must not contain any hard links,  [#]_ device special
> -files, sockets or setuid or setgid files.. [#]_
> +The source package must not contain device special files, sockets, or
> +setuid or setgid files. [#]_
>  
>  .. _s-debianrules:
>  
> @@ -918,13 +918,6 @@ must not exist a file ``debian/patches/foo.series`` for 
> any ``foo``.
> would be nice if the modification time of the upstream source would
> be preserved.
>  
> -.. [#]
> -   This is not currently detected when building source packages, but
> -   only when extracting them.
> -
> -   Hard links may be permitted at some point in the future, but would
> -   require a fair amount of work.
> -
>  .. [#]
> Setgid directories are allowed.
>  

Seconded.

Helmut


signature.asc
Description: PGP signature

Bug#970234: consider dropping "No hard links in source packages"

[Russ Allbery]

Russ Allbery  writes:

> The fact that this has gone unnoticed in a source package in an existing
> release makes a pretty strong argument that nothing in Debian cares and
> we should just remove the constraint.

Here is a patch dropping the restriction on hard links in source packages
that I think is ready for seconds.  I'm copying Guillem for his review, in
case there's some dpkg concern.

-- 
Russ Allbery (r...@debian.org)  

>From 12b014c4b930577a728dfb1254b64aac6a5eb1e0 Mon Sep 17 00:00:00 2001
From: Russ Allbery 
Date: Thu, 22 Sep 2022 19:15:52 -0700
Subject: [PATCH] Allow hard links in source packages

It's not clear why this restriction was in place, and Debian
included a package containing hard links without anyone noticing
in the last release.
---
 policy/ch-source.rst | 11 ++-
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index c7415fc..a7df539 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -282,8 +282,8 @@ source files in a package, as far as is reasonably possible.  [#]_
 Restrictions on objects in source packages
 --
 
-The source package must not contain any hard links,  [#]_ device special
-files, sockets or setuid or setgid files.. [#]_
+The source package must not contain device special files, sockets, or
+setuid or setgid files. [#]_
 
 .. _s-debianrules:
 
@@ -918,13 +918,6 @@ must not exist a file ``debian/patches/foo.series`` for any ``foo``.
would be nice if the modification time of the upstream source would
be preserved.
 
-.. [#]
-   This is not currently detected when building source packages, but
-   only when extracting them.
-
-   Hard links may be permitted at some point in the future, but would
-   require a fair amount of work.
-
 .. [#]
Setgid directories are allowed.
 
-- 
2.37.2

Bug#970234: consider dropping "No hard links in source packages"

[Sam Hartman]

> "Russ" == Russ Allbery  writes:

Russ> Sam Hartman  writes:

>> I think that hard links in a source package are fine provided
>> that breaking the hard links would not either break the build or
>> provide an unreasonable space multiplier.

Russ> I agree with you that those would be undesirable properties,
Russ> but are they likely and important enough to have it be worth
Russ> retaining a section talking about this, as opposed to using
Russ> the "not every bug is a Policy violation" rule?  We do pay a
Russ> (small) complexity cost for each additional requirement we put
Russ> in Policy, so I'm tempted to just drop this entirely and bank
Russ> the complexity reduction.

You have me sold on not likely enough to matter.
I do think that having the build break (or worse produce bad results) on
hard link breaking would be a really important problem because of how
much it violates the principle of least surprise, but I agree that it is
unlikely enough that it need not be covered in policy.

--Sam

Bug#970234: consider dropping "No hard links in source packages"

[Russ Allbery]

Helmut Grohne  writes:

> Jakub stumbled into the "No hard links in source packages" requirement
> added around 1996 and couldn't make sense of it. Neither could Christoph
> nor myself. tar does support hard links just fine. lintian does not
> check this property. sugar-log-activity/38 is an example package
> violating the property. It is shipped in buster and technically rc-buggy
> though no bug is filed about it.

> I believe that the requriement needs a rationale. Failing that, it
> should be dropped.

I'm inclined to agree with you on this, but it's probably worth mentioning
somewhere in this bug that the AFS file system is still used and still, so
far as I know, does not support cross-directory hard links because of its
per-directory ACL system.  I'm not sure what tar does in this situation:
whether it fails or whether it just silently duplicates the file.

I'm not sure that we care, but that sort of concern (along with a general
dislike of hard links) is probably the original rationale.

The fact that this has gone unnoticed in a source package in an existing
release makes a pretty strong argument that nothing in Debian cares and we
should just remove the constraint.

Sam Hartman  writes:

> I think that hard links in a source package are fine provided that
> breaking the hard links would not either break the build or provide an
> unreasonable space multiplier.

I agree with you that those would be undesirable properties, but are they
likely and important enough to have it be worth retaining a section
talking about this, as opposed to using the "not every bug is a Policy
violation" rule?  We do pay a (small) complexity cost for each additional
requirement we put in Policy, so I'm tempted to just drop this entirely
and bank the complexity reduction.

-- 
Russ Allbery (r...@debian.org)  

Bug#970234: consider dropping "No hard links in source packages"

[Sam Hartman]

> "Bill" == Bill Allombert  writes:


Bill> I am pretty sure we were concerned about source packages being
Bill> unpackable on non Debian systems, though.

And I think we probably still are.  I was trying to capture the concerns
there in the part of my message you trimmed.


My rationale is that I don't think we want to work around an upstream
build system or repack sources just because it has hard links.  On the
other hand I also don't think we want to depend on hard links being
preserved.

Bug#970234: consider dropping "No hard links in source packages"

[Bill Allombert]

On Tue, Oct 13, 2020 at 09:44:42AM -0400, Sam Hartman wrote:
> > "Giacomo" == Giacomo Catenazzi  writes:
> 
> Giacomo> The rationale was probably similar so symlinks: they may
> Giacomo> fail across different filesystems, and we supported to have
> Giacomo> e.g. / /usr /usr/share /usr/local /var (and various /var/*)
> Giacomo> /home /tmp /boot etc on different file systems. Now we are
> Giacomo> more strict on where we can split filesystems (and disk are
> Giacomo> larger, and LVM simplified much of filesystem handling).
> 
> But I think even in 1996, we anticipated a single source package
> (*source package*) being unpacked on a single filesystem.
> Perhaps we were worried about filesystems like umsdos?

It is good to see I am not the only one left who remember about umsdos!

I am pretty sure we were concerned about source packages being
unpackable on non Debian systems, though.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#970234: consider dropping "No hard links in source packages"

[Sam Hartman]

> "Giacomo" == Giacomo Catenazzi  writes:

Giacomo> The rationale was probably similar so symlinks: they may
Giacomo> fail across different filesystems, and we supported to have
Giacomo> e.g. / /usr /usr/share /usr/local /var (and various /var/*)
Giacomo> /home /tmp /boot etc on different file systems. Now we are
Giacomo> more strict on where we can split filesystems (and disk are
Giacomo> larger, and LVM simplified much of filesystem handling).

But I think even in 1996, we anticipated a single source package
(*source package*) being unpacked on a single filesystem.
Perhaps we were worried about filesystems like umsdos?


I think that hard links in a source package are fine provided that
breaking the hard links would not either break the build or provide an
unreasonable space multiplier.

Bug#970234: consider dropping "No hard links in source packages"

[Giacomo Catenazzi]

Hello Helmut

On 12.10.2020 19:30, Helmut Grohne wrote:


You appear to be talking about binary packages. This bug is about source
packages. When you unpack a source package, you are creating a directory
hiearchy rooted at the point where you start unpacking. There is not
possibly any reasonable way to split your source package into multiple
file systems. This is very different from binary packages where the
underlying hiearchy is shared with other packages and directories
frequently already exist.


Your are totally right. And it is also on the subject line.

So: I have not reasonable explanation.

ciao
cate

Bug#970234: consider dropping "No hard links in source packages"

[Helmut Grohne]

Hi cate,

On Mon, Oct 12, 2020 at 04:10:00PM +0200, Giacomo Catenazzi wrote:
> The rationale was probably similar so symlinks: they may fail across
> different filesystems, and we supported to have e.g. / /usr /usr/share
> /usr/local /var (and various /var/*) /home /tmp /boot etc on different file
> systems. Now we are more strict on where we can split filesystems (and disk
> are larger, and LVM simplified much of filesystem handling).

You appear to be talking about binary packages. This bug is about source
packages. When you unpack a source package, you are creating a directory
hiearchy rooted at the point where you start unpacking. There is not
possibly any reasonable way to split your source package into multiple
file systems. This is very different from binary packages where the
underlying hiearchy is shared with other packages and directories
frequently already exist.

> I think a hardlink on same directory should be fine, or within directories
> which must be on the same filesystem.

I argue that all files within a source package are always located on the
same filesystem, because the unpack step creates the source package root
directory on one file system and everything else resides on that very
filesystem.

For binary packages, restricting the use of symlinks makes a lot more
sense to me.

Helmut

Bug#970234: consider dropping "No hard links in source packages"

[Andrey Rahmatullin]

On Mon, Oct 12, 2020 at 05:05:43PM +0200, Giacomo Catenazzi wrote:
> > > Now we are more strict on where we can split filesystems
> > What do you mean?
> 
> If I remember correctly, now we do not support / and /usr to be on a
> different filesystems
Not really, please read
https://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken/

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Bug#970234: consider dropping "No hard links in source packages"

[Giacomo Catenazzi]

On 12.10.2020 16:22, Andrey Rahmatullin wrote:

On Mon, Oct 12, 2020 at 04:10:00PM +0200, Giacomo Catenazzi wrote:

Now we are more strict on where we can split filesystems

What do you mean?


If I remember correctly, now we do not support / and /usr to be on a 
different filesystems, and I think there were few other corner cases 
which were forbidden.


ciao
cate

Bug#970234: consider dropping "No hard links in source packages"

[Andrey Rahmatullin]

On Mon, Oct 12, 2020 at 04:10:00PM +0200, Giacomo Catenazzi wrote:
> Now we are more strict on where we can split filesystems
What do you mean?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Bug#970234: consider dropping "No hard links in source packages"

[Giacomo Catenazzi]

On 13.09.2020 12:52, Helmut Grohne wrote:

Package: debian-policy
Version: 4.5.0.3
Severity: wishlist

Jakub stumbled into the "No hard links in source packages" requirement
added around 1996 and couldn't make sense of it. Neither could Christoph
nor myself. tar does support hard links just fine. lintian does not
check this property. sugar-log-activity/38 is an example package
violating the property. It is shipped in buster and technically
rc-buggy though no bug is filed about it.

I believe that the requriement needs a rationale. Failing that, it
should be dropped.



The rationale was probably similar so symlinks: they may fail across 
different filesystems, and we supported to have e.g. / /usr /usr/share 
/usr/local /var (and various /var/*) /home /tmp /boot etc on different 
file systems. Now we are more strict on where we can split filesystems 
(and disk are larger, and LVM simplified much of filesystem handling).


I think a hardlink on same directory should be fine, or within 
directories which must be on the same filesystem.


ciao
cate


[for symlinks we have the problem with relative links (containing "../") 
passing filesystem boundaries]

Bug#970234: consider dropping "No hard links in source packages"

[Helmut Grohne]

Package: debian-policy
Version: 4.5.0.3
Severity: wishlist

Jakub stumbled into the "No hard links in source packages" requirement
added around 1996 and couldn't make sense of it. Neither could Christoph
nor myself. tar does support hard links just fine. lintian does not
check this property. sugar-log-activity/38 is an example package
violating the property. It is shipped in buster and technically
rc-buggy though no bug is filed about it.

I believe that the requriement needs a rationale. Failing that, it
should be dropped.

Helmut