Bug#1050709: tar-ignore debian/source/options
Sean Whitton: Hello Niels, [...] dgit will never let you accidentally perform an upload containing something that's in the tree but not in git, and you can use .git/info/exclude as a local way to ensure it's never in git. So a simple policy of always doing uploads with dgit might achieve what you want, without introducing all these deltas between source packages, git, local trees etc.? Hi Sean Appreciate the suggestion. The suggestion assumes that I am using dgit or ready to migrate my workflow to use dgit, which I am not at this stage. I was open to doing to making my package work better for people that use dgit if it has a "near zero cost" to my current packaging work flow. It is regular that other people upload debhelper and some of them use dgit. The proposal of replacing `tar-ignore` with its expanded version minus `tar-ignore=.gitignore` sadly does not fit that "near zero cost" bill for me. For now, I am afraid my packages will not be compatible with dgit until the feature interaction between dpkg-source -I/tar-ignore and dgit is resolved. Best regards, Niels
Bug#1050709: tar-ignore debian/source/options
Hello Niels, On Mon 28 Aug 2023 at 06:56pm +02, Niels Thykier wrote: > So this is a common pattern in my packages although it sometimes appears in > d/s/local-options rather than d/s/options. > > Basically, the issue is when you have something you want to have locally, not > in git and also not in the archive. In one my other packages, I have a > "local" directory filled with local work items (including a full copy of the > sudo deb package for testing purposes). Here the "local" directory is listed > both in .gitignore and in "tar-ignore", because I do not want it in git nor in > the archive when I do an upload. > > To solve this, I add `tar-ignore=...` (for native packages) to > debian/source/(local-)options. However, if you add that option, you end up > with the entire *.git* directory in the tarball. To avoid that, I add the > `tar-ignore` on its own to get the sane defaults back. > > This then breaks dgit leading to this bug, which is kind of a catch-22 if you > want local specific things (IDE files or local prototype stuff) that you > guarantee is excluded from git and dpkg-source automatically and also support > dgit at the same time. dgit will never let you accidentally perform an upload containing something that's in the tree but not in git, and you can use .git/info/exclude as a local way to ensure it's never in git. So a simple policy of always doing uploads with dgit might achieve what you want, without introducing all these deltas between source packages, git, local trees etc.? -- Sean Whitton signature.asc Description: PGP signature
Bug#1050709: tar-ignore debian/source/options
Control: clone -1 -2
Control: reassign -2 debhelper
Control: retitle -2 Please adjust debian/source/options
Hi.
FTR, I didn't intend to draw this bug to your attention as the
debhelper maintainer. #1050709 was addressed to dgit maintainers
about about how to deal with this situation in dgit.
I suggested in dgit that we might do one or more of the following
(numbering addded):
| 1. File a bug or MR against debhelper
| 2. Somehow ask that dpkg-source do something about this, but what ?
| 3. Have dgit detect this situation and at least explain it to the user
| 4. Have dgit generate a non-roundtrippable source package
|(probably very hard and also undesirable)
I am working on (3). (4) seems wrong. As for (2), that's already
#908742 and #908747.
Since you've engaged, let me do (1) now, and clone this bug
and address you as debhelper maintainer:
The fundamental principle of operation of dgit is that the git tree[1]
is *identical*[2] to result of dpkg-source -x.
Niels Thykier writes ("Bug#1050709: tar-ignore debian/source/options"):
> So this is a common pattern in my packages although it sometimes appears
> in d/s/local-options rather than d/s/options.
dgit refuses to work with d/s/local-options, because they cannot be
included in the source package. In debhelper it seems that someone
moved the ignores to d/s/options, probably for this reason.
> Basically, the issue is when you have something you want to have
> locally, not in git and also not in the archive.
That's fine. The correct approach is to make dpkg-source and git
agree with each other. The problem with plain tar-ignore is that it
ignores .gitignore, which *is* in your git tree and is not ignored by
git.
See https://bugs.debian.org/908747
> Here the "local" directory is listed both in .gitignore and in
> "tar-ignore", because I do not want it in git nor in the archive when I
> do an upload.
Yes, that makes sense.
> To solve this, I add `tar-ignore=...` (for native packages) to
> debian/source/(local-)options. However, if you add that option, you end
> up with the entire *.git* directory in the tarball. To avoid that, I
> add the `tar-ignore` on its own to get the sane defaults back.
Please change this to --tar-ignore=.git. I.e., I think,
`tar-ignore=.git` in d/s/options.
I think that will cause dpkg-source to do the right things. I tested
this locally and it seems to fix the problem.
Patch attached.
> PS: Not sure if this problem is specific to native packages, which I
> happen to maintain a lot of (relative to other maintainers).
It's somewhat worse with non-native packages because things are more
complicated and there's all the different git workflows, but the basic
issue is the same.
Thanks,
Ian.
[1] The git tree uploaded to the dgit git server and seen via "dgit
clone". There are split view modes for handling some, but not all,
deviations from this principle, notably patches-unapplied git trees.
[2] Except that there is no .pc directory in the git tree.
>From a1d7a8a5ef09938b372d3ae28b954552f6dd9e8e Mon Sep 17 00:00:00 2001
From: Ian Jackson
Date: Sat, 2 Sep 2023 12:40:40 +0100
Subject: [PATCH] Update d/s/options to use less broad tarignore syntax
---
debian/source/options | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/debian/source/options b/debian/source/options
index edbbfb5b..2ac64523 100644
--- a/debian/source/options
+++ b/debian/source/options
@@ -1,2 +1,2 @@
-tar-ignore
+tar-ignore=.git
tar-ignore=debhelper/.idea
--
2.20.1
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Bug#1050709: tar-ignore debian/source/options
On Mon, 28 Aug 2023 13:53:22 +0100 Ian Jackson wrote: Package: dgit Version: 11.3 debhelper (at least in sid) has a debian/source/options with; tar-ignore tar-ignore=debhelper/.idea And it has a .gitignore. This has the following effects: 1. It defeats dgit's passing of a -I and -i options to dpkg-source, so the dgit-generated source package does not contain .gitignore 2. If one were to somehow manage to make a proper source package that corresponds to the git tree, it wouldn't round trip through dpkg-source. So IMO there is no way to make a DFSG-compliant source package of Debian's current debhelper package. I was just assisting another DD contributor with this and other problems; I was going to try to help sort out a mess by doing what amounted to a sponsored upload. However, this turned out to be impossible. I have done a non-dgit-based non-DFSG-compliant upload instead. What should we do about this ? Options (not mutually exclusive) could include: * File a bug or MR against debhelper * Somehow ask that dpkg-source do something about this, but what ? * Have dgit detect this situation and at least explain it to the user * Have dgit generate a non-roundtrippable source package (probably very hard and also undesirable) Ian. [...] Hi, So this is a common pattern in my packages although it sometimes appears in d/s/local-options rather than d/s/options. Basically, the issue is when you have something you want to have locally, not in git and also not in the archive. In one my other packages, I have a "local" directory filled with local work items (including a full copy of the sudo deb package for testing purposes). Here the "local" directory is listed both in .gitignore and in "tar-ignore", because I do not want it in git nor in the archive when I do an upload. To solve this, I add `tar-ignore=...` (for native packages) to debian/source/(local-)options. However, if you add that option, you end up with the entire *.git* directory in the tarball. To avoid that, I add the `tar-ignore` on its own to get the sane defaults back. This then breaks dgit leading to this bug, which is kind of a catch-22 if you want local specific things (IDE files or local prototype stuff) that you guarantee is excluded from git and dpkg-source automatically and also support dgit at the same time. Best regards, Niels PS: Not sure if this problem is specific to native packages, which I happen to maintain a lot of (relative to other maintainers). PPS: Another example is debputy/experimental. In there I also included some python related artefacts because the early drafts do not have a proper clean rule for all the "local only" tooling I use, such as python coverage. The official builds do not use these things, so the clean rule should work fine, but I do not want a left over ".coverage" appearing in the source package as it is just cruft.
Bug#1050709: tar-ignore debian/source/options
Package: dgit Version: 11.3 debhelper (at least in sid) has a debian/source/options with; tar-ignore tar-ignore=debhelper/.idea And it has a .gitignore. This has the following effects: 1. It defeats dgit's passing of a -I and -i options to dpkg-source, so the dgit-generated source package does not contain .gitignore 2. If one were to somehow manage to make a proper source package that corresponds to the git tree, it wouldn't round trip through dpkg-source. So IMO there is no way to make a DFSG-compliant source package of Debian's current debhelper package. I was just assisting another DD contributor with this and other problems; I was going to try to help sort out a mess by doing what amounted to a sponsored upload. However, this turned out to be impossible. I have done a non-dgit-based non-DFSG-compliant upload instead. What should we do about this ? Options (not mutually exclusive) could include: * File a bug or MR against debhelper * Somehow ask that dpkg-source do something about this, but what ? * Have dgit detect this situation and at least explain it to the user * Have dgit generate a non-roundtrippable source package (probably very hard and also undesirable) Ian. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
