Bug#1005765: dgit doesn't handle upstream files with CRLF well
Control: close -1 Thanks for the report. But, having reviewed this bug, I think all of dgit's behaviour here is correct. What's wrong is that the git tree and .orig have mismatched line ending conventions. I've conjectured that this is tolerated by other tooling (eg git-buildpackage) because git has been configured to do automatic line ending conversion. That's not a good approach, because that configuration is local to particular git trees, so different people see different working trees for the same commit. Probably, the git tree should be changed to match the upstream source code. The other alternative is not to use the upstream .orig, but rather use one that you've converted to Unix line endings. Ian. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Bug#1005765: dgit doesn't handle upstream files with CRLF well
Stéphane Glondu writes ("Re: Bug#1005765: dgit doesn't handle upstream files
with CRLF well"):
> I had trouble uploading opam/2.1.2-1 with dgit. Eventually, I gave up
> and did a regular upload.
>
> The commit id is 225dd4102cfd4391fab166d7cc220d020efedafd on:
>
>https://salsa.debian.org/ocaml-team/opam
Thanks. That is helpful information. I observe that:
1. In opam_2.1.2-1.dsc, appveyor_build.cmd has CRLF line endings;
2. In 225dd4102cfd4391fab166d7cc220d020efedafd it has unix line endings.
3. There is nothing in debian/patches to represent this change.
I see this:
opam$ git checkout 225dd4102cfd4391fab166d7cc220d020efedafd
HEAD is now at 225dd41 Prepare upload to unstable
opam$ dgit --gbp quilt-fixup
Format `3.0 (quilt)', need to check/update patch stack
gzip: warning: GZIP environment variable is deprecated; use an alias or script
dgit: split brain (separate dgit view) may be needed (--quilt=gbp).
examining quilt state (multiple patches, gbp mode)
dgit: base trees orig=1aba501b88d166b9b63f o+d/p=4c4f0f75326134d939d1
dgit: quilt differences: src: ## orig ## gitignores: == orig ==
dgit: quilt differences: HEAD ## o+d/p HEAD == o+d/p
dgit: error: --quilt=gbp specified, implying patches-unapplied git tree
dgit: but git tree differs from orig in upstream files.
dgit: For full diff showing the problem(s), type:
dgit: git diff 1aba501b88d166b9b63f363a1583c6b6a4ac111e HEAD -- :/ ':!debian'
':!/.gitignore' ':!*/.gitignore'
opam$ git --no-pager diff --stat 1aba501b88d166b9b63f363a1583c6b6a4ac111e HEAD
-- :/ ':!debian' ':!/.gitignore' ':!*/.gitignore'
appveyor_build.cmd | 416
++--
1 file changed, 208 insertions(+), 208 deletions(-)
opam$
You said
> > I couldn't find a combination of checked-in files and git crlf
> > handling that would please both git and dgit.
You seem to be using tarball imports for your git tree. I don't know
what tool you are using for your tarball imports, but I think you need
to tell it not to convert line endings.
I don't think git converts line endings by default, so I think you
probably enabled some conversion feature. I recommend against that.
If some tool you are using (eg for your tarball imports) converted the
line endings, then I think it's a bug in that tool.
I did this experiment:
opam$ git checkout 225dd4102cfd4391fab166d7cc220d020efedafd
HEAD is now at 225dd41 Prepare upload to unstable
opam$ unix2dos appveyor_build.cmd
unix2dos: converting file appveyor_build.cmd to DOS format...
opam$ git commit -m fix appveyor_build.cmd
[detached HEAD 761e246] fix
1 file changed, 208 insertions(+), 208 deletions(-)
opam$ dgit --gbp quilt-fixup
Format `3.0 (quilt)', need to check/update patch stack
gzip: warning: GZIP environment variable is deprecated; use an alias or script
dgit: split brain (separate dgit view) may be needed (--quilt=gbp).
examining quilt state (multiple patches, gbp mode)
dgit: base trees orig=1aba501b88d166b9b63f o+d/p=4c4f0f75326134d939d1
dgit: quilt differences: src: == orig ## gitignores: == orig ==
dgit: quilt differences: HEAD ## o+d/p HEAD == o+d/p
dgit view: creating patches-applied version using gbp pq
dgit view: created (commit id 795bce795ad5b5f6164f757b13892275968b3b05)
opam$
> > Ideally, a run with dgit -D would be helpful.
> I will try when I update the package.
(FTAOD this isn't needed now.)
> > If you have a situation that doesn't match the criteria above, but you
> > think should be able to work, and currently doesn't, please also let
> > us know.
>
> I'm not sure whether my situation matches the criteria above, but I
> guess dgit should be able to do any upload to the official Debian
> archive, shouldn't it?
dgit ought to be able to upload any source package that the archive
will accept, yes. But dgit requires that the git branch you are using
corresponds, *precisely*, to the source package. Otherwise people who
ran "dgit clone" might get a different tree to people who did "apt-get
source", for the very same upload, which would be really very bad.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Bug#1005765: dgit doesn't handle upstream files with CRLF well
Hello, Le 03/09/2022 à 18:43, Ian Jackson a écrit : I wrote: I think that if - your upstream files have CRLF in the orig - your upstream files have CRLF in your git tree - you do not expect (or enable) line ending conversions in git then - dgit will not complain - the source package you produce will have files with CRLF - everything should work If you have a repro that demonstrates the contrary, please let us know. Typically a repro would consist of the following elements: * The precise git commit that you had as HEAD in your working tree We need the git hash; hopefully I can obtain the data from salsa. * The precise contents of all relevant .origs If they were already previously uploaded, we can get them from the archive or snapshot.d.o. Otherwise please supply their sha256sums, as well as instructions for download or creation (eg pristine-tar branch). * The precise dgit command line, and any configuration settings applied to any of the relevant tools. I had trouble uploading opam/2.1.2-1 with dgit. Eventually, I gave up and did a regular upload. The commit id is 225dd4102cfd4391fab166d7cc220d020efedafd on: https://salsa.debian.org/ocaml-team/opam Ideally, a run with dgit -D would be helpful. I will try when I update the package. If you have a situation that doesn't match the criteria above, but you think should be able to work, and currently doesn't, please also let us know. I'm not sure whether my situation matches the criteria above, but I guess dgit should be able to do any upload to the official Debian archive, shouldn't it? Thanks for all your work on dgit! Cheers, -- Stéphane
Bug#1005765: dgit doesn't handle upstream files with CRLF well
Control: tags -1 moreinfo I wrote: > I think that if > - your upstream files have CRLF in the orig > - your upstream files have CRLF in your git tree > - you do not expect (or enable) line ending conversions in git > then > - dgit will not complain > - the source package you produce will have files with CRLF > - everything should work If you have a repro that demonstrates the contrary, please let us know. Typically a repro would consist of the following elements: * The precise git commit that you had as HEAD in your working tree We need the git hash; hopefully I can obtain the data from salsa. * The precise contents of all relevant .origs If they were already previously uploaded, we can get them from the archive or snapshot.d.o. Otherwise please supply their sha256sums, as well as instructions for download or creation (eg pristine-tar branch). * The precise dgit command line, and any configuration settings applied to any of the relevant tools. Ideally, a run with dgit -D would be helpful. If you have a situation that doesn't match the criteria above, but you think should be able to work, and currently doesn't, please also let us know. Thanks, Ian. -- Ian JacksonThese opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
Bug#1005765: dgit doesn't handle upstream files with CRLF well
Stéphane Glondu writes ("Bug#1005765: dgit doesn't handle upstream files with
CRLF well"):
> I just tried to upload opam with dgit, and I couldn't because of
> pretended differences in upstream files. However, these differences
> consist of line endings only. I couldn't find a combination of
> checked-in files and git crlf handling that would please both git and
> dgit.
>
> Maybe dgit should call "git diff" with --ignore-cr-at-eol?
Hi. I'm sorry you're having trouble.
I'm not sure why things don't just work for you, although I have some
guesses/suppositions.
I think that if
- your upstream files have CRLF in the orig
- your upstream files have CRLF in your git tree
- you do not expect (or enable) line ending conversions in git
then
- dgit will not complain
- the source package you produce will have files with CRLF
- everything should work
I suspect that maybe you are trying to create a .dsc containing
CRLFs where the git tree contains LFs, or vice versa ?
That's not going to work with dgit, because dgit's purpose is to give
someone *the very exact same source tree* via git, as they would via
"apt-get source". So dgit checks that the git tree and the dsc are
identical - and it tries to to ensure that this check is not affected
by (and thereby perhaps defeated by) any git automatic line ending
conversions.
I think perhaps you are trying to
- work with files with LF line endings in git
- nevertheless use an upstream .orig tarball with CRLF line endings
- not declare any kind of delta to represent this anomaly
?
I may be able to advise more helpfully if you were to share your git
HEAD and your orig (or means to reproduce or download it - eg
pristine-tar branch, or url, or whatever).
Ian.
Bug#1005765: dgit doesn't handle upstream files with CRLF well
Package: dgit Version: 9.15 Severity: important Dear Maintainer, I just tried to upload opam with dgit, and I couldn't because of pretended differences in upstream files. However, these differences consist of line endings only. I couldn't find a combination of checked-in files and git crlf handling that would please both git and dgit. Maybe dgit should call "git diff" with --ignore-cr-at-eol? Cheers, -- Stéphane -- System Information: Debian Release: bookworm/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.15.0-3-amd64 (SMP w/4 CPU threads) Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages dgit depends on: ii apt2.3.15 ii ca-certificates20211016 ii coreutils 8.32-4.1 ii curl 7.81.0-1 ii devscripts 2.22.1 ii dpkg-dev 1.21.1 ii dput 1.1.0 ii git [git-core] 1:2.34.1-1 ii git-buildpackage 0.9.25 ii libdpkg-perl 1.21.1 ii libjson-perl 4.04000-1 ii liblist-moreutils-perl 0.430-2 ii liblocale-gettext-perl 1.07-4+b2 ii libtext-csv-perl 2.01-1 ii libtext-glob-perl 0.11-2 ii libtext-iconv-perl 1.7-7+b2 ii libwww-curl-perl 4.17-7+b2 ii perl [libdigest-sha-perl] 5.34.0-3 Versions of packages dgit recommends: ii distro-info-data 0.52 ii liburi-perl 5.10-1 ii openssh-client [ssh-client] 1:8.7p1-4 Versions of packages dgit suggests: ii cowbuilder 0.89 ii pbuilder0.231 -- no debconf information
