Bug#891033: dgit: Please integrate with pristine-tar

[Felipe Sateler]

On Sat, Feb 24, 2018 at 4:20 PM, Sean Whitton  wrote:
> Hello Ian, Felipe,
>
> Before I get into details, let me mention origtargz(1) from devscripts.
> It will extract origs from pristine-tar branches if those exist, or pull
> from the archive.
>
> Whenever I get an error about a missing orig and it's not a new upstream
> version, I just type `origtargz` and I don't recall it ever failing to
> fix the problem :)

Ah, interesting tool. I think it is a good complement to dgit.

>
> On Wed, Feb 21 2018, Ian Jackson wrote:
>
>> I think there is probably a gbp rune that will just provide the
>> upstream source tarball from the pristine tar branch.  I'm afraid I
>> don't know how it's spelled, but you can probably use that as a
>> workaround in the meantime.
>>
>> If you let me know what it is, that will be a small step towards
>> implementing this feature :-).
>
> It's --git-pristine-tar.  Look for that in gbp-buildpackage(1).

That works when using gbp directly, but it did not work for me when
running it through dgit.


>> Sean may disagree, but I don't think automatically generating missing
>> origs from pristine tar branches is something that dgit should to by
>> default.
>
> I do not disagree.  I would like to see dgit's defaults remain suitable
> for the subset of pure dgit workflows to which pristine-tar is
> orthogonal.
>
> (By a pure dgit workflow I just mean one that may involve simple scripts
> like git-deborig but not a big tool like gbp or git-dpm.  pristine-tar
> is orthogonal to a subset of the pure dgit workflows, which I consider
> to be the best pure dgit workflows.  I'll avoid explaining that
> orthogonality here, unless asked.)
>
>> But it's certainly something we could have as an option, and maybe it
>> should even be implied by --gbp.  And dgit could prompt the user to
>> specify the relevant options, if it detected an pristine-tar or
>> upstream branch.
>
> Right.  We certainly want to make dgit+gbp convenient.
>
> I wonder if we might just have an option that causes dgit to call
> `origtargz` whenever it would error out due to a missing orig, and then
> try again?  Or suggest to the user running that command?  I think I'd
> prefer the latter, if the hint is not buried in other output.

I would prefer an option to have dgit do it for me. This way I could
enable it globally on my system, since all my packages use
pristine-tar. Usually when firing up a build I then go do something
else while the build proceeds. Removing a possible interaction in the
process sounds good to me.

I think opportunistically calling origtargz (ie, enabling by default
this option if origtargz is installed) would be very nice. I can't
think of problems that may be created by enabling this option, but
that may be just failure of my imagination :)

> That would do what's needed for this bug, and it centralises logic about
> finding missing orig scripts to that script, rather than adding more
> code to dgit.

I think outsourcing the job to a tool that can integrate with various
sources is the right path forward.




-- 

Saludos,
Felipe Sateler

Bug#891033: dgit: Please integrate with pristine-tar

[Sean Whitton]

Hello Ian, Felipe,

Before I get into details, let me mention origtargz(1) from devscripts.
It will extract origs from pristine-tar branches if those exist, or pull
from the archive.

Whenever I get an error about a missing orig and it's not a new upstream
version, I just type `origtargz` and I don't recall it ever failing to
fix the problem :)

On Wed, Feb 21 2018, Ian Jackson wrote:

> I think there is probably a gbp rune that will just provide the
> upstream source tarball from the pristine tar branch.  I'm afraid I
> don't know how it's spelled, but you can probably use that as a
> workaround in the meantime.
>
> If you let me know what it is, that will be a small step towards
> implementing this feature :-).

It's --git-pristine-tar.  Look for that in gbp-buildpackage(1).

> Sean may disagree, but I don't think automatically generating missing
> origs from pristine tar branches is something that dgit should to by
> default.

I do not disagree.  I would like to see dgit's defaults remain suitable
for the subset of pure dgit workflows to which pristine-tar is
orthogonal.

(By a pure dgit workflow I just mean one that may involve simple scripts
like git-deborig but not a big tool like gbp or git-dpm.  pristine-tar
is orthogonal to a subset of the pure dgit workflows, which I consider
to be the best pure dgit workflows.  I'll avoid explaining that
orthogonality here, unless asked.)

> But it's certainly something we could have as an option, and maybe it
> should even be implied by --gbp.  And dgit could prompt the user to
> specify the relevant options, if it detected an pristine-tar or
> upstream branch.

Right.  We certainly want to make dgit+gbp convenient.

I wonder if we might just have an option that causes dgit to call
`origtargz` whenever it would error out due to a missing orig, and then
try again?  Or suggest to the user running that command?  I think I'd
prefer the latter, if the hint is not buried in other output.

That would do what's needed for this bug, and it centralises logic about
finding missing orig scripts to that script, rather than adding more
code to dgit.

> Related to this is #865905 "want better msg for missing .orig (eg due
> to accidental upstream version bump)".  #865905 mentions git-deborig
> which is also a thing that dgit should maybe be willing to use for
> this, when requested/configured.

git-deborig has the potential to generate an orig with a different hash
to the one in the archive and/or the pristine-tar branch (though
git-deborig's tarball would be identical to an orig downloaded from an
upstream's github releases page).  So I'm not sure it should happen
automatically.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#891033: dgit: Please integrate with pristine-tar

[Ian Jackson]

Control: retitle -1 on push/build, generate missing orig from pristine-tar

Felipe Sateler writes ("Re: Bug#891033: dgit: Please integrate with 
pristine-tar"):
> On Wed, Feb 21, 2018 at 8:32 PM, Ian Jackson
> > Do you mean when the user says "dgit clone" ?  Or do you mean when
> > building a source package ?
> 
> When building a source package. In my (gbp-based) workflow, I
> don't always have the current tarball in `..`, for the following
> reasons:

I see, yes.

> 1. I use gbp's export-dir option to run builds in a tmpfs, so the
> tarball lives somewhere in /tmp.
> 2. Even if not doing that, I might have pulled a git tree with an
> upstream tarball imported by a teammate. Pristine-tar avoids hash
> mismatch problems by ensuring we both use the same tarball.
> 
> Note that I currently faced this error when using `dgit push-source`.
> Perhaps other build options do not have this problem due to gbp
> integration (gbp already extracts the tarbal when it needs to).

I think that's likely.

I think there is probably a gbp rune that will just provide the
upstream source tarball from the pristine tar branch.  I'm afraid I
don't know how it's spelled, but you can probably use that as a
workaround in the meantime.

If you let me know what it is, that will be a small step towards
implementing this feature :-).

> > It's possible that dgit could be extended somehow in these directions
> > but I don't think it's entirely trivial, depending what is wanted.
> 
> /me hopes it is not difficult.

Now I understand what you're asking for, yes, I think this would be
possible and not even too difficult.

Sean may disagree, but I don't think automatically generating missing
origs from pristine tar branches is something that dgit should to by
default.

But it's certainly something we could have as an option, and maybe it
should even be implied by --gbp.  And dgit could prompt the user to
specify the relevant options, if it detected an pristine-tar or
upstream branch.

Related to this is #865905 "want better msg for missing .orig (eg due
to accidental upstream version bump)".  #865905 mentions git-deborig
which is also a thing that dgit should maybe be willing to use for
this, when requested/configured.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#891033: dgit: Please integrate with pristine-tar

[Felipe Sateler]

On Wed, Feb 21, 2018 at 8:32 PM, Ian Jackson
<ijack...@chiark.greenend.org.uk> wrote:
> Felipe Sateler writes ("Bug#891033: dgit: Please integrate with 
> pristine-tar"):
>> It would be great if dgit would detect a pristine-tar enabled repo, and
>> in that case try to checkout the upstream tarball if it is not present
>> in `..`.
>
> Hi.  I'm not sure exactly when you think it should do this.

Sorry for being too terse.

>
> Do you mean when the user says "dgit clone" ?  Or do you mean when
> building a source package ?

When building a source package. In my (gbp-based) workflow, I
don't always have the current tarball in `..`, for the following
reasons:

1. I use gbp's export-dir option to run builds in a tmpfs, so the
tarball lives somewhere in /tmp.
2. Even if not doing that, I might have pulled a git tree with an
upstream tarball imported by a teammate. Pristine-tar avoids hash
mismatch problems by ensuring we both use the same tarball.

Note that I currently faced this error when using `dgit push-source`.
Perhaps other build options do not have this problem due to gbp
integration (gbp already extracts the tarbal when it needs to).

> It's possible that dgit could be extended somehow in these directions
> but I don't think it's entirely trivial, depending what is wanted.

/me hopes it is not difficult.

-- 

Saludos,
Felipe Sateler

Bug#891033: dgit: Please integrate with pristine-tar

[Ian Jackson]

Felipe Sateler writes ("Bug#891033: dgit: Please integrate with pristine-tar"):
> It would be great if dgit would detect a pristine-tar enabled repo, and
> in that case try to checkout the upstream tarball if it is not present
> in `..`.

Hi.  I'm not sure exactly when you think it should do this.

Do you mean when the user says "dgit clone" ?  Or do you mean when
building a source package ?

It's possible that dgit could be extended somehow in these directions
but I don't think it's entirely trivial, depending what is wanted.

Ian.
(for my reference: #869675 is perhaps related)

Bug#891033: dgit: Please integrate with pristine-tar

[Felipe Sateler]

Package: dgit
Version: 4.3
Severity: wishlist
Tags: upstream

Hi,

It would be great if dgit would detect a pristine-tar enabled repo, and
in that case try to checkout the upstream tarball if it is not present
in `..`.

-- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'unstable'), (1, 
'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.14.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dgit depends on:
ii  apt   1.6~alpha7
ii  ca-certificates   20170717
ii  coreutils 8.28-1
ii  curl  7.58.0-2
ii  devscripts2.17.12
ii  dpkg-dev  1.19.0.5
ii  dput  1.0.1
ii  git [git-core]1:2.16.1-1
ii  git-buildpackage  0.9.7
ii  libdpkg-perl  1.19.0.5
ii  libjson-perl  2.97001-1
ii  liblist-moreutils-perl0.416-1+b3
ii  libperl5.26 [libdigest-sha-perl]  5.26.1-4+b1
ii  libtext-glob-perl 0.10-1
ii  libtext-iconv-perl1.7-5+b6
ii  libwww-perl   6.31-1
ii  perl  5.26.1-4+b1

Versions of packages dgit recommends:
ii  openssh-client [ssh-client]  1:7.6p1-4

Versions of packages dgit suggests:
ii  sbuild  0.73.0-4

-- no debconf information