Bug#871809: Please allow to store detached tarball signatures as well

[Chris Lamb]

Hi Tomasz,

> Thanks, merged in the git repo. It will be released in the new
> release, before we sort out #871938 which I consider to be a blocking
> bug.

Great; looking forward to release. Alas, I fear #871938 is a little
beyond me :)


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Bug#871809: Please allow to store detached tarball signatures as well

[Tomasz Buchert]

On 20/08/17 17:40, Chris Lamb wrote:
>
> (I accidentally left a debugging statement in; please use the attached file)
>

Thanks, merged in the git repo. It will be released in the new
release, before we sort out #871938 which I consider to be a blocking
bug.

Thanks a lot,
Tomasz


signature.asc
Description: PGP signature

Bug#871809: Please allow to store detached tarball signatures as well

[Chris Lamb]

(I accidentally left a debugging statement in; please use the attached file)


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-
>From ec2403ec3e48db91cd0f8c22eed9a7ac66677d1e Mon Sep 17 00:00:00 2001
From: Chris Lamb 
Date: Sun, 20 Aug 2017 16:25:45 -0700
Subject: [PATCH] Support storing and retrieval of upstream signatures.
 (Closes: #871809)

This commit adds support for optionally storing and regenerating an
upstream signature with the tarball so that it can be verified by,
for example, dpkg-source(1).

Regardless of the original signature filename provided, it is always
stored alongside the .delta and .id files as .sig for deterministic
retrieval.

The existing behaviour of pristine-tar is unchanged unless you specify
the `-s` option; in particular, extraction of signatures is not performed
by default - one must specify the filename. This is to prevent breaking
existing behaviour.
---
 README |  3 ++
 debian/control |  3 ++
 pristine-tar   | 62 --
 test/samples/signatures/foo-1.0.tar.gz.asc | 16 
 test/test_checkout.sh  | 14 +++
 5 files changed, 87 insertions(+), 11 deletions(-)
 create mode 100644 test/samples/signatures/foo-1.0.tar.gz.asc

diff --git a/README b/README
index c792882..710c2ff 100644
--- a/README
+++ b/README
@@ -9,3 +9,6 @@ The delta file is designed to be checked into revision control along-side the
 upstream branch, thus allowing Debian packages to be built entirely using
 sources in revision control, without the need to keep copies of upstream
 tarballs. See `delta-format.txt` for details on the format of the delta file.
+
+An optional upstream signature may be attached to tarballs for verification
+by, for example, dpkg-source(1).
diff --git a/debian/control b/debian/control
index 35e3a4a..dab3b3f 100644
--- a/debian/control
+++ b/debian/control
@@ -37,3 +37,6 @@ Description: regenerate pristine tarballs
  the upstream branch, thus allowing Debian packages to be built entirely
  using sources in revision control, without the need to keep copies of
  upstream tarballs.
+ .
+ An optional upstream signature may be attached to tarballs for verification
+ by, for example, dpkg-source(1).
diff --git a/pristine-tar b/pristine-tar
index d4f4b0e..1c4eaf0 100755
--- a/pristine-tar
+++ b/pristine-tar
@@ -10,7 +10,7 @@ B [-vdk] gendelta I I
 
 B [-vdk] gentar I I
 
-B [-vdk] [-m message] commit I [I]
+B [-vdk] [-m message] [-s signaturefile] commit I [I]
 
 B [-vdk] checkout I
 
@@ -120,6 +120,14 @@ Don't clean up the temporary directory on exit.
 
 Use this option to specify a custom commit message to pristine-tar commit.
 
+=item -s signaturefile
+
+=item --signature-file=signaturefile
+
+Use this option to optionally commit or checkout an upstream signature
+file for the tarball. Note that extraction of signatures is not
+performed by default.
+
 =back
 
 =head1 EXAMPLES
@@ -198,6 +206,7 @@ use Pristine::Tar;
 use Pristine::Tar::Delta;
 use Pristine::Tar::Formats;
 use Pristine::Tar::DeltaTools;
+use File::Copy;
 use File::Path;
 use File::Basename;
 use Cwd qw{getcwd abs_path};
@@ -226,7 +235,7 @@ use constant {
   XDELTA_LONG => "2.0"
 };
 
-my $message;
+my ($message, $signature_file);
 my $genversion =
   version_from_env(XDELTA3, "xdelta" => XDELTA, "xdelta3" => XDELTA3);
 
@@ -243,7 +252,8 @@ dispatch(
 verify   => [ \, 1 ],
   },
   options => {
-"m|message=s" => \$message,
+"m|message=s"=> \$message,
+"s|signature-file=s" => \$signature_file,
   },
 );
 
@@ -251,8 +261,9 @@ sub usage {
   print STDERR "Usage: pristine-tar [-vdk] gendelta tarball delta\n";
   print STDERR "   pristine-tar [-vdk] gentar delta tarball\n";
   print STDERR
-"   pristine-tar [-vdk] [-m message] commit tarball [upstream]\n";
-  print STDERR "   pristine-tar [-vdk] checkout tarball\n";
+"   pristine-tar [-vdk] [-m message] [-s signaturefile] commit tarball [upstream]\n";
+  print STDERR
+"   pristine-tar [-vdk] [-s signaturefile] checkout tarball\n";
   print STDERR "   pristine-tar [-vdk] verify tarball\n";
   print STDERR "   pristine-tarlist\n";
   exit 1;
@@ -780,8 +791,9 @@ sub checkoutdelta {
   my $branch= "pristine-tar";
   my $deltafile = basename($tarball) . ".delta";
   my $idfile= basename($tarball) . ".id";
+  my $sigfile   = basename($tarball) . ".asc";
 
-  my ($delta, $id);
+  my ($delta, $id, $signature);
 
   my $vcs = vcstype();
   if ($vcs eq "git") {
@@ -810,11 +822,19 @@ sub checkoutdelta {
 if (!length $id) {
   error "git show $branch:$idfile returned no id";
 }
+if (defined $signature_file) {
+  # We only extract the signature if the user specifically requested
+  # it and we assume the data will fit comfortably into memory.
+  $signature = 

Bug#871809: Please allow to store detached tarball signatures as well

[Chris Lamb]

Hi Tomasz,

> A quick glimpse tells me that it should be ok. Would you mind adding a
> test to cover this functionality?

No problem — updated patch attached:

  commit d71b37d49e57dd6e31b4d6db5752dcdc607a2dd1
  Author: Chris Lamb 
  Date:   Sun Aug 20 16:25:45 2017 -0700
  
  Support storing and retrieval of upstream signatures. (Closes: #871809)
  
  This commit adds support for optionally storing and regenerating an
  upstream signature with the tarball so that it can be verified by,
  for example, dpkg-source(1).
  
  Regardless of the original signature filename provided, it is always
  stored alongside the .delta and .id files as .sig for deterministic
  retrieval.
  
  The existing behaviour of pristine-tar is unchanged unless you specify
  the `-s` option; in particular, extraction of signatures is not performed
  by default - one must specify the filename. This is to prevent breaking
  existing behaviour.
  
   README |  3 ++
   debian/control |  3 ++
   pristine-tar   | 62 
--
   test/samples/signatures/foo-1.0.tar.gz.asc | 16 
   test/test_checkout.sh  | 14 +++
   5 files changed, 87 insertions(+), 11 deletions(-)

> Thanks, the amount of love pristine-tar is getting these days must
> make it blush.

:) :)


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-
>From d71b37d49e57dd6e31b4d6db5752dcdc607a2dd1 Mon Sep 17 00:00:00 2001
From: Chris Lamb 
Date: Sun, 20 Aug 2017 16:25:45 -0700
Subject: [PATCH] Support storing and retrieval of upstream signatures.
 (Closes: #871809)

This commit adds support for optionally storing and regenerating an
upstream signature with the tarball so that it can be verified by,
for example, dpkg-source(1).

Regardless of the original signature filename provided, it is always
stored alongside the .delta and .id files as .sig for deterministic
retrieval.

The existing behaviour of pristine-tar is unchanged unless you specify
the `-s` option; in particular, extraction of signatures is not performed
by default - one must specify the filename. This is to prevent breaking
existing behaviour.
---
 README |  3 ++
 debian/control |  3 ++
 pristine-tar   | 62 --
 test/samples/signatures/foo-1.0.tar.gz.asc | 16 
 test/test_checkout.sh  | 14 +++
 5 files changed, 87 insertions(+), 11 deletions(-)
 create mode 100644 test/samples/signatures/foo-1.0.tar.gz.asc

diff --git a/README b/README
index c792882..710c2ff 100644
--- a/README
+++ b/README
@@ -9,3 +9,6 @@ The delta file is designed to be checked into revision control along-side the
 upstream branch, thus allowing Debian packages to be built entirely using
 sources in revision control, without the need to keep copies of upstream
 tarballs. See `delta-format.txt` for details on the format of the delta file.
+
+An optional upstream signature may be attached to tarballs for verification
+by, for example, dpkg-source(1).
diff --git a/debian/control b/debian/control
index 35e3a4a..dab3b3f 100644
--- a/debian/control
+++ b/debian/control
@@ -37,3 +37,6 @@ Description: regenerate pristine tarballs
  the upstream branch, thus allowing Debian packages to be built entirely
  using sources in revision control, without the need to keep copies of
  upstream tarballs.
+ .
+ An optional upstream signature may be attached to tarballs for verification
+ by, for example, dpkg-source(1).
diff --git a/pristine-tar b/pristine-tar
index d4f4b0e..1c4eaf0 100755
--- a/pristine-tar
+++ b/pristine-tar
@@ -10,7 +10,7 @@ B [-vdk] gendelta I I
 
 B [-vdk] gentar I I
 
-B [-vdk] [-m message] commit I [I]
+B [-vdk] [-m message] [-s signaturefile] commit I [I]
 
 B [-vdk] checkout I
 
@@ -120,6 +120,14 @@ Don't clean up the temporary directory on exit.
 
 Use this option to specify a custom commit message to pristine-tar commit.
 
+=item -s signaturefile
+
+=item --signature-file=signaturefile
+
+Use this option to optionally commit or checkout an upstream signature
+file for the tarball. Note that extraction of signatures is not
+performed by default.
+
 =back
 
 =head1 EXAMPLES
@@ -198,6 +206,7 @@ use Pristine::Tar;
 use Pristine::Tar::Delta;
 use Pristine::Tar::Formats;
 use Pristine::Tar::DeltaTools;
+use File::Copy;
 use File::Path;
 use File::Basename;
 use Cwd qw{getcwd abs_path};
@@ -226,7 +235,7 @@ use constant {
   XDELTA_LONG => "2.0"
 };
 
-my $message;
+my ($message, $signature_file);
 my $genversion =
   version_from_env(XDELTA3, "xdelta" => XDELTA, "xdelta3" => XDELTA3);
 
@@ -243,7 +252,8 @@ dispatch(
 verify   => [ \, 1 ],
   },
   options => {
-"m|message=s" => \$message,
+   

Bug#871809: Please allow to store detached tarball signatures as well

[Tomasz Buchert]

On 20/08/17 16:42, Chris Lamb wrote:
> tags 871809 + patch
> thanks
>
> Hi,
>
> > I will implement this soon, this doesn't seem to be too hard
> > to do.
>
> Beat you to it, I think! I've attached:
>
>   commit 24549c61be4c0eea1495e3508377bf46d162230f
>   Author: Chris Lamb 
>   Date:   Sun Aug 20 16:25:45 2017 -0700
>
>   Support storing and retrieval of upstream signatures. (Closes: #871809)
>
>   This commit adds support for optionally storing and regenerating an
>   upstream signature with the tarball so that it can be verified by,
>   for example, dpkg-source(1).
>
>   Regardless of the original signature filename provided, it is always
>   stored alongside the .delta and .id files as .sig for deterministic
>   retrieval.
>
>   The existing behaviour of pristine-tar is unchanged unless you specify
>   the `-s` option; in particular, extraction of signatures is not 
> performed
>   by default - one must specify the filename. This is to prevent breaking
>   existing behaviour.
>
>README |  3 +++
>debian/control |  3 +++
>pristine-tar   | 59 
> +++---
>3 files changed, 54 insertions(+), 11 deletions(-)
>
>
> Best wishes,

Thanks, the amount of love pristine-tar is getting these days must
make it blush.

A quick glimpse tells me that it should be ok. Would you mind adding a
test to cover this functionality?

Tomasz


signature.asc
Description: PGP signature

Bug#871809: Please allow to store detached tarball signatures as well

[Chris Lamb]

tags 871809 + patch
thanks

Hi,

> I will implement this soon, this doesn't seem to be too hard
> to do.

Beat you to it, I think! I've attached:

  commit 24549c61be4c0eea1495e3508377bf46d162230f
  Author: Chris Lamb 
  Date:   Sun Aug 20 16:25:45 2017 -0700
  
  Support storing and retrieval of upstream signatures. (Closes: #871809)
  
  This commit adds support for optionally storing and regenerating an
  upstream signature with the tarball so that it can be verified by,
  for example, dpkg-source(1).
  
  Regardless of the original signature filename provided, it is always
  stored alongside the .delta and .id files as .sig for deterministic
  retrieval.
  
  The existing behaviour of pristine-tar is unchanged unless you specify
  the `-s` option; in particular, extraction of signatures is not performed
  by default - one must specify the filename. This is to prevent breaking
  existing behaviour.
  
   README |  3 +++
   debian/control |  3 +++
   pristine-tar   | 59 
+++---
   3 files changed, 54 insertions(+), 11 deletions(-)


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-
>From 24549c61be4c0eea1495e3508377bf46d162230f Mon Sep 17 00:00:00 2001
From: Chris Lamb 
Date: Sun, 20 Aug 2017 16:25:45 -0700
Subject: [PATCH] Support storing and retrieval of upstream signatures.
 (Closes: #871809)

This commit adds support for optionally storing and regenerating an
upstream signature with the tarball so that it can be verified by,
for example, dpkg-source(1).

Regardless of the original signature filename provided, it is always
stored alongside the .delta and .id files as .sig for deterministic
retrieval.

The existing behaviour of pristine-tar is unchanged unless you specify
the `-s` option; in particular, extraction of signatures is not performed
by default - one must specify the filename. This is to prevent breaking
existing behaviour.
---
 README |  3 +++
 debian/control |  3 +++
 pristine-tar   | 59 +++---
 3 files changed, 54 insertions(+), 11 deletions(-)

diff --git a/README b/README
index c792882..710c2ff 100644
--- a/README
+++ b/README
@@ -9,3 +9,6 @@ The delta file is designed to be checked into revision control along-side the
 upstream branch, thus allowing Debian packages to be built entirely using
 sources in revision control, without the need to keep copies of upstream
 tarballs. See `delta-format.txt` for details on the format of the delta file.
+
+An optional upstream signature may be attached to tarballs for verification
+by, for example, dpkg-source(1).
diff --git a/debian/control b/debian/control
index 35e3a4a..dab3b3f 100644
--- a/debian/control
+++ b/debian/control
@@ -37,3 +37,6 @@ Description: regenerate pristine tarballs
  the upstream branch, thus allowing Debian packages to be built entirely
  using sources in revision control, without the need to keep copies of
  upstream tarballs.
+ .
+ An optional upstream signature may be attached to tarballs for verification
+ by, for example, dpkg-source(1).
diff --git a/pristine-tar b/pristine-tar
index d4f4b0e..fe61388 100755
--- a/pristine-tar
+++ b/pristine-tar
@@ -10,7 +10,7 @@ B [-vdk] gendelta I I
 
 B [-vdk] gentar I I
 
-B [-vdk] [-m message] commit I [I]
+B [-vdk] [-m message] [-s signaturefile] commit I [I]
 
 B [-vdk] checkout I
 
@@ -120,6 +120,14 @@ Don't clean up the temporary directory on exit.
 
 Use this option to specify a custom commit message to pristine-tar commit.
 
+=item -s signaturefile
+
+=item --signature-file=signaturefile
+
+Use this option to optionally commit or checkout an upstream signature
+file for the tarball. Note that extraction of signatures is not
+performed by default.
+
 =back
 
 =head1 EXAMPLES
@@ -198,6 +206,7 @@ use Pristine::Tar;
 use Pristine::Tar::Delta;
 use Pristine::Tar::Formats;
 use Pristine::Tar::DeltaTools;
+use File::Copy;
 use File::Path;
 use File::Basename;
 use Cwd qw{getcwd abs_path};
@@ -226,7 +235,7 @@ use constant {
   XDELTA_LONG => "2.0"
 };
 
-my $message;
+my ($message, $signature_file);
 my $genversion =
   version_from_env(XDELTA3, "xdelta" => XDELTA, "xdelta3" => XDELTA3);
 
@@ -243,7 +252,8 @@ dispatch(
 verify   => [ \, 1 ],
   },
   options => {
-"m|message=s" => \$message,
+"m|message=s"=> \$message,
+"s|signature-file=s" => \$signature_file,
   },
 );
 
@@ -251,8 +261,9 @@ sub usage {
   print STDERR "Usage: pristine-tar [-vdk] gendelta tarball delta\n";
   print STDERR "   pristine-tar [-vdk] gentar delta tarball\n";
   print STDERR
-"   pristine-tar [-vdk] [-m message] commit tarball [upstream]\n";
-  print STDERR "   pristine-tar [-vdk] checkout tarball\n";
+"   pristine-tar [-vdk] [-m message] [-s signaturefile] commit tarball 

Bug#871809: Please allow to store detached tarball signatures as well

[Guido Günther]

Hi,
On Sat, Aug 12, 2017 at 01:07:26AM +0200, Tomasz Buchert wrote:
> On 11/08/17 16:36, Guido Günther wrote:
> > Package: pristine-tar
> > Version: 1.40
> > Severity: wishlist
> >
> > Hi,
> > as proposed by maxy on debian-devel it would be great if pristine-tar
> > would store the tarball signtures as well:
> >
> > 
> > https://lists.debian.org/msgid-search/20170731145720.6jccnhgmyr4gc...@neoptolemo.gnuservers.com.ar
> >
> > pristine-tar could commit the orig.tar.{$ext}.{asc,pgp} right away by
> > default if present or we'd extend the command line to
> >
> > pristine-tar [-vdk] [-m message] [-s signaturefile] commit tarball 
> > [upstream]
> >
> > Cheers and thanks for this very useful tool!
> >  -- Guido
> 
> Hi Guido,
> I think the most backwards compatible flow is to explicitly specify
> "signaturefile", both during the commit into the pristine-tar branch
> and during checkout. Will this work for you?

Works, sure.

> I will implement this soon, this doesn't seem to be too hard to do.

Great. So I'll hack on more gbp bugs in the maintime.
Thanks a lot!
 -- Guido

Bug#871809: Please allow to store detached tarball signatures as well

[Tomasz Buchert]

On 11/08/17 16:36, Guido Günther wrote:
> Package: pristine-tar
> Version: 1.40
> Severity: wishlist
>
> Hi,
> as proposed by maxy on debian-devel it would be great if pristine-tar
> would store the tarball signtures as well:
>
> 
> https://lists.debian.org/msgid-search/20170731145720.6jccnhgmyr4gc...@neoptolemo.gnuservers.com.ar
>
> pristine-tar could commit the orig.tar.{$ext}.{asc,pgp} right away by
> default if present or we'd extend the command line to
>
> pristine-tar [-vdk] [-m message] [-s signaturefile] commit tarball 
> [upstream]
>
> Cheers and thanks for this very useful tool!
>  -- Guido

Hi Guido,
I think the most backwards compatible flow is to explicitly specify
"signaturefile", both during the commit into the pristine-tar branch
and during checkout. Will this work for you?

I will implement this soon, this doesn't seem to be too hard to do.

Tomasz

signature.asc
Description: PGP signature

Bug#871809: Please allow to store detached tarball signatures as well

[Guido Günther]

Package: pristine-tar
Version: 1.40
Severity: wishlist

Hi,
as proposed by maxy on debian-devel it would be great if pristine-tar
would store the tarball signtures as well:


https://lists.debian.org/msgid-search/20170731145720.6jccnhgmyr4gc...@neoptolemo.gnuservers.com.ar

pristine-tar could commit the orig.tar.{$ext}.{asc,pgp} right away by
default if present or we'd extend the command line to

pristine-tar [-vdk] [-m message] [-s signaturefile] commit tarball 
[upstream]

Cheers and thanks for this very useful tool!
 -- Guido

-- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 
'testing-debug'), (500, 'oldoldstable'), (500, 'unstable'), (500, 'stable'), 
(1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.11.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), 
LANGUAGE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages pristine-tar depends on:
ii  libbz2-1.0  1.0.6-8.1
ii  libc6   2.24-12
ii  perl5.26.0-4
ii  tar 1.29b-2
ii  xdelta  1.1.3-9.1+b1
ii  xdelta3 3.0.11-dfsg-1+b1
ii  zlib1g  1:1.2.8.dfsg-5

Versions of packages pristine-tar recommends:
ii  bzip2 1.0.6-8.1
ii  pbzip21.1.9-1+b1
ii  xz-utils  5.2.2-1.3

pristine-tar suggests no packages.

-- no debconf information