Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-08 Thread Ulrich Mueller
 On Sun, 07 Dec 2014, Zac Medico wrote:

 On 12/07/2014 07:06 PM, Christoph Junghans wrote:
 I know, I am late to the party, I just wanted to say that in
 unpacker.eclass I implemented a variant which uses neither deb2tags
 nor ar on prefix, but just bash's read and head.

The code uses head -c which is a GNU extension:

   $ man 1p head
   HEAD(1P)POSIX Programmer's ManualHEAD(1P)
   [...]
   SYNOPSIS
  head [−n number] [file...]
   [...]

For example, on OpenBSD and HPUX there is no -c option, or it is used
differently. Also Busybox doesn't have it.

Are we better off by requiring GNU head instead of GNU ar?

 Very nice. We should borrow that code and use it to eliminate the
 need for deb2targz in portage.

In PMS, this would have to go into EAPI 6, I suppose?

Ulrich


pgpfesq0ME4CD.pgp
Description: PGP signature


Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-08 Thread Michał Górny
Dnia 2014-12-07, o godz. 19:50:24
Zac Medico zmed...@gentoo.org napisał(a):

 On 12/07/2014 07:06 PM, Christoph Junghans wrote:
  I know, I am late to the party, I just wanted to say that in
  unpacker.eclass I implemented a variant which uses neither deb2tags
  nor ar on prefix, but just bash's read and head.
 
 Very nice. We should borrow that code and use it to eliminate the need
 for deb2targz in portage.

You shouldn't borrow code written by vapier. Ever. We have enough work
on our shoulders having to fix it in one location.

-- 
Best regards,
Michał Górny


pgpFIoLYEHMy3.pgp
Description: OpenPGP digital signature


Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-08 Thread Zac Medico
On 12/08/2014 12:45 AM, Ulrich Mueller wrote:
 On Sun, 07 Dec 2014, Zac Medico wrote:
 
 On 12/07/2014 07:06 PM, Christoph Junghans wrote:
 I know, I am late to the party, I just wanted to say that in
 unpacker.eclass I implemented a variant which uses neither deb2tags
 nor ar on prefix, but just bash's read and head.
 
 The code uses head -c which is a GNU extension:
 
$ man 1p head
HEAD(1P)POSIX Programmer's ManualHEAD(1P)
[...]
SYNOPSIS
   head [−n number] [file...]
[...]
 
 For example, on OpenBSD and HPUX there is no -c option, or it is used
 differently. Also Busybox doesn't have it.
 
 Are we better off by requiring GNU head instead of GNU ar?

That's a good question. We might use dd instead of head, or port the
shell code to python.

 Very nice. We should borrow that code and use it to eliminate the
 need for deb2targz in portage.
 
 In PMS, this would have to go into EAPI 6, I suppose?

Yes, I suppose so, since PMS currently says deb2targz is required when
ar is not compatible with GNU ar. However, earlier you said that other
package managers currently use ar unconditionally. Also, I have to
wonder if any of the other package managers are used on AIX. So, maybe a
retroactive change to PMS would be acceptable?

Maybe it's not worth the trouble, and we should simply keep the
deb2targz usage for AIX.
-- 
Thanks,
Zac



Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Ulrich Mueller
 On Sat, 06 Dec 2014, Zac Medico wrote:

 The PMS people should be *very* interested in any changes to unpack
 behavior like this. It supports behavior that will lead to failures for
 older versions of portage and other package managers.

Some remarks:

- The upstream deb2targz program supports only data.tar.gz. Only the
  Gentoo version has been patched up to support other file types.

- There is only a single package in the tree with a dependency on
  app-arch/deb2targz, and this only on AIX [1]. So on most users'
  systems deb2targz will not be installed, and ar x will be used for
  unpacking.

- Paludis and Pkgcore unconditionally use ar x for unpacking.

- PMS says [2]:

 deb packages (*.deb). Ebuilds must ensure that the deb2targz
 program is installed on those platforms where the GNU binutils
 ar program is not available and the installed ar program is
 incompatible with GNU archives. Otherwise, ebuilds must ensure
 that GNU binutils is installed.

  One could conclude that GNU binutils ar should be used on platforms
  where it is available. deb2targz should be used only where ar is
  incompatible with the GNU version.

So in any case, this is a problem of Portage only. I'd suggest to
change the logic in unpack() as follows:

if ar is GNU; then
ar x ...
elif deb2targz installed; then
deb2targz ...
else
ar x ...
fi

And then the deb2targz case could be patched as proposed. After all,
it is only a fallback that should be compatible with ar x
extraction.

Ulrich

[1] http://qa-reports.gentoo.org/output/genrdeps/dindex/app-arch/deb2targz
[2] http://dev.gentoo.org/~ulm/pms/5/pms.html#x1-14100011.3.3.13


pgph26utPMmUN.pgp
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Zac Medico
On 12/07/2014 03:04 AM, Ulrich Mueller wrote:
 On Sat, 06 Dec 2014, Zac Medico wrote:
 
 The PMS people should be *very* interested in any changes to unpack
 behavior like this. It supports behavior that will lead to failures for
 older versions of portage and other package managers.
 
 Some remarks:
 
 - The upstream deb2targz program supports only data.tar.gz. Only the
   Gentoo version has been patched up to support other file types.
 
 - There is only a single package in the tree with a dependency on
   app-arch/deb2targz, and this only on AIX [1]. So on most users'
   systems deb2targz will not be installed, and ar x will be used for
   unpacking.
 
 - Paludis and Pkgcore unconditionally use ar x for unpacking.
 
 - PMS says [2]:
 
  deb packages (*.deb). Ebuilds must ensure that the deb2targz
  program is installed on those platforms where the GNU binutils
  ar program is not available and the installed ar program is
  incompatible with GNU archives. Otherwise, ebuilds must ensure
  that GNU binutils is installed.
 
   One could conclude that GNU binutils ar should be used on platforms
   where it is available. deb2targz should be used only where ar is
   incompatible with the GNU version.
 
 So in any case, this is a problem of Portage only. I'd suggest to
 change the logic in unpack() as follows:
 
 if ar is GNU; then
 ar x ...
 elif deb2targz installed; then
 deb2targz ...
 else
 ar x ...
 fi
 
 And then the deb2targz case could be patched as proposed. After all,
 it is only a fallback that should be compatible with ar x
 extraction.
 
 Ulrich
 
 [1] http://qa-reports.gentoo.org/output/genrdeps/dindex/app-arch/deb2targz
 [2] http://dev.gentoo.org/~ulm/pms/5/pms.html#x1-14100011.3.3.13

Okay, so we may want to examine this AIX case more closely, because
dropping support for AIX would solve our problem with the lack of xz
support in deb2targz.

Anyway, Brian's patch for xz support with debt2targz appears to be
compatible with other package managers as well as AIX, so that seems
like a good way to go.
-- 
Thanks,
Zac



Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Zac Medico
On 12/06/2014 04:52 PM, Brian Dolbec wrote:
 
 From 4cb661d994cf8503c4459b8f7da7ee0f739a9826 Mon Sep 17 00:00:00 2001
 From: Brian Dolbec dol...@gentoo.org
 Date: Sat, 6 Dec 2014 14:51:13 -0800
 Subject: [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz 
 types
 MIME-Version: 1.0
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: 8bit

LGTM, given that other package managers default to 'ar x' which already
works with xz files.
-- 
Thanks,
Zac



Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Ulrich Mueller
 On Sun, 07 Dec 2014, Zac Medico wrote:

 Anyway, Brian's patch for xz support with debt2targz appears to be
 compatible with other package managers as well as AIX, so that seems
 like a good way to go.

I still think that ar should always be used on platforms where GNU
binutils is installed. Currently unpack uses deb2targz when it finds
the program, which is not a well defined behaviour.

Ulrich


pgppXonqP1VjG.pgp
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Zac Medico
On 12/07/2014 10:05 AM, Ulrich Mueller wrote:
 On Sun, 07 Dec 2014, Zac Medico wrote:
 
 Anyway, Brian's patch for xz support with debt2targz appears to be
 compatible with other package managers as well as AIX, so that seems
 like a good way to go.
 
 I still think that ar should always be used on platforms where GNU
 binutils is installed. Currently unpack uses deb2targz when it finds
 the program, which is not a well defined behaviour.

Okay, I guess we can default to ar if [[ $(ar --version 21) == GNU
ar* ]] and otherwise fall back to deb2targz.
-- 
Thanks,
Zac



Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Zac Medico
On 12/07/2014 09:54 AM, Zac Medico wrote:
 On 12/06/2014 04:52 PM, Brian Dolbec wrote:

 From 4cb661d994cf8503c4459b8f7da7ee0f739a9826 Mon Sep 17 00:00:00 2001
 From: Brian Dolbec dol...@gentoo.org
 Date: Sat, 6 Dec 2014 14:51:13 -0800
 Subject: [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz 
 types
 MIME-Version: 1.0
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: 8bit
 
 LGTM, given that other package managers default to 'ar x' which already
 works with xz files.

Based on discussion with Ulrich Mueller I think we should default to ar
if [[ $(ar --version 21) == GNU ar* ]] and otherwise fall back to
deb2targz.
-- 
Thanks,
Zac



Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Zac Medico
On 12/07/2014 11:23 AM, Fabian Groffen wrote:
 On 07-12-2014 10:49:30 -0800, Zac Medico wrote:
 On 12/07/2014 10:47 AM, Ulrich Mueller wrote:
 On Sun, 07 Dec 2014, Zac Medico wrote:

 On 12/07/2014 10:37 AM, Ulrich Mueller wrote:
 It's sort of trivial, but here is a patch:

 From c53e7057f94728d6e0c7d16c675702ca831b9a5a Mon Sep 17 00:00:00 2001
 From: Ulrich Müller u...@gentoo.org
 Date: Sun, 7 Dec 2014 19:33:52 +0100
 Subject: [PATCH] Prefer GNU ar when unpacking .deb packages.

 [...]

 LGTM.

 So shall I push it?

 Yes, please do.
 
 FYI:
 
 % portageq envvar CHOST
 x86_64-apple-darwin13
 % ar --version
 ar: illegal option -- -
 usage:  ar -d [-TLsv] archive file ...
 ar -m [-TLsv] archive file ...
 ar -m [-abiTLsv] position archive file ...
 ar -p [-TLsv] archive [file ...]
 ar -q [-cTLsv] archive file ...
 ar -r [-cuTLsv] archive file ...
 ar -r [-abciuTLsv] position archive file ...
 ar -t [-TLsv] archive [file ...]
 ar -x [-ouTLsv] archive [file ...]
 % ar x $EPREFIX/usr/portage/distfiles/realpath_1.16_i386.deb
 % ls
 control.tar.gz  data.tar.gz  debian-binary
 
 deb2targz should work (but I don't have it installed, so not sure if tested).
 
 Fabian
 

It's hard to whitelist it if doesn't support --version, so it seems
reasonable to require deb2targz whenever ar is not GNU ar.
-- 
Thanks,
Zac



Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Ulrich Mueller
 On Sun, 07 Dec 2014, Zac Medico wrote:

 On 12/07/2014 11:23 AM, Fabian Groffen wrote:
 FYI:
 
 % portageq envvar CHOST
 x86_64-apple-darwin13
 % ar --version
 ar: illegal option -- -
 usage:  ar -d [-TLsv] archive file ...
 [...]

 It's hard to whitelist it if doesn't support --version, so it seems
 reasonable to require deb2targz whenever ar is not GNU ar.

unpack will still use ar if deb2targz is not installed. So on Darwin,
behaviour should be the same as before.

Ulrich


pgpPQEfcOLwSJ.pgp
Description: PGP signature


Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-07 Thread Zac Medico
On 12/07/2014 11:41 AM, Ulrich Mueller wrote:
 On Sun, 07 Dec 2014, Zac Medico wrote:
 
 On 12/07/2014 11:23 AM, Fabian Groffen wrote:
 FYI:

 % portageq envvar CHOST
 x86_64-apple-darwin13
 % ar --version
 ar: illegal option -- -
 usage:  ar -d [-TLsv] archive file ...
 [...]
 
 It's hard to whitelist it if doesn't support --version, so it seems
 reasonable to require deb2targz whenever ar is not GNU ar.
 
 unpack will still use ar if deb2targz is not installed. So on Darwin,
 behaviour should be the same as before.
 
 Ulrich
 

Ah, that's a nice feature. That way, we can just leave it to platforms
like AIX to pull in deb2targz when necessary.
-- 
Thanks,
Zac



Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types

2014-12-06 Thread Zac Medico
On 12/06/2014 04:52 PM, Brian Dolbec wrote:
 
 From 4cb661d994cf8503c4459b8f7da7ee0f739a9826 Mon Sep 17 00:00:00 2001
 From: Brian Dolbec dol...@gentoo.org
 Date: Sat, 6 Dec 2014 14:51:13 -0800
 Subject: [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz 
 types
 MIME-Version: 1.0
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: 8bit
 
 Issue reported by: mva in irc
 
 Unpacking source...
 Unpacking 2gis_4.14.11-0trusty1+shv139+r191_amd64.deb to 
 /var/tmp/portage/app-misc/2gis-4.14.11.139.191/work
 deb2targz: converting '2gis_4.14.11-0trusty1+shv139+r191_amd64.deb' ...
 deb2targz: skipping section 'debian-binary'
 deb2targz: skipping section 'control.tar.gz'
 deb2targz: wrote '2gis_4.14.11-0trusty1+shv139+r191_amd64.tar.xz'
 mv: cannot stat ‘2gis_4.14.11-0trusty1+shv139+r191_amd64.tar.gz’: No such 
 file or directory
 ---
  bin/phase-helpers.sh | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
 
 diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh
 index 6e437da..91f80c9 100644
 --- a/bin/phase-helpers.sh
 +++ b/bin/phase-helpers.sh
 @@ -475,8 +475,9 @@ unpack() {
   rm -f $y
   fi
   if ! mv -f ${y%.deb}.tar.gz 
 data.tar.gz; then
 - __helpers_die $myfail
 - return 1
 + if ! mv -f ${y%.deb}.tar.xz 
 data.tar.xz; then
 + __helpers_die $myfail
 + return 1
   fi
   else
   if ! ar x $srcdir$x; then
 

The PMS people should be *very* interested in any changes to unpack
behavior like this. It supports behavior that will lead to failures for
older versions of portage and other package managers.
-- 
Thanks,
Zac