Re: [gentoo-dev] Gentoo-CI can now filter on maintainers (also verbose output)

2019-07-25 Thread Michał Górny
Dnia July 26, 2019 5:17:31 AM UTC, Georgy Yakovlev  
napisał(a):
>On Thursday, July 25, 2019 7:05:16 AM PDT Michał Górny wrote:
>> Hi,
>> 
>> Just a quick bit of news on gentoo-ci.
>> 
>> Firstly, I've implemented support for quiet/verbose reports.  So now
>> the default report seen at [1] is not that huge, and if somebody
>wants
>> all the details at [2].
>> 
>
>This is great! Now I need to RSS-feed it somehow.

That should be just a matter of adding a new jinja template.

>
>btw, there are some BadRestricts warnings for preserve-libs,
>pkgcheck/pkgcore does not support it, but portage does.
>Are you leaving this on intentionally?

I'm enabling everything in pkgcheck except for exp profiles and git checks (for 
performance reasons). Non-verbose reports include everything that does not 
result in huge number of matches.

Radhermit takes care of pkgcore & co. I only deploy CI and write new checks 
sometimes.

>
>trivial patch I did sometime ago
>https://patch-diff.githubusercontent.com/raw/pkgcore/pkgcheck/pull/75.patch


--
Best regards, 
Michał Górny



Re: [gentoo-dev] Gentoo-CI can now filter on maintainers (also verbose output)

2019-07-25 Thread Georgy Yakovlev
On Thursday, July 25, 2019 7:05:16 AM PDT Michał Górny wrote:
> Hi,
> 
> Just a quick bit of news on gentoo-ci.
> 
> Firstly, I've implemented support for quiet/verbose reports.  So now
> the default report seen at [1] is not that huge, and if somebody wants
> all the details at [2].
> 

This is great! Now I need to RSS-feed it somehow.

btw, there are some BadRestricts warnings for preserve-libs,
pkgcheck/pkgcore does not support it, but portage does.
Are you leaving this on intentionally?

trivial patch I did sometime ago
https://patch-diff.githubusercontent.com/raw/pkgcore/pkgcheck/pull/75.patch


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags

2019-07-25 Thread desultory
On 07/25/19 04:14, Kent Fredric wrote:
> On Thu, 25 Jul 2019 00:10:49 -0400
> desultory  wrote:
> 
>> The user-side effects pf the proposal in question, were it to become
>> policy, would be that anyone seeking to not install what is presently
>> optional documentation would either be:
>> (1) wasting build time and space (and, depending on implementation,
>> dependencies) on their build system (potentially  masking the files from
>> being installed);
>> (2) wasting the space in their installed image(s) (if they did not mask
>> the files which would not currently be installed anyway); or
>> (3) wasting their own time working around what the developers would be
>> required by policy to implement by repackaging the software themselves
>> to avoid the time and space drawbacks (though this would generally be
>> expected to be a rather exceptional case, as it would be relatively
>> extreme to avoid what would be a distfile and some file masking from the
>> user side).
> 
> Those concerns as per current policy is unrelated to the
> dependency-control-of-USE issue presented here.
> 
> Its already established not to use a USE flag to toggle building of man
> pages if it doesn't require additional dependencies.
> 
> These are _man_ pages we're talking about, not general purpose
> documentation.
> 
> End users who don't like man pages are encouraged to use the relevant
> FEATURES to strip them from the installed image, or use INSTALL_MASK or
> something. ( FEATURES=noman )
> 
> The GOAL here is to provide man pages in *all* circumstances, and, if
> additional dependencies are required to build them, to ALWAYS install
> them, or NEVER install them, and then, find a secondary way to obtain
> man pages (prebuilt)
> 
> The Total Cost of man pages as pure files on the target system is tiny,
> and has practically no risk with regards to complexity.
> 
> But the cost of *dependencies* has risk, and can inflate to complexity,
> causing much more real problems for more users than a few kb of .1.bz2
> files.
> 
> Hence, why we gate them with USE flags: Because it provides an out for
> that complexity ( at the cost of giving a different kind of complexity,
> and a reduction in utility if somebody has to opt-out to get around
> that initial complexity )
> 
So, we more or less concur on those points, or are you attempting to
convey some other meaning by restating points?

> And hence why the counter-proposal to USE flags to solve that
> complexity a different way is "prebuild them yourself and ship
> them" ( as that eliminates all the dependency complexity and USE flag
> complexity user side, while also giving them the man pages -> Quality )
> 
> Just the current mechanisms for this counter-proposal shift that
> inescapable complexity to a place where it becomes harder to manage in
> a standardized way.
> 
Are you referring to a QA mandate to package or build man pages,
regardless as a counter proposal to the status quo? If so, why? It is a
proposal; the status quo is, at the risk of redundancy, the present
state of things.

> And I don't think shifting this complexity to maintainers is the right
> step, even though I want the same deliverables.
> 
> That's why I'd rather a way to shift this complexity to a build
> service, ... albeit, it introduces some complexity of its own with the
> building and distribution of these man pages.
> 
As I have noted twice before in discussion of this proposal, such a
build service once existed, and indeed could alternately be provided as
a side effect of one or more of the tinderboxes still in operation, and
could with some additional scripting automatically generate such packages.

> Complexity is a pain-in-the-ass, you can't get rid of it, only shift it
> around till it stops hurting.
> 
> However, all that said, If we're going to shoot some kind of
> documentation in the face for the pain its dependency-complexity
> introduces, let it be "info".
> 
> - Far fewer people enjoy or can actually get useful information out of
>   info pages
> - Its build tooling frequently has dizzying problems in dependency
>   complexity and fragility, frequently made worse by portage getting
>   build ordering wrong after perl upgrades.[1]
> 
> (Hopefully we've fixed the worst of that, but this is plutonium, a gift
> that keeps giving cancer)
> 
> 1: https://bugs.gentoo.org/buglist.cgi?quicksearch=texinfo
> 
Since when is anyone proposing extirpating man pages on the whole? I am
simply making the rather simple suggestion that pulling in more packages
to support presently optional documentation as newly mandated
documentation when such documentation is neither expected nor desired by
the users of systems onto which it would be installed is not a net
benefit to anyone. Even default on USE flags would be a better "fix" for
the purported problem then making maintainers generate, package, and
publish man pages themselves.



Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes

2019-07-25 Thread Michael Orlitzky
On 7/25/19 4:29 PM, Michał Górny wrote:
>>
>> * In the md5-cache entry, maybe use a common prefix like EXT_ for the
>> extra keys in order to distinguish them from normal keys.
> 
> Yeah, I was thinking of something like '__ext_foo', or '__ext[foo]'.
> 

What are the pros/cons of this? The names refer to global variables, so
they should already be safely namespaced, right?.

There is a possibility that an eclass variable name (e.g. PATCHES) could
become standardized at a later date. If that happens, we could wind up
with both FOO and __ext_FOO in the cache, and tools would have to figure
out what to do with zero, one, or both present. (This has happened in
email/web protocols when an X-Foo header was standardized.) It's not the
end of the world, but someone would have to stop and think about it.

Finally, just having the name be predictable so that I can grep '^FOO='
without having to care where it came from is nice.

OTOH for testing, and for figuring out why these weird variables are
showing up in my cache, the prefix would help.




Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes

2019-07-25 Thread Michał Górny
On Thu, 2019-07-25 at 12:57 -0700, Zac Medico wrote:
> On 7/25/19 5:20 AM, Michał Górny wrote:
> > Hi,
> > 
> > TL;DR: I'd like to make it possible for ebuilds to define additional
> > variables that will be stored in md5-cache.  This would be useful for CI
> > and other tooling that right now has to parse ebuilds for other data.
> > 
> > 
> > The idea is to add a new incremental ebuild/eclass variable (technical
> > name: QA_EXTRA_CACHE_VARS) that would define additional data to be
> > stored in cache.  For example, python*-r1 eclasses would define
> > 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc.
> > 
> > When regenerating cache, the PM would read this variable, and store
> > the values of all defined variables into md5-cache.  As a result,
> > programs needing those variables can get them straight from cache
> > without having to attempt to run or parse ebuilds (which is both slow
> > and prone to bugs).
> > 
> > This would benefit e.g. gpyutils that right now need to attempt to parse
> > PYTHON_COMPAT from ebuilds.  It would also benefit writing future
> > pkgcheck checks for user/group ID collisions.
> > 
> > 
> > Notes:
> > 
> > - since md5-cache uses key-value format and allows for future
> > extensions, the new values can be added without breaking anything;
> > 
> > - md5-cache is not specified in the PMS, and the whole thing can be
> > implemented without need for EAPI bump,
> > 
> > - I would like to have this implemented consistently both in Portage
> > and pkgcore,
> > 
> > - we will need to clearly define how to dump arrays.
> > 
> > 
> > What do you think?
> 
> Sounds good. Some thoughts:
> 
> * Maybe omit QA from the variable name, since it can be could be
> generally useful for things that are unrelated to QA.
> 
> * In the md5-cache entry, maybe use a common prefix like EXT_ for the
> extra keys in order to distinguish them from normal keys.

Yeah, I was thinking of something like '__ext_foo', or '__ext[foo]'.

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes

2019-07-25 Thread Zac Medico
On 7/25/19 5:20 AM, Michał Górny wrote:
> Hi,
> 
> TL;DR: I'd like to make it possible for ebuilds to define additional
> variables that will be stored in md5-cache.  This would be useful for CI
> and other tooling that right now has to parse ebuilds for other data.
> 
> 
> The idea is to add a new incremental ebuild/eclass variable (technical
> name: QA_EXTRA_CACHE_VARS) that would define additional data to be
> stored in cache.  For example, python*-r1 eclasses would define
> 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc.
> 
> When regenerating cache, the PM would read this variable, and store
> the values of all defined variables into md5-cache.  As a result,
> programs needing those variables can get them straight from cache
> without having to attempt to run or parse ebuilds (which is both slow
> and prone to bugs).
> 
> This would benefit e.g. gpyutils that right now need to attempt to parse
> PYTHON_COMPAT from ebuilds.  It would also benefit writing future
> pkgcheck checks for user/group ID collisions.
> 
> 
> Notes:
> 
> - since md5-cache uses key-value format and allows for future
> extensions, the new values can be added without breaking anything;
> 
> - md5-cache is not specified in the PMS, and the whole thing can be
> implemented without need for EAPI bump,
> 
> - I would like to have this implemented consistently both in Portage
> and pkgcore,
> 
> - we will need to clearly define how to dump arrays.
> 
> 
> What do you think?

Sounds good. Some thoughts:

* Maybe omit QA from the variable name, since it can be could be
generally useful for things that are unrelated to QA.

* In the md5-cache entry, maybe use a common prefix like EXT_ for the
extra keys in order to distinguish them from normal keys.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Gentoo-CI can now filter on maintainers (also verbose output)

2019-07-25 Thread Kent Fredric
On Thu, 25 Jul 2019 16:05:16 +0200
Michał Górny  wrote:

> Thirdly, I've implemented filtering by maintainer.
> 
> To get reports for developer foo, append ';maintainer=foo', e.g.:

Thanks very much :). This will come in *very* handy for me :)


pgp_LVJov6dDk.pgp
Description: OpenPGP digital signature


[gentoo-dev] Gentoo-CI can now filter on maintainers (also verbose output)

2019-07-25 Thread Michał Górny
Hi,

Just a quick bit of news on gentoo-ci.

Firstly, I've implemented support for quiet/verbose reports.  So now
the default report seen at [1] is not that huge, and if somebody wants
all the details at [2].

Secondly, the reports now explicitly list maintainers for the reported
packages.

Thirdly, I've implemented filtering by maintainer.

To get reports for developer foo, append ';maintainer=foo', e.g.:

https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=foo

For non-devs, you need full e-mail address:

https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=f...@gentoo.org

You can also append ';include-projects' to include packages maintainer
indirectly by a project you're member of:

https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=fo
o;include-projects


[1] https://qa-reports.gentoo.org/output/gentoo-ci/output.html
[2] https://qa-reports.gentoo.org/output/gentoo-ci/output.verbose.html

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


[gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes

2019-07-25 Thread Michał Górny
Hi,

TL;DR: I'd like to make it possible for ebuilds to define additional
variables that will be stored in md5-cache.  This would be useful for CI
and other tooling that right now has to parse ebuilds for other data.


The idea is to add a new incremental ebuild/eclass variable (technical
name: QA_EXTRA_CACHE_VARS) that would define additional data to be
stored in cache.  For example, python*-r1 eclasses would define
'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc.

When regenerating cache, the PM would read this variable, and store
the values of all defined variables into md5-cache.  As a result,
programs needing those variables can get them straight from cache
without having to attempt to run or parse ebuilds (which is both slow
and prone to bugs).

This would benefit e.g. gpyutils that right now need to attempt to parse
PYTHON_COMPAT from ebuilds.  It would also benefit writing future
pkgcheck checks for user/group ID collisions.


Notes:

- since md5-cache uses key-value format and allows for future
extensions, the new values can be added without breaking anything;

- md5-cache is not specified in the PMS, and the whole thing can be
implemented without need for EAPI bump,

- I would like to have this implemented consistently both in Portage
and pkgcore,

- we will need to clearly define how to dump arrays.


What do you think?

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


[gentoo-dev] [PATCH v2] user.eclass: Fix egetgroups bash compliance, and make it simpler

2019-07-25 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 eclass/user.eclass | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/eclass/user.eclass b/eclass/user.eclass
index fdf98caa6099..9dc15fa75d23 100644
--- a/eclass/user.eclass
+++ b/eclass/user.eclass
@@ -445,11 +445,12 @@ egetgroups() {
local egroups_arr
read -r -a egroups_arr < <(id -G -n "$1")
 
-   local defgroup=${egroups_arr[0]}
+   local g groups=${egroups_arr[0]}
# sort supplementary groups to make comparison possible
-   readarray -t exgroups_arr < <(printf '%s\n' "${egroups_arr[@]:1}" | 
sort)
-   local exgroups=${exgroups_arr[*]}
-   echo "${defgroup}${exgroups:+,${exgroups// /,}}"
+   while read -r g; do
+   [[ -n ${g} ]] && groups+=",${g}"
+   done < <(printf '%s\n' "${egroups_arr[@]:1}" | sort)
+   echo "${groups}"
 }
 
 # @FUNCTION: esethome
-- 
2.22.0




Re: [gentoo-dev] [PATCH] user.eclass: Fix egetgroups bash compliance, and make it simpler

2019-07-25 Thread Michał Górny
On Thu, 2019-07-25 at 10:14 +0200, Ulrich Mueller wrote:
> > > > > > On Thu, 25 Jul 2019, Michał Górny wrote:
> > -   local defgroup=${egroups_arr[0]}
> > +   local g groups=( "${egroups_arr[0]}" )
> > # sort supplementary groups to make comparison possible
> > -   readarray -t exgroups_arr < <(printf '%s\n' "${egroups_arr[@]:1}" | 
> > sort)
> > -   local exgroups=${exgroups_arr[*]}
> > -   echo "${defgroup}${exgroups:+,${exgroups// /,}}"
> > +   while read -r g; do
> > +   [[ -n ${g} ]] && groups+=( "${g}" )
> > +   done < <(printf '%s\n' "${egroups_arr[@]:1}" | sort)
> > +   groups=${groups[*]}
> > +   echo "${groups// /,}"
> >  }
>  
> Why don't you make groups a scalar variable (i.e., a comma separated
> list) from the very beginning, if you're converting to it later anyway?
> 
>   local g groups=${egroups_arr[0]}
>   # sort supplementary groups to make comparison possible
>   while read -r g; do
>   [[ -n ${g} ]] && groups+=",${g}"
>   done < <(printf '%s\n' "${egroups_arr[@]:1}" | sort)
>   echo "${groups}"

Indeed.  Will do.

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags

2019-07-25 Thread Kent Fredric
On Thu, 25 Jul 2019 00:10:49 -0400
desultory  wrote:

> The user-side effects pf the proposal in question, were it to become
> policy, would be that anyone seeking to not install what is presently
> optional documentation would either be:
> (1) wasting build time and space (and, depending on implementation,
> dependencies) on their build system (potentially  masking the files from
> being installed);
> (2) wasting the space in their installed image(s) (if they did not mask
> the files which would not currently be installed anyway); or
> (3) wasting their own time working around what the developers would be
> required by policy to implement by repackaging the software themselves
> to avoid the time and space drawbacks (though this would generally be
> expected to be a rather exceptional case, as it would be relatively
> extreme to avoid what would be a distfile and some file masking from the
> user side).

Those concerns as per current policy is unrelated to the
dependency-control-of-USE issue presented here.

Its already established not to use a USE flag to toggle building of man
pages if it doesn't require additional dependencies.

These are _man_ pages we're talking about, not general purpose
documentation.

End users who don't like man pages are encouraged to use the relevant
FEATURES to strip them from the installed image, or use INSTALL_MASK or
something. ( FEATURES=noman )

The GOAL here is to provide man pages in *all* circumstances, and, if
additional dependencies are required to build them, to ALWAYS install
them, or NEVER install them, and then, find a secondary way to obtain
man pages (prebuilt)

The Total Cost of man pages as pure files on the target system is tiny,
and has practically no risk with regards to complexity.

But the cost of *dependencies* has risk, and can inflate to complexity,
causing much more real problems for more users than a few kb of .1.bz2
files.

Hence, why we gate them with USE flags: Because it provides an out for
that complexity ( at the cost of giving a different kind of complexity,
and a reduction in utility if somebody has to opt-out to get around
that initial complexity )

And hence why the counter-proposal to USE flags to solve that
complexity a different way is "prebuild them yourself and ship
them" ( as that eliminates all the dependency complexity and USE flag
complexity user side, while also giving them the man pages -> Quality )

Just the current mechanisms for this counter-proposal shift that
inescapable complexity to a place where it becomes harder to manage in
a standardized way.

And I don't think shifting this complexity to maintainers is the right
step, even though I want the same deliverables.

That's why I'd rather a way to shift this complexity to a build
service, ... albeit, it introduces some complexity of its own with the
building and distribution of these man pages.

Complexity is a pain-in-the-ass, you can't get rid of it, only shift it
around till it stops hurting.

However, all that said, If we're going to shoot some kind of
documentation in the face for the pain its dependency-complexity
introduces, let it be "info".

- Far fewer people enjoy or can actually get useful information out of
  info pages
- Its build tooling frequently has dizzying problems in dependency
  complexity and fragility, frequently made worse by portage getting
  build ordering wrong after perl upgrades.[1]

(Hopefully we've fixed the worst of that, but this is plutonium, a gift
that keeps giving cancer)

1: https://bugs.gentoo.org/buglist.cgi?quicksearch=texinfo


pgpyG9eHivbrG.pgp
Description: OpenPGP digital signature


Re: [gentoo-dev] [PATCH] user.eclass: Fix egetgroups bash compliance, and make it simpler

2019-07-25 Thread Ulrich Mueller
> On Thu, 25 Jul 2019, Michał Górny wrote:

> - local defgroup=${egroups_arr[0]}
> + local g groups=( "${egroups_arr[0]}" )
>   # sort supplementary groups to make comparison possible
> - readarray -t exgroups_arr < <(printf '%s\n' "${egroups_arr[@]:1}" | 
> sort)
> - local exgroups=${exgroups_arr[*]}
> - echo "${defgroup}${exgroups:+,${exgroups// /,}}"
> + while read -r g; do
> + [[ -n ${g} ]] && groups+=( "${g}" )
> + done < <(printf '%s\n' "${egroups_arr[@]:1}" | sort)
> + groups=${groups[*]}
> + echo "${groups// /,}"
>  }
 
Why don't you make groups a scalar variable (i.e., a comma separated
list) from the very beginning, if you're converting to it later anyway?

local g groups=${egroups_arr[0]}
# sort supplementary groups to make comparison possible
while read -r g; do
[[ -n ${g} ]] && groups+=",${g}"
done < <(printf '%s\n' "${egroups_arr[@]:1}" | sort)
echo "${groups}"


signature.asc
Description: PGP signature


[gentoo-dev] [PATCH] user.eclass: Fix egetgroups bash compliance, and make it simpler

2019-07-25 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 eclass/user.eclass | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/eclass/user.eclass b/eclass/user.eclass
index fdf98caa6099..b237abdda045 100644
--- a/eclass/user.eclass
+++ b/eclass/user.eclass
@@ -445,11 +445,13 @@ egetgroups() {
local egroups_arr
read -r -a egroups_arr < <(id -G -n "$1")
 
-   local defgroup=${egroups_arr[0]}
+   local g groups=( "${egroups_arr[0]}" )
# sort supplementary groups to make comparison possible
-   readarray -t exgroups_arr < <(printf '%s\n' "${egroups_arr[@]:1}" | 
sort)
-   local exgroups=${exgroups_arr[*]}
-   echo "${defgroup}${exgroups:+,${exgroups// /,}}"
+   while read -r g; do
+   [[ -n ${g} ]] && groups+=( "${g}" )
+   done < <(printf '%s\n' "${egroups_arr[@]:1}" | sort)
+   groups=${groups[*]}
+   echo "${groups// /,}"
 }
 
 # @FUNCTION: esethome
-- 
2.22.0




Re: [gentoo-dev] [PATCH v2 4/5] vcs-snapshot.eclass: Detect and report invalid directory structure

2019-07-25 Thread Michał Górny
On Thu, 2019-07-25 at 09:37 +0200, Michał Górny wrote:
> Detect when the archive does not contain a single top-level directory,
> and abort in that case.  Otherwise, --strip-components would result
> in unpredictable mess.
> 
> Signed-off-by: Michał Górny 
> ---
>  eclass/vcs-snapshot.eclass | 17 +++--
>  1 file changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
> index 312e9a4611e1..33abd9a7c190 100644
> --- a/eclass/vcs-snapshot.eclass
> +++ b/eclass/vcs-snapshot.eclass
> @@ -1,87 +1,100 @@
>  # Copyright 1999-2019 Gentoo Authors
>  # Distributed under the terms of the GNU General Public License v2
>  
>  # @ECLASS: vcs-snapshot.eclass
>  # @MAINTAINER:
>  # mgo...@gentoo.org
>  # @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
>  # @BLURB: support eclass for unpacking VCS snapshot tarballs
>  # @DESCRIPTION:
>  # THIS ECLASS IS NOT NECESSARY FOR MODERN GITHUB AND GITLAB SNAPSHOTS.
>  # THEIR DIRECTORY STRUCTURE IS ENTIRELY PREDICTABLE, SO UPDATE YOUR
>  # EBUILD TO USE /ARCHIVE/ URI AND SET S IF NECESSARY.
>  #
>  # This eclass provides a convenience src_unpack() which does unpack all
>  # the tarballs in SRC_URI to locations matching their (local) names,
>  # discarding the original parent directory.
>  #
>  # The typical use case are VCS tag snapshots coming from BitBucket
>  # (but not GitHub or GitLab).  They have hash appended to the directory
>  # name which makes extracting them a painful experience.  But if you are
>  # using a SRC_URI arrow to rename them (which quite likely you have to
>  # do anyway), vcs-snapshot will just extract them into matching
>  # directories.
>  #
>  # Please note that this eclass handles only tarballs (.tar, .tar.gz,
>  # .tar.bz2 & .tar.xz).  For any other file format (or suffix) it will
>  # fall back to regular unpack.  Support for additional formats may be
>  # added in the future if necessary.
>  #
>  # @EXAMPLE:
>  #
>  # @CODE
>  # EAPI=7
>  # inherit vcs-snapshot
>  #
>  # SRC_URI="
>  #https://bitbucket.org/foo/bar/get/${PV}.tar.bz2 -> ${P}.tar.bz2
>  #https://bitbucket.org/foo/bar-otherstuff/get/${PV}.tar.bz2
>  #-> ${P}-otherstuff.tar.bz2"
>  # @CODE
>  #
>  # and however the tarballs were originally packed, all files will appear
>  # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
>  
>  case ${EAPI:-0} in
>   0|1|2|3|4|5|6|7) ;;
>   *) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
>  esac
>  
>  EXPORT_FUNCTIONS src_unpack
>  
>  # @FUNCTION: vcs-snapshot_src_unpack
>  # @DESCRIPTION:
>  # Extract all the archives from ${A}. The .tar, .tar.gz, .tar.bz2
>  # and .tar.xz archives will be unpacked to directories matching their
>  # local names. Other archive types will be passed down to regular
>  # unpack.
>  vcs-snapshot_src_unpack() {
>   debug-print-function ${FUNCNAME} "${@}"
>  
>   local f
>  
>   for f in ${A}
>   do
>   case "${f}" in
>   *.tar|*.tar.gz|*.tar.bz2|*.tar.xz)
>   local destdir=${WORKDIR}/${f%.tar*}
>  
>   debug-print "${FUNCNAME}: unpacking ${f} to 
> ${destdir}"
>  
> - # XXX: check whether the directory structure 
> inside is
> - # fine? i.e. if the tarball has actually a 
> parent dir.
> + local l topdirs=()
> + while read l; do

Oh, and please imagine you see '-r' here.

> + topdirs+=( "${l}" )
> + done < <(tar -t -f "${DISTDIR}/${f}" | cut -d/ 
> -f1 | sort -u)
> + if [[ ${#topdirs[@]} -gt 1 ]]; then
> + eerror "The archive ${f} contains 
> multiple or no top directory."
> + eerror "It is impossible for 
> vcs-snapshot to unpack this correctly."
> + eerror "Top directories found:"
> + local d
> + for d in "${topdirs[@]}"; do
> + eerror "${d}"
> + done
> + die "${FUNCNAME}: Invalid directory 
> structure in archive ${f}"
> + fi
> +
>   mkdir "${destdir}" || die
>   # -o (--no-same-owner) to avoid restoring 
> original owner
>   einfo "Unpacking ${f}"
>   tar -C "${destdir}" -x -o --strip-components 1 \
>   -f "${DISTDIR}/${f}" || die
>   ;;
>   *)
>   debug-print "${FUNCNAME}: falling back to 
> unpack for ${f}"
>  
>   # fall back to 

[gentoo-dev] [PATCH v2 3/5] vcs-snapshot.eclass: Add verbose einfo for unpacking

2019-07-25 Thread Michał Górny
Add an einfo call to make explicit notice of each archive being
unpacked.

Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 2e734c509d1a..312e9a4611e1 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -1,86 +1,87 @@
 # Copyright 1999-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: vcs-snapshot.eclass
 # @MAINTAINER:
 # mgo...@gentoo.org
 # @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
 # THIS ECLASS IS NOT NECESSARY FOR MODERN GITHUB AND GITLAB SNAPSHOTS.
 # THEIR DIRECTORY STRUCTURE IS ENTIRELY PREDICTABLE, SO UPDATE YOUR
 # EBUILD TO USE /ARCHIVE/ URI AND SET S IF NECESSARY.
 #
 # This eclass provides a convenience src_unpack() which does unpack all
 # the tarballs in SRC_URI to locations matching their (local) names,
 # discarding the original parent directory.
 #
 # The typical use case are VCS tag snapshots coming from BitBucket
 # (but not GitHub or GitLab).  They have hash appended to the directory
 # name which makes extracting them a painful experience.  But if you are
 # using a SRC_URI arrow to rename them (which quite likely you have to
 # do anyway), vcs-snapshot will just extract them into matching
 # directories.
 #
 # Please note that this eclass handles only tarballs (.tar, .tar.gz,
 # .tar.bz2 & .tar.xz).  For any other file format (or suffix) it will
 # fall back to regular unpack.  Support for additional formats may be
 # added in the future if necessary.
 #
 # @EXAMPLE:
 #
 # @CODE
 # EAPI=7
 # inherit vcs-snapshot
 #
 # SRC_URI="
 #https://bitbucket.org/foo/bar/get/${PV}.tar.bz2 -> ${P}.tar.bz2
 #https://bitbucket.org/foo/bar-otherstuff/get/${PV}.tar.bz2
 #-> ${P}-otherstuff.tar.bz2"
 # @CODE
 #
 # and however the tarballs were originally packed, all files will appear
 # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
 
 case ${EAPI:-0} in
0|1|2|3|4|5|6|7) ;;
*) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
 esac
 
 EXPORT_FUNCTIONS src_unpack
 
 # @FUNCTION: vcs-snapshot_src_unpack
 # @DESCRIPTION:
 # Extract all the archives from ${A}. The .tar, .tar.gz, .tar.bz2
 # and .tar.xz archives will be unpacked to directories matching their
 # local names. Other archive types will be passed down to regular
 # unpack.
 vcs-snapshot_src_unpack() {
debug-print-function ${FUNCNAME} "${@}"
 
local f
 
for f in ${A}
do
case "${f}" in
*.tar|*.tar.gz|*.tar.bz2|*.tar.xz)
local destdir=${WORKDIR}/${f%.tar*}
 
debug-print "${FUNCNAME}: unpacking ${f} to 
${destdir}"
 
# XXX: check whether the directory structure 
inside is
# fine? i.e. if the tarball has actually a 
parent dir.
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
+   einfo "Unpacking ${f}"
tar -C "${destdir}" -x -o --strip-components 1 \
-f "${DISTDIR}/${f}" || die
;;
*)
debug-print "${FUNCNAME}: falling back to 
unpack for ${f}"
 
# fall back to the default method
unpack "${f}"
;;
esac
done
 }
-- 
2.22.0




[gentoo-dev] [PATCH v2 5/5] vcs-snapshot.eclass: Detect unnecessary usage and complain

2019-07-25 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 12 
 1 file changed, 12 insertions(+)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 33abd9a7c190..dad586c9e40b 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -1,100 +1,112 @@
 # Copyright 1999-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: vcs-snapshot.eclass
 # @MAINTAINER:
 # mgo...@gentoo.org
 # @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
 # THIS ECLASS IS NOT NECESSARY FOR MODERN GITHUB AND GITLAB SNAPSHOTS.
 # THEIR DIRECTORY STRUCTURE IS ENTIRELY PREDICTABLE, SO UPDATE YOUR
 # EBUILD TO USE /ARCHIVE/ URI AND SET S IF NECESSARY.
 #
 # This eclass provides a convenience src_unpack() which does unpack all
 # the tarballs in SRC_URI to locations matching their (local) names,
 # discarding the original parent directory.
 #
 # The typical use case are VCS tag snapshots coming from BitBucket
 # (but not GitHub or GitLab).  They have hash appended to the directory
 # name which makes extracting them a painful experience.  But if you are
 # using a SRC_URI arrow to rename them (which quite likely you have to
 # do anyway), vcs-snapshot will just extract them into matching
 # directories.
 #
 # Please note that this eclass handles only tarballs (.tar, .tar.gz,
 # .tar.bz2 & .tar.xz).  For any other file format (or suffix) it will
 # fall back to regular unpack.  Support for additional formats may be
 # added in the future if necessary.
 #
 # @EXAMPLE:
 #
 # @CODE
 # EAPI=7
 # inherit vcs-snapshot
 #
 # SRC_URI="
 #https://bitbucket.org/foo/bar/get/${PV}.tar.bz2 -> ${P}.tar.bz2
 #https://bitbucket.org/foo/bar-otherstuff/get/${PV}.tar.bz2
 #-> ${P}-otherstuff.tar.bz2"
 # @CODE
 #
 # and however the tarballs were originally packed, all files will appear
 # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
 
 case ${EAPI:-0} in
0|1|2|3|4|5|6|7) ;;
*) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
 esac
 
 EXPORT_FUNCTIONS src_unpack
 
 # @FUNCTION: vcs-snapshot_src_unpack
 # @DESCRIPTION:
 # Extract all the archives from ${A}. The .tar, .tar.gz, .tar.bz2
 # and .tar.xz archives will be unpacked to directories matching their
 # local names. Other archive types will be passed down to regular
 # unpack.
 vcs-snapshot_src_unpack() {
debug-print-function ${FUNCNAME} "${@}"
 
+   local renamed_any=
local f
 
for f in ${A}
do
case "${f}" in
*.tar|*.tar.gz|*.tar.bz2|*.tar.xz)
local destdir=${WORKDIR}/${f%.tar*}
 
debug-print "${FUNCNAME}: unpacking ${f} to 
${destdir}"
 
local l topdirs=()
while read l; do
topdirs+=( "${l}" )
done < <(tar -t -f "${DISTDIR}/${f}" | cut -d/ 
-f1 | sort -u)
if [[ ${#topdirs[@]} -gt 1 ]]; then
eerror "The archive ${f} contains 
multiple or no top directory."
eerror "It is impossible for 
vcs-snapshot to unpack this correctly."
eerror "Top directories found:"
local d
for d in "${topdirs[@]}"; do
eerror "${d}"
done
die "${FUNCNAME}: Invalid directory 
structure in archive ${f}"
fi
+   [[ ${topdirs[0]} != ${f%.tar*} ]] && 
renamed_any=1
 
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
einfo "Unpacking ${f}"
tar -C "${destdir}" -x -o --strip-components 1 \
-f "${DISTDIR}/${f}" || die
;;
*)
debug-print "${FUNCNAME}: falling back to 
unpack for ${f}"
 
# fall back to the default method
unpack "${f}"
;;
esac
done
+
+   if [[ ! ${renamed_any} ]]; then
+   local w=eerror
+   [[ ${EAPI} == [0123456] ]] && w=eqawarn
+   "${w}" "${FUNCNAME} did not find any archives that needed 
renaming."
+   "${w}" "Please verify that its usage is really necessary, and 
remove"
+   "${w}" "the inherit if it is not."
+
+   [[ ${w} == 

[gentoo-dev] [PATCH v2 4/5] vcs-snapshot.eclass: Detect and report invalid directory structure

2019-07-25 Thread Michał Górny
Detect when the archive does not contain a single top-level directory,
and abort in that case.  Otherwise, --strip-components would result
in unpredictable mess.

Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 17 +++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 312e9a4611e1..33abd9a7c190 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -1,87 +1,100 @@
 # Copyright 1999-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: vcs-snapshot.eclass
 # @MAINTAINER:
 # mgo...@gentoo.org
 # @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
 # THIS ECLASS IS NOT NECESSARY FOR MODERN GITHUB AND GITLAB SNAPSHOTS.
 # THEIR DIRECTORY STRUCTURE IS ENTIRELY PREDICTABLE, SO UPDATE YOUR
 # EBUILD TO USE /ARCHIVE/ URI AND SET S IF NECESSARY.
 #
 # This eclass provides a convenience src_unpack() which does unpack all
 # the tarballs in SRC_URI to locations matching their (local) names,
 # discarding the original parent directory.
 #
 # The typical use case are VCS tag snapshots coming from BitBucket
 # (but not GitHub or GitLab).  They have hash appended to the directory
 # name which makes extracting them a painful experience.  But if you are
 # using a SRC_URI arrow to rename them (which quite likely you have to
 # do anyway), vcs-snapshot will just extract them into matching
 # directories.
 #
 # Please note that this eclass handles only tarballs (.tar, .tar.gz,
 # .tar.bz2 & .tar.xz).  For any other file format (or suffix) it will
 # fall back to regular unpack.  Support for additional formats may be
 # added in the future if necessary.
 #
 # @EXAMPLE:
 #
 # @CODE
 # EAPI=7
 # inherit vcs-snapshot
 #
 # SRC_URI="
 #https://bitbucket.org/foo/bar/get/${PV}.tar.bz2 -> ${P}.tar.bz2
 #https://bitbucket.org/foo/bar-otherstuff/get/${PV}.tar.bz2
 #-> ${P}-otherstuff.tar.bz2"
 # @CODE
 #
 # and however the tarballs were originally packed, all files will appear
 # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
 
 case ${EAPI:-0} in
0|1|2|3|4|5|6|7) ;;
*) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
 esac
 
 EXPORT_FUNCTIONS src_unpack
 
 # @FUNCTION: vcs-snapshot_src_unpack
 # @DESCRIPTION:
 # Extract all the archives from ${A}. The .tar, .tar.gz, .tar.bz2
 # and .tar.xz archives will be unpacked to directories matching their
 # local names. Other archive types will be passed down to regular
 # unpack.
 vcs-snapshot_src_unpack() {
debug-print-function ${FUNCNAME} "${@}"
 
local f
 
for f in ${A}
do
case "${f}" in
*.tar|*.tar.gz|*.tar.bz2|*.tar.xz)
local destdir=${WORKDIR}/${f%.tar*}
 
debug-print "${FUNCNAME}: unpacking ${f} to 
${destdir}"
 
-   # XXX: check whether the directory structure 
inside is
-   # fine? i.e. if the tarball has actually a 
parent dir.
+   local l topdirs=()
+   while read l; do
+   topdirs+=( "${l}" )
+   done < <(tar -t -f "${DISTDIR}/${f}" | cut -d/ 
-f1 | sort -u)
+   if [[ ${#topdirs[@]} -gt 1 ]]; then
+   eerror "The archive ${f} contains 
multiple or no top directory."
+   eerror "It is impossible for 
vcs-snapshot to unpack this correctly."
+   eerror "Top directories found:"
+   local d
+   for d in "${topdirs[@]}"; do
+   eerror "${d}"
+   done
+   die "${FUNCNAME}: Invalid directory 
structure in archive ${f}"
+   fi
+
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
einfo "Unpacking ${f}"
tar -C "${destdir}" -x -o --strip-components 1 \
-f "${DISTDIR}/${f}" || die
;;
*)
debug-print "${FUNCNAME}: falling back to 
unpack for ${f}"
 
# fall back to the default method
unpack "${f}"
;;
esac
done
 }
-- 
2.22.0




[gentoo-dev] [PATCH v2 2/5] vcs-snapshot.eclass: Update 'unusage' warning

2019-07-25 Thread Michał Górny
Nowadays both GitHub and GitLab snapshot do not need this eclass,
so make it clear it's needed only for BitBucket.  Also make the warning
uppercase because people still don't read it.

Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 33 ++---
 1 file changed, 18 insertions(+), 15 deletions(-)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 321e307b894d..2e734c509d1a 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -1,83 +1,86 @@
 # Copyright 1999-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: vcs-snapshot.eclass
 # @MAINTAINER:
 # mgo...@gentoo.org
 # @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
+# THIS ECLASS IS NOT NECESSARY FOR MODERN GITHUB AND GITLAB SNAPSHOTS.
+# THEIR DIRECTORY STRUCTURE IS ENTIRELY PREDICTABLE, SO UPDATE YOUR
+# EBUILD TO USE /ARCHIVE/ URI AND SET S IF NECESSARY.
+#
 # This eclass provides a convenience src_unpack() which does unpack all
 # the tarballs in SRC_URI to locations matching their (local) names,
 # discarding the original parent directory.
 #
-# The typical use case are VCS snapshots, coming from bitbucket
-# and similar services. They have hash appended to the directory name
-# which makes extracting them a painful experience. But if you just use
-# a SRC_URI arrow to rename it (which you're likely have to do anyway),
-# vcs-snapshot will just extract it into a matching directory.
+# The typical use case are VCS tag snapshots coming from BitBucket
+# (but not GitHub or GitLab).  They have hash appended to the directory
+# name which makes extracting them a painful experience.  But if you are
+# using a SRC_URI arrow to rename them (which quite likely you have to
+# do anyway), vcs-snapshot will just extract them into matching
+# directories.
 #
 # Please note that this eclass handles only tarballs (.tar, .tar.gz,
-# .tar.bz2 & .tar.xz). For any other file format (or suffix) it will
-# fall back to regular unpack. Support for additional formats may be
-# added at some point so please keep your SRC_URIs clean.
-#
-# Note: this eclass is no longer needed with the new-style 'archive'
-# GitHub URLs. They have sane directory names and stable contents,
-# so you should really prefer them.
+# .tar.bz2 & .tar.xz).  For any other file format (or suffix) it will
+# fall back to regular unpack.  Support for additional formats may be
+# added in the future if necessary.
 #
 # @EXAMPLE:
 #
 # @CODE
-# EAPI=6
+# EAPI=7
 # inherit vcs-snapshot
 #
-# SRC_URI="https://github.com/example/${PN}/tarball/v${PV} -> ${P}.tar.gz
-#  https://github.com/example/${PN}-otherstuff/tarball/v${PV} -> 
${P}-otherstuff.tar.gz"
+# SRC_URI="
+#https://bitbucket.org/foo/bar/get/${PV}.tar.bz2 -> ${P}.tar.bz2
+#https://bitbucket.org/foo/bar-otherstuff/get/${PV}.tar.bz2
+#-> ${P}-otherstuff.tar.bz2"
 # @CODE
 #
 # and however the tarballs were originally packed, all files will appear
 # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
 
 case ${EAPI:-0} in
0|1|2|3|4|5|6|7) ;;
*) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
 esac
 
 EXPORT_FUNCTIONS src_unpack
 
 # @FUNCTION: vcs-snapshot_src_unpack
 # @DESCRIPTION:
 # Extract all the archives from ${A}. The .tar, .tar.gz, .tar.bz2
 # and .tar.xz archives will be unpacked to directories matching their
 # local names. Other archive types will be passed down to regular
 # unpack.
 vcs-snapshot_src_unpack() {
debug-print-function ${FUNCNAME} "${@}"
 
local f
 
for f in ${A}
do
case "${f}" in
*.tar|*.tar.gz|*.tar.bz2|*.tar.xz)
local destdir=${WORKDIR}/${f%.tar*}
 
debug-print "${FUNCNAME}: unpacking ${f} to 
${destdir}"
 
# XXX: check whether the directory structure 
inside is
# fine? i.e. if the tarball has actually a 
parent dir.
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
tar -C "${destdir}" -x -o --strip-components 1 \
-f "${DISTDIR}/${f}" || die
;;
*)
debug-print "${FUNCNAME}: falling back to 
unpack for ${f}"
 
# fall back to the default method
unpack "${f}"
;;
esac
done
 }
-- 
2.22.0




[gentoo-dev] [PATCH v2 1/5] vcs-snapshot.eclass: Allow EAPI 7

2019-07-25 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 316a37773d43..321e307b894d 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -1,83 +1,83 @@
-# Copyright 1999-2016 Gentoo Foundation
+# Copyright 1999-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: vcs-snapshot.eclass
 # @MAINTAINER:
 # mgo...@gentoo.org
-# @SUPPORTED_EAPIS: 0 1 2 3 4 5 6
+# @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
 # This eclass provides a convenience src_unpack() which does unpack all
 # the tarballs in SRC_URI to locations matching their (local) names,
 # discarding the original parent directory.
 #
 # The typical use case are VCS snapshots, coming from bitbucket
 # and similar services. They have hash appended to the directory name
 # which makes extracting them a painful experience. But if you just use
 # a SRC_URI arrow to rename it (which you're likely have to do anyway),
 # vcs-snapshot will just extract it into a matching directory.
 #
 # Please note that this eclass handles only tarballs (.tar, .tar.gz,
 # .tar.bz2 & .tar.xz). For any other file format (or suffix) it will
 # fall back to regular unpack. Support for additional formats may be
 # added at some point so please keep your SRC_URIs clean.
 #
 # Note: this eclass is no longer needed with the new-style 'archive'
 # GitHub URLs. They have sane directory names and stable contents,
 # so you should really prefer them.
 #
 # @EXAMPLE:
 #
 # @CODE
 # EAPI=6
 # inherit vcs-snapshot
 #
 # SRC_URI="https://github.com/example/${PN}/tarball/v${PV} -> ${P}.tar.gz
 #  https://github.com/example/${PN}-otherstuff/tarball/v${PV} -> 
${P}-otherstuff.tar.gz"
 # @CODE
 #
 # and however the tarballs were originally packed, all files will appear
 # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
 
 case ${EAPI:-0} in
-   0|1|2|3|4|5|6) ;;
+   0|1|2|3|4|5|6|7) ;;
*) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
 esac
 
 EXPORT_FUNCTIONS src_unpack
 
 # @FUNCTION: vcs-snapshot_src_unpack
 # @DESCRIPTION:
 # Extract all the archives from ${A}. The .tar, .tar.gz, .tar.bz2
 # and .tar.xz archives will be unpacked to directories matching their
 # local names. Other archive types will be passed down to regular
 # unpack.
 vcs-snapshot_src_unpack() {
debug-print-function ${FUNCNAME} "${@}"
 
local f
 
for f in ${A}
do
case "${f}" in
*.tar|*.tar.gz|*.tar.bz2|*.tar.xz)
local destdir=${WORKDIR}/${f%.tar*}
 
debug-print "${FUNCNAME}: unpacking ${f} to 
${destdir}"
 
# XXX: check whether the directory structure 
inside is
# fine? i.e. if the tarball has actually a 
parent dir.
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
tar -C "${destdir}" -x -o --strip-components 1 \
-f "${DISTDIR}/${f}" || die
;;
*)
debug-print "${FUNCNAME}: falling back to 
unpack for ${f}"
 
# fall back to the default method
unpack "${f}"
;;
esac
done
 }
-- 
2.22.0




Re: [gentoo-dev] [PATCH 4/5] vcs-snapshot.eclass: Detect and report invalid directory structure

2019-07-25 Thread Michał Górny
On Thu, 2019-07-25 at 09:15 +0200, Ulrich Mueller wrote:
> > > > > > On Thu, 25 Jul 2019, Michał Górny wrote:
> > +   readarray -t topdirs \
> 
> Not legal in EAPIs 0 to 5 which use bash-3.2.
> 

I knew there was a reason I was doing it manually in all these eclasses!

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-dev] [PATCH 4/5] vcs-snapshot.eclass: Detect and report invalid directory structure

2019-07-25 Thread Ulrich Mueller
> On Thu, 25 Jul 2019, Ulrich Mueller wrote:

>> +readarray -t topdirs \

> Not legal in EAPIs 0 to 5 which use bash-3.2.

In fact, not legal in EAPIs 6 and 7 either, because the -t option
doesn't exist in bash-4.2.


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH 4/5] vcs-snapshot.eclass: Detect and report invalid directory structure

2019-07-25 Thread Ulrich Mueller
> On Thu, 25 Jul 2019, Michał Górny wrote:

> + readarray -t topdirs \

Not legal in EAPIs 0 to 5 which use bash-3.2.

Ulrich


signature.asc
Description: PGP signature


[gentoo-dev] [PATCH 1/3] ruby-fakegem.eclass: enable EAPI 7

2019-07-25 Thread Hans de Graaff
Signed-off-by: Hans de Graaff 
---
 eclass/ruby-fakegem.eclass | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/eclass/ruby-fakegem.eclass b/eclass/ruby-fakegem.eclass
index 1e8620c166d4..ab7b36eb7ae7 100644
--- a/eclass/ruby-fakegem.eclass
+++ b/eclass/ruby-fakegem.eclass
@@ -8,7 +8,7 @@
 # Author: Diego E. Pettenò 
 # Author: Alex Legler 
 # Author: Hans de Graaff 
-# @SUPPORTED_EAPIS: 4 5 6
+# @SUPPORTED_EAPIS: 4 5 6 7
 # @BLURB: An eclass for installing Ruby packages to behave like RubyGems.
 # @DESCRIPTION:
 # This eclass allows to install arbitrary Ruby libraries (including Gems),
@@ -109,7 +109,7 @@ RUBY_FAKEGEM_BINDIR="${RUBY_FAKEGEM_BINDIR-bin}"
 case "${EAPI:-0}" in
0|1|2|3)
die "Unsupported EAPI=${EAPI} (too old) for 
ruby-fakegem.eclass" ;;
-   4|5|6)
+   4|5|6|7)
;;
*)
die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
@@ -173,6 +173,13 @@ 
SRC_URI="mirror://rubygems/${RUBY_FAKEGEM_NAME}-${RUBY_FAKEGEM_VERSION}${RUBY_FA
 
 ruby_add_bdepend virtual/rubygems
 ruby_add_rdepend virtual/rubygems
+case ${EAPI} in
+   4|5|6)
+   ;;
+   *)
+   ruby_add_depend virtual/rubygems
+   ;;
+esac
 
 # @FUNCTION: ruby_fakegem_gemsdir
 # @RETURN: Returns the gem data directory
-- 
2.21.0




[gentoo-dev] [PATCH 2/3] ruby-fakegem.eclass: change default DOC recipe to use rdoc

2019-07-25 Thread Hans de Graaff
The previous default was "rake" but this turned out to be a poor
choice because many packages do not implement "rake doc" and even if
they do there are usually many local development environment
assumption attached to that task. Using a consistent "rdoc" call that
is handled by the eclass gets more consistent results at the code of
missing out on specific rdoc options set by packages.
---
 eclass/ruby-fakegem.eclass | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/eclass/ruby-fakegem.eclass b/eclass/ruby-fakegem.eclass
index ab7b36eb7ae7..33a9e453f564 100644
--- a/eclass/ruby-fakegem.eclass
+++ b/eclass/ruby-fakegem.eclass
@@ -57,7 +57,14 @@ RUBY_FAKEGEM_TASK_TEST="${RUBY_FAKEGEM_TASK_TEST-test}"
 #  - rdoc (calls `rdoc-2`, adds dev-ruby/rdoc to the dependencies);
 #  - yard (calls `yard`, adds dev-ruby/yard to the dependencies);
 #  - none
-RUBY_FAKEGEM_RECIPE_DOC="${RUBY_FAKEGEM_RECIPE_DOC-rake}"
+case ${EAPI} in
+   4|5|6)
+   RUBY_FAKEGEM_RECIPE_DOC="${RUBY_FAKEGEM_RECIPE_DOC-rake}"
+   ;;
+   *)
+   RUBY_FAKEGEM_RECIPE_DOC="${RUBY_FAKEGEM_RECIPE_DOC-rdoc}"
+   ;;
+esac
 
 # @ECLASS-VARIABLE: RUBY_FAKEGEM_DOCDIR
 # @DEFAULT_UNSET
-- 
2.21.0




[gentoo-dev] [PATCH 3/3] ruby-fakegem.eclass: warn about using the fallback gemspec

2019-07-25 Thread Hans de Graaff
The fallback gemspec does not contain dependencies so it will only
work for packages without any runtime gem dependencies. It is easy to
use it by mistake when switching from a gem to a source-based archive,
because the source-based archive does not contain the generated
metadata, but RUBY_FAKEGEM_GEMSPEC has not been set yet. This warning
alerts developers to this situation and encourages them to set
RUBY_FAKEGEM_GEMSPEC instead.

Signed-off-by: Hans de Graaff 
---
 eclass/ruby-fakegem.eclass | 9 +
 1 file changed, 9 insertions(+)

diff --git a/eclass/ruby-fakegem.eclass b/eclass/ruby-fakegem.eclass
index 33a9e453f564..a6a7654f9e6d 100644
--- a/eclass/ruby-fakegem.eclass
+++ b/eclass/ruby-fakegem.eclass
@@ -295,6 +295,15 @@ ruby_fakegem_metadata_gemspec() {
 # See RUBY_FAKEGEM_NAME and RUBY_FAKEGEM_VERSION for setting name and version.
 # See RUBY_FAKEGEM_REQUIRE_PATHS for setting extra require paths.
 ruby_fakegem_genspec() {
+   case ${EAPI} in
+   4|5|6) ;;
+   *)
+   eqawarn "Generating generic fallback gemspec *without* 
dependencies"
+   eqawarn "This will only work when there are no runtime 
dependencies"
+   eqawarn "Set RUBY_FAKEGEM_GEMSPEC to generate a proper 
specifications file"
+   ;;
+   esac
+
local required_paths="'lib'"
for path in ${RUBY_FAKEGEM_REQUIRE_PATHS}; do
required_paths="${required_paths}, '${path}'"
-- 
2.21.0




[gentoo-dev] [PATCH 4/5] vcs-snapshot.eclass: Detect and report invalid directory structure

2019-07-25 Thread Michał Górny
Detect when the archive does not contain a single top-level directory,
and abort in that case.  Otherwise, --strip-components would result
in unpredictable mess.

Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 16 ++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 312e9a4611e1..dbca6fd586d2 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -68,8 +68,20 @@ vcs-snapshot_src_unpack() {
 
debug-print "${FUNCNAME}: unpacking ${f} to 
${destdir}"
 
-   # XXX: check whether the directory structure 
inside is
-   # fine? i.e. if the tarball has actually a 
parent dir.
+   local topdirs=()
+   readarray -t topdirs \
+   < <(tar -t -f "${DISTDIR}/${f}" | cut 
-d/ -f1 | sort -u)
+   if [[ ${#topdirs[@]} -gt 1 ]]; then
+   eerror "The archive ${f} contains 
multiple or no top directory."
+   eerror "It is impossible for 
vcs-snapshot to unpack this correctly."
+   eerror "Top directories found:"
+   local d
+   for d in "${topdirs[@]}"; do
+   eerror "${d}"
+   done
+   die "${FUNCNAME}: Invalid directory 
structure in archive ${f}"
+   fi
+
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
einfo "Unpacking ${f}"
-- 
2.22.0




[gentoo-dev] [PATCH 5/5] vcs-snapshot.eclass: Detect unnecessary usage and complain

2019-07-25 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 12 
 1 file changed, 12 insertions(+)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index dbca6fd586d2..c01cf01f052c 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -58,6 +58,7 @@ EXPORT_FUNCTIONS src_unpack
 vcs-snapshot_src_unpack() {
debug-print-function ${FUNCNAME} "${@}"
 
+   local renamed_any=
local f
 
for f in ${A}
@@ -81,6 +82,7 @@ vcs-snapshot_src_unpack() {
done
die "${FUNCNAME}: Invalid directory 
structure in archive ${f}"
fi
+   [[ ${topdirs[0]} != ${f%.tar*} ]] && 
renamed_any=1
 
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
@@ -96,4 +98,14 @@ vcs-snapshot_src_unpack() {
;;
esac
done
+
+   if [[ ! ${renamed_any} ]]; then
+   local w=eerror
+   [[ ${EAPI} == [0123456] ]] && w=eqawarn
+   "${w}" "${FUNCNAME} did not find any archives that needed 
renaming."
+   "${w}" "Please verify that its usage is really necessary, and 
remove"
+   "${w}" "the inherit if it is not."
+
+   [[ ${w} == eerror ]] && die "${FUNCNAME}: Unnecessary usage 
detected"
+   fi
 }
-- 
2.22.0




[gentoo-dev] [PATCH 3/5] vcs-snapshot.eclass: Add verbose einfo for unpacking

2019-07-25 Thread Michał Górny
Add an einfo call to make explicit notice of each archive being
unpacked.

Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 2e734c509d1a..312e9a4611e1 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -72,6 +72,7 @@ vcs-snapshot_src_unpack() {
# fine? i.e. if the tarball has actually a 
parent dir.
mkdir "${destdir}" || die
# -o (--no-same-owner) to avoid restoring 
original owner
+   einfo "Unpacking ${f}"
tar -C "${destdir}" -x -o --strip-components 1 \
-f "${DISTDIR}/${f}" || die
;;
-- 
2.22.0




[gentoo-dev] [PATCH 1/5] vcs-snapshot.eclass: Allow EAPI 7

2019-07-25 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 316a37773d43..321e307b894d 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -1,10 +1,10 @@
-# Copyright 1999-2016 Gentoo Foundation
+# Copyright 1999-2019 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2
 
 # @ECLASS: vcs-snapshot.eclass
 # @MAINTAINER:
 # mgo...@gentoo.org
-# @SUPPORTED_EAPIS: 0 1 2 3 4 5 6
+# @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
 # This eclass provides a convenience src_unpack() which does unpack all
@@ -40,7 +40,7 @@
 # in ${WORKDIR}/${P} and ${WORKDIR}/${P}-otherstuff respectively.
 
 case ${EAPI:-0} in
-   0|1|2|3|4|5|6) ;;
+   0|1|2|3|4|5|6|7) ;;
*) die "vcs-snapshot.eclass API in EAPI ${EAPI} not yet established."
 esac
 
-- 
2.22.0




[gentoo-dev] [PATCH 2/5] vcs-snapshot.eclass: Update 'unusage' warning

2019-07-25 Thread Michał Górny
Nowadays both GitHub and GitLab snapshot do not need this eclass,
so make it clear it's needed only for BitBucket.  Also make the warning
uppercase because people still don't read it.

Signed-off-by: Michał Górny 
---
 eclass/vcs-snapshot.eclass | 33 ++---
 1 file changed, 18 insertions(+), 15 deletions(-)

diff --git a/eclass/vcs-snapshot.eclass b/eclass/vcs-snapshot.eclass
index 321e307b894d..2e734c509d1a 100644
--- a/eclass/vcs-snapshot.eclass
+++ b/eclass/vcs-snapshot.eclass
@@ -7,33 +7,36 @@
 # @SUPPORTED_EAPIS: 0 1 2 3 4 5 6 7
 # @BLURB: support eclass for unpacking VCS snapshot tarballs
 # @DESCRIPTION:
+# THIS ECLASS IS NOT NECESSARY FOR MODERN GITHUB AND GITLAB SNAPSHOTS.
+# THEIR DIRECTORY STRUCTURE IS ENTIRELY PREDICTABLE, SO UPDATE YOUR
+# EBUILD TO USE /ARCHIVE/ URI AND SET S IF NECESSARY.
+#
 # This eclass provides a convenience src_unpack() which does unpack all
 # the tarballs in SRC_URI to locations matching their (local) names,
 # discarding the original parent directory.
 #
-# The typical use case are VCS snapshots, coming from bitbucket
-# and similar services. They have hash appended to the directory name
-# which makes extracting them a painful experience. But if you just use
-# a SRC_URI arrow to rename it (which you're likely have to do anyway),
-# vcs-snapshot will just extract it into a matching directory.
+# The typical use case are VCS tag snapshots coming from BitBucket
+# (but not GitHub or GitLab).  They have hash appended to the directory
+# name which makes extracting them a painful experience.  But if you are
+# using a SRC_URI arrow to rename them (which quite likely you have to
+# do anyway), vcs-snapshot will just extract them into matching
+# directories.
 #
 # Please note that this eclass handles only tarballs (.tar, .tar.gz,
-# .tar.bz2 & .tar.xz). For any other file format (or suffix) it will
-# fall back to regular unpack. Support for additional formats may be
-# added at some point so please keep your SRC_URIs clean.
-#
-# Note: this eclass is no longer needed with the new-style 'archive'
-# GitHub URLs. They have sane directory names and stable contents,
-# so you should really prefer them.
+# .tar.bz2 & .tar.xz).  For any other file format (or suffix) it will
+# fall back to regular unpack.  Support for additional formats may be
+# added in the future if necessary.
 #
 # @EXAMPLE:
 #
 # @CODE
-# EAPI=6
+# EAPI=7
 # inherit vcs-snapshot
 #
-# SRC_URI="https://github.com/example/${PN}/tarball/v${PV} -> ${P}.tar.gz
-#  https://github.com/example/${PN}-otherstuff/tarball/v${PV} -> 
${P}-otherstuff.tar.gz"
+# SRC_URI="
+#https://bitbucket.org/foo/bar/get/${PV}.tar.bz2 -> ${P}.tar.bz2
+#https://bitbucket.org/foo/bar-otherstuff/get/${PV}.tar.bz2
+#-> ${P}-otherstuff.tar.bz2"
 # @CODE
 #
 # and however the tarballs were originally packed, all files will appear
-- 
2.22.0