Re: [gentoo-portage-dev] [PATCH] RepoConfigLoader: allow subsitution of variables like ROOT in repos.conf

[Brian Dolbec]

On Thu, 22 Oct 2015 18:36:03 -0700
Zac Medico  wrote:

> This makes it possible to sync repositories inside $ROOT.
> 
> X-Gentoo-Bug: 563836
> X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=563836
> ---
>  pym/portage/repository/config.py | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/pym/portage/repository/config.py
> b/pym/portage/repository/config.py index a90a994..1060bc7 100644
> --- a/pym/portage/repository/config.py
> +++ b/pym/portage/repository/config.py
> @@ -620,7 +620,12 @@ class RepoConfigLoader(object):
>   treemap = {}
>   ignored_map = {}
>   ignored_location_map = {}
> - default_opts = {}
> + default_opts = {
> + "EPREFIX" : settings["EPREFIX"],
> + "EROOT" : settings["EROOT"],
> + "PORTAGE_CONFIGROOT" :
> settings["PORTAGE_CONFIGROOT"],
> + "ROOT" : settings["ROOT"],
> + }
>  
>   if "PORTAGE_REPOSITORIES" in settings:
>   portdir = ""


This looks good to me.
-- 
Brian Dolbec 

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Anthony G. Basile]

On 10/28/15 7:16 AM, hasufell wrote:

On 10/28/2015 07:23 AM, Ryan Hill wrote:

Agreed.  If there's one choice then "ssl" should be used.  openssl/libressl/etc
should really be considered sub-flags of ssl.


This is what I did with curl.  USE=ssl means one and exactly one ssl 
provider must be specified.  I suggested making it a model gentoo wide, 
but there were criticisms, I forget what, but the made sense to me at 
teh time.




I really wish we had some way of specifying this to make things clearer to the
user, so they could see exactly how these flags interact with each other.
Something like (emerge -pv):


The problem here is that we introduced REQUIRED_USE foo for these cases
which is again highly ambigous instead of making the PM aware that this
is an actual sub-USE flag.


A properly designed sub-USE flag would be useful here and clearly better 
than our REQUIRED_USE.  I think REQUIRED_USE is fine for heterogeneous 
cases, but not when you have something like curl where you can either 
turn ssl on or off.  If it is off, nothing more needs to be specified, 
if it is on, then you must further specify one and exactly one ssl provider.




This is outside of the scope of this thread, but there are already
distros that have fixed this:
1. NixOS [0] with truly declarative configuration format, e.g. something
like:
packages.ssl.provider = openssl;

or somesuch (just an example)

2. Exherbo partly [1] with providers syntax:
*/* providers: -openssl libressl

and the exheres would then do something like:
DEPENDENCIES="
 build+run:
 providers:openssl?  ( dev-libs/openssl:0 )
 providers:libressl? ( dev-libs/libressl )
"

which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
have arbitrary meanings.

But NIH will prevent us from learning here I guess.


[0] https://nixos.org/nixos/manual/
[1] http://exherbo.org/docs/eapi/providers-and-virtuals.html




--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 12:23 PM, Anthony G. Basile wrote:
> 
> A properly designed sub-USE flag would be useful here and clearly better
> than our REQUIRED_USE.  I think REQUIRED_USE is fine for heterogeneous
> cases, but not when you have something like curl where you can either
> turn ssl on or off.  If it is off, nothing more needs to be specified,
> if it is on, then you must further specify one and exactly one ssl
> provider.
> 

Uhm, curl makes use of REQUIRED_USE heavily, otherwise the use_expand
would not work:
> REQUIRED_USE="
>   curl_ssl_winssl? ( elibc_Winnt )
>   threads? ( !adns )
>   ssl? (
>   ^^ (
>   curl_ssl_axtls
>   curl_ssl_gnutls
>   curl_ssl_libressl
>   curl_ssl_openssl
>   curl_ssl_nss
>   curl_ssl_polarssl
>   curl_ssl_winssl
>   )
>   )"

With the providers syntax from exherbo, this monster would basically be
gone.

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Kristian Fiskerstrand]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 10/28/2015 12:23 PM, Anthony G. Basile wrote:
> On 10/28/15 7:16 AM, hasufell wrote:
>> On 10/28/2015 07:23 AM, Ryan Hill wrote:
>>> Agreed.  If there's one choice then "ssl" should be used. 
>>> openssl/libressl/etc should really be considered sub-flags of
>>> ssl.
> 
> This is what I did with curl.  USE=ssl means one and exactly one
> ssl provider must be specified.  I suggested making it a model
> gentoo wide, but there were criticisms, I forget what, but the made
> sense to me at teh time.

Such a setup makes sense to me as well, although likely want it to be
more generic and maybe use a prioritized list rather than a use expand
per package / group of packages. Maybe something similar to
python_compat can be used for the TLS / crypto provider


- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-BEGIN PGP SIGNATURE-

iQEcBAEBCgAGBQJWMLIvAAoJECULev7WN52Fdb8IALLlM/763gnsx3iEc7NIh7IL
HWDfBwfY2vuDKWGGVlKIXn2CZfIrDhcqOZxtprFcrK073q+Gu6ZDqVd+og+EWTJw
Erv7aq5Mi4poBPtcQT6P3CJG77RXCQPdzgPG6K8FaWyVPh+RkKFr7VsJ615visk6
UmRbwpehU51Vb/qTWl94J1z0SvIo3gusSzeK9FpkSFimrZQtsByWFtIijWYeToLH
Mv4hl7BVwpHWctTwLDObuqkl+zWEITapJ3CIYz93jrfnyWZXi/DFvOyWcK3YOES2
vDcIDWGKt0SunZgka0VTJ+CTbfCcuvxb0sXUJfJNdy6P/p7aJyzZllPEZ2ciVVk=
=iN64
-END PGP SIGNATURE-

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 09:36 AM, Alexis Ballier wrote:
> On Wed, 28 Oct 2015 03:06:59 +0100
> hasufell  wrote:
>> A is not that difficult. Most uses of 'openssl' can just be replaced
>> with 'ssl', others probably with '!gnutls?' even. A few exotic ones
>> might stay and we will have to advice users to set USE="openssl
>> libressl" instead of USE="-openssl libressl".
>> B will definitely be more work, but ofc is also a lot cleaner and
>> totally unambigous.
> 
> 
> You haven't taken into consideration the licence incompatibilities:
> http://www.gnu.org/licenses/license-list.en.html#OpenSSL
> it gets really messy for libraries: a gpl binary linking against a
>library linking against openssl means the binary can be
>redistributed, but not with such a library linked against openssl...
> 
> the point of the 'openssl' useflag is to have something that is not
> enabled by default and that can be used in RESTRICT="openssl?
> ( bindist )" expressions...
> 


You can just do RESTRICT="ssl? ( bindist )" and disable bindist by
default or vote for solution B.

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Kristian Fiskerstrand]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 10/28/2015 07:23 AM, Ryan Hill wrote:

> 
> Agreed.  If there's one choice then "ssl" should be used.
> openssl/libressl/etc should really be considered sub-flags of ssl.
> 

If we are introducing a new and proper way to define this it might
make sense to not refer to ssl (is anything actually using SSL these
days? I certainly hope not), tls is the natural replacement unless we
want to go for something more generic.

- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-BEGIN PGP SIGNATURE-

iQEcBAEBCgAGBQJWMK+QAAoJECULev7WN52FFesH/2uEi1vBt5QpR/OEusr2EFVi
IlkM9+hNNxkFO2pFriB63+iQX90k1+p6jB5X/0ARzXaaL4jfnQlq7XtvBmN6HFvI
ROBuWIpqtotM6Bm2EKDyDABXxExGcWSNPHPQS4hkyNVSfaredCoQMaweMExDPsip
Ief8T1R6orqPTv2iDN4Q7uypHUbOi0ogF7r1SAD/CQGL0a9WwFtYVzXBhj90O2F/
4OYTnzMirhDtyypPZb++H3J4U2CMm2kcoRtPomXybghTGjuWwz7We2lF61QrNeVA
IPODRSUDcDPgU8wp2knMGOmKxUQ6Ny8DH9IdQtY4BYdztDYVEV6jCRHGur1gVag=
=4KHj
-END PGP SIGNATURE-

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Rich Freeman]

On Wed, Oct 28, 2015 at 7:16 AM, hasufell  wrote:
>
> This is outside of the scope of this thread, but there are already
> distros that have fixed this:
> 1. NixOS [0] with truly declarative configuration format, e.g. something
> like:
> packages.ssl.provider = openssl;

Well, we can accomplish this in our syntax.  Just RDEPEND on openssl,
and set USE requirements for openssl on any dependencies that offer
both.

NixOS is still bound by the constraint that the two libraries have
colliding namespace, so a package needs to have a dependency chain
that exclusively uses one or the other.

However, assuming all your packages are able to work with either
library the thing NixOS does have going for it is that it would let
you have apache using openssl and postfix using libressl on the same
system, with side-by-side versions of any shared dependencies built
against each.

Their approach (as I understand it) is basically that every process is
almost containerized on the same filesystem.

>
> which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
> have arbitrary meanings.
>

Well, I think we can accomplish all of the above using the tools we
already have, but I agree that we tend to do it in one namespace while
other distros are using more than one.  That is probably a good idea
just to improve consistency.

We should probably pursue both.  For ssl we need the best solution we
can implement today.  However, for a future EAPI we should pursue a
better way to handle this.

-- 
Rich

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 07:23 AM, Ryan Hill wrote:
> 
> Agreed.  If there's one choice then "ssl" should be used.  
> openssl/libressl/etc
> should really be considered sub-flags of ssl.
> 
> I really wish we had some way of specifying this to make things clearer to the
> user, so they could see exactly how these flags interact with each other.
> Something like (emerge -pv):
> 

The problem here is that we introduced REQUIRED_USE foo for these cases
which is again highly ambigous instead of making the PM aware that this
is an actual sub-USE flag.

This is outside of the scope of this thread, but there are already
distros that have fixed this:
1. NixOS [0] with truly declarative configuration format, e.g. something
like:
packages.ssl.provider = openssl;

or somesuch (just an example)

2. Exherbo partly [1] with providers syntax:
*/* providers: -openssl libressl

and the exheres would then do something like:
DEPENDENCIES="
build+run:
providers:openssl?  ( dev-libs/openssl:0 )
providers:libressl? ( dev-libs/libressl )
"

which is a lot cleaner than USE_EXPAND + REQUIRED_USE, which still can
have arbitrary meanings.

But NIH will prevent us from learning here I guess.


[0] https://nixos.org/nixos/manual/
[1] http://exherbo.org/docs/eapi/providers-and-virtuals.html

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[hasufell]

On 10/28/2015 12:20 PM, Kristian Fiskerstrand wrote:
> On 10/28/2015 07:23 AM, Ryan Hill wrote:
> 
> 
>> Agreed.  If there's one choice then "ssl" should be used.
>> openssl/libressl/etc should really be considered sub-flags of ssl.
> 
> 
> If we are introducing a new and proper way to define this it might
> make sense to not refer to ssl (is anything actually using SSL these
> days? I certainly hope not), tls is the natural replacement unless we
> want to go for something more generic.
> 

That's nitpick for no gain and will break more configurations than any
of the previously suggested formats.

Re: [gentoo-portage-dev] [PATCH] vardbapi.aux_get: treat cache as valid if mtime is truncated (bug 564222)

[Alexander Berntsen]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Sheesh. That expression barely fits in a tweet. ;) Please just use
intermediate statements or an if or something. Otherwise LGTM.

- -- 
Alexander
berna...@gentoo.org
https://secure.plaimi.net/~alexander
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCgAGBQJWMLUHAAoJENQqWdRUGk8BdkkQANcyuwqNAp1kmj1OPBvE9Nw/
HUFZ/mKrtSgAQCMLkG9awS/8h+cq/GPru+KMHmh3uS8EIcSKPkDKcN1Zrb8dl3sv
RFa4GqwlGP/QRk8yUmwTCrdX9GQ2NYSJ048JGZ3HkpvwZiOhUPqXprrUMm64qkov
Ou1Ks2erkEptQqQTD9Ezcd+ylSSLaNx4fnAypMc92aR99fChmf8YNR28KQjI2ghw
b9CIN3r4F/D1mu3sUsw2aHsqvessiI4dgMqYbYnqn024bLIILznbHXQFpsNjYC38
aUrrfkn1aapzM8+92bpzizJmBbH9Cj4Dp21suDivOHXqWJBNdkmVAz5RXepG9/0F
OEZ8QJJelLICzunnIJ0dvepSJj0av+24UWe3WoPOx5MTuSaELs/d9QXWQUc9QM0c
494PQsTNNy9Zshro4Yv6PyfyA6zvT9xRK06/Ed7edywjAJCtcgQ5P5svfw1MJ9V4
JChGtcvasXcTi4mWgUqohrFGfwePoONlXMAY29WSicOx/LyFZiPrLFQuZmpH+ml7
twVcK2rPcO8yGtCcl1MxRiSnr6VUFGPbSFVmUFjVXJN7NoLQtJyfHofSBYrCzWFF
LvzNj4Xt3+SPehh0NOPPCyTtjUinOjuTILGRJWTxE7TsK+cI7iSqJsX4tNYR2WIi
rzqoClu1tUQbHgmLEU4Y
=Lzt8
-END PGP SIGNATURE-

Re: [gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Anthony G. Basile]

On 10/28/15 7:30 AM, hasufell wrote:

On 10/28/2015 12:23 PM, Anthony G. Basile wrote:

A properly designed sub-USE flag would be useful here and clearly better
than our REQUIRED_USE.  I think REQUIRED_USE is fine for heterogeneous
cases, but not when you have something like curl where you can either
turn ssl on or off.  If it is off, nothing more needs to be specified,
if it is on, then you must further specify one and exactly one ssl
provider.


Uhm, curl makes use of REQUIRED_USE heavily, otherwise the use_expand
would not work:

REQUIRED_USE="
curl_ssl_winssl? ( elibc_Winnt )
threads? ( !adns )
ssl? (
^^ (
curl_ssl_axtls
curl_ssl_gnutls
curl_ssl_libressl
curl_ssl_openssl
curl_ssl_nss
curl_ssl_polarssl
curl_ssl_winssl
)
)"

With the providers syntax from exherbo, this monster would basically be
gone.

Yes, we're in agreement.  I'm saying I'm *trying* to do what exherbo 
does the best I can but really dont like REQUIRED_USE for this.


--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

[gentoo-dev] Re: ssl vs openssl vs libressl vs gnutls USE flag foo

[Ryan Hill]

On Tue, 27 Oct 2015 22:46:35 -0400
Rich Freeman  wrote:

> On Tue, Oct 27, 2015 at 10:06 PM, hasufell  wrote:
> >
> > B) 1 feature flag, 3 strict provider flags
> > * ssl: enable any sort of SSL/TLS support
> > * gnutls: only to enable gnutls provided ssl support in case there
> >   is a choice
> > * openssl: only to enable openssl provided ssl support in case
> >there is a choice (should not be implemented as !gnutls?)
> > * libressl: only to enable libressl provided ssl support in case there
> > is a choice, must conflict with 'openssl' USE flag
> >
> > consequences:
> > * REQUIRED_USE="^^ ( openssl libressl )" is not only allowed, it is
> >   _mandatory_
> > * packages like media-video/ffmpeg _must_ switch the USE flag
> >   openssl->ssl to avoid breaking global USE flags
> > * !gnutls? ( dev-libs/openssl:0 ) will be bad form or even disallowed
> >
> > B will definitely be more work, but ofc is also a lot cleaner and
> > totally unambigous.
> >  
> 
> ++
> 
> The pain is for a short time.  Then we have to live with this for a
> long time.  USE flags should have one meaning.  The fact that this
> isn't the case right now is already a bug.  We don't need to
> perpetuate it.
> 
> Honestly, this just seems like "the right thing" so if there isn't
> opposition then I'd suggest to "just do it" and commit fixes to
> ebuilds that need the fix (ie if maintainer doesn't respond to bug
> quickly just take care of it).  If people object they should speak up
> now, and we can take it up at the next council meeting if necessary
> (which is right around the corner).

Agreed.  If there's one choice then "ssl" should be used.  openssl/libressl/etc
should really be considered sub-flags of ssl.

I really wish we had some way of specifying this to make things clearer to the
user, so they could see exactly how these flags interact with each other.
Something like (emerge -pv):

ssl [ (openssl) libressl gnutls ] 
 - if USE ssl then pick one of, default openssl if none chosen

ssl [[ (openssl) libressl gnutls ]]
 - if USE ssl then one or more of... etc.

But I suppose that's another topic.


-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


pgp616RCHiArF.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] ssl vs openssl vs libressl vs gnutls USE flag foo

[Alexis Ballier]

On Wed, 28 Oct 2015 03:06:59 +0100
hasufell  wrote:
> A is not that difficult. Most uses of 'openssl' can just be replaced
> with 'ssl', others probably with '!gnutls?' even. A few exotic ones
> might stay and we will have to advice users to set USE="openssl
> libressl" instead of USE="-openssl libressl".
> B will definitely be more work, but ofc is also a lot cleaner and
> totally unambigous.


You haven't taken into consideration the licence incompatibilities:
http://www.gnu.org/licenses/license-list.en.html#OpenSSL
it gets really messy for libraries: a gpl binary linking against a
   library linking against openssl means the binary can be
   redistributed, but not with such a library linked against openssl...

the point of the 'openssl' useflag is to have something that is not
enabled by default and that can be used in RESTRICT="openssl?
( bindist )" expressions...

Re: [gentoo-portage-dev] [PATCH] vardbapi.aux_get: treat cache as valid if mtime is truncated (bug 564222)

[Zac Medico]

On 10/28/2015 04:44 AM, Alexander Berntsen wrote:
> Sheesh. That expression barely fits in a tweet. ;) Please just use
> intermediate statements or an if or something. Otherwise LGTM.

I've split it with an if, and also eliminated a redundant comparison
since long(cache_mtime) == mydir_stat.st_mtime implies that
mydir_stat.st_mtime == mydir_stat[stat.ST_MTIME], so here's what I've
pushed:

if cache_mtime == mydir_stat.st_mtime:
cache_valid = True

# Handle truncated mtime in order to avoid cache
# invalidation for livecd squashfs (bug 564222).
elif long(cache_mtime) == mydir_stat.st_mtime:
cache_valid = True

-- 
Thanks,
Zac