Re: [gentoo-dev] Questions on overlays, repositories and PMS

[Brian Dolbec]

On Fri, 23 Feb 2018 18:36:07 +0100
Michael Lienhardt  wrote:

> I started refactoring my solver to make it more modular, to fix some
> details w.r.t. the PMS and to manage different repositories. I thus
> have several questions on how multiple repositories work in portage.
> 
> 1. My understanding was that /etc/portage/repos.conf replaced the
> PORTDIR_OVERLAY variable, however this variable is still documented
> (e.g. in
> https://devmanual.gentoo.org/general-concepts/overlay/index.html).
> Was my intuition right? Or in other word, it is enough to only look
> at /etc/portage/repos.conf? In general, an overlay is a repository,
> i.e., a valid tree layout for the PMS, right (as stated in
> https://devmanual.gentoo.org/general-concepts/overlay/index.html)?
> 

Yes, you should not have to look at the old make.conf variables, yes,
they do work, but mostly for backward compatibility with some old third
party tools.  Those really need to be updated.  I forget now how many
there are, but if I remember correctly only 2 or 3 are left.  And yes,
we have been slow to get them updated or replaced.


> 2. the PMS states that any valid repository has a profiles folder
> which can contain profiles and a package.mask file.
>   - can the profiles in a repository different from DEFAULT be
> selected?
>   - is the package.mask file apply only on the packages of that
> repository, or on every packages of every repositories listed
> in /etc/portage/repos.conf?
> 
> 3. many repositories do not have an eclass folder, and miss many
> (optional) configuration files in the profiles folder (like
> arch.list, categories):
>   - is such information implicitly inherited from the DEFAULT
> repository (even though
> https://wiki.gentoo.org/wiki//etc/portage/repos.conf states that it
> is not)? the brother overlay
> (https://github.com/stefan-langenmaier/brother-overlay) does not
> specify any masters
>   - when the eclass folder, profiles/arch.list and such are present,
> is the data from the DEFAULT repository still implicitly inherited?
>   - when the eclass folder, profiles/arch.list and such are present,
> are they visible globally (i.e., a package from another repository
> can use a keyword of the arch.list and inherit from one of the
> eclass)?
> 
> 4. is the "masters" attribute in /etc/portage/repos.conf make the
> repository inherit other data than the eclasses?
> 
> 5. since every repos can have a profiles/categories file, is the
> file /etc/portage/categories obsolete (or should it be)?
> 
> 
> Best Regards,
> Michael Lienhardt
> 
> 



-- 
Brian Dolbec 

[gentoo-dev] details on the PMS

[Michael Lienhardt]

I noticed that the following files are not mentioned, or stated optional in the 
PMS:
 - profiles/repo_name
 - metadata/layout.conf

It seems to me that without these files, portage complains. Shouldn't them be 
declared as mandatory?

 - profiles/package.use and profiles/make.defaults 
(https://wiki.gentoo.org/wiki/USE_ORDER)
These files are not documented in the PMS: should the package manager recognize 
these files? (if so, I think it would be clearer to document them in the PMS)

Also, in https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-430004.7.1 the 
directory is named metadata/cache while it is named metadata/md5-cache in my 
repository.


Should I report something on the Gentoo Bugzilla?

Best,
Michael Lienhardt

Re: [gentoo-dev] Questions on overlays, repositories and PMS

[Michael Lienhardt]

Il 23/02/2018 20:37, Alec Warner ha scritto:

My general observation is that Gentoo is not successful as an organization 
about deprecating and removing things. One area where Gentoo has done well is 
in EAPI and in PMS itself, with mostly-clear versioning and standards and 
whatnot. But in general if something worked 15 years ago, it probably still 
works today (doubly so for sys-apps/portage).

There is a different question when building a tool like yours if it is worth the effort to support things that are 15 years old and are possibly not used (particularly in cases where functionality was replaced). I'd recommend starting with the basic implementation and adding support for the 'older' formats when users ask for them; but this is mostly a trade-off in efforts. If your goal is to build 
a "100% compatible" tool then you will probably need to support these edge cases.


You have a very good point.
I'd like to be complete (it's a side effect of working in formal methods), but 
it's quite unrealistic as I am the only developer in this project, and it's 
true that there are few technical design choices that were made in portage that 
I'd be happier not to implement.
I'd like to implement the /etc/portage/repos.conf system to remove as many hard 
coded references to /usr/portage in my code as possible.
Moreover, the /etc/portage/repos.conf system looks nice, modular with explicit 
dependencies and it almost unifies all the repositories (I don't really 
understand the need of a DEFAULT section).

If possible, I'd rather avoid implementing things that are deprecated, but like 
you pointed out, few are (portage seems to be always expanding with 
new/alternative functionalities).
The ones that are, like the /etc/portage/package.keywords file, seem to be 
still used (I've got a request to support it in my get_installation.sh script).
Additionally, there are two systems that I did not want to implement but had 
to: the IUSE_IMPLICIT and USE_EXPAND.
I didn't find any good documentation on these systems (nor the PMS nor 
https://dev.gentoo.org/%7Ezmedico/portage/doc/man/portage.5.html are very clear 
on the subject -- the PMS is still clearer), I tested a lot and looked at the 
portage implementation...
I don't understand the reason to implement these systems with bash variables 
expanded with prefixes, while many of the USE flag manipulation is done with 
dedicated files (use.*, package.use.*).
It really felt like an old design choice kept there because it worked, but 
which could be simplified.

On a similar topic, does anyone still have USE-related variables in his 
/etc/env.d folder? (https://wiki.gentoo.org/wiki/USE_ORDER)
It seems to me that portage's current effort is to have all configuration files 
in /etc/portage or in the profile.

Best,
Michael Lienhardt

Re: [gentoo-dev] Questions on overlays, repositories and PMS

[Michael Lienhardt]

Thank you for your answers :).
I have few followup questions.

Il 23/02/2018 19:09, Vadim A. Misbakh-Soloviov ha scritto:

Or in other word, it is enough to only look at /etc/portage/repos.conf?

No


What else should I look for?


- is the package.mask file apply only on the packages of that repository, or on 
every packages of
every repositories listed in /etc/portage/repos.conf?

Actually, I can't remember the correct answer right now, but definitelly it 
have the effect on repos, that states this repo as master.


If you get the correct answer, I'm really interested.

Also, what does mean "that states this repo as master"?
 - every repo that have a profiles/package.mask file is implicitly the master 
of every other repository?
 - there can be only one repo with a profiles/package.mask file?
 - other possible meaning?


is such information implicitly inherited from the DEFAULT repository (even
though https://wiki.gentoo.org/wiki//etc/portage/repos.conf states that it
is not)?

Usually, that info is inerited from `master` repo of the current repo (that is 
stated in the layout conf file)


You're right, I forgot about the metadata/layout.conf

This makes me think: a repository can have four different names:
 - the name of the file in /etc/portage/repos.conf/
 - the name specified in the section of the /etc/portage/repos.conf/
 - the name in the metadata/layout.conf
 - the name in the profiles/repo_name

All these names should be equal, but what happens if they aren't?
In particular, which of the four names is used to resolve the masters 
dependency?


4. is the "masters" attribute in /etc/portage/repos.conf make the repository
inherit other data than the eclasses?

Yes, but that attribute is usually not recommended for general use.


I guess the masters attribute in /etc/portage/repos.conf overrides the one in 
the metadata/layout.conf, and that's why it is not recommended, right?
Which other data is inherited?

Also, what happen when some inherited data is also declared locally?
 - for instance when a master and the local repository both share the same 
eclass?
 - or if the profiles/categories is inherited but also declared locally: does 
the local declaration override the one from the masters, or all the category 
lists are merged together?


5. since every repos can have a profiles/categories file, is the file
/etc/portage/categories obsolete (or should it be)?

Why?


Because this file is useless if every repository declare its own categories 
(that's what I understand from the documentation 
https://wiki.gentoo.org/wiki//etc/portage/categories).
So I guess it is not deprecated.

My experience is that it is far more maintainable to have the repositories not 
depending on unreliable external data (the /etc/portage/categories file) to 
work.
Hence, the only case I can see where it is not necessary for a repository to 
have its own local profiles/categories file (or having it inherited from some 
master) is when the repository is user-defined and not distributed to anyone.
But in this case too, it makes more sense to me to have repository-specific 
data declared locally in the repository itself.


Best,
Michael Lienhardt

Re: [gentoo-dev] Questions on overlays, repositories and PMS

[Alec Warner]

On Fri, Feb 23, 2018 at 12:36 PM, Michael Lienhardt <
michael.lienha...@laposte.net> wrote:

> I started refactoring my solver to make it more modular, to fix some
> details w.r.t. the PMS and to manage different repositories.
> I thus have several questions on how multiple repositories work in portage.
>
> 1. My understanding was that /etc/portage/repos.conf replaced the
> PORTDIR_OVERLAY variable, however this variable is still documented (e.g.
> in https://devmanual.gentoo.org/general-concepts/overlay/index.html).
> Was my intuition right?
> Or in other word, it is enough to only look at /etc/portage/repos.conf?
> In general, an overlay is a repository, i.e., a valid tree layout for the
> PMS, right (as stated in https://devmanual.gentoo.org/g
> eneral-concepts/overlay/index.html)?
>
> 2. the PMS states that any valid repository has a profiles folder which
> can contain profiles and a package.mask file.
>  - can the profiles in a repository different from DEFAULT be selected?
>  - is the package.mask file apply only on the packages of that repository,
> or on every packages of every repositories listed in
> /etc/portage/repos.conf?
>
> 3. many repositories do not have an eclass folder, and miss many
> (optional) configuration files in the profiles folder (like arch.list,
> categories):
>  - is such information implicitly inherited from the DEFAULT repository
> (even though https://wiki.gentoo.org/wiki//etc/portage/repos.conf states
> that it is not)?
>  the brother overlay (https://github.com/stefan-lan
> genmaier/brother-overlay) does not specify any masters
>  - when the eclass folder, profiles/arch.list and such are present, is the
> data from the DEFAULT repository still implicitly inherited?
>  - when the eclass folder, profiles/arch.list and such are present, are
> they visible globally (i.e., a package from another repository can use a
> keyword of the arch.list and inherit from one of the eclass)?
>
> 4. is the "masters" attribute in /etc/portage/repos.conf make the
> repository inherit other data than the eclasses?
>
> 5. since every repos can have a profiles/categories file, is the file
> /etc/portage/categories obsolete (or should it be)?
>
>
My general observation is that Gentoo is not successful as an organization
about deprecating and removing things. One area where Gentoo has done well
is in EAPI and in PMS itself, with mostly-clear versioning and standards
and whatnot. But in general if something worked 15 years ago, it probably
still works today (doubly so for sys-apps/portage).

There is a different question when building a tool like yours if it is
worth the effort to support things that are 15 years old and are possibly
not used (particularly in cases where functionality was replaced). I'd
recommend starting with the basic implementation and adding support for the
'older' formats when users ask for them; but this is mostly a trade-off in
efforts. If your goal is to build a "100% compatible" tool then you will
probably need to support these edge cases.

-A


>
> Best Regards,
> Michael Lienhardt
>
>
>

Re: [gentoo-dev] Questions on overlays, repositories and PMS

[Vadim A. Misbakh-Soloviov]

> Or in other word, it is enough to only look at /etc/portage/repos.conf?
No

> In general, an overlay is a repository, i.e., a valid tree layout for the
Yes
 
> - can the profiles in a repository different from DEFAULT be selected?
Yes

> - is the package.mask file apply only on the packages of that repository, or 
> on every packages of
> every repositories listed in /etc/portage/repos.conf?
Actually, I can't remember the correct answer right now, but definitelly it 
have the effect on repos, that states this repo as master.

> is such information implicitly inherited from the DEFAULT repository (even
> though https://wiki.gentoo.org/wiki//etc/portage/repos.conf states that it
> is not)?
Usually, that info is inerited from `master` repo of the current repo (that is 
stated in the layout conf file)

> the brother overlay (https://github.com/stefan-langenmaier/brother-overlay) 
> does not specify
> any masters
Eeeerm?
https://github.com/stefan-langenmaier/brother-overlay/blob/master/metadata/layout.conf#L1

> - when the eclass folder, profiles/arch.list and such are
> present, is the data from the DEFAULT repository still implicitly
> inherited?
I still insist on inheritance from master repo.

> - when the eclass folder, profiles/arch.list and such are
> present, are they visible globally (i.e., a package from another repository
> can use a keyword of the arch.list and inherit from one of the eclass)?
AFAIRC, depends on the repos relative priority.

> 4. is the "masters" attribute in /etc/portage/repos.conf make the repository
> inherit other data than the eclasses?
Yes, but that attribut is usually not recommended for general use.

> 5. since every repos can have a profiles/categories file, is the file
> /etc/portage/categories obsolete (or should it be)?
Why?

[gentoo-dev] Questions on overlays, repositories and PMS

[Michael Lienhardt]

I started refactoring my solver to make it more modular, to fix some details 
w.r.t. the PMS and to manage different repositories.
I thus have several questions on how multiple repositories work in portage.

1. My understanding was that /etc/portage/repos.conf replaced the 
PORTDIR_OVERLAY variable, however this variable is still documented (e.g. in 
https://devmanual.gentoo.org/general-concepts/overlay/index.html).
Was my intuition right?
Or in other word, it is enough to only look at /etc/portage/repos.conf?
In general, an overlay is a repository, i.e., a valid tree layout for the PMS, 
right (as stated in 
https://devmanual.gentoo.org/general-concepts/overlay/index.html)?

2. the PMS states that any valid repository has a profiles folder which can 
contain profiles and a package.mask file.
 - can the profiles in a repository different from DEFAULT be selected?
 - is the package.mask file apply only on the packages of that repository, or 
on every packages of every repositories listed in /etc/portage/repos.conf?

3. many repositories do not have an eclass folder, and miss many (optional) 
configuration files in the profiles folder (like arch.list, categories):
 - is such information implicitly inherited from the DEFAULT repository (even 
though https://wiki.gentoo.org/wiki//etc/portage/repos.conf states that it is 
not)?
 the brother overlay 
(https://github.com/stefan-langenmaier/brother-overlay) does not specify any 
masters
 - when the eclass folder, profiles/arch.list and such are present, is the data 
from the DEFAULT repository still implicitly inherited?
 - when the eclass folder, profiles/arch.list and such are present, are they 
visible globally (i.e., a package from another repository can use a keyword of 
the arch.list and inherit from one of the eclass)?

4. is the "masters" attribute in /etc/portage/repos.conf make the repository 
inherit other data than the eclasses?

5. since every repos can have a profiles/categories file, is the file 
/etc/portage/categories obsolete (or should it be)?


Best Regards,
Michael Lienhardt

Re: [gentoo-dev] Its time to mask sys-libs/uclibc

[Anthony G. Basile]

On 2/23/18 11:22 AM, Matthias Maier wrote:
> 
> On Fri, Feb 23, 2018, at 10:01 CST, "Anthony G. Basile"  
> wrote:
> 
>> [...]
> 
>> I'm not even sure a news item is needed here.  What do people think?  If
>> you think so, who do I even direct it at?
> 
> If there is no action needed on user side and an upgrade and migration
> from uclibc to uclibc-ng happens automatically, I'd say no news item is
> necessary.
> 
> Further, seeing "uclibc-ng" being emerged and "uclibc" unmerged should
> be pretty self explanatory.
> 
> Best,
> Matthias
> 

I already sent a news item for migrating uclibc -> uclibc-ng some years
ago.  After 6 years, anyone still on uclibc has a seriously broken
system.  I doubt migration is even possible at this point.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Its time to mask sys-libs/uclibc

[Matthias Maier]

On Fri, Feb 23, 2018, at 10:01 CST, "Anthony G. Basile"  
wrote:

> [...]

> I'm not even sure a news item is needed here.  What do people think?  If
> you think so, who do I even direct it at?

If there is no action needed on user side and an upgrade and migration
from uclibc to uclibc-ng happens automatically, I'd say no news item is
necessary.

Further, seeing "uclibc-ng" being emerged and "uclibc" unmerged should
be pretty self explanatory.

Best,
Matthias

[gentoo-dev] Its time to mask sys-libs/uclibc

[Anthony G. Basile]

Hi everyone,

So if anyone has been following uclibc, you know that its development
stalled with its last official release in 2012 (https://uclibc.org/).
It was forked into uclibc-ng (https://uclibc-ng.org/) which has been
actively maintained since.  All my uclibc work has been done using
sys-libs/uclibc-ng although the profile names still retain /uclibc/ in them.

Its time to mask and remove sys-libs/uclibc in favor of
sys-libs/uclibc-ng.  This email is just an alert to the community that
I'm going to do that soon.

I'm not even sure a news item is needed here.  What do people think?  If
you think so, who do I even direct it at?

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

[gentoo-dev] Last rites: sys-cluster/ganglia-web

[Thomas Deutschmann]

# Thomas Deutschmann  (23 Feb 2018)
# Unmaintained in Gentoo, known critical vulnerabilities like
# auth bypass. Removal in 30 days. Bugs #559658, #592080.
sys-cluster/ganglia-web


If you are using sys-cluster/ganglia-web or want to keep the package in
official Gentoo repository, it is now _your_ time to offer your help or
this package will be removed in 30 days.

Non-Gentoo developer who wants to help can contribute through the Gentoo
Proxy Maintainers project. See
https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers#Getting_Started
for more information.


-- 
Regards,
Thomas Deutschmann / Gentoo Security Team
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5



signature.asc
Description: OpenPGP digital signature