Re: [gentoo-dev] [RFC] multiversion ebuilds

[Georgy Yakovlev]

On Saturday, May 12, 2018 5:20:30 AM PDT Gerion Entrup wrote:
> Hi,
> 
> just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the ebuild
> itself: ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```
You may also be interested checking out exherbo's approach:

exheres (ebuild) holds some metadata and version and sometimes some code 
overrides.
exlib (similar to eclass, but not quite so) holds the most of the code an 
logic. can be per-package and per-category.


examples:
https://git.exherbo.org/arbor.git/tree/packages/net-wireless/wpa_supplicant
https://git.exherbo.org/arbor.git/tree/packages/sys-libs/glibc

--
Regards,
Georgy.

signature.asc
Description: This is a digitally signed message part.

[gentoo-dev] Re: Monthly x11@ project status for May 2018

[Mike Gilbert]

On Mon, May 7, 2018 at 4:49 PM, Matt Turner  wrote:
> On Mon, May 7, 2018 at 1:38 PM, Matt Turner  wrote:
>> == Convert media-libs/mesa ebuild to build with Meson ==
>> https://bugs.gentoo.org/652762
>>
>> Ebuild posted. Blocked on problems surrounding LLVM (this is the story of
>> maintaining media-libs/mesa).
>>
>> The problem is that Meson has no way of specifying a different llvm-config
>> binary without a cross file, which our meson.eclass does not generate for
>> 32-bit x86 builds on amd64. We've filed a Meson issue here [4] but haven't
>> seen
>> any progress towards a solution.
>
> Just wanted to highlight this section for the Meson maintainers specifically.
>
> Why do we not generate cross files for 32-bit builds on 64-bit? As far
> as I understand that would be sufficient for us. Do you have any
> suggestions?

Using a cross file triggers cross-compile mode in meson, which makes
it behave differently in some ways. It is hard to say exactly what the
differences are; this aspect of meson if very poorly documented.

As an experiment, I implemented some changes in meson.eclass to have
it always use a cross file.

https://github.com/gentoo/gentoo/pull/8379

This seems to work ok for building systemd with multilib enabled.

Once difference I spotted is that the CFLAGS/LDFLAGS given in the
cross file get appended at the end of the compile/link commands,
instead of in the middle. That could be problematic since link
arguments like --as-needed are sensitive to their position on the
command line.

Anyway, it would be great if meson would just look at the LLVM_CONFIG
environment var instead of forcing us to use a cross file here.

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Dennis Schridde]

On Saturday, 12 May 2018 16:36:13 CEST Gerion Entrup wrote:
> Am Samstag, 12. Mai 2018, 16:21:26 CEST schrieb Ulrich Mueller:
> > > On Sat, 12 May 2018, Gerion Entrup wrote:
> > > - The size of the tree reduces.
> > 
> > I very much doubt that (or at least it remains to be proven).
> > 
> > Currently, when an ebuild is copied for a version bump, it will reuse
> > the same blob in the Git repository. With the scheme above, you would
> > have to modify the ebuild, which would add a new blob for every
> > version bump.
> 
> You are right, I've not thought about that. However, this is only true for
> the repository not for the rsync copy most(?) users have and not for a
> checkout.

As I understand it, rsync will indeed re-use blocks from existing files on the 
receiver [1], and hence copy files with the same content only once.

--Dennis

[1]: http://www.anchor.com.au/blog/2013/08/out-tridging-tridge/

signature.asc
Description: This is a digitally signed message part.

[gentoo-dev] Last rites: net-dns/dnssec-lookup, net-dns/dnssec-nodes, net-dns/dnssec-system-tray

[Andreas Sturmlechner]

# Andreas Sturmlechner  (12 May 2018)
# Depends on dead Qt4, upstream TRAC and SVN dead. No maint. response.
# Masked for removal in 30 days, bugs #645418, #591866, #574244.
net-dns/dnssec-lookup
net-dns/dnssec-nodes
net-dns/dnssec-system-tray

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

Am Samstag, 12. Mai 2018, 16:24:00 CEST schrieb R0b0t1:
> On Sat, May 12, 2018 at 7:20 AM, Gerion Entrup  wrote:
> > - The size of the tree reduces.
> >
> 
> If this is a big concern you may be able to mount the portage tree
> under a compressed loopback filesystem. It may even be worth
> considering that as a recommended-by-handbook default as the benefit
> for compressing the ebuilds is likely huge anyway.
This seems reasonable. However, first duplicating code and then deduplicate it 
at file system level seems weird.

Gerion


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

Am Samstag, 12. Mai 2018, 16:21:26 CEST schrieb Ulrich Mueller:
> > On Sat, 12 May 2018, Gerion Entrup wrote:
> 
> > just an idea for now. But what you think about multiversion ebuilds?
> > Technically this could be realized with the following line in the
> > ebuild itself:
> > ```
> > VERSIONS=( 3.0.11 3.0.12 3.1 )
> > ```
> 
> > [...]
> 
> > The advantages of this idea I see are:
> > - Ebuilds are written in a multiversion manner anyway, and then get
> > copied (or linked?), so it can be made explicit.
> > - The diffs between different versions of ebuilds and the commit
> > history are way more readable.
> 
> That depends on the options you specify for git diff (e.g.,
> --find-renames, --find-copies, or --find-copies-harder).
This does not apply to the online diffs (see e.g. [1]). I assume that most 
users don't have
the git repository checked out.

> 
> > - The size of the tree reduces.
> 
> I very much doubt that (or at least it remains to be proven).
> 
> Currently, when an ebuild is copied for a version bump, it will reuse
> the same blob in the Git repository. With the scheme above, you would
> have to modify the ebuild, which would add a new blob for every
> version bump.
You are right, I've not thought about that. However, this is only true for
the repository not for the rsync copy most(?) users have and not for a checkout.

Gerion

[1] 
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d9397ab8d48feb4b1360be614da35fa2faae44d9



signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] [RFC] multiversion ebuilds

[R0b0t1]

On Sat, May 12, 2018 at 7:20 AM, Gerion Entrup  wrote:
> - The size of the tree reduces.
>

If this is a big concern you may be able to mount the portage tree
under a compressed loopback filesystem. It may even be worth
considering that as a recommended-by-handbook default as the benefit
for compressing the ebuilds is likely huge anyway.

Cheers,
 R0b0t1

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Ulrich Mueller]

> On Sat, 12 May 2018, Gerion Entrup wrote:

> just an idea for now. But what you think about multiversion ebuilds?
> Technically this could be realized with the following line in the
> ebuild itself:
> ```
> VERSIONS=( 3.0.11 3.0.12 3.1 )
> ```

> [...]

> The advantages of this idea I see are:
> - Ebuilds are written in a multiversion manner anyway, and then get
> copied (or linked?), so it can be made explicit.
> - The diffs between different versions of ebuilds and the commit
> history are way more readable.

That depends on the options you specify for git diff (e.g.,
--find-renames, --find-copies, or --find-copies-harder).

> - The size of the tree reduces.

I very much doubt that (or at least it remains to be proven).

Currently, when an ebuild is copied for a version bump, it will reuse
the same blob in the Git repository. With the scheme above, you would
have to modify the ebuild, which would add a new blob for every
version bump.

Ulrich


pgpi3VK4FdPgZ.pgp
Description: PGP signature

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

Am Samstag, 12. Mai 2018, 15:47:57 CEST schrieb Rich Freeman:
> On Sat, May 12, 2018 at 8:20 AM Gerion Entrup 
> wrote:
> 
> 
> > Different version keywording can be done as before:
> > ```
> > if [[ ${PV} == "3.1" ]] ; then
> >  KEYWORDS="~amd64 ~x86"
> > else
> >  KEYWORDS="amd64 x86"
> > fi
> > ```
> 
>  From a readability standpoint I could see this getting out of hand if we
> aren't careful. 
I have kind of copied this code from a package, where a  ebuild exists. So 
it is already there.

> Also, we'd want to keep this simple enough that the simple
> act of stabilizing a package doesn't introduce regressions (perhaps even in
> versions that we don't intend to touch).  We'd also need rules about
> conditionals so that the metadata cache works.
Maybe this rule can be added:
4. Stable ebuilds are never multiversioned.

So once an ebuild get stable it is split out of the multiversion ebuild (aka 
copied) and not touched anymore.
Maybe this rule can be smoothed by moving the split to the point where new 
features are introduced in the ebuild:
4. If a multiversion ebuild contains a stable version and is changed 
featurewise (e.g. a new useflag), then the stable version is split out of the 
multiversion ebuild.

I don't know the internals of the metadata cache, so I can't say something to 
that.


> You also would need to be careful about ebuild revisions in general, since
> you're potentially affecting multiple versions at the same time.
You are right. But maybe it's the same fix, that is applied to all versions at 
once.

> How would revbumps fit into this?  All kinds of bad things can happen by
> editing ebuilds in place, and this would compound the issue.
Maybe it is possible to solve with the following method:
1. If the fix affects all ebuild versions, revbump all versions in the ebuild.
2. If the fix affects only one version, split out this version of the ebuild 
and make it an explicit one (together with the fix).

The thing is that it is an additional concept. It does not replace the current 
mechanisms and at best reduces overhead for ebuilds that are identical.

Gerion


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] [RFC] multiversion ebuilds

[Rich Freeman]

On Sat, May 12, 2018 at 8:20 AM Gerion Entrup 
wrote:


> Different version keywording can be done as before:
> ```
> if [[ ${PV} == "3.1" ]] ; then
>  KEYWORDS="~amd64 ~x86"
> else
>  KEYWORDS="amd64 x86"
> fi
> ```

 From a readability standpoint I could see this getting out of hand if we
aren't careful.  Also, we'd want to keep this simple enough that the simple
act of stabilizing a package doesn't introduce regressions (perhaps even in
versions that we don't intend to touch).  We'd also need rules about
conditionals so that the metadata cache works.

You also would need to be careful about ebuild revisions in general, since
you're potentially affecting multiple versions at the same time.

How would revbumps fit into this?  All kinds of bad things can happen by
editing ebuilds in place, and this would compound the issue.

(Don't get me wrong - the concept is interesting...)

-- 
Rich

[gentoo-dev] [RFC] multiversion ebuilds

[Gerion Entrup]

Hi,

just an idea for now. But what you think about multiversion ebuilds?
Technically this could be realized with the following line in the ebuild itself:
```
VERSIONS=( 3.0.11 3.0.12 3.1 )
```

and the filename without version:
//.ebuild

together with this set of rules:
1. If there is an ebuild with had a version in its name, this ebuild is 
preferred.
 e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and 
"foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
2. If the ebuild has the variable VERSIONS specified but also a version in its 
name, the version in its name is taken.
3. There can be only one multiversioned ebuild per package.

Different version keywording can be done as before:
```
if [[ ${PV} == "3.1" ]] ; then
KEYWORDS="~amd64 ~x86"
else
KEYWORDS="amd64 x86"
fi
```

The resolution of versions can be done as before, with the difference that one 
ebuild can represent multiple versions.

The "ebuild" tool needs some adjustments. Maybe it tries to download and build 
all version per default and has an additional flag to specify a single version.

The advantages of this idea I see are:
- Ebuilds are written in a multiversion manner anyway, and then get copied (or 
linked?), so it can be made explicit.
- The diffs between different versions of ebuilds and the commit history are 
way more readable.
- The size of the tree reduces.

Regards,
Gerion



signature.asc
Description: This is a digitally signed message part.