Re: [gentoo-dev] New eclass: openvdb.eclass

2020-06-15 Thread Gerion Entrup
Friendly ping (or have I missed any discussion channel?).
As a user, I'm really looking forward to see blender >=2.80 in the tree.
According to https://bugs.gentoo.org/689740 this eclass is the main blocker.

Gerion

Am Mittwoch, 27. Mai 2020, 03:40:04 CEST schrieb Adrian Grigo:
> Hello, I am a proxied maintainer of blender and openvdb and wish to propose a 
> new eclass and USE_EXPAND variable to ensure the same version of the openvdb 
> abi is used when building the blender, openvdb and openimageio packages.
> 
> The tree currently contains openvdb 4 and 5, and I plan to add 6 and 7 soon 
> to support my new blender PR. Each major release of openvdb has a new ABI, 
> and can also support legacy versions. However only one version can be built 
> at a time, which is passed to cmake through OPENVDB_ABI_VERSION_NUMBER, and 
> produces a library with functions named xxx_7abi5 when building openvdb 7 
> with support for abi 5. Client packages like blender and openimageio which 
> depend on openvdb and also need to be compiled with the same version number 
> in order to link correctly.
> 
> To enforce this I propose to add a new USE_EXPAND variable called 
> OPENVDB_ABI, which is set to the version number the user wants to use in the 
> system eg 5, which expands to openvdb_abi_5. The packages can then append 
> -DOPENVDB_ABI_VERSION_NUMBER=${OPENVDB_ABI} to ensure they pass the same 
> value to cmake.
> 
> To reduce the boilerplate code required to ensure that only one supported 
> value is set in OPENVDB_ABI and used by all packages, so I propose to add an 
> eclass to maintain this in one place. Similar to PYTHON_COMPAT, each ebuilds 
> set OPENVDB_COMPAT to specify which legacy ABI they can support. The 
> openvdb.eclass uses it to set global variables OPENVDB_REQUIRED_USE and 
> OPENVDB_SINGLE_USEDEP and verify in pkg_prepare that only one of the 
> openvdb_abi_X flags is set.
> 
> Each ebuild sets REQUIRED_USE="${OPENVDB_REQUIRED_USE}" which evaluates to ^^ 
> ( openvdb_abi_3 openvdb_abi_4 ... ) to ensure only one abi is enabled from 
> those it supports.
> They set RDEPEND="openvdb? ( media-gfx/openvdb[${OPENVDB_SINGLE_USEDEP}] )" 
> which evaluates to openvdb_abi_3(-)?,openvdb_abi_4(-)?,... to allow portage 
> to enforce the same abi version is built between all packages.
> 
> With a new release of openvdb, I will only need to add it to the 
> openvdb_abi.desc file, to an eclass internal list _OPENVDB_ALL_ABI, and to 
> OPENVDB_COMPAT for each ebuild which supports the new version.
> 
> Please review my proposed eclass below. I have tested it works when 
> OPENVDB_ABI is missing, set to invalid values, or set to values that cannot 
> satisfy all package requirements simultaneously, and that the system 
> recompiles all packages when the ABI is changed. It also works when set to 
> appropriate values.
> 
> I considered other alternatives during development. It is possible to 
> manually add the expanded REQUIRED_USE and RDEPEND strings to each package, 
> but with each new version of openvdb and multiple client packages to maintain 
> this becomes messy and error prone. The user also needs to set the same 
> openvdb_abi_X flag for each package. While currently there are only three 
> packages which need to be synchronised, any other package which depends on 
> openvdb in future will benefit from this infrastructure.
> 
> Also without the USE_EXPAND variable the ebuilds do not have access to the 
> version number to pass to the build system, and need to generate it by 
> testing each USE flag. So I think this is the cleanest solution.
> 
> I also looked into slotting openvdb as then packages could specify the 
> slot/subslot required eg openvdb:7/5 but don't think this is possible. This 
> would be a major undertaking as it has a static as well as dynamic core 
> library, another library for python, and the cmake modules that find openvdb 
> use the output of a binary at /usr/bin/vdb_print to determine which version 
> to link against. It would also not centralise the boilerplate code.
> 
> For further information and my discussion during development see 
> https://github.com/redchillipadi/ebuild-overlay/issues/4
> 
> Please let me know if there is a better solution or improvements are required.
> 
> Kind Regards,
> Adrian Grigo
> 
> 
> # Copyright 1999-2020 Gentoo Authors
> # Distributed under the terms of the GNU General Public License v2
> 
> # @ECLASS: openvdb.eclass
> # @MAINTAINER:
> # Adrian Grigo 
> # @ AUTHOR:
> # Author: Adrian Grigo 
> # Based on work of: Michał Górny  and Krzysztof Pawlik 
> 
> # @SUPPORTED_EAPIS: 5 6 7
> # @BLURB: An eclass for OpenVDB to control which ABI version is compiled
> # @DESCRIPTION:
> # The OpenVDB package is a library for sorting and manipulating sparse
> # data structures with dynamic topology as required for use in volume
> # rendering for computer graphics.
> #
> # Each major version of OpenVDB provides an updated ABI, as well as
> # the ability to compile 

Re: [gentoo-dev] [kde overlay] Up for grabs: dev-util/arcanist

2020-05-19 Thread Gerion Entrup
Am Dienstag, 19. Mai 2020, 15:56:53 CEST schrieb Andreas Sturmlechner:
> KDE project has moved away from the unloved Phabricator instance to GitLab 
> (invent.kde.org), so there is no reason for us to keep this package in kde 
> overlay for much longer.
> 
> If for some reason you still rely on kde overlay providing this package, 
> consider picking it up now and maintain it in gentoo.git or your own place.
> 
> Regards,
> Andreas

I use this tool for my work. Is it in a fashion to push it to gentoo.git?
If yes, I can proxy maintain it.

If I get it right, there is not real installation, but only "copy all in opt 
and symlink the binary".
Is is meaningful to make a fake version like arcanist-2020.05.19?

Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] [gentoostats continued] Collected data and justification for it

2020-05-09 Thread Gerion Entrup
Am Donnerstag, 7. Mai 2020, 09:29:36 CEST schrieb Michał Górny:
> I'm going to start with the data and uses I can think of.  Please reply
> with other things you can think of.
> 
> 
> 1) list of selected packages (@world)
> 
> We would use this to determine the popularity of individual packages,
> plus by scanning their dependencies we would be able to make combined
> statistics for direct usage + dependencies of other selected packages. 
> This would allow us to judge which packages need more of our attention.
> 
> For example, as we port Python packages to Python 3.8 the packages with
> more declared users would be ported first.

You may want to collect packages installed per sets, too.
I mainly do what Hans mentioned in his mail to this thread but with sets.
For example I have a KDE-PIM set that installs only my needed subset of
the KDE PIM suite. (I also use this as common workaround for yet missing
runtime dependencies / suggestions made by pkg_postinst.)

Retrieval of this packages would be straight forward: Look at world_sets
and collect all packages that are installed by the set.

 
> 2) USE flags on installed packages (disabled/default/enabled)
> 
> This would allow us to determine which flags users are most likely to
> actually rely on.  This could determine tested flag combinations,
> defaults, and required level of support for individual flags.
> 
> For example, if OCaml bindings on some package are broken and require
> a lot of work, I would find useful to know how likely it is that anyone
> is using it.  Or if a lot of people are enabling 'frobnicate' flag,
> I could consider employing USE defaults.

I'm not sure, if Portage is capable of this, but a distinction in USE
flags needed to fulfil some dependency of another package and USE flags
actively activated by the user could be useful.

Dependency use flags should be treated with a higher priority in my
opinion, since they enable the installation of another package (tree),
while use flags that enable a certain feature that is not used elsewhere
are more "nice to have".


Best
Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] rfc: noarch keyword

2020-03-19 Thread Gerion Entrup
Am Donnerstag, 19. März 2020, 02:59:56 CET schrieb Kent Fredric:
> On Wed, 18 Mar 2020 17:52:25 +
> James Le Cuirot  wrote:
> 
> > Not quite. Tools like repoman will need to be updated to understand
> > that an ebuild with KEYWORDS="amd64" can depend on another ebuild with
> > only KEYWORDS="noarch". I do think the idea has merit though.
> 
> But the inverse is _not_ true, an ebuild with KEYWORDS="noarch"
> *cannot* depend on another ebuild with only KEYWORDS="amd64".
Maybe I misunderstand something but shouldn't that be the normal case?
Every single Python package (candidates for noarch) for example depends
on the Python interpreter, which must have non noarch keywords.


> Otherwise it breaks the entire stabilization graph.
The condition would be: If the interpreter is stable for an arch, all
it's interpreted code is also stable for this arch.


Best,
Gerion



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Packaging changes in LLVM 10

2020-03-16 Thread Gerion Entrup
Am Montag, 16. März 2020, 09:23:49 CET schrieb Michał Górny:
> Hi,
> 
> Since 10.0.0 final is around the corner, I'd like to take a minute to
> inform developers of packaging changes in Gentoo that affect its
> revdeps.  Following frequently repeated requests from Gentoo developers
> and users, LLVM 10 is stopping to use BUILD_SHARED_LIBS=ON, i.e. split
> shared libraries.  Instead, we'll be using the 'dylib' model recommended
> upstream.
> 
> This means that sys-devel/{llvm,clang} no longer install those small
> libLLVM*.so and libclang*.so files.  Instead, one big libLLVM-${ver}.so
> and libclang-cpp.so.${sover} are installed (yes, I know this historical
> inconsistency sucks).  Also note that Clang continues installing
> libclang.so which is a *C* (not C++) library.
> 
> Most of LLVM revdeps should not have problem with LLVM dylib, as it was
> the de-facto standard on other distros.  llvm-config handles it
> transparently.  If you're dealing with a custom build system that
> doesn't handle it, it's as simple as trying '-lLLVM' first (note that
> you will '-L' due to slotting).
> 
> Clang revdeps may have trouble with clang-cpp library since it's
> a fairly recent addition.  Again, the solution is simple: '-lclang-cpp'
> (repeating: do not confuse it with -lclang which is a different library
> for historical reasons).
> 
> sys-devel/lld does not install any libraries anymore.  If you ever need
> them, please ping me and we'll try to come up with something upstream.
> 
> dev-util/lldb installs liblldb.so.  However, I'm not familiar with that
> library and I don't know if it has any consumers.
> 
> If you need any help, please don't hesitate to ping me.

Hi,

when I compile LLVM for myself, I also always choose SHARED_LIBS simply
because of RAM usage when linking the libraries. In the past, I was not
able to link the single library on my 16 GB machine (I have not tested
it now and this could be dependent on other circumstances). What is the
current expected RAM usage for LLVM 10?

Best,
Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Ebuild Generators

2020-02-11 Thread Gerion Entrup
Am Dienstag, 11. Februar 2020, 16:18:44 CET schrieb Tom Gillespie:
> For historical curiosity there was also
> https://github.com/domenkozar/g-pypi at one point (similar to
> https://github.com/rafaelmartins/g-octave). Having used g-octave, the
> primary issue is as Michał says, there are a lot of corner cases that
> the generation doesn't handle correctly which lead to broken ebuilds
> and maintenance headaches. Speaking for python, catching and
> correcting the use of setup_requires and other insanity visited upon
> us by the wonders of setuptools makes this a non-starter
That's essentially my whole point. Generators are _not_ able to catch all
corner cases but should be able to lower the initial ebuild creation
cost. That could be such things like figure out the license, try to
fill the source url, 
In principal: Try to automate what is possible and leave the rest to
the maintainer.

But in that state they are a tool for the Gentoo developer not for the
user.

Gerion



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Ebuild Generators

2020-02-03 Thread Gerion Entrup
Am Montag, 3. Februar 2020, 13:19:52 CET schrieb Benda Xu:
> Hi Gerion,
> 
> Gerion Entrup  writes:
> 
> >> Yes, that makes a lot of sense.  The R overlay follows this model.  Most
> >> of the ebuilds are automated.  When an ebuild generation fails, we add
> >> the ebuild manually, understand it and then update the generator to
> >> cover it in the future.
> >
> > Is this possible in all cases? I think of adding custom patches,
> > appropriate mapping of dependencies, check for things like desktop
> > icon cache...
> 
> That's too complex to handle automatically.  Luckily, in R overlay, such
> packages are less than 5%.  An ebuild generator is based on the
> observation that many language-specific packages are trivial to fetch,
> compile and install.
> 
> >> > I'm only "maintaining" an overlay so maybe I'm  missing experience
> >> > but I often have wished a tool that automatically parses the language 
> >> > specific
> >> > packaging files and is able to generate a primitive ebuild out of that.
> >> > Maybe it even can do this in an interactive way:
> >> > "Hey, upstream needs the dependency 'foo'. In the Gentoo packages I have 
> >> > found
> >> > 'dev-bar/foo' and 'dev-util/foo'. What is the correct one?"
> >> 
> >> Yes, that's the way R overlay is working.  And I have a similar plan and
> >> proof-of-concept solution for the Java Maven overlay.
> >
> > Nice to hear. I think, it is meaningful to solve all generation with one
> > tool. Maybe it can even "recognize" the used build system and package
> > database. Is this your plan, too?
> 
> No, I don't think it possible as far as I can see...  That would be a
> strong AI.
I mean only on a primitive base:
```
if link contains "pypi":
# it's a Python package from pypi
handle_pypi()
elif work_tree contains "setup.py":
# it's a Python package
handle_generic_python()
elif work_tree contains "meson.build":
handle_meson_package()
...
```

Best,
Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Ebuild Generators (Was: GSoC 2020: Call for mentors and project ideas)

2020-02-03 Thread Gerion Entrup
Am Montag, 3. Februar 2020, 05:20:42 CET schrieb Benda Xu:
> Gerion Entrup  writes:
> 
> > I saw the idea „Big Data Infrastructure by Gentoo“ and found it kind of
> > interesting. However, I have a little bit the fear that a full automation
> > won't be possible and the whole project becomes a little bit like g-sorcery
> > (gs-pypi, gs-elpa) or g-octave: a really cool project but not used at a
> > large scale.
> 
> Yes, that's true.  I share the same observation and concern with you.
> 
> This is one exception: the CRAN ebuild generator powered R overlay has
> been running well for 8 years.
> 
>   https://wiki.gentoo.org/wiki/Project:Science/Overlay/R
Hmm, interesting, thank you.


> > What do you think of the idea to not do this fully automated but supervised
> > by a maintainer? With that I mean an ebuild generator that generates only
> > the parts of the ebuild that it can easily parse and then present the ebuild
> > draft to a maintainer who completes it to an full ebuild. As far a I know no
> > tool like this exists. I think the focus shift helps a lot:
> > Developing a tool for the Gentoo maintainer not the Gentoo user.
> 
> Yes, that makes a lot of sense.  The R overlay follows this model.  Most
> of the ebuilds are automated.  When an ebuild generation fails, we add
> the ebuild manually, understand it and then update the generator to
> cover it in the future.
Is this possible in all cases? I think of adding custom patches,
appropriate mapping of dependencies, check for things like desktop icon
cache...


> > I'm only "maintaining" an overlay so maybe I'm  missing experience
> > but I often have wished a tool that automatically parses the language 
> > specific
> > packaging files and is able to generate a primitive ebuild out of that.
> > Maybe it even can do this in an interactive way:
> > "Hey, upstream needs the dependency 'foo'. In the Gentoo packages I have 
> > found
> > 'dev-bar/foo' and 'dev-util/foo'. What is the correct one?"
> 
> Yes, that's the way R overlay is working.  And I have a similar plan and
> proof-of-concept solution for the Java Maven overlay.
Nice to hear. I think, it is meaningful to solve all generation with one
tool. Maybe it can even "recognize" the used build system and package
database. Is this your plan, too?


Best,
Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] GSoC 2020: Call for mentors and project ideas

2020-02-02 Thread Gerion Entrup
Am Sonntag, 2. Februar 2020, 12:47:55 CET schrieb Benda Xu:
> Dear Fellows,
> 
> alicef  writes:
> 
> > As always, Gentoo plans to participate in the Google Summer of Code
> > 2020. We are looking for new project ideas and are always open for
> > new mentors.
> > Google Summer of Code is a big opportunity for making Gentoo project more
> > visible and get more people interested to join Gentoo and helping out.
> >
> > [...]
> 
> This year's GSoC organization application deadline is on Feb 5.  The
> more project ideas, the better Gentoo will show itself to be prepared.
> If you have been thinking of adding projects to our GSoC 2020 list, this
> is a good chance to do so.
> 
> Cheers,
> Benda
> 

Hi,

I saw the idea „Big Data Infrastructure by Gentoo“ and found it kind of
interesting. However, I have a little bit the fear that a full automation
won't be possible and the whole project becomes a little bit like g-sorcery
(gs-pypi, gs-elpa) or g-octave: a really cool project but not used at a
large scale.

What do you think of the idea to not do this fully automated but supervised
by a maintainer? With that I mean an ebuild generator that generates only
the parts of the ebuild that it can easily parse and then present the ebuild
draft to a maintainer who completes it to an full ebuild. As far a I know no
tool like this exists. I think the focus shift helps a lot:
Developing a tool for the Gentoo maintainer not the Gentoo user.

I'm only "maintaining" an overlay so maybe I'm  missing experience
but I often have wished a tool that automatically parses the language specific
packaging files and is able to generate a primitive ebuild out of that.
Maybe it even can do this in an interactive way:
"Hey, upstream needs the dependency 'foo'. In the Gentoo packages I have found
'dev-bar/foo' and 'dev-util/foo'. What is the correct one?"

With a not fully automatic tool also packages can be parsed that are not
in a complete closed ecosystem, like a 'meson.build' file or cmake files for
C++/C programs. But of course package databases like Maven/Cargo/Pypi are
also candidates.

Unfortunately, I have no time currently to participate in the GSOC. I just
want to mention this here as an idea. Please comment or correct me, if
such a tool already exists.

Best,
Gerion



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Needs ideas: Upcoming circular dependency: expat <> CMake

2019-12-20 Thread Gerion Entrup
Am Donnerstag, 19. Dezember 2019, 19:43:37 CET schrieb Sebastian Pipping:
> On 19.12.19 18:37, Michał Górny wrote:
> > We have a better alternative that lets us limit the impact on the users.
> > Why not use it?
> 
> Which one?  The CMake bootstrap copy?  The adding to stage3 one?

Is it possible to show a specific error message when running in a
circular dependency?

When I get it right, the problem occurs only one time when solved
with a bundled use flag. As soon as expat and/or cmake are installed,
follow up versions can be merge without problems.

If an error message can be shown, maybe this is enough as a hint:
"expat and cmake have circular dependencies. Emerge it the first time with:
USE=bundled-cmake emerge -1 cmake expat
and then just don't care anymore about this use flag."

Of course, the same applies for other known circular dependencies, too.
At least for me as a user, this would be enough.

Best,
Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] [PATCH] package.deprecated: Create initial template

2019-12-05 Thread Gerion Entrup
Am Donnerstag, 5. Dezember 2019, 19:04:50 CET schrieb Michał Górny:
> On Thu, 2019-12-05 at 18:59 +0100, Alexis Ballier wrote:
> > unless i missed something, repoman is still the standard for pre-commit 
> > checks and raising everyone's attention on potential
> > improvements/issues;
> 
> Nope.  Per quite recent Council meeting pkgcheck is fully acceptable
> alternative, and to my knowledge a number of devs have switched already.
> Because they value their time and good package quality.

As a side note:
In this case, can someone with knowledge how to use pkgcheck please update
https://wiki.gentoo.org/wiki/GitHub_Pull_Requests#Step_4:_User_updates_the_local_repository

I have always used it as first address when making (occasional) pull requests.

Thanks,
Gerion



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Last rites: dev-python/* leaf packages

2019-12-05 Thread Gerion Entrup
Am Donnerstag, 5. Dezember 2019, 01:15:48 CET schrieb Aaron Bauman:
> dev-python/pycdio

Has Python 3 support since 2.1 (released in August this year). Developed by 
libcdio itself.

> app-text/pdfshuffler

Was last rited a few day ago. As I said, pdfarranger 
(https://github.com/jeromerobert/pdfarranger) seems to be the modern follow up 
project.

Gerion




signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Last rites: app-text/pdfshuffler

2019-12-02 Thread Gerion Entrup
Am Sonntag, 1. Dezember 2019, 21:09:50 CET schrieb Michał Górny:
> # Michał Górny  (2019-12-01)
> # Unmaintained.  Last release in 2012.  Buggy ebuild.
> # Removal in 30 days.  Bug #658302.
> app-text/pdfshuffler

This is also Python 2 only and GTK 2 (with very experimental GTK 3 support).
However, there seems to be a modern (active) fork with Python 3 and
GTK 3:
https://github.com/jeromerobert/pdfarranger

Gerion

signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags

2019-07-23 Thread Gerion Entrup
Am Dienstag, 23. Juli 2019, 04:00:07 CEST schrieb Kent Fredric:
> On Mon, 22 Jul 2019 21:08:51 -0400
> Aaron Bauman  wrote:
> 
> > 1. I want some documentation
> > 2. It doesn't ship from upstream (without crazy extra deps)
> > 3. Gentoo guy hooked me up and packaged it pre-built with it
> > 4. Thanks!
> 
> The proposal as-stated is:
> 
> 1. Documentation requires even 1 additional dep
> 2. Thou may not use a USE flag for this
> 3. Thus, if you want to elide the dependency from *any* merge graph,
>you must elide it from *all* merge graphs.
> 4. Thus, you must locally perform some non-standard hackery that will
>be different for every package to produce these, work out where to put
>it which is also not standardised, and also prohibit the user from
>being able to update these themselves via a revision bump, _AND_ you
>will need to put in place non-standard mechanisms to ensure it gets
>updated when you update the package, in order for the documentation
>not to diverge from the sources.
What about a compromise?:
Deliver a (prebuild) manpage as package maintainer by default, but keep
a use flag "man-build" (or whatever) that builds the man page for everyone
(also the maintainer herself)  with use of the crazy extra deps. So a user can
do (incomplete) version bumps and gets a manpage and the maintainer
gets the prebuild manpage in a defined way.


> 
> There's a lot of "U, thats bad" in point 4.
> 
> Hence, counter-proposals are trying to look at a way to achieve points
> 2 & 3 in your list, without resorting to barbaric torture and inherent
> fragility.
> 
> We understand the /achieve, but the mechanism proposed doesn't suit, as
> stated.

Regards,
Gerion 



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Packages up for grabs: net-misc/sobby, net-libs/obby, net-libs/libinfinity, net-libs/net6

2018-11-27 Thread Gerion Entrup
Am Dienstag, 27. November 2018, 13:47:25 CET schrieb Ulrich Mueller:
> > On Mon, 26 Nov 2018, Tiziano Müller wrote:
> 
> > the following packages are up for grabs since I have no use for them
> > anymore:
> 
> > net-misc/sobby
> > net-libs/obby
> > net-libs/libinfinity
> > net-libs/net6
> 
> > They are unfortunately not up-to-date, and 3 minor bugs are open for
> > net-libs/libinfinity:
> 
> > https://bugs.gentoo.org/527158
> > https://bugs.gentoo.org/537488
> > https://bugs.gentoo.org/670610
> 
> > Upstream seems to be still around for the editor itself (gobby, not in
> > the tree anymore).
> 
> > If nobody picks them up, they are likely to get tree-cleaned since all
> > except one are libraries nobody seems to be using.
> 
> Actually, sobby is the server counterpart for app-emacs/rudel. So unless
> anyone else is interested, the Emacs team will take these three:
> 
>net-misc/sobby
>net-libs/obby
>net-libs/net6
> 
> Ulrich

It seems that sobby was deprecated by the Gobby team (also last commit
from 2012). Libinifity also provides an obby server (infinoted). Does
app-emacs/rudel run with infinoted as well?

Gerion



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: rfc: why are we still distributing the portage tree via rsync?

2018-07-05 Thread Gerion Entrup
Am Donnerstag, 5. Juli 2018, 14:03:36 CEST schrieb Martin Vaeth:
> Matt Turner  wrote:
> > The ebuild tree is 600MB with rsync and cannot fit on the partition
> > with git.
> >
> > I'd be happy to switch if the space requirements were similar.
> 
> If one git repacks every few syncs one needs currently about 800 MB.
> 
> With additionally squashfs (zstd) (+ overlayfs) the full
> archive size is currently <600 MB.
> 
> In both cases, the temporary disk space is slightly more, of course.
> For a 1GB reserved partition I'd use the partition for the temporary
> mounting and store the archive somewhere else, but I think chances are
> good that you also come through with only a git repack after
> every sync. A difficulty might be the very first git sync.

Would it possible to take the bare repo (< 600 MB) and only mount the latest 
checkout (with fuse eg)?

Gerion






Re: [gentoo-dev] [RFC] multiversion ebuilds

2018-05-17 Thread Gerion Entrup
Am Mittwoch, 16. Mai 2018, 09:38:30 CEST schrieb Michał Górny:
> W dniu sob, 12.05.2018 o godzinie 14∶20 +0200, użytkownik Gerion Entrup
> napisał:
> > Hi,
> > 
> > just an idea for now. But what you think about multiversion ebuilds?
> > Technically this could be realized with the following line in the ebuild 
> > itself:
> > ```
> > VERSIONS=( 3.0.11 3.0.12 3.1 )
> > ```
> > 
> > and the filename without version:
> > //.ebuild
> > 
> > together with this set of rules:
> > 1. If there is an ebuild with had a version in its name, this ebuild is 
> > preferred.
> >  e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and 
> > "foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
> > 2. If the ebuild has the variable VERSIONS specified but also a version in 
> > its name, the version in its name is taken.
> > 3. There can be only one multiversioned ebuild per package.
> > 
> > Different version keywording can be done as before:
> > ```
> > if [[ ${PV} == "3.1" ]] ; then
> > KEYWORDS="~amd64 ~x86"
> > else
> > KEYWORDS="amd64 x86"
> > fi
> > ```
> > 
> > The resolution of versions can be done as before, with the difference that 
> > one ebuild can represent multiple versions.
> > 
> > The "ebuild" tool needs some adjustments. Maybe it tries to download and 
> > build all version per default and has an additional flag to specify a 
> > single version.
> > 
> > The advantages of this idea I see are:
> > - Ebuilds are written in a multiversion manner anyway, and then get copied 
> > (or linked?), so it can be made explicit.
> > - The diffs between different versions of ebuilds and the commit history 
> > are way more readable.
> > - The size of the tree reduces.
> > 
> 
> In my opinion, this puts more effort into inventing a problem than
> solving one.  In order to consider a particular idea thouroughly, you
> should also consider potential disadvantages, rather than focusing
> purely on advantages as you see them.
> 
> While one might think that this will help developers, it is going to be
> a maintenance nightmare.  Just compare the workflow.
> 
> Currently, adding a patch affecting runtime involves copying the ebuild,
> and applying the patch.  Later on, the old revision is just removed. 
> With your solution, you need to adjust VERSIONS, add conditional; later
> on, you need to adjust VERSIONS, find all conditionals, remove them. 
> Not only it involves more work but also increases the risk of accidental
> breakage.
I'm not sure, if I understand you correctly, so I try to express myself:
If an ebuild (say foobar-1.0.ebuild) needs a patch affecting runtime, the
current workflow is to copy the ebuild, add the code that applies the patch
and do a revbump (to foobar-1.0-r1.ebuild). The old ebuild remains
untouched.
With the VERSIONS approach, the VERSIONS need to be adjusted to contain
the revbump version as well, and add a conditional that applies only in the
revbump version the patch and otherwise not.

The VERSIONS approach does not break the old mechanism. So if a patch
needs to be applied, the multiversion ebuild (that contains other versions
as well, say foobar.ebuild with VERSIONS="1.0 1.1 1.2") can just be copied,
renamed with the revbump number (to foobar-1.0-r1.ebuild) and completed
with the apply line. Once the old versions should be deleted the VERSIONS
variable can be adjusted (to VERSIONS="1.1 1.2").


> The arch team work is going to become a nightmare.  Currently, they just
> use 'ekeyword' tool to mass-edit ebuilds.  With your solution, they're
> now have to find the correct conditional (or add one), and update
> keywords there.  I can already imagine monsters like:
> 
>   if pv1; then
> KEYWORDS="~amd64"
>   elif pv2; then
> KEYWORDS="amd64 ~arm64 x86"
>   elif pv3; then
> KEYWORDS="~amd64 ~arm64 ~x86"
>   elif pv4; then
> KEYWORDS="amd64 ~arm64 ~x86"
>   fi
> 
> Basically, any action requiring >1 arch team would involve creating
> a new version at least temporarily.  And I seriously doubt arch teams
> would really want to spend time collapsing those afterwards.
You are right, I have not thought about that. Maybe the approach of R0b0t1
would work.


> The algorithm you presented might look nice at first.  But remember that
> the developers will now have to do the same thing -- i.e. discover which
> ebuild should they look at.  You're moving for a clear 1 ebuild : 1 file
> mapping to N ebuilds : 1 file.

Gerion







Re: [gentoo-dev] [RFC] multiversion ebuilds

2018-05-12 Thread Gerion Entrup
Am Samstag, 12. Mai 2018, 16:24:00 CEST schrieb R0b0t1:
> On Sat, May 12, 2018 at 7:20 AM, Gerion Entrup <gerion.ent...@flump.de> wrote:
> > - The size of the tree reduces.
> >
> 
> If this is a big concern you may be able to mount the portage tree
> under a compressed loopback filesystem. It may even be worth
> considering that as a recommended-by-handbook default as the benefit
> for compressing the ebuilds is likely huge anyway.
This seems reasonable. However, first duplicating code and then deduplicate it 
at file system level seems weird.

Gerion


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] [RFC] multiversion ebuilds

2018-05-12 Thread Gerion Entrup
Am Samstag, 12. Mai 2018, 16:21:26 CEST schrieb Ulrich Mueller:
> >>>>> On Sat, 12 May 2018, Gerion Entrup wrote:
> 
> > just an idea for now. But what you think about multiversion ebuilds?
> > Technically this could be realized with the following line in the
> > ebuild itself:
> > ```
> > VERSIONS=( 3.0.11 3.0.12 3.1 )
> > ```
> 
> > [...]
> 
> > The advantages of this idea I see are:
> > - Ebuilds are written in a multiversion manner anyway, and then get
> > copied (or linked?), so it can be made explicit.
> > - The diffs between different versions of ebuilds and the commit
> > history are way more readable.
> 
> That depends on the options you specify for git diff (e.g.,
> --find-renames, --find-copies, or --find-copies-harder).
This does not apply to the online diffs (see e.g. [1]). I assume that most 
users don't have
the git repository checked out.

> 
> > - The size of the tree reduces.
> 
> I very much doubt that (or at least it remains to be proven).
> 
> Currently, when an ebuild is copied for a version bump, it will reuse
> the same blob in the Git repository. With the scheme above, you would
> have to modify the ebuild, which would add a new blob for every
> version bump.
You are right, I've not thought about that. However, this is only true for
the repository not for the rsync copy most(?) users have and not for a checkout.

Gerion

[1] 
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d9397ab8d48feb4b1360be614da35fa2faae44d9



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] [RFC] multiversion ebuilds

2018-05-12 Thread Gerion Entrup
Am Samstag, 12. Mai 2018, 15:47:57 CEST schrieb Rich Freeman:
> On Sat, May 12, 2018 at 8:20 AM Gerion Entrup <gerion.ent...@flump.de>
> wrote:
> 
> 
> > Different version keywording can be done as before:
> > ```
> > if [[ ${PV} == "3.1" ]] ; then
> >  KEYWORDS="~amd64 ~x86"
> > else
> >  KEYWORDS="amd64 x86"
> > fi
> > ```
> 
>  From a readability standpoint I could see this getting out of hand if we
> aren't careful. 
I have kind of copied this code from a package, where a  ebuild exists. So 
it is already there.

> Also, we'd want to keep this simple enough that the simple
> act of stabilizing a package doesn't introduce regressions (perhaps even in
> versions that we don't intend to touch).  We'd also need rules about
> conditionals so that the metadata cache works.
Maybe this rule can be added:
4. Stable ebuilds are never multiversioned.

So once an ebuild get stable it is split out of the multiversion ebuild (aka 
copied) and not touched anymore.
Maybe this rule can be smoothed by moving the split to the point where new 
features are introduced in the ebuild:
4. If a multiversion ebuild contains a stable version and is changed 
featurewise (e.g. a new useflag), then the stable version is split out of the 
multiversion ebuild.

I don't know the internals of the metadata cache, so I can't say something to 
that.


> You also would need to be careful about ebuild revisions in general, since
> you're potentially affecting multiple versions at the same time.
You are right. But maybe it's the same fix, that is applied to all versions at 
once.

> How would revbumps fit into this?  All kinds of bad things can happen by
> editing ebuilds in place, and this would compound the issue.
Maybe it is possible to solve with the following method:
1. If the fix affects all ebuild versions, revbump all versions in the ebuild.
2. If the fix affects only one version, split out this version of the ebuild 
and make it an explicit one (together with the fix).

The thing is that it is an additional concept. It does not replace the current 
mechanisms and at best reduces overhead for ebuilds that are identical.

Gerion


signature.asc
Description: This is a digitally signed message part.


[gentoo-dev] [RFC] multiversion ebuilds

2018-05-12 Thread Gerion Entrup
Hi,

just an idea for now. But what you think about multiversion ebuilds?
Technically this could be realized with the following line in the ebuild itself:
```
VERSIONS=( 3.0.11 3.0.12 3.1 )
```

and the filename without version:
//.ebuild

together with this set of rules:
1. If there is an ebuild with had a version in its name, this ebuild is 
preferred.
 e.g. is a tree consists of "foobar/foobar-1.1.ebuild" and 
"foobar/foobar.ebuild" for version 1.1 the specific ebuild is taken.
2. If the ebuild has the variable VERSIONS specified but also a version in its 
name, the version in its name is taken.
3. There can be only one multiversioned ebuild per package.

Different version keywording can be done as before:
```
if [[ ${PV} == "3.1" ]] ; then
KEYWORDS="~amd64 ~x86"
else
KEYWORDS="amd64 x86"
fi
```

The resolution of versions can be done as before, with the difference that one 
ebuild can represent multiple versions.

The "ebuild" tool needs some adjustments. Maybe it tries to download and build 
all version per default and has an additional flag to specify a single version.

The advantages of this idea I see are:
- Ebuilds are written in a multiversion manner anyway, and then get copied (or 
linked?), so it can be made explicit.
- The diffs between different versions of ebuilds and the commit history are 
way more readable.
- The size of the tree reduces.

Regards,
Gerion



signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-project] Re: [gentoo-dev] [RFC] Splitting developer-oriented and expert user mailing lists

2017-12-03 Thread Gerion Entrup
Am Sonntag, 3. Dezember 2017, 22:43:19 CET schrieb Michał Górny:
> W dniu nie, 03.12.2017 o godzinie 21∶30 +0100, użytkownik Dirkjan
> Ochtman napisał:
> > On Sun, Dec 3, 2017 at 12:18 AM, Michał Górny  wrote:
> > 
> > > 1. Posting to gentoo-dev@ and gentoo-project@ mailing lists will be
> > > initially restricted to active Gentoo developers.
> > > 
> > > 1a. Subscription (reading) and archives will still be open.
> > > 
> > > 1b. Active Gentoo contributors will be able to obtain posting access
> > > upon being vouched for by an active Gentoo developer.
> > > 
> > 
> > On the face of it, I like this proposal. On the other hand, wouldn't it be
> > better if we just had more active list moderators? That is, moderators who
> > move problematic user's posts to moderated by default, and then withhold
> > the specific posts if necessary?
> 
> I don't think this is really technically feasible. I don't know if mlmmj
> has the specific feature you're asking for, and even if it did,
> moderation with mlmmj is practically impossible to use. Even for low-
> traffic channel like gentoo-dev-announce@ it's not working well.
> 
> > 
> > 2. A new mailing list 'gentoo-expert' will be formed to provide
> > > a discussion medium for expert Gentoo users and developers.
> > > 
> > > 2a. gentoo-expert will have open posting access like gentoo-dev has now.
> > > 
> > 
> > I'm not sure this will be worth it. Who exactly do you think is the
> > audience for this mailing list? What is the goal? How is it different from
> > existing mailing lists?
> 
> The audience is expect users who usually don't need basic support
> but instead want to discuss the development of Gentoo and want to have
> some impact on where it goes.
> 
> The main goal is to be able to restore more developers to gentoo-dev@,
> and be able to focus it on feedback and reviews.
> 
> In other words, the goal is that if the attitude on gentoo-expert
> becomes impossible to bear, the developers can unsubscribe from that
> list without actually losing the ability to give feedback on important
> Gentoo issues.
If core Gentoo developers don't read the expert list, I'm not seeing a high
value in such a list.

I'm a long term Gentoo user, but have read this list a few month only, so
correct me, if I'm wrong. I've seen the main usage of this list in three
aspects:
1. Review and discussion of new (technical) features (eclasses, EAPI, package
manager specs).
2. Information about unmaintained packages.
3. Input and proposals from users.

Splitting the list would reduce the meaning of gentoo-dev to the first point.
The second point has to be handled on the expert list (or both lists), so
proxy maintainers can reply. The third point can only be handled on the expert
list, but core developers have to read it, otherwise the whole point would be
meaningless.

In other projects with similar problems but the technical possibility to 
moderate
some "code of conduct" was adopted, so moderators can ban users on that base
for a fixed amount of time.

Gerion


> 
> > 
> > Cheers,
> > 
> > Dirkjan
> 
> 



signature.asc
Description: This is a digitally signed message part.