[gentoo-user] Re: Changing dependencies without upping version ??

[Kai Krakow]

Am Tue, 26 Sep 2017 18:30:33 -0700
schrieb Ian Zimmerman :

> On 2017-09-27 02:38, Kai Krakow wrote:
> 
> > If you don't want (or cannot) upgrade, you have two options:
> > 
> >   1. Prepare to maintain your own overlay and deal with it
> > 
> >   2. Don't use a rolling release distribution
> > 
> > Personally, and since you seem to know enough to manage your own
> > overlay, I'd stick to #1.  
> 
> I do so already, and in fact my initial workaround was to fork the
> ebuild in my repo, pretty much like you recommend.
> 
> But I didn't know that this was the official way of stopping upgrades.
> I thought package.mask was that, and I think that's what the Handbook
> (or maybe some other part of the wiki) recommends.

Yes, masking of course. But at some point in time, the ebuild would be
dropped. And you may want to keep it around for rebuilds.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: Changing dependencies without upping version ??

[Ian Zimmerman]

On 2017-09-27 02:38, Kai Krakow wrote:

> If you don't want (or cannot) upgrade, you have two options:
> 
>   1. Prepare to maintain your own overlay and deal with it
> 
>   2. Don't use a rolling release distribution
> 
> Personally, and since you seem to know enough to manage your own
> overlay, I'd stick to #1.

I do so already, and in fact my initial workaround was to fork the
ebuild in my repo, pretty much like you recommend.

But I didn't know that this was the official way of stopping upgrades.
I thought package.mask was that, and I think that's what the Handbook
(or maybe some other part of the wiki) recommends.

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
Do obvious transformation on domain to reply privately _only_ on Usenet.

[gentoo-user] Re: Thunderbird build failure

[Kai Krakow]

Am Sat, 23 Sep 2017 17:53:25 +0200 (CEST)
schrieb Christoph Böhmwalder :

> Hey all,
> 
> I've been trying to get Thunderbird to build for a few days now.
> Since I'm pretty much out of ideas at this point, I figured I'd ask
> on here if someone has an idea on what my problem is.
> 
> I've attached the output of `emerge --info
> '=mail-client/thunderbird-52.3.0::gentoo'`, `emerge -pqv
> '=mail-client/thunderbird-52.3.0::gentoo'`, and the complete build
> log.
> 
> From the error message I can see that it likely has something to do
> with either libpng or zlib. I have both of those installed:
> 
> $ emerge --info libpng
> --- >8 ---  
> media-libs/libpng-1.6.29::gentoo was built with the following:
> USE="apng (-neon) -static-libs" ABI_X86="32 (64) (-x32)"
> CPU_FLAGS_X86="sse"
> 
> $ emerge --info zlib
> --- >8 ---  
> sys-libs/zlib-1.2.11::gentoo was built with the following:
> USE="minizip -static-libs" ABI_X86="32 (64) (-x32)"
> 
> 
> I noticed that I have zlib-1.2.11 installed, though Thunderbird (or
> libpng?) is apparently trying to reference a symbol from zlib 1.2.9.
> I tried downgrading to zlib 1.2.9, but...
> 
> # emerge --ask \=sys-libs/zlib-1.2.9
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> 
> emerge: there are no ebuilds to satisfy "=sys-libs/zlib-1.2.9".
> 
> 
> I'd really appreciate any hints on what I'm doing wrong here. Thanks!

Without having looked at the logs, did you try:

# emerge -DNua world --changed-deps

and/or

# emerge -1a @preserved-rebuild

(not necessarily in that order)


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: Changing dependencies without upping version ??

[Kai Krakow]

Am Tue, 26 Sep 2017 11:45:45 -0700
schrieb Ian Zimmerman :

> On 2017-09-26 22:01, Michael Palimaka wrote:
> 
> > If the only argument is you don't want to upgrade, I'm afraid
> > there's not much we can do to help you.  
> 
> You're right that I don't want to upgrade, and I have already
> explained my workaround for that.  But that is _not_ what I'm
> complaining about in this thread.  Rather, my complaint is that such
> a major change is hidden in an ebuild edit with no version/revision
> bump, which means I cannot use the normal means (ie. package.mask) to
> prevent it.  Before I decided to drop Qt completely, I had to make a
> local package of qtcustomplot in my own repo.

If you don't want (or cannot) upgrade, you have two options:

  1. Prepare to maintain your own overlay and deal with it

  2. Don't use a rolling release distribution


Personally, and since you seem to know enough to manage your own
overlay, I'd stick to #1.


> Surely there are other reasons against this kind of thing?  What if
> someone reports a bug in the package?  Now you don't know from the
> version/rev number if it's linked with Qt4 or Qt5.  Is that not
> important?

The problem seems to be that while the package can be built against
both qt4 and qt5, qt4 wasn't present at all.

A proper way I'm sure you could have arranged with, would have been to
introduce both useflags, then mask the qt4 useflag and mark it for
removal during the next version bump. That would have given you an easy
opportunity to properly react to the change, by either unmasking the
flag and pinning the version, or copy the ebuild from db/pkg to your
own overlay.

I don't know how Gentoo policy suggests but I'm pretty sure this is one
of the official ways to prevent exactly your problem.

In the long way, tho, due to qt4 breakage, the qt5 useflag had to be
introduced, and qt4 support had to be dropped. But maybe not in just
one single step without revbump.

The change causes rebuilds for most qt users anyways, as either you had
one of the flags enabled and that resulted in a useflag change thus
rebuild, or: None of the useflags was set, and then you were not
affected (which probably was the "short sighted" decision for doing it
without a revbump in the first place).


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: distributed emerge

[Kai Krakow]

Am Wed, 27 Sep 2017 02:04:12 +0200
schrieb Kai Krakow :

> Am Mon, 25 Sep 2017 21:35:02 +1000
> schrieb Damo Brisbane :
> 
> > Can someone point where I might go for parallel @world build, it is
> > really for my own curiositynat this time. Currently I stage binaries
> > for multiple machines on a single nfs share, but the assumption is
> > to use instead some distributed filesystem. So I think I just need a
> > recipie, pointers or ideas on how to distribute emerge on an @world
> > set? I am thinking granular first, ie per package rather than eg
> > distributed gcc within a single package.  
> 
> As others already pointed out, distcc introduces more headache then it
> solves.
> 
> If you are searching for a solution due to performance of package
> building, you get most profit from building on tmpfs.
> 
> Then, I also suggest going breadth first, thus building more packages
> at the same time.
> 
> Your question implies depth first which means having more compiler
> processes running at a time for a single package. But most build
> processes do not scale out very well for the following reasons:
> 
>   1. Configure phases are serial processes
> 
>   2. Dependencies in Makefile are often buggy or incomplete
> 
>   3. Dependencies between source files often allow parallel
>  building only for short burst throughout the complete
>  build and are serial otherwise
> 
> Building packages in parallel instead solves all these problems: Each
> build phase can one in parallel to every other build phase. So while a
> serialized configure phase is running or package is bundled/merged,
> another package can have multiple gccs running while a third package
> maybe builds serialized due to source file deps.
> 
> Also, emerge is very IO bound. Resorting to distcc won't solve this,
> as a lot of compiler internals need to be copied back and forth
> between the peers. It may even create more IO than building locally
> only. Using tmpfs instead solves this much better.
> 
> I'm using the following settings and have 100% on all eight cores
> almost all the time during emerge, while IO is idle most of the time:
> 
> MAKEOPTS="-s -j9 -l8"
> FEATURES="sfperms parallel-fetch parallel-install protect-owned \
> userfetch splitdebug fail-clean cgroup compressdebug buildpkg \
> binpkg-multi-instance clean-logs userpriv usersandbox"
> EMERGE_DEFAULT_OPTS="--binpkg-respect-use=y --binpkg-changed-deps=y \
> --jobs=10 --load-average 8 --keep-going --usepkg"
> 
> $ fgrep portage /etc/fstab
> none /var/tmp/portage tmpfs
> noauto,x-systemd.automount,x-systemd.idle-timeout=60,size=32G,mode=770,uid=portage,gid=portage
> 
> Have either enough swap or lower the tmpfs allocation.
> 
> Using FEATURES buildpkg pinpkg-multi-instance allows to reuse packages
> on different but similar machines. EMERGE_DEFAULT_OPTS makes use of
> this. /usr/portage/{distfiles,packages} is on shared media.
> 
> Also, I'm usually building world upgrades with --changed-deps to
> rebuild dependers and update the bin packages that way.
> 
> I'm not sure, tho, if running emerge in parallel on two machines would
> pickup newly appearing binpkgs during the process... I guess, not. I
> usually don't do that except the dep tree looks independent between
> both machines.
> 
> If your machine cannot saturate the CPU throughout the whole emerge
> process (as long as there are parallel ebuild running), then distcc
> will clearly not help you, make the complete process slower due to
> waiting on remote resources, and even increase the load. Only very
> few, huge projects, with Makefile deps very clearly optimized or
> specially crafted for distributed builds can benefit from distcc.
> Most projects aren't of this type, even Chromium and LibreOffice
> don't. Exactly, those projects have way to much meta data to
> transport between the distcc peers.
> 
> But YMMV. I'd say, try a different path first.

I imagine one case where distcc could help you: If the building machine
(that one running emerge) is very constraint on system resources. But
in that case, the much better performing option is still staging the
builds on another machine and using binary install on that low-resource
machine.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: distributed emerge

[Kai Krakow]

Am Mon, 25 Sep 2017 21:35:02 +1000
schrieb Damo Brisbane :

> Can someone point where I might go for parallel @world build, it is
> really for my own curiositynat this time. Currently I stage binaries
> for multiple machines on a single nfs share, but the assumption is to
> use instead some distributed filesystem. So I think I just need a
> recipie, pointers or ideas on how to distribute emerge on an @world
> set? I am thinking granular first, ie per package rather than eg
> distributed gcc within a single package.

As others already pointed out, distcc introduces more headache then it
solves.

If you are searching for a solution due to performance of package
building, you get most profit from building on tmpfs.

Then, I also suggest going breadth first, thus building more packages
at the same time.

Your question implies depth first which means having more compiler
processes running at a time for a single package. But most build
processes do not scale out very well for the following reasons:

  1. Configure phases are serial processes

  2. Dependencies in Makefile are often buggy or incomplete

  3. Dependencies between source files often allow parallel
 building only for short burst throughout the complete
 build and are serial otherwise

Building packages in parallel instead solves all these problems: Each
build phase can one in parallel to every other build phase. So while a
serialized configure phase is running or package is bundled/merged,
another package can have multiple gccs running while a third package
maybe builds serialized due to source file deps.

Also, emerge is very IO bound. Resorting to distcc won't solve this, as
a lot of compiler internals need to be copied back and forth between
the peers. It may even create more IO than building locally only. Using
tmpfs instead solves this much better.

I'm using the following settings and have 100% on all eight cores
almost all the time during emerge, while IO is idle most of the time:

MAKEOPTS="-s -j9 -l8"
FEATURES="sfperms parallel-fetch parallel-install protect-owned \
userfetch splitdebug fail-clean cgroup compressdebug buildpkg \
binpkg-multi-instance clean-logs userpriv usersandbox"
EMERGE_DEFAULT_OPTS="--binpkg-respect-use=y --binpkg-changed-deps=y \
--jobs=10 --load-average 8 --keep-going --usepkg"

$ fgrep portage /etc/fstab
none /var/tmp/portage tmpfs 
noauto,x-systemd.automount,x-systemd.idle-timeout=60,size=32G,mode=770,uid=portage,gid=portage

Have either enough swap or lower the tmpfs allocation.

Using FEATURES buildpkg pinpkg-multi-instance allows to reuse packages
on different but similar machines. EMERGE_DEFAULT_OPTS makes use of
this. /usr/portage/{distfiles,packages} is on shared media.

Also, I'm usually building world upgrades with --changed-deps to
rebuild dependers and update the bin packages that way.

I'm not sure, tho, if running emerge in parallel on two machines would
pickup newly appearing binpkgs during the process... I guess, not. I
usually don't do that except the dep tree looks independent between
both machines.

If your machine cannot saturate the CPU throughout the whole emerge
process (as long as there are parallel ebuild running), then distcc
will clearly not help you, make the complete process slower due to
waiting on remote resources, and even increase the load. Only very few,
huge projects, with Makefile deps very clearly optimized or specially
crafted for distributed builds can benefit from distcc. Most projects
aren't of this type, even Chromium and LibreOffice don't. Exactly,
those projects have way to much meta data to transport between the
distcc peers.

But YMMV. I'd say, try a different path first.


-- 
Regards,
Kai

Replies to list-only preferred.

[gentoo-user] Re: Changing dependencies without upping version ??

[Ian Zimmerman]

On 2017-09-26 22:01, Michael Palimaka wrote:

> If the only argument is you don't want to upgrade, I'm afraid there's
> not much we can do to help you.

You're right that I don't want to upgrade, and I have already explained
my workaround for that.  But that is _not_ what I'm complaining about in
this thread.  Rather, my complaint is that such a major change is hidden
in an ebuild edit with no version/revision bump, which means I cannot
use the normal means (ie. package.mask) to prevent it.  Before I decided
to drop Qt completely, I had to make a local package of qtcustomplot in
my own repo.

Surely there are other reasons against this kind of thing?  What if
someone reports a bug in the package?  Now you don't know from the
version/rev number if it's linked with Qt4 or Qt5.  Is that not
important?

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
Do obvious transformation on domain to reply privately _only_ on Usenet.

Re: [gentoo-user] Masked by unknown ?

[Neil Bothwick]

On Tue, 26 Sep 2017 18:31:59 +0200, tu...@posteo.de wrote:

> while updateing I got this:
> 
> These are the packages that would be merged, in reverse order:
> 
> Calculating dependencies... done!
> 
> Total: 0 packages, Size of downloads: 0 KiB
> 
> !!! The following update has been skipped due to unsatisfied
> dependencies:
> 
> dev-libs/boost:0
> 
> selected: (dev-libs/boost-1.63.0:0/1.63.0::gentoo, installed)
> skipped: (dev-libs/boost-1.65.0:0/1.65.0::gentoo, ebuild scheduled
> for merge) (see unsatisfied dependency below)
> 
> !!! All ebuilds that could satisfy "=dev-util/boost-build-1.65*"
> have been masked. !!! One of the following masked packages is required
> to complete your request:
> - dev-util/boost-build-1.65.0::gentoo (masked by: )
> 
> (dependency required by "dev-libs/boost-1.65.0::gentoo" [ebuild])
> For more information, see the MASKED PACKAGES section in the emerge
> man page or refer to the Gentoo Handbook.
> 
> I would interpret this as follows:
> emerge would like to upgrade to dev-libs/boost-1.65.0:0/1.65.0 and
> needs =dev-util/boost-build-1.65* for that, of which in turn emerge
> thinks, it has been (cite) "masked by:  ".
> 
> Since ":" is to less of a text to mask a whole package it must be the
> "  " after that ... :)
> 
> Seriously...as root I did 
> 
> /etc>grep -r boost-build  
> [1]26987 exit 1 egrep --color -r boost-build
> /etc>  
> 
> ...and as emerge before...I did not find anything.
> 
> What is masking boost-build ?
> Where can I search else?
> 
> Thanks a lot in advance for unmasking this riddle !

This seems to happen on every boost update and "emerge -1a boost-build
boost" updates both with no problems.


-- 
Neil Bothwick

"You want us to do WHAT?" - Ancient Chinese wall engineer.


pgpAu0CpyFW6i.pgp
Description: OpenPGP digital signature

[gentoo-user] Masked by unknown ?

[tuxic]

Hi,

while updateing I got this:

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 KiB

!!! The following update has been skipped due to unsatisfied dependencies:

dev-libs/boost:0

selected: (dev-libs/boost-1.63.0:0/1.63.0::gentoo, installed)
skipped: (dev-libs/boost-1.65.0:0/1.65.0::gentoo, ebuild scheduled for 
merge) (see unsatisfied dependency below)

!!! All ebuilds that could satisfy "=dev-util/boost-build-1.65*" have been 
masked.
!!! One of the following masked packages is required to complete your 
request:
- dev-util/boost-build-1.65.0::gentoo (masked by: )

(dependency required by "dev-libs/boost-1.65.0::gentoo" [ebuild])
For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

I would interpret this as follows:
emerge would like to upgrade to dev-libs/boost-1.65.0:0/1.65.0 and
needs =dev-util/boost-build-1.65* for that, of which in turn emerge
thinks, it has been (cite) "masked by:  ".

Since ":" is to less of a text to mask a whole package it must be the
"  " after that ... :)

Seriously...as root I did 

/etc>grep -r boost-build
[1]26987 exit 1 egrep --color -r boost-build
/etc>

...and as emerge before...I did not find anything.

What is masking boost-build ?
Where can I search else?

Thanks a lot in advance for unmasking this riddle !
Cheers
Meino

Re: [gentoo-user] zfs emerge failure (solved)

[John Blinka]

Rich Freeman had the right clue.

Some time ago, after successfully installing zfs, I changed root's
umask to 0027.  This had the effect of changing the permissions on
/lib/modules/X.Y.Z-gentoo to drwxr-x--- on a subsequent kernel
upgrade.  This prevents emerge (once it switches to user:group
portage:portage) from being able to explore the contents of
/lib/modules/X.Y.Z-gentoo.  Unfortunately for me, spl's configure
script locates the current kernel source by following the
/lib/modules/X.Y.Z-gentoo/build soft link.  And it couldn't do that
with the overly restrictive umask.  The solution was simple: eliminate
the 0027 umask for root, and chmod o+rx /lib/modules/X.Y.Z-gentoo.

Thanks for all the suggestions.  They all helped.

John Blinka

[gentoo-user] Re: Changing dependencies without upping version ??

[Michael Palimaka]

On 09/26/2017 03:03 AM, Ian Zimmerman wrote:
> On 2017-09-25 22:24, Michael Palimaka wrote:
> 
>> I see a few complaints in this thread, but nobody so far has
>> elaborated on the problem they have with this change.
> 
> The problem is that if I want to complete the upgrade the way portage
> suggests, I have to (newly) allow in and time-consumingly build _all_
> the qt5 core libraries, since they depend on one another in nearly
> circular fashion, and the updated qtcustomplot becomes the "camel's
> nose".
> 
> I dealt with this by unmerging the few qt using apps I had installed and
> finding alternatives for them.  Some of the alternatives are inferior,
> but it beats this "eternal transition" qt stuff.  I'll make a prediction
> but I don't expect anyone to bet: by the time all useful packages
> migrate to qt5, the qt6 transition will already be in full swing.
> 

If the only argument is you don't want to upgrade, I'm afraid there's
not much we can do to help you.

The reality is that Qt 4 has not been maintained for over 2 years and is
starting to break in worse and worse ways. As we do not have the
resources to maintain a local fork, we have no choice but to follow
upstream's decision to kill it off.

Re: [gentoo-user] no more googleearth in portage

[Raffaele Belardi]

Urs Schütz wrote:
> On 09/20/17 07:55, Raffaele Belardi wrote:
>> I suppose it's due to Google's choice to support only Chrome, although I 
>> missed the Gentoo
>> news bit if there was one.
>>
>> For Android there is the really good Open Street Map application, are there 
>> any desktop
>> alternatives in Portage for non-Chrome users? I know OSM has a web interface 
>> but I'd
>> prefer a standalone application.
>>
> 
> Maybe  sci-geosciences/viking ?
> Try to change the map from MapQuest to MapNik to get a first impression.
> 

Thanks to all for replies, I think I'll start with viking, qgis looks more 
complicated for
a beginner. But as always (mapquest vs mapnik? why there are two? why should  
choose?)
when you scrape the surface complexity immediately emerges so it will take some 
time and
study.

raffaele

Re: [gentoo-user] distributed emerge

[R0b0t1]

On Tue, Sep 26, 2017 at 1:20 AM, Damo Brisbane  wrote:
> Thanks, digesting it!
>
> On Tue, Sep 26, 2017 at 12:59 AM, Andrés Becerra Sandoval
>  wrote:
>>
>>
>>
>> 2017-09-25 6:35 GMT-05:00 Damo Brisbane :
>>>
>>> hi,
>>>
>>> Can someone point where I might go for parallel @world build, it is
>>> really for my own curiositynat this time. Currently I stage binaries for
>>> multiple machines on a single nfs share, but the assumption is to use
>>> instead some distributed filesystem. So I think I just need a recipie,
>>> pointers or ideas on how to distribute emerge on an @world set? I am
>>> thinking granular first, ie per package rather than eg distributed gcc
>>> within a single package.
>>>
>>> thank you
>>
>>
>>
>> Hello,
>>
>> I think distcc might be what you look for the merging part:
>> https://wiki.gentoo.org/wiki/Distcc
>>

Distcc seems to work for some people, but other setups have lots of
issues with it. Not all build scripts provide all of the hooks
necessary to do certain things. Sometimes there is corruption when
communicating results, or subtle flag or optimization differences that
lead to bugs.

When distcc works it works well, but a single package host may be the
more robust solution. You could write some shell script to take
/var/lib/portage/world and only build e.g. half the packages.

Cheers,
 R0b0t1

Re: [gentoo-user] distributed emerge

[Damo Brisbane]

Thanks, digesting it!

On Tue, Sep 26, 2017 at 12:59 AM, Andrés Becerra Sandoval <
andres.bece...@gmail.com> wrote:

>
>
> 2017-09-25 6:35 GMT-05:00 Damo Brisbane :
>
>> hi,
>>
>> Can someone point where I might go for parallel @world build, it is
>> really for my own curiositynat this time. Currently I stage binaries for
>> multiple machines on a single nfs share, but the assumption is to use
>> instead some distributed filesystem. So I think I just need a recipie,
>> pointers or ideas on how to distribute emerge on an @world set? I am
>> thinking granular first, ie per package rather than eg distributed gcc
>> within a single package.
>>
>> thank you
>>
>
>
> ​Hello,
>
> I think distcc might be what you look for the merging part:
> ​https://wiki.gentoo.org/wiki/Distcc
>
> --
>   Andrés Becerra Sandoval
>
>