[gentoo-user] silencing distcc with systemd
Hi all, I've moved a couple of machines from openrc to systemd. I have discovered this odd problem. On openrc, distcc was quiet during building packages. It would obey environment variable set in /etc/env.d: DISTCC_DIR=/var/distcc DISTCC_ENABLE_DISCREPANCY_EMAIL= DISTCC_FALLBACK=1 DISTCC_SAVE_TEMPS=0 DISTCC_SSH= DISTCC_TCP_CORK= DISTCC_VERBOSE=0 This currently shows up in the enviroment (checked with `set`.) However, on systemd it spams messages in build logs: distccd[290662] (dcc_check_compiler_masq) Warning: x86_64-pc-linux-gnu-g++ on distccd's path is /usr/lib/distcc/bin/x86_64-pc-linux-gnu-g++ and really a link to /usr/bin/distcc distcc[293558] (dcc_trace_version) distcc 3.4 x86_64-pc-linux-gnu; built Mar 28 2024 05:05:38 distcc[293558] (dcc_recursion_safeguard) safeguard: 1 distcc[293558] (dcc_recursion_safeguard) safeguard level=1 distcc[293558] (main) compiler name is "x86_64-pc-linux-gnu-g++" distcc[293558] (dcc_set_path) setting PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin distcc[293558] exec on localhost: x86_64-pc-linux-gnu-g++ -O2 -pipe -march=znver3 -felide-constructors -fno-strict-aliasing -pie -fPIC -fstack-protector --param=ssp-buffer-size=4 -Wconversion -Wno-sign-conversion -fno-omit-frame-pointer -Wall -Wenum-compare -Wenum-conversion -Wextra -Wformat-security -Wmissing-braces -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare -Wno-unused-parameter -Woverloaded-virtual -Wnon-virtual-dtor -Wvla -Wwrite-strings -std=gnu++11 -fvisibility=hidden -o /tmp/distccd_9988a064.o -c /tmp/distccd_998ea064.ii distcc[293558] (dcc_mkdir) ERROR: mkdir '/dev/null/.distcc' failed: Not a directory distcc[293558] (dcc_spawn_child) forking to execute: x86_64-pc-linux-gnu-g++ -O2 -pipe -march=znver3 -felide-constructors -fno-strict-aliasing -pie -fPIC -fstack-protector --param=ssp-buffer-size=4 -Wconversion -Wno-sign-conversion -fno-omit-frame-pointer -Wall -Wenum-compare -Wenum-conversion -Wextra -Wformat-security -Wmissing-braces -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare -Wno-unused-parameter -Woverloaded-virtual -Wnon-virtual-dtor -Wvla -Wwrite-strings -std=gnu++11 -fvisibility=hidden -o /tmp/distccd_9988a064.o -c /tmp/distccd_998ea064.ii distcc[293558] (dcc_spawn_child) child started as pid293559 distcc[293559] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=2 /var/tmp/portage/dev-db/mariadb-10.6.17/work/mysql/storage/innobase/fsp/fsp0file.cc: In member function ‘dberr_t Datafile::validate_for_recovery()’: /var/tmp/portage/dev-db/mariadb-10.6.17/work/mysql/storage/innobase/fsp/fsp0file.cc:419:16: warning: this statement may fall through [-Wimplicit-fallthrough=] /var/tmp/portage/dev-db/mariadb-10.6.17/work/mysql/storage/innobase/fsp/fsp0file.cc:429:2: note: here distcc[293558] (dcc_collect_child) cc child 293559 terminated with status 0 distcc[293558] (dcc_collect_child) cc times: user 0.00s, system 0.00s, 0 minflt, 0 majflt distcc[293558] compile (null) on localhost completed ok distcc[293558] elapsed compilation time 1.651658s distcc[293558] (dcc_exit) exit: code 0; self: 0.000832 user 0.000832 sys; children: 1.523681 user 0.134703 sys distcc[293558] (dcc_mkdir) ERROR: mkdir '/dev/null/.distcc' failed: Not a directory distcc[293558] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files I was trying to solve this problem and it's normally set in the environment as above. I read systemd uses a different mechanism, but it doesn't work. I did set /etc/systemd/system/distccd.service.d/00gentoo.conf as per instructions I found on the wiki: Environment="DISTCC_VERBOSE=0" Environment="DISTCC_SAVE_TEMPS=0" Environment="DISTCC_DIR=/var/distcc" Reloaded using `systemctl daemon-reload` and restarted distccd - although this is happening on the client side and it's not really the daemon (as far as I can tell.) The output is showing up while compiling using emerge. I can see it's sourcing this file: distccd.service - Distccd: A Distributed Compilation Server Loaded: loaded (/etc/systemd/system/distccd.service; enabled; preset: disabled) Drop-In: /etc/systemd/system/distccd.service.d └─00gentoo.conf Active: active (running) since Fri 2024-03-29 22:04:12 PDT; 29min ago Is there some other place that needs this environment set? It clearly isn't getting the environment set somewhere in systemd as this error is solved with the DISTCC_DIR in the file sourced above: distcc[293558] (dcc_mkdir) ERROR: mkdir '/dev/null/.distcc' failed: Not a directory ...and the DISTCC_VERBOSE=0 is supposed to silence build messages. Any insight would be helpful... as this problem is easily solved in openrc but it seem to be rather difficult with systemd. Dan
Re: [gentoo-user] masked packages
On 2024.03.29 15:53, n952162 wrote: I'd like to emerge *radicale*, but see it's masked for amd, etc. I looked at the portage meta data and the ebuild to see if I could find out why it should be masked - it's just a python program, supposedly. But I can't find out anything. This warnings are unequivocal about unmasking a package if you don't know why it's masked (https://wiki.gentoo.org/wiki/Knowledge_Base:Unmasking_a_package). Does anybody have any ideas? I see www-apps/radicale-3.1.8 marked as testing, but not masked. The place to look for masking reasons is /usr/portage/profiles/package.mask (or wherever your portage tree lives.) However, if I search for radicale, I only see the one package, and the associated acct-group and acct-user, so I don't know if you just used * to quote the name, or if there are related packages I'm not seeing.
[gentoo-user] masked packages
Hello. I'd like to emerge *radicale*, but see it's masked for amd, etc. I looked at the portage meta data and the ebuild to see if I could find out why it should be masked - it's just a python program, supposedly. But I can't find out anything. This warnings are unequivocal about unmasking a package if you don't know why it's masked (https://wiki.gentoo.org/wiki/Knowledge_Base:Unmasking_a_package). Does anybody have any ideas?
Re[2]: [gentoo-user] merge-usr and SPF implementations
Michael wrote on Friday, 29. März 2024 18:53: > On Friday, 29 March 2024 16:05:29 GMT Stefan Schmiedl wrote: >> Greetings. >> After updating profiles, I decided to try switching to merged-user, too, >> following the wiki page at https://wiki.gentoo.org/wiki/Merge-usr >> One server reported during the dry run: >> ERROR: Conflict for file '/usr/sbin/spfd': [Errno 17] File exists: >> '/usr/bin/spfd' >> # equery belongs /usr/bin/spfd >> * Searching for /usr/bin/spfd ... >> mail-filter/libspf2-1.2.11 (/usr/bin/spfd) >> # equery belongs /usr/sbin/spfd >> * Searching for /usr/sbin/spfd ... >> dev-perl/Mail-SPF-2.9.0-r3 (/usr/sbin/spfd) >> That does put me in a bit of a pickle, as both are active dependencies >> pulled in by essential software: >> # emerge -cav Mail-SPF >> Calculating dependencies ... done! >> dev-perl/Mail-SPF-2.9.0-r3 pulled in by: >> mail-filter/spamassassin-4.0.0-r4 requires dev-perl/Mail-SPF > No packages selected for removal by depclean >> # emerge -cav libspf2 >> Calculating dependencies... done! >> mail-filter/libspf2-1.2.11 pulled in by: >> mail-filter/opendmarc-1.4.1.1-r5 requires mail-filter/libspf2 >> mail-mta/exim-4.97.1-r5 requires >=mail-filter/libspf2-1.2.5-r1 >> What is the recommended way to proceed in this scenario? >> Regards, >> Stefan > My guess and this is only a guess, is the two binaries are in separate > subdirectories of /usr and therefore there shouldn't be a problem. Before > you > progress with this you could raise a bug, or try to seek a dev's advice on > IRC. A fresh backup before you make any changes is definitely a good idea. Quoting the wiki page: "In addition, the script applies the "sbin merge" at the same time where /sbin and /usr/sbin are both actually merged to /usr/bin." So while it is not a problem with split-usr, those two packages will clash with merged-usr. https://bugs.gentoo.org/928140 Thanks, s.
Re: [gentoo-user] merge-usr and SPF implementations
On Friday, 29 March 2024 16:05:29 GMT Stefan Schmiedl wrote: > Greetings. > > After updating profiles, I decided to try switching to merged-user, too, > following the wiki page at https://wiki.gentoo.org/wiki/Merge-usr > > One server reported during the dry run: > ERROR: Conflict for file '/usr/sbin/spfd': [Errno 17] File exists: > '/usr/bin/spfd' > > # equery belongs /usr/bin/spfd > * Searching for /usr/bin/spfd ... > mail-filter/libspf2-1.2.11 (/usr/bin/spfd) > # equery belongs /usr/sbin/spfd > * Searching for /usr/sbin/spfd ... > dev-perl/Mail-SPF-2.9.0-r3 (/usr/sbin/spfd) > > That does put me in a bit of a pickle, as both are active dependencies > pulled in by essential software: > > # emerge -cav Mail-SPF > > Calculating dependencies ... done! >dev-perl/Mail-SPF-2.9.0-r3 pulled in by: > mail-filter/spamassassin-4.0.0-r4 requires dev-perl/Mail-SPF > > >>> No packages selected for removal by depclean > > # emerge -cav libspf2 > > Calculating dependencies... done! >mail-filter/libspf2-1.2.11 pulled in by: > mail-filter/opendmarc-1.4.1.1-r5 requires mail-filter/libspf2 > mail-mta/exim-4.97.1-r5 requires >=mail-filter/libspf2-1.2.5-r1 > > > What is the recommended way to proceed in this scenario? > > Regards, > Stefan My guess and this is only a guess, is the two binaries are in separate subdirectories of /usr and therefore there shouldn't be a problem. Before you progress with this you could raise a bug, or try to seek a dev's advice on IRC. A fresh backup before you make any changes is definitely a good idea. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] New profiles 23.0
On Tuesday, 26 March 2024 13:01:16 GMT Michael wrote: > On Tuesday, 26 March 2024 10:32:05 GMT William KENWORTHY wrote: > > I have a question about binaries and the new profile: I have a number of > > almost identical architectures that I build binaries for and share across > > the similar sytems e.g. arm, aarch64, amd64 etc. > > > > Is deleting the bin host storage (rm -rf ) enough on the > > buildhost so I can share/use the binaries for the "emerge -e" on the > > other hosts? Or does it have to be a native emerge? BillK > > I'm about to try this out in the next couple of days. I will clean binpkgs > on both host and guest, transfer the freshly compile packages with the 23.0 > profile to the guest and then emerge them there as binaries, but following > the profile migration guide, i.e. toolchain first followed by world. > Should things break I'll report back. All went according to plan. Cleared out all files in binpks on both build-host and the guest. Carried out the steps for the migration to profile 23.0 on the host, transferred binaries to the guest, migrated guest to profile 23.0 using binaries. Then merged /usr on both. All is good following this upgrade, but as always YMMV. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] After Update grub does not work anymore
On 3/29/24 03:55, Alexander Puchmayr wrote: Hi, After upgrading two Lenovo Laptops (UEFI, secure boot disabled), grub does not work anymore; instead it says "Welcome to Grub" ... And then immediately boots into bios setup. What did go wrong? I did the usual things: * emerge update world * emerge --config gentoo-kernel * grub-mkconfig -o /boot/grub/grub.cfg Everything went fine without errors, but then the final reboot, only bios setup. And these steps do not alter the efi boot loaders Any ideas? How to get out of this? Alex You'll have to boot off of a rescue disk and chroot in to fix this problem. I had this exact problem yesterday. I hadn't upgraded grub in quite some time and I forgot that particular machine will not boot using: EFI/gentoo/grubx64.efi ...it silently fails to boot and removes the boot entry (efibootmgr confirmed this.) What happened is I forgot to run two grub install commands (I only ran the first one): grub-install --efi-directory=/efi grub-install --efi-directory=/efi --removeable This created a conundrum as only EFI/gentoo/grubx64.efi was updated, which my BIOS refuses to boot from. After I chroot'ed in, I ran both commands so that both EFI/gentoo/grubx64.efi AND EFI/BOOT/BOOTX64.EFI were updated to the new versions. After that my system booted normally. (I rebooted from ssh only to have the machine not come up, when I hooked up the monitor it said something about something missing.) Dan
[gentoo-user] merge-usr and SPF implementations
Greetings. After updating profiles, I decided to try switching to merged-user, too, following the wiki page at https://wiki.gentoo.org/wiki/Merge-usr One server reported during the dry run: ERROR: Conflict for file '/usr/sbin/spfd': [Errno 17] File exists: '/usr/bin/spfd' # equery belongs /usr/bin/spfd * Searching for /usr/bin/spfd ... mail-filter/libspf2-1.2.11 (/usr/bin/spfd) # equery belongs /usr/sbin/spfd * Searching for /usr/sbin/spfd ... dev-perl/Mail-SPF-2.9.0-r3 (/usr/sbin/spfd) That does put me in a bit of a pickle, as both are active dependencies pulled in by essential software: # emerge -cav Mail-SPF Calculating dependencies ... done! dev-perl/Mail-SPF-2.9.0-r3 pulled in by: mail-filter/spamassassin-4.0.0-r4 requires dev-perl/Mail-SPF >>> No packages selected for removal by depclean # emerge -cav libspf2 Calculating dependencies... done! mail-filter/libspf2-1.2.11 pulled in by: mail-filter/opendmarc-1.4.1.1-r5 requires mail-filter/libspf2 mail-mta/exim-4.97.1-r5 requires >=mail-filter/libspf2-1.2.5-r1 What is the recommended way to proceed in this scenario? Regards, Stefan
Re: [gentoo-user] openrc - parallel start - timeouts
On Friday, 29 March 2024 13:30:23 GMT J. Roeleveld wrote: > Hi All, > > To improve the bootup time of my server, I want to enable "parallel", > however, I run into an issue where some of the services take longer than 60 > seconds to start, causing this to be classed as "not started", which then > kills the entire boot sequence. > > Boot, obviously, goes fine with parallel off, but I am certain I can reduce > the boot time significantly using parallel. I also spent quite some time > checking dependencies between the services to ensure they are all present. > > Now, how do I configure a longer timeout? Preferably on a per-service basis, > but at the very least, globally. > > Many thanks in advance, > > Joost Take a look in /etc/rc.conf, the section titled "SERVICE CONFIGURATION VARIABLES". signature.asc Description: This is a digitally signed message part.
[gentoo-user] openrc - parallel start - timeouts
Hi All, To improve the bootup time of my server, I want to enable "parallel", however, I run into an issue where some of the services take longer than 60 seconds to start, causing this to be classed as "not started", which then kills the entire boot sequence. Boot, obviously, goes fine with parallel off, but I am certain I can reduce the boot time significantly using parallel. I also spent quite some time checking dependencies between the services to ensure they are all present. Now, how do I configure a longer timeout? Preferably on a per-service basis, but at the very least, globally. Many thanks in advance, Joost
Re: [gentoo-user] How to synchronise between 2 locations
On Wednesday, 27 March 2024 19:58:47 CET J. Roeleveld wrote: > Hi all, > > I am looking for a way to synchronise a filesystem between 2 servers. > Changes can occur on both sides which means I need to have it synchronise > in both directions. > > Does anyone have any thoughts on this? > > Also, both servers are connected using a slow VPN link, which is why I can't > simply access files on the remote server. > > -- > Joost All, Many thanks to everyone. I will be investigating and testing "net-misc/unison" and "net-p2p/syncthing" to see which fits my requirement best. Thanks again, Joost
Re: [gentoo-user] How to synchronise between 2 locations
Am Thu, Mar 28, 2024 at 09:26:33PM -0500 schrieb Grant Taylor: > On 3/27/24 13:58, J. Roeleveld wrote: > > Hi all, > > Hi, > > > I am looking for a way to synchronise a filesystem between 2 servers. > > Changes > > can occur on both sides which means I need to have it synchronise in both > > directions. > > What sort of turn around time are you looking for? seconds, minus, hours, > longer? > > > Does anyone have any thoughts on this? > > I would wonder about using rsync. Rsync can’t handle file moves. Given: > > Also, both servers are connected using a slow VPN link, which is why I can't > > simply access files on the remote server. it would be beneficial to conserve traffic as much as possible. While you can use the -u flag to only overwrite if the source is newer than the destination, AFAIK rsync can’t detect if the destination has also been altered since the last sync, so it might clobber important changes. That’s why sync tools use a metadata cache to remember last edit timestamps. -- Grüße | Greetings | Salut | Qapla’ Please do not share anything from, with or about me on any social network. In the 60's people took acid to make the world weird. Now the world is weird and people take Prozac to make it normal. signature.asc Description: PGP signature
Re: [gentoo-user] How to synchronise between 2 locations
Am Thu, Mar 28, 2024 at 05:33:43PM +0100 schrieb ralfconn: > Il 28/03/24 07:30, J. Roeleveld ha scritto: > > > Unison creates a local index of all files it syncronised. So when you > > > move a > > > file around on one end, Unison will notice that because the file at the > > > new > > > location has the same hash as the file at the old location. As a result, > > > it > > > does not transmit the file anew to the remote host, but instead copies it > > > locally on the remote host. > > > > > > Since Unison uses ssh underneath, you can use ssh’s transparent > > > compression > > > to speed up the transfer. > > Unison sounds interesting. How does it handle conflicts (eg, file is > > changed on > > both sides?) > > > I use Unison GUI on one of the two machines (on the other peer it's just a > program invoked from the ssh). When the analysis is complete, the GUI shows > what it would do to sync the machines, indicating the conflicts and giving > you the chance to choose what to do. > > I believe it can be used from the command line or maybe even in batch mode > instead of GUI but I never did it that way. You can set up a merge command to solve conflicts on the cmdline, such as vimdiff. But when I set that, it blocks the GUI. Maybe I did something wrong with the setup. Anyways, when I get a conflict, I make a backup of the file locally, overwrite it with the remote and then do a conflict resolution with vim. In my every-day workflow, I usually only get conflicts in text files (logs, notes, and so on). Binary conflicts are rare and usually due to recent actions, such as editing an image or music file. In that case I can decide on a per-case-basis which version to keep. -- Grüße | Greetings | Salut | Qapla’ Please do not share anything from, with or about me on any social network. How does the Heisenberg compensator work? – Thank you, fine. signature.asc Description: PGP signature
Re: [gentoo-user] How to synchronise between 2 locations
Am Thu, Mar 28, 2024 at 01:08:03AM +0100 schrieb Alarig Le Lay: > On Wed 27 Mar 2024 20:37:27 GMT, Frank Steinmetzger wrote: > > +1 for Unison. I’ve been using it for many years now to synchronise between > > the four PC systems in my household. > > > > Unison creates a local index of all files it syncronised. So when you move > > a > > file around on one end, Unison will notice that because the file at the new > > location has the same hash as the file at the old location. As a result, it > > does not transmit the file anew to the remote host, but instead copies it > > locally on the remote host. > > > > Since Unison uses ssh underneath, you can use ssh’s transparent compression > > to speed up the transfer. > > I’ve been thinking about using it to synchronise dovecot maildir folders, > since dsync is now deprecated. But I’m not sure about it as I never used > it under “heavy” loads. Do you have any thoughts about it? Among the many files I sync is my Mail folder. It currently contains around 106k files. My music collection is another 50k files. If there are no changes and I did a recent run, then another run to look for changes takes only very few seconds. -- Grüße | Greetings | Salut | Qapla’ Please do not share anything from, with or about me on any social network. A computer only crashes if your haven’t saved your text for a long time. signature.asc Description: PGP signature
Re: [gentoo-user] After Update grub does not work anymore
Am 29.03.24 um 11:55 schrieb Alexander Puchmayr: Hi, After upgrading two Lenovo Laptops (UEFI, secure boot disabled), grub does not work anymore; instead it says "Welcome to Grub" ... And then immediately boots into bios setup. What did go wrong? I did the usual things: * emerge update world * emerge --config gentoo-kernel * grub-mkconfig -o /boot/grub/grub.cfg Everything went fine without errors, but then the final reboot, only bios setup. And these steps do not alter the efi boot loaders Any ideas? How to get out of this? Alex Hi again, to get out of this booting a live usb or CD and chroot will most likely be your best way to fix this. The installation handbook explains how to chroot. Then you can start fixing the problem. Regards
Re: [gentoo-user] After Update grub does not work anymore
Am 29.03.24 um 11:55 schrieb Alexander Puchmayr: Hi, After upgrading two Lenovo Laptops (UEFI, secure boot disabled), grub does not work anymore; instead it says "Welcome to Grub" ... And then immediately boots into bios setup. What did go wrong? I did the usual things: * emerge update world * emerge --config gentoo-kernel * grub-mkconfig -o /boot/grub/grub.cfg Everything went fine without errors, but then the final reboot, only bios setup. And these steps do not alter the efi boot loaders Any ideas? How to get out of this? Alex Hi, die you follow the announcement about grub? eselect news read 2024-02-01-grub-upgrades Title GRUB upgrades AuthorMike Gilbert Posted2024-02-01 Revision 2 When booting with GRUB, it is important that the core image and modules have matching versions. Usually, running grub-install is sufficient to ensure this. On the UEFI platform, grub-install allows the core image to be placed in two different locations: EFI/gentoo/grubx64.efi This is the location used by grub-install without options. EFI/BOOT/BOOTX64.EFI This is the location used by grub-install --removable. On upgrades, it is common for users to mismatch the grub-install options they used for the current and previous versions of grub. This will cause a stale core image to exist. For example: /boot/efi/EFI/BOOT/BOOTX64.EFI (grub 2.06 core image) /boot/efi/EFI/gentoo/grubx64.efi (grub 2.12 core image) /boot/grub/x86_64-efi/*.mod (grub 2.12 modules) Booting this system using BOOTX64.EFI image would likely fail due to a symbol mismatch between the core image and modules. [1] Re-runing grub-install both with and without the --removable option should ensure a working GRUB installation. However, this will clobber any BOOTX64.EFI image provided by other loaders. If dual-booting using another boot loader, users must take care not to replace BOOTX64.EFI if it is not provided by GRUB. References: [1] https://bugs.gentoo.org/920708
[gentoo-user] After Update grub does not work anymore
Hi, After upgrading two Lenovo Laptops (UEFI, secure boot disabled), grub does not work anymore; instead it says "Welcome to Grub" ... And then immediately boots into bios setup. What did go wrong? I did the usual things: * emerge update world * emerge --config gentoo-kernel * grub-mkconfig -o /boot/grub/grub.cfg Everything went fine without errors, but then the final reboot, only bios setup. And these steps do not alter the efi boot loaders Any ideas? How to get out of this? Alex
