[gentoo-dev] ncurses-5.9 -> ncurses-6.0 step-by-step?
I might've missed it, but do we have a step-by-step procedure to take a rootfs that uses ncurses-5.9 and upgrade it to ncurses-6.0 WITHOUT breaking anything? I, like others, ran into the problem of emerge yanking libncurses.so.5 libs out when upgrading to ncurses-6.0, which broke bash and a few other packages. Took some kludging but I got that fixed on my primary systems. However I have a few chroots that need to be refreshed for catalyst runs, and I would like to just automate the ncurses upgrade for them somehow. I know there's a few nucrses metapackages in the tree that facilitate this, but the -rX numbers make it confusing as to which packages need to be merged in which order (without causing blocks) so that the libncurses.so.[56] libs stay in place until an 'emerge @preserved-libs' run can rebuild the packages that still link against the old libncurses.so.5. Thanks!, -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org 6144R/F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] Determenistic system group and user id
On Sun, Dec 13, 2015 at 02:23:29PM -0800, Alec Warner wrote: > 1) Why do you need deterministic uid / gid's? > 2) If you do need deterministic uid / gid's, I would recommend storing them > all in the same place. They are ALL for system users and groups. TL;DR: if you're sharing data/config for system users/groups between multiple systems based on UID/GID (not username), you need consistent generation. Data on NFSv[23], with a shared apache/nginx user was one of the original examples. I agree since then, that the data should NOT be owned by apache/nginx anymore (NFSv4 also solves the problem). A much newer example, is let's consider the system group 'plugdev'. It's one that is created dynamically at the moment. If I want to put my user in that group LDAP-wide, and have an LDAP environment, I need to make sure the plugdev GID is the same on all systems (actually it also varies slightly depending which LDAP group membership model you're using for NSS data). > For example, you typically want a deterministic UID for a user. To > accomplish this, you add that user to LDAP, give them a UID in LDAP, and > then either add LDAP to nssswitch or use something like nsscache to sync > the ldap UID's into the local system. > > 3) If you need deterministic GID's I would recommend storing them all in > LDAP and syncing the group memberships locally. So you want to define the group twice? Both in LDAP and locally? > I never understood why people would think the distro should handle unique > gid / uids. Plus you usually end up running: > > 1) More than one distro. > 2) More than one 'flavor' of a single distro where for whatever reason, uid > and gid decisions differed (they renumbered, etc.) Here's the work LSB did on it, with further references to what more distros do: https://github.com/LinuxStandardBase/lsb/blob/master/documents/wip/userNaming.txt Here's the debian central database for it: https://anonscm.debian.org/cgit/users/cjwatson/base-passwd.git/tree/README > So if you want a consistent GID for a group, store the group name and gid > in ldap and sync it; do not rely on your distro to do it. IMHO doing so is > a design error. Which is incompatible with NFSv3. > > [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' | > > grep -v eclass | sort -u | wc -l > > 443 > > So there not so much gid uids needed There are definitely entries like these, so be very careful in your counting. enewgroup $PN enewuser ${PN} -1 -1 /var/lib/${PN} ${PN} Based on counting unique tuples of ($CAT/$PN, $ARGS, I count 410+ of each enewgroup and enewuser calls. $ git grep -e 'enewuser ' -e 'enewgroup ' | \ sed -r -e 's,/[^/]+:[[:space:]]*,/: ,g' -e 's,#.*,,g' | \ grep -e ': enew' |sort |uniq Also watch out for packages that create MULTIPLE users/groups for privilege separation (qmail is notorious for this). -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 12/13/2015 07:00 PM, Brian Dolbec wrote: > [snip] > > I just wanted to say that I didn't have (too) much trouble setting up my key when I was getting started back in May. I couldn't have done it without your assistance; I believe one other person helped me out, too. I went to #gentoo-keys and you guys went over it step by step. It looks like my key is still good, too. If you need any help with the project, Brian, I wouldn't mind contributing. Specifically the documentation. I'd just need an idea of where to start. - -- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6 -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCAAGBQJWbmBTAAoJEAEkDpRQOeFwYNEQAId2kLObVWr8RmK/jhD8iD5a nmTdGi54SeDBYLVufYAq6u6O6xE9RJT0bb8uI2/Yy2ux+ZFGyALNIf736AjS9j15 vCmcnC0QrGXIMCfRMGtbRLdacqFGU0Ov59MyF3mzuFVElqW2YafFDGE0s6uZjt8m ElC9Q/tewBHAFYbALUxQcYsY6V5xdVkxRBsBRKfE2HaOEnYma6oRcdefYuVHfhEm Jtn9X+E+pAT8fSoskyQhTMmu4NUBonHcqeZBaWrZmIzKs8w8fdK8Z2Jw4QqzT+Fc 6QclX/BBmrEM8V9649ywaINYXWlOGKVe9PKUlMyr29qJ54azmGFW6LuakTaIpmtc W2q7JpdL3bX7IREEsLl2+DHAcGIf4GRihbNvEgT5JTGcRH2kf/NtDjHUP+pKdye6 NVlNmIQU8ZwB57n7fWniQFTzEZ5xoD0GGIPHCPAtku4+mnGhRinvJt4zrViPi+Mw IwKnknKt72fs9kvmPO5o0zna9NdydkxWySU6Cq5jt6gMHcBwacztj+YXazcxL14y Sb8ozV5mG2IlKL3ghJJhWuEbpMnWuW4XAu4EES5ZB2keXxFxVbpSNL6IDf2hyxxY AUYHLyTD+aaab0dPsGrddg8iQt+pbH+WxKduhqtGFg8QdYT9AaE5iAZxGMehg+kA Vpo7Lk1Glb9qYRVCJ+Lg =EdQ8 -END PGP SIGNATURE-
Re: [gentoo-dev] Use GLEP27!
On Mon, Dec 14, 2015 at 07:49:42AM +0300, Alexey Shvetsov wrote: > Hi! > > Ok. Since there is GLEP27 we should make it reality. To do so i think we > should > 1. Have some list of system uid/gid (on wiki for example). Also we need > to agree on uid/gid numbers for services This database was already started, prior to GLEP27. In CVS, you want gentoo-src/eid_database/ > 2. Add uid/gid from list to existing ebuilds > 3. Make a repoman (or may be eclass) check, that will no allow to commit > ebuilds with enewuser enewgroup calls with undefined uids I think in the original discussion, there were concerns that there were cases where this was going to be valid. I think this check needs to come later, after we rule those out. It should however start to warn about them ASAP. > 4. Make some script or howto to migrate to determenistic uids/gids from Much of the work was implemented for GSOC2006, "Creandus" by developer pioto. Cardoe did more work on it later on. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Determenistic system group and user id
Hi Alec! Alec Warner писал 14-12-2015 01:23: On Sun, Dec 13, 2015 at 10:03 AM, Alexey Shvetsov wrote: Hi all! We trying to use ldap for users @work, many of our workstations running binary gentoo based distro called Calculate linux. However if we wanna have wide use of ldap there is a need for determenistic system group gids names and user uids. Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next available parameter)[1]. However it will be much better to set distro wide deterministic uid and gid for system service name. So for example ldap users may have determenistic groups like video, audio, plugdev, etc.. So the first question I normally ask here is: 1) Why do you need deterministic uid / gid's? for exmaple plugdev group may have random gid from range 10-1000+ (i have some systems when it have gid >1000) so if you're ldap user want to mount external drive on workstation you dont know what gid it should have.. 2) If you do need deterministic uid / gid's, I would recommend storing them all in the same place. For example, you typically want a deterministic UID for a user. To accomplish this, you add that user to LDAP, give them a UID in LDAP, and then either add LDAP to nssswitch or use something like nsscache to sync the ldap UID's into the local system. 3) If you need deterministic GID's I would recommend storing them all in LDAP and syncing the group memberships locally. Syncing groups localy is major design error if you have more then 10+ systems. I never understood why people would think the distro should handle unique gid / uids. Plus you usually end up running: 1) More than one distro. Its not the case. Most time there only one 'supported' distro by local IT stuff. 2) More than one 'flavor' of a single distro where for whatever reason, uid and gid decisions differed (they renumbered, etc.) So if you want a consistent GID for a group, store the group name and gid in ldap and sync it; do not rely on your distro to do it. IMHO doing so is a design error. -A [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' | grep -v eclass | sort -u | wc -l 443 So there not so much gid uids needed -- Best Regards, Alexey 'Alexxy' Shvetsov Best Regards, Alexey 'Alexxy' Shvetsov, PhD Department of Molecular and Radiation Biophysics FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Leningrad region, Gatchina, Russia Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru -- Best Regards, Alexey 'Alexxy' Shvetsov Best Regards, Alexey 'Alexxy' Shvetsov, PhD Department of Molecular and Radiation Biophysics FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Leningrad region, Gatchina, Russia Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] Use GLEP27!
Hi! Ok. Since there is GLEP27 we should make it reality. To do so i think we should 1. Have some list of system uid/gid (on wiki for example). Also we need to agree on uid/gid numbers for services 2. Add uid/gid from list to existing ebuilds 3. Make a repoman (or may be eclass) check, that will no allow to commit ebuilds with enewuser enewgroup calls with undefined uids 4. Make some script or howto to migrate to determenistic uids/gids from 1 Robin H. Johnson писал 13-12-2015 23:41: On Sun, Dec 13, 2015 at 09:03:51PM +0300, Alexey Shvetsov wrote: Hi all! We trying to use ldap for users @work, many of our workstations running binary gentoo based distro called Calculate linux. However if we wanna have wide use of ldap there is a need for determenistic system group gids names and user uids. Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next available parameter)[1]. However it will be much better to set distro wide deterministic uid and gid for system service name. So for example ldap users may have determenistic groups like video, audio, plugdev, etc.. GLEP27 was approved for this, however it is barely used. Convert the rest of the tree to use it, and then you'll be done, aside from the existing mess on user systems. -- Best Regards, Alexey 'Alexxy' Shvetsov Best Regards, Alexey 'Alexxy' Shvetsov, PhD Department of Molecular and Radiation Biophysics FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Leningrad region, Gatchina, Russia Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
On Sun, 13 Dec 2015 22:20:01 +0300 Andrew Savchenko wrote: > Hi, > > On Sun, 13 Dec 2015 18:36:51 +0100 Patrick Lauer wrote: > > Oh hey. We're in the future. Let's try to commit something to > > repo/gentoo.git! > > > > So apparently we're signing things with gpg now, so let's read the > > official documentation. > > The [1] wiki seems to be the canonical location for such things. ... > > Since signing is mandatory since the git migration, ahem, this means > > that no one in the last 5 months(!) actually followed the > > documentation (because that does NOT work!). I'm almost impressed, > > but, wow, this is enterprisey. > > It is absolutely possible to create correct gpg key, put it into > LDAP according to GLEP and to sign commits and pushes properly. > What is not currently possible is to verify all tree automatically. > > I agree that gkeys needs more work. But we are all volunteers here. > You may help them if you are that interested into this > functionality. > > What worries me more that we still have no way for rsync users to > verify the portage tree (or Gentoo tree in the newspeak someone > prefers here). And most users use rsync. > > > So, what can we do to make this whole story of 'commit (and push) to > > repo/gentoo.git' make sense? And why do I appear to be the only one > > to notice this chain of breakage?! > > We need to complete gkeys project, right? That's not all of the > story, but a start. So send patches :) As for the full story, we > still need to somehow verify rsync tree. For now only snapshots are > verified. > > Best regards, > Andrew Savchenko Thank you Andrew. Let me first say, that while I am leading the gentoo-keys project. I have not been doing much with making progress to get another release out. My time is mostly taken up by full time work, add some health issues (not serious), plus with coding full time now (it was just a hobby before). I am finding it difficult to switch codebases to keep development going in a non work project. There is a certain amount of time needed to get back into the code to make any significant progress. But, one of the biggest things keeping me from doing more work on it when I do have some time, is the fact that barely any of the devs seem to care (other than the OP, who just seems to bitch about everything not working for him). Since the GLEP 63 spec has been approved. Barely any of the gentoo developers have even tried to update their gpg key or generate a new one that does meet the spec. For that reason, I have not endeavored to get more done in it. I've been trying to keep the gentoo-devs seed file reasonably up to date, but since there are few devs actually fixing or generating new keys, it is not needed that often. In fact weeks go by before there is a change in LDAP in regards to gpg keys. As Andrew pointed out in another reply, there is a fairly decent document about generating new gpg keys either directly using gpg or using gkeys-gen (gkeys-gen-) has the most troublesome bugs fixed in it btw). But lets get back to the OP's gpg keys. dolsen@vulture /var/lib/gkeys $ gkeys spec-check -C gentoo-devs -n patrick Checking keys... patrick, Patrick Lauer: 0x10F17899A28441CC, 0xA8447784E52864CE == -- Fingerprint..: 0A0901B8F018D5D6A4A31A4410F17899A28441CC Key type : PUBCapabilities.: sc Algorithm: Pass Bit Length...: Fail Create Date..: Pass Expire Date..: Pass Key Version..: Pass Validity.: e, Expired Days till expiry.: 0 Capability...: Pass Qualified ID.: Fail <== '@gentoo.org' user id not found This primary key.: Fail -- Fingerprint..: 043F1AB5B35382E666BDBEA4058F9332F0BAE0B1 Key type : SUBCapabilities.: e Algorithm: Bit Length...: Create Date..: Pass Expire Date..: Pass Key Version..: Pass Validity.: e, Expired Days till expiry.: 0 Capability...: Pass Qualified ID.: Fail <== '@gentoo.org' user id not found This subkey..: Fail Key summary primary..: Fail signing subkey: Fail encryption subkey: Noauthentication subkey: No SPEC requirements: Fail -- Fingerprint..: F941D86BAA0D851BFE411493A8447784E52864CE Key type : PUBCapabilities.: scaESCA Algorithm: Pass Bit Length...: Fail Create Date..: Pass Expire Date..: Fail Key Version..: Pass Validity.: -, Unknown Days till expiry.: infinite <== Exceeds specification Capability...: Pass Qualified ID.: Pass This primary key.: Fail -- Fingerprint..: 8FE9C051CD0859ED2E03274104711EEBFBF3D64A Key type : SUBCapabilities.: e encrypt Algorithm: Bit Length...
Re: [gentoo-dev] Breakage and frustration
On Sun, Dec 13, 2015 at 12:36 PM, Patrick Lauer wrote: > In the long run I am considering just creating my own clone of all > infrastructure bits so I can fix things I just wanted to comment that things like this should never be viewed as a bad thing. Many contributions to Gentoo arose because somebody basically took something they saw and made an unofficial fork they improved upon. That may or may not have later become official, but it is often beneficial all the same. I'd just suggest that anybody doing this document what they do and publish their sources, so that others can do the same more easily. My ideal state for Gentoo would be one where very little of what we do is centrally housed at all, and infra is just a bunch of servers we tend to use as a matter of convention, but anybody can clone one at any time and set up their own. Federated authentication could let people login to various services without having to actually have access to anything sensitive. This would make it much easier for anybody to offer new services, or to offer patches to infra, and then infra could perhaps be more in the role of a trusted gatekeeper and spend less time having to implement every little thing. That said, it is something that would need to be worked towards. My understanding is that infra is already trying to keep the sensitive stuff separate from the mundane in their newer work, but I'm sure they have a ton of old stuff that hasn't been migrated in this fashion. As with many teams they're probably a bit overloaded, so a big question is how to make it happen without just throwing complaints on the folks who are trying their best to keep it all going. -- Rich
[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2015-12-13 23:59 UTC
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2015-12-13 23:59 UTC. Removals: dev-db/jxtray 20151210-08:14 monsieurp ee8ad7c dev-java/dbunit 20151210-08:13 monsieurp 4959819 dev-java/poi 20151210-08:13 monsieurp 4959819 net-p2p/phex 20151210-09:46 monsieurp b3fd801 net-proxy/webscarab 20151210-09:44 monsieurp 6ee5bd6 Additions: app-admin/filebeat-bin20151207-12:29 tmozes c846729 dev-libs/libb64 20151212-14:21 mgorny f32ba92 dev-perl/Gtk2-AppIndicator20151211-23:24 dilfridge 402c3be dev-python/abstract_rendering 20151209-14:41 jlec e22cfaf dev-python/python-engineio20151207-21:13 zmedico6e61044 dev-python/python-socketio20151207-21:25 zmedico4f82f53 dev-python/reno 20151207-18:14 prometheanfire e8b71bb dev-ruby/aws-sdk-core 20151213-07:12 graaff baa1b16 dev-ruby/aws-sdk-resources20151213-07:15 graaff 46b58df kde-plasma/breeze-gtk 20151208-16:15 kensington 9ef59ad kde-plasma/kscreenlocker 20151208-16:15 kensington 9ef59ad media-libs/virglrenderer 20151208-03:48 vapier f413b11 net-p2p/bitcoinxt-qt 20151213-07:11 blueness d39af53 sys-fs/ctmg 20151210-17:34 zx2c4 cc0ddb8 -- Robin Hugh Johnson Gentoo Linux Developer E-Mail : robb...@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 Removed Packages: net-p2p/phex,removed,monsieurp,20151210-09:46,b3fd801 net-proxy/webscarab,removed,monsieurp,20151210-09:44,6ee5bd6 dev-db/jxtray,removed,monsieurp,20151210-08:14,ee8ad7c dev-java/dbunit,removed,monsieurp,20151210-08:13,4959819 dev-java/poi,removed,monsieurp,20151210-08:13,4959819 Added Packages: dev-ruby/aws-sdk-resources,added,graaff,20151213-07:15,46b58df dev-ruby/aws-sdk-core,added,graaff,20151213-07:12,baa1b16 net-p2p/bitcoinxt-qt,added,blueness,20151213-07:11,d39af53 dev-libs/libb64,added,mgorny,20151212-14:21,f32ba92 dev-perl/Gtk2-AppIndicator,added,dilfridge,20151211-23:24,402c3be sys-fs/ctmg,added,zx2c4,20151210-17:34,cc0ddb8 dev-python/abstract_rendering,added,jlec,20151209-14:41,e22cfaf kde-plasma/breeze-gtk,added,kensington,20151208-16:15,9ef59ad kde-plasma/kscreenlocker,added,kensington,20151208-16:15,9ef59ad media-libs/virglrenderer,added,vapier,20151208-03:48,f413b11 dev-python/python-socketio,added,zmedico,20151207-21:25,4f82f53 dev-python/python-engineio,added,zmedico,20151207-21:13,6e61044 dev-python/reno,added,prometheanfire,20151207-18:14,e8b71bb app-admin/filebeat-bin,added,tmozes,20151207-12:29,c846729 Done.
Re: [gentoo-dev] Determenistic system group and user id
* Alec Warner schrieb am 13.12.15 um 23:23 Uhr: [...] > I never understood why people would think the distro should handle unique > gid / uids. Plus you usually end up running: > > 1) More than one distro. not in most places > 2) More than one 'flavor' of a single distro where for whatever reason, uid > and gid decisions differed (they renumbered, etc.) > > So if you want a consistent GID for a group, store the group name and gid > in ldap and sync it; do not rely on your distro to do it. IMHO doing so is > a design error. I disagree here. Most (enterprise) environments use just one distro. And its just very useful if you have sticky UIDs for daemon users for example. One example: You build an apache two-node cluster using DRBD (and pacemaker...). If you happen to install some daemons in random order on both nodes you might end up with apache having different UIDs which will break things. This is a PITA. ANd you do not want another central LDAP-Cluster just to have apache UIDs in sync ;) Red Hat for example has unique distro UIDs for many years now. I would strongly vote for making GLEP27 reality. It makes life easier in many places. -Marc -- 0x35A64134 - 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134 signature.asc Description: Digital signature
Re: [gentoo-dev] Determenistic system group and user id
On Sun, Dec 13, 2015 at 10:03 AM, Alexey Shvetsov wrote: > Hi all! > > We trying to use ldap for users @work, many of our workstations running > binary gentoo based distro called Calculate linux. However if we wanna have > wide use of ldap there is a need for determenistic system group gids names > and user uids. > > Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next > available parameter)[1]. However it will be much better to set distro wide > deterministic uid and gid for system service name. So for example ldap > users may have determenistic groups like video, audio, plugdev, etc.. > So the first question I normally ask here is: 1) Why do you need deterministic uid / gid's? 2) If you do need deterministic uid / gid's, I would recommend storing them all in the same place. For example, you typically want a deterministic UID for a user. To accomplish this, you add that user to LDAP, give them a UID in LDAP, and then either add LDAP to nssswitch or use something like nsscache to sync the ldap UID's into the local system. 3) If you need deterministic GID's I would recommend storing them all in LDAP and syncing the group memberships locally. I never understood why people would think the distro should handle unique gid / uids. Plus you usually end up running: 1) More than one distro. 2) More than one 'flavor' of a single distro where for whatever reason, uid and gid decisions differed (they renumbered, etc.) So if you want a consistent GID for a group, store the group name and gid in ldap and sync it; do not rely on your distro to do it. IMHO doing so is a design error. -A > > [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' | > grep -v eclass | sort -u | wc -l > 443 > So there not so much gid uids needed > > -- > Best Regards, > Alexey 'Alexxy' Shvetsov > Best Regards, > Alexey 'Alexxy' Shvetsov, PhD > Department of Molecular and Radiation Biophysics > FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, > Leningrad region, Gatchina, Russia > Gentoo Team Ru > Gentoo Linux Dev > mailto:alexx...@gmail.com > mailto:ale...@gentoo.org > mailto:ale...@omrb.pnpi.spb.ru > >
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
On Sun, 13 Dec 2015 16:30:06 -0500 Mike Gilbert wrote: > On Sun, Dec 13, 2015 at 2:20 PM, Andrew Savchenko wrote: > >> Since signing is mandatory since the git migration, ahem, this means > >> that no one in the last 5 months(!) actually followed the documentation > >> (because that does NOT work!). I'm almost impressed, but, wow, this is > >> enterprisey. > > > > It is absolutely possible to create correct gpg key, put it into > > LDAP according to GLEP and to sign commits and pushes properly. > > If gkeys is as broken as Patrick describes, it might be helpful to > have step-by-step instructions for manually generating an appropriate > key. This would help new developers. GLEP 63 already contains detailed instructions of how to do this: https://wiki.gentoo.org/wiki/GLEP:63#Specifications_for_GnuPG_keys New devs only have to run $ gpg --gen-key follow instructions from GLEP 63, and upload key using $ gpgp --send-key $key_id Best regards, Andrew Savchenko pgp0A7TsaOHNI.pgp Description: PGP signature
Re: [gentoo-dev] Breakage and frustration
TL;DR summary: Yes, stuff has broken, but I'd call them reasonable teething issues well distributed through the stack, and to be compared to the CVS server moves from a decade ago, rather than CVS just before the Git switch. On Sun, Dec 13, 2015 at 06:36:41PM +0100, Patrick Lauer wrote: ... (mail re-ordered for related issues) > Once that was 'fixed' there was the fun of [2], which made emerge --sync > very expensive because it refetched lots of files. Every time. The fix for this caused these two bugs: > And fixing that introduces [7] some more regressions that broke updating > @system for about 3.5 days. > - a few days of grub being uninstallable (iow, making installing > impossible for many users) > And the manifest issues are still [9] making life exciting. Bug #567920 describes the issue very succinctly (mtime of a deleted file needs to be included in the new Manifest mtime calculation). Both of them can be worked around if the entire path (all staging nodes, servers and end users) uses --checksum, but that's even more expensive. I have another work-around idea, and that's simply appending a comment of the latest commit per directory to the changelog, because that will trigger the manifest being different ;-). > The fix to that fix (notice a pattern here?) broke rsync for *all* users > [8]. Almost as if no one ever tests things in a test environment ... but > hey, we're agile, let's fix stuff in production! We still never figured out how .git came to be added to the outgoing data. It was NOT the rsync into staging directory, because it was only the directory structure, and none of the files. --exclude=.git WAS used in most of the ryncs, but not the final ones from staging to tree distribution. > - about 3 months of emerge --changelog being broken, just to be broken > in a different way This change (order of changelog entries) was explicitly to reduce your complaint of prior excess traffic. Why Portage's parsing of the ChangeLogs is still not handling it is an open question. > - 3.5 days of emerge @system being broken > - about a day of emerge --sync needing manual interaction to be able to > update again You missed one: - rsync generation now halts if somebody committed some breakage to the tree (missing DIST entry, bad eclass). > > So all in all emerge --sync && emerge -uND @system being down for >10% > of the time. > > Now, I don't know if you use Gentoo, but I do, and I use it at work, so > having this level of randomization happen is not really useful. > > Tell me then, please - what can I/we do so that this kind of breakage > stops, and we can actually aim at having a most excellent distro? In the > long run I am considering just creating my own clone of all > infrastructure bits so I can fix things, but it's an option that is > needlessly braindead, wasting effort, and not really useful to users > that are not me. Your own infra option would NOT have fixed [2]/[7]/[9]. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
On Sun, Dec 13, 2015 at 2:20 PM, Andrew Savchenko wrote: >> Since signing is mandatory since the git migration, ahem, this means >> that no one in the last 5 months(!) actually followed the documentation >> (because that does NOT work!). I'm almost impressed, but, wow, this is >> enterprisey. > > It is absolutely possible to create correct gpg key, put it into > LDAP according to GLEP and to sign commits and pushes properly. If gkeys is as broken as Patrick describes, it might be helpful to have step-by-step instructions for manually generating an appropriate key. This would help new developers.
Re: [gentoo-dev] Breakage and frustration
On Sun, 13 Dec 2015 18:36:41 +0100 Patrick Lauer wrote: > Broken breakage > > > tl;dr: Stuff is broken, and no one seems to care > ... > [1] https://bugs.gentoo.org/show_bug.cgi?id=557184 RESOLVED > [2] https://bugs.gentoo.org/show_bug.cgi?id=557192 RESOLVED > [3] https://bugs.gentoo.org/show_bug.cgi?id=557344 RESOLVED > [4] https://bugs.gentoo.org/show_bug.cgi?id=557400 RESOLVED > [5] https://bugs.gentoo.org/show_bug.cgi?id=557826 RESOLVED > [6] https://bugs.gentoo.org/show_bug.cgi?id=565574 RESOLVED > [7] https://bugs.gentoo.org/show_bug.cgi?id=565694 RESOLVED > [8] https://bugs.gentoo.org/show_bug.cgi?id=567074 RESOLVED > [9] https://bugs.gentoo.org/show_bug.cgi?id=567830 RESOLVED Tried to follow the email and failed to find unfixed problems. I believe focusing on actual issues would help. What is broken currently? Do relevant people know they are broken (= are there open bugs)? Or is it an email thread about how not to break things ever in future? -- Sergei signature.asc Description: PGP signature
Re: [gentoo-dev] Determenistic system group and user id
On Sun, Dec 13, 2015 at 1:03 PM, Alexey Shvetsov wrote: > We trying to use ldap for users @work, many of our workstations running > binary gentoo based distro called Calculate linux. However if we wanna have > wide use of ldap there is a need for determenistic system group gids names > and user uids. I think you are looking for GLEP 27 and the related bug 53269. Unfortunately, it seems there has been no movement on that bug in several years. Maybe you could help implement something? https://wiki.gentoo.org/wiki/GLEP:27 https://bugs.gentoo.org/show_bug.cgi?id=53269
[gentoo-dev] Use GLEP27!
On Sun, Dec 13, 2015 at 09:03:51PM +0300, Alexey Shvetsov wrote: > Hi all! > > We trying to use ldap for users @work, many of our workstations running > binary gentoo based distro called Calculate linux. However if we wanna > have wide use of ldap there is a need for determenistic system group > gids names and user uids. > > Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next > available parameter)[1]. However it will be much better to set distro > wide deterministic uid and gid for system service name. So for example > ldap users may have determenistic groups like video, audio, plugdev, > etc.. GLEP27 was approved for this, however it is barely used. Convert the rest of the tree to use it, and then you'll be done, aside from the existing mess on user systems. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
Hi, On Sun, 13 Dec 2015 18:36:51 +0100 Patrick Lauer wrote: > Oh hey. We're in the future. Let's try to commit something to > repo/gentoo.git! > > So apparently we're signing things with gpg now, so let's read the > official documentation. > The [1] wiki seems to be the canonical location for such things. > > Oh dear. The layout is VERY broken. See [2]. Which redirects to [3], > which is a duplicate of [4], which has been closed because apparently > the persons responsible don't understand how to internet. > Since this bug is only about a year old I don't expect any progress soon > - but fetching random crap from untrusted hosts is not a sane option. > Especially since there is already a webserver, which is also trusted, so > I'm confused why we're still having this conversation. > > But hey, let's blindly fetch CSS from unknown, just to notice that this > 'theme' needs JavaScript to display properly. Because reasons. > > Why would I want to blindly execute code when reading the text of a > wiki? Because, reasons. Because, future! I agree with you that wikification of the documentation brings security risks, especially due to sourcing of not-so-trusted resources. But anyway wiki is just docs, one can read them in any isolation environment of choise. Of course, javascript powered L3 cache attack may extract ones git key, this kind of attack may happen from any js-enabled site. So if someone prefers to go for such high security levels, a physically isolated box should be used for git purposes only — and this is what Linus does IIRC. Rackcdn js is not an additional risk in real-life conditions IMO. Also wiki is barely readable in the lightweigth (and rather secure due to lack of extra functions) browsers like elinks or lynx. This irritates me, but is still tolerable in this imperfect world. > Since signing is mandatory since the git migration, ahem, this means > that no one in the last 5 months(!) actually followed the documentation > (because that does NOT work!). I'm almost impressed, but, wow, this is > enterprisey. It is absolutely possible to create correct gpg key, put it into LDAP according to GLEP and to sign commits and pushes properly. What is not currently possible is to verify all tree automatically. I agree that gkeys needs more work. But we are all volunteers here. You may help them if you are that interested into this functionality. What worries me more that we still have no way for rsync users to verify the portage tree (or Gentoo tree in the newspeak someone prefers here). And most users use rsync. > So, what can we do to make this whole story of 'commit (and push) to > repo/gentoo.git' make sense? And why do I appear to be the only one to > notice this chain of breakage?! We need to complete gkeys project, right? That's not all of the story, but a start. So send patches :) As for the full story, we still need to somehow verify rsync tree. For now only snapshots are verified. Best regards, Andrew Savchenko pgpdUcOcxpXcW.pgp Description: PGP signature
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
On 12/13/2015 07:50 PM, Andrew Savchenko wrote: > Hi, > > On Sun, 13 Dec 2015 18:38:55 +0100 Patrick Lauer wrote: >> On 12/13/2015 06:36 PM, Patrick Lauer wrote: >>> So apparently we're signing things with gpg now >> And a related question: >> >> How would I actually verify the signatures in a meaningful way? > git log --show-signature does this using GnuPG. That's not very automated or effective. I'd assume 'emerge' has such functionality included ...? > > Of course, in order to gpg to work one have to mark dev keys as > trusted, they can be verified using ldap or several public > keyservers. LDAP is more reliable, of course, but this method works > only for devs (and probably some stuff members) having an access > here. That's what the app-crypt/gkeys thing is for, as far as I can tell.
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
Hi, On Sun, 13 Dec 2015 18:38:55 +0100 Patrick Lauer wrote: > On 12/13/2015 06:36 PM, Patrick Lauer wrote: > > So apparently we're signing things with gpg now > > And a related question: > > How would I actually verify the signatures in a meaningful way? git log --show-signature does this using GnuPG. Of course, in order to gpg to work one have to mark dev keys as trusted, they can be verified using ldap or several public keyservers. LDAP is more reliable, of course, but this method works only for devs (and probably some stuff members) having an access here. > ... and why is that not default then. Best regards, Andrew Savchenko pgpaR8EFsY5mi.pgp Description: PGP signature
[gentoo-dev] Determenistic system group and user id
Hi all! We trying to use ldap for users @work, many of our workstations running binary gentoo based distro called Calculate linux. However if we wanna have wide use of ldap there is a need for determenistic system group gids names and user uids. Many ebuilds in tree uses enewgroup and enewuser with -1 (aka next available parameter)[1]. However it will be much better to set distro wide deterministic uid and gid for system service name. So for example ldap users may have determenistic groups like video, audio, plugdev, etc.. [1] $ egrep '(enewgroup|enewuser)' * -R | awk -F '/' '{print $1 "/" $2}' | grep -v eclass | sort -u | wc -l 443 So there not so much gid uids needed -- Best Regards, Alexey 'Alexxy' Shvetsov Best Regards, Alexey 'Alexxy' Shvetsov, PhD Department of Molecular and Radiation Biophysics FSBI Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, Leningrad region, Gatchina, Russia Gentoo Team Ru Gentoo Linux Dev mailto:alexx...@gmail.com mailto:ale...@gentoo.org mailto:ale...@omrb.pnpi.spb.ru
Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging
On 12/13/2015 06:36 PM, Patrick Lauer wrote: > So apparently we're signing things with gpg now And a related question: How would I actually verify the signatures in a meaningful way? ... and why is that not default then.
[gentoo-dev] repo/gentoo.git, or how committing is challenging
Oh hey. We're in the future. Let's try to commit something to repo/gentoo.git! So apparently we're signing things with gpg now, so let's read the official documentation. The [1] wiki seems to be the canonical location for such things. Oh dear. The layout is VERY broken. See [2]. Which redirects to [3], which is a duplicate of [4], which has been closed because apparently the persons responsible don't understand how to internet. Since this bug is only about a year old I don't expect any progress soon - but fetching random crap from untrusted hosts is not a sane option. Especially since there is already a webserver, which is also trusted, so I'm confused why we're still having this conversation. But hey, let's blindly fetch CSS from unknown, just to notice that this 'theme' needs JavaScript to display properly. Because reasons. Why would I want to blindly execute code when reading the text of a wiki? Because, reasons. Because, future! Sigh. I'll just live with the breakage then. But anyway, we find [5] the right document, and ... hit [6]. Can't install, bug is over half a year old, so I have to consider upstream dead. But we can easily patch the ebuild and somehow install app-crypt/gkeys. Well, we can install it, but won't be able to use it because [7][8] it's TOFU. Totally Fine and Usable! Nothing some random stabbing won't fix, eh, but now we're an hour in just trying to get dependencies of dependencies installed. Sigh. Now that gkeys is out of the way, let's try to use gkeys-gen! [9][10][11] Nope. Nope nope, you don't get to play! So there's no way to actually *use* this software in the default config (how was this ever released?!), and upstream has not fixed any of these issues in almost a year. This parrot is an ex-parrot! Let's capitulate, err, repudiate. Wait, wrong word. Recapitulate! That's it. Let's recapitulate: The official docs are running on an unmaintained broken platform. If you manage to read them they are wrong. And the software to use has been abandoned a year ago, but is still suggested as default in the docs. Since signing is mandatory since the git migration, ahem, this means that no one in the last 5 months(!) actually followed the documentation (because that does NOT work!). I'm almost impressed, but, wow, this is enterprisey. So, what can we do to make this whole story of 'commit (and push) to repo/gentoo.git' make sense? And why do I appear to be the only one to notice this chain of breakage?! [1] http://wiki.gentoo.org [2] https://bugs.gentoo.org/show_bug.cgi?id=559530 [3] https://bugs.gentoo.org/show_bug.cgi?id=547536 [4] https://bugs.gentoo.org/show_bug.cgi?id=536744 [5] https://wiki.gentoo.org/wiki/Project:Gentoo-keys/Generating_GLEP_63_based_OpenPGP_keys [6] https://bugs.gentoo.org/show_bug.cgi?id=550848 [7] https://bugs.gentoo.org/show_bug.cgi?id=536338 [8] https://bugs.gentoo.org/show_bug.cgi?id=557090 [9] https://bugs.gentoo.org/show_bug.cgi?id=567768 [10] https://bugs.gentoo.org/show_bug.cgi?id=566782 [11] https://bugs.gentoo.org/show_bug.cgi?id=536316
[gentoo-dev] Breakage and frustration
Broken breakage tl;dr: Stuff is broken, and no one seems to care In August the git "migration" happened, moving our main repository from old stupid cvs to modern shiny git. Well, migration is not the word I'd use, because this was an untested forced migration that is now, months later, still suffering from regressions and failures. But, hey, no more cvs! That's good because, reasons. So at first there was [1] the lack of proper Manifests. Which broke things for all rsync users for a few hours. Once that was 'fixed' there was the fun of [2], which made emerge --sync very expensive because it refetched lots of files. Every time. The 'fix' to the fix of the fix for that is still in progress ... And a few little things like [3] happened. Oopsie. Also there seems to be confusion how git works, leading to hilarity like [4]. Now [5] was reported. Who needs ChangeLogs, this is git! Except for all users, who don't get ChangeLogs. Well, let's add them and [6] not test what happens. Guess what? Stuff breaks more. And they are added backwards so that emerge --changelog fails in a different way. No, I didn't want to read all changlog except the part I cared about. And fixing that introduces [7] some more regressions that broke updating @system for about 3.5 days. The fix to that fix (notice a pattern here?) broke rsync for *all* users [8]. Almost as if no one ever tests things in a test environment ... but hey, we're agile, let's fix stuff in production! And the manifest issues are still [9] making life exciting. So to summarize, in about 5 months there was user-visible breakage: - ~1 day downtime for git migration (no updates) - 8h no Manifests (no updates possible) - a few days of emerge --sync being stupidly slow - a few days of emerge-webrsync not updating - about 3 months of emerge --changelog being broken, just to be broken in a different way - 3.5 days of emerge @system being broken - about a day of emerge --sync needing manual interaction to be able to update again - a few days of grub being uninstallable (iow, making installing impossible for many users) So all in all emerge --sync && emerge -uND @system being down for >10% of the time. Now, I don't know if you use Gentoo, but I do, and I use it at work, so having this level of randomization happen is not really useful. Tell me then, please - what can I/we do so that this kind of breakage stops, and we can actually aim at having a most excellent distro? In the long run I am considering just creating my own clone of all infrastructure bits so I can fix things, but it's an option that is needlessly braindead, wasting effort, and not really useful to users that are not me. [1] https://bugs.gentoo.org/show_bug.cgi?id=557184 [2] https://bugs.gentoo.org/show_bug.cgi?id=557192 [3] https://bugs.gentoo.org/show_bug.cgi?id=557344 [4] https://bugs.gentoo.org/show_bug.cgi?id=557400 [5] https://bugs.gentoo.org/show_bug.cgi?id=557826 [6] https://bugs.gentoo.org/show_bug.cgi?id=565574 [7] https://bugs.gentoo.org/show_bug.cgi?id=565694 [8] https://bugs.gentoo.org/show_bug.cgi?id=567074 [9] https://bugs.gentoo.org/show_bug.cgi?id=567830
Re: [gentoo-dev] [RFC] name of smartcard support USE flag
On 13 December 2015 at 19:30, Alon Bar-Lev wrote: > On 13 December 2015 at 19:28, Gilles Dartiguelongue wrote: >> Le dimanche 13 décembre 2015 à 18:25 +0200, Alon Bar-Lev a écrit : >>> On 13 December 2015 at 18:20, Gilles Dartiguelongue >>> wrote: >>> > >>> > I was trying to cleanup my local USE flag settings and stumbled on >>> > the >>> > following three: smartcard, pcsc-lite and pkcs11. >>> > >>> > Knowing all 3 are related, I greped use.local.desc to see what each >>> > meant for different packages. To sum up what I found: >>> > * pcsc-lite is basically: enable smartcard support through >>> > sys- >>> > apps/pcsc-lite >>> > * pkcs11: enabled PKCS#11 (smartcard) via $pkg >>> > >>> > These look like the same thing to me so I propose we merge them all >>> > into USE=smartcard as this is the feature being enabled, not the >>> > lib or >>> > the standard being used to access the hardware if any. >>> >>> pcsc-lite and PKCS#11 interfaces are both related to smartcards but >>> different unrelated interfaces. I am unsure merging them will serve >>> the purpose for applications that are capable of supporting more than >>> one interface. >> >>> also, please notice that PKCS#11 is not all about smartcards, but an >>> interface to any cryptographic hardware. >> >> I agree with your points, my point is that it seems most of the time, >> both use flags are used in place of smartcard (or another name if this >> one does not fit in your opinion). >> >> According to local description, app-mobilephone/gnoki, net- >> libs/libosmocore and net-misc/rdesktop at least should be using >> USE=smartcard instead of USE=pcsc-lite > > rdesktop - I agree. BTW: why don't you use net-misc/freerdp, works better to me and does have smartcard USE :) > I do not know the other packages.
Re: [gentoo-dev] [RFC] name of smartcard support USE flag
On 13 December 2015 at 19:28, Gilles Dartiguelongue wrote: > Le dimanche 13 décembre 2015 à 18:25 +0200, Alon Bar-Lev a écrit : >> On 13 December 2015 at 18:20, Gilles Dartiguelongue >> wrote: >> > >> > I was trying to cleanup my local USE flag settings and stumbled on >> > the >> > following three: smartcard, pcsc-lite and pkcs11. >> > >> > Knowing all 3 are related, I greped use.local.desc to see what each >> > meant for different packages. To sum up what I found: >> > * pcsc-lite is basically: enable smartcard support through >> > sys- >> > apps/pcsc-lite >> > * pkcs11: enabled PKCS#11 (smartcard) via $pkg >> > >> > These look like the same thing to me so I propose we merge them all >> > into USE=smartcard as this is the feature being enabled, not the >> > lib or >> > the standard being used to access the hardware if any. >> >> pcsc-lite and PKCS#11 interfaces are both related to smartcards but >> different unrelated interfaces. I am unsure merging them will serve >> the purpose for applications that are capable of supporting more than >> one interface. > >> also, please notice that PKCS#11 is not all about smartcards, but an >> interface to any cryptographic hardware. > > I agree with your points, my point is that it seems most of the time, > both use flags are used in place of smartcard (or another name if this > one does not fit in your opinion). > > According to local description, app-mobilephone/gnoki, net- > libs/libosmocore and net-misc/rdesktop at least should be using > USE=smartcard instead of USE=pcsc-lite rdesktop - I agree. I do not know the other packages.
Re: [gentoo-dev] [RFC] name of smartcard support USE flag
Le dimanche 13 décembre 2015 à 18:25 +0200, Alon Bar-Lev a écrit : > On 13 December 2015 at 18:20, Gilles Dartiguelongue > wrote: > > > > I was trying to cleanup my local USE flag settings and stumbled on > > the > > following three: smartcard, pcsc-lite and pkcs11. > > > > Knowing all 3 are related, I greped use.local.desc to see what each > > meant for different packages. To sum up what I found: > > * pcsc-lite is basically: enable smartcard support through > > sys- > > apps/pcsc-lite > > * pkcs11: enabled PKCS#11 (smartcard) via $pkg > > > > These look like the same thing to me so I propose we merge them all > > into USE=smartcard as this is the feature being enabled, not the > > lib or > > the standard being used to access the hardware if any. > > pcsc-lite and PKCS#11 interfaces are both related to smartcards but > different unrelated interfaces. I am unsure merging them will serve > the purpose for applications that are capable of supporting more than > one interface. > also, please notice that PKCS#11 is not all about smartcards, but an > interface to any cryptographic hardware. I agree with your points, my point is that it seems most of the time, both use flags are used in place of smartcard (or another name if this one does not fit in your opinion). According to local description, app-mobilephone/gnoki, net- libs/libosmocore and net-misc/rdesktop at least should be using USE=smartcard instead of USE=pcsc-lite -- Gilles Dartiguelongue Gentoo signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [RFC] name of smartcard support USE flag
On 13 December 2015 at 18:20, Gilles Dartiguelongue wrote: > > I was trying to cleanup my local USE flag settings and stumbled on the > following three: smartcard, pcsc-lite and pkcs11. > > Knowing all 3 are related, I greped use.local.desc to see what each > meant for different packages. To sum up what I found: > * pcsc-lite is basically: enable smartcard support through sys- > apps/pcsc-lite > * pkcs11: enabled PKCS#11 (smartcard) via $pkg > > These look like the same thing to me so I propose we merge them all > into USE=smartcard as this is the feature being enabled, not the lib or > the standard being used to access the hardware if any. pcsc-lite and PKCS#11 interfaces are both related to smartcards but different unrelated interfaces. I am unsure merging them will serve the purpose for applications that are capable of supporting more than one interface. also, please notice that PKCS#11 is not all about smartcards, but an interface to any cryptographic hardware. Regards, Alon
[gentoo-dev] [RFC] name of smartcard support USE flag
I was trying to cleanup my local USE flag settings and stumbled on the following three: smartcard, pcsc-lite and pkcs11. Knowing all 3 are related, I greped use.local.desc to see what each meant for different packages. To sum up what I found: * pcsc-lite is basically: enable smartcard support through sys- apps/pcsc-lite * pkcs11: enabled PKCS#11 (smartcard) via $pkg These look like the same thing to me so I propose we merge them all into USE=smartcard as this is the feature being enabled, not the lib or the standard being used to access the hardware if any. -- Gilles Dartiguelongue Gentoo signature.asc Description: This is a digitally signed message part