Re: [gentoo-dev] Re: udev and /usr
On Friday, September 16, 2011 06:06:35 PM Duncan wrote: Joost Roeleveld posted on Fri, 16 Sep 2011 10:36:27 +0200 as excerpted: I agree, I just used this example to explain that it shouldn't be necessary to force an initramfs on all users just because there is a small group who wants to have an extreme setup. Careful with the extreme. As you no doubt realize by now, the udev folks apparently consider anyone wanting a separate /usr but not an initr* extreme. That'd certainly apply double if said admin (since no simple user cares about such stuff, in this view) had /usr on lvm. Yes, I've noticed that. Except that Redhat and Centos use LVM by default. Which will also mean that simple users also end up using LVM. Then again, they also end up with an initr* and a generic kernel for everything under the sun. I haven't properly looked at the kernel-configs from redhat lately, but I don't think they include all the possible hardware options be default? -- Joost
Re: [gentoo-dev] Re: udev and /usr
On Sat, Sep 17, 2011 at 2:16 AM, Joost Roeleveld jo...@antarean.org wrote: Except that Redhat and Centos use LVM by default. Which will also mean that simple users also end up using LVM. Then again, they also end up with an initr* and a generic kernel for everything under the sun. I haven't properly looked at the kernel-configs from redhat lately, but I don't think they include all the possible hardware options be default? The typical mainstream binary distro approach is to: 1. Build every module under the sun that won't cause more support headaches than benefits. 2. Build a really smart initramfs that can find root whether it is on raid, lvm, luks, or on a SAN behind luks and a VPN (ok, I'm stretching it a little). 3. Deploy that on everything. With an initramfs you can essentially build a completely modular kernel, since the kernel can rely on any module it wishes right from the start. However, once the initramfs is done the kernel is still a minimal size since unused modules don't occupy space (the initramfs wipes itself out of ram as its last step, unless in a debug mode). Sure, the fancy initramfs takes work, but since the install image is one-size-fits-all it is less maintenance in the long haul. Plus you can replace your motherboard and still boot the same drive image. The downside is that it might take an extra second or two to boot, but dracut is pretty fast. Honestly, if I were running a server setup I'd probably consider using an intiramfs. They're a lot less susceptible to being messed up if for whatever reason the drives get re-ordered in the BIOS (root=UUID, or with LVM the device names can usually be trusted). I once booted off of a rescue CD that for whatever reason changed the major numbers assigned to my raid devices and for a while I could never predict what /dev/md# my root would end up with. That is what started my quest to get dracut working, which I'll continue whenever git.kernel.org is back up... Rich
Re: [gentoo-dev] udev and /usr
On Fri, Sep 16, 2011 at 10:36:27AM +0200, Joost Roeleveld wrote: Either udev does this already and the execution sequence is always the same. In which case my suggestion above would follow the same sequence as the queue would be on a First-in First-out basis. Or, if udev doesn't do this yet, udev will end up having the same problem. It doesn't do it presently. The worst case I've seen is having an early rule that succeeds, but gives different results w/ /var mounted vs. not mounted. This rule didn't actual fail, so it does not get retried... And here is my main concern with this. The udev team don't list all the possible filesystems where things can go wrong. They only mention /usr. We know that there are rules that invoke various parts of /usr, and rules that have state storage in various parts of /var. I agree that a lot more granularity would be useful, to help those that might have multiple mountpoints within each of /usr and /var (eg, I have /usr/lib64/debug in a seperate LV). This is why I would suggest the actiond process to be started after localmount. That's too late. What about all the udev rules required to get the device nodes for localmount to succeed? Shouldn't these already exist for currently working setups? Specifically, I meant, how do you know which rules to run immediately before localmount, and which to postpone via actiond? The upstream discussions I've been party to previously (both on lists and in person), have been trying to avoid needing a full dependency system in udev, because it's a huge degree of additional complexity. I don't see why it would not be possible to pause actioning of these scripts till the boot-process says all required mounts are available. You still have to declare which scripts need to be paused, and/or which rules inside the scripts need to be paused. Even worse are cases where mounting some of localmount entries requires those scripts to have completed. In other words, a dependency on the rules would be needed? Yes, as I noted in my initial response to you, there's going to be interdependencies between udev rules and other parts of the system. (The other reason I think systemd and udev might merge at some point, or at least have good IPC between them, because there is a potential for speed gains there). udev runs that rule as soon as the hardware turns up, which is often before localmount. I have doubts about having all these things handled by udev. As you said, there is an init-script that handles this. Is the ultimate goal to get rid of init-scripts and have everything done automagically? The rule is really useful important if you plug in a USB or Firewire sound card at some point after boot. If you already had it configured a previous time, that rule will restore your volume settings :-). The other parts of that init script are valuable still, but the volume restore is just a crutch for failing to load the volume when the device was first detected. If you have a soundcard that makes a pop when your system boots, that's a bug caused by this. Just because there are no visible errors, doesn't mean that they don't exist. This move to encourage initramfs is to ensure that there isn't any major breakage incidents soon. What if udev upstream suddenely starts hard requiring /usr to mounted, and not doing retries at all. How many systems are going to break, and users going to complain about needing to use livecds to recover? A lot. And those will be very vocal. I have a few goals with this thread and one of them is to try to figure out how best to avoid users to get affected by this. For now, users having an initramfs ahead of time is the best option to avoid future breakages. DEVTMPFS creates the first batch, and udev creates the rest. The deciding case then becomes: - Is the device for your /usr entry in fstab created by udev or something else? MD: done by devtmpfs LVM: done by udev+lvm by-uuid/by-label: done by udev by-uuid and by-label present a lot of annoyance to the minimal initramfs. We have to ensure that we explicitly support them, which has increased the complexity of the initramfs. My /usr is on LVM. That requires udev. My understanding is: - udev needs /usr to be mounted to work Correct. - udev is needed to sort out LVM to get access to /usr Incorrect. But perhaps not for the reason that you think. Using genkernel's initramfs here as an example, this is the sequence of LVM commands that run: vgscan vgchange -ay --sysinit That --sysinit part is important, as it tells LVM to avoid locking (and some interaction with udev), and then LVM has fallback code to create the symlinks and other device nodes in /dev. What udev CAN do is create all of the by-label/by-uuid bits above. The fallback code in LVM is very complex, just for the case of handling udev not being there yet. And why can't this be implemented in
[gentoo-dev] Infrastructure Team survey
Hello, My name is Alec and I am a recent addition to the Gentoo Infrastructure Team. I think our Infra team does a decent job of holding the fort. However I am curious what developers and users think about our efforts. As such I have a short (4 question) survey[1] for you. You do not need to be a developer to answer the survey. [1] https://docs.google.com/a/scriptkitty.com/spreadsheet/viewform?formkey=dC1BVVdkSDlnR1RKdE9LMlY2SWEwV0E6MQ#gid=0 PS: Yes the survey is a poopy Google Doc. No you should not need a Google Account to respond to it. Yes Google will store your IP and browser string. This is one of the easiest ways to do a survey. If you really hate Google you can just email your replies to me (anta...@gentoo.org) and I'll add them to the spreadsheet by hand. -A
Re: [gentoo-dev] new `usex` helper
On 09/16/2011 02:06 AM, Brian Harring wrote: On Thu, Sep 15, 2011 at 10:00:19PM -0500, Donnie Berkholz wrote: On 17:29 Wed 14 Sep , Brian Harring wrote: On Wed, Sep 14, 2011 at 02:16:41PM -0500, Donnie Berkholz wrote: On 19:14 Tue 13 Sep , Brian Harring wrote: On Tue, Sep 13, 2011 at 09:02:28PM -0500, Donnie Berkholz wrote: On 17:56 Tue 13 Sep , Mike Frysinger wrote: useful enough for EAPI ? or should i just stick it into eutils.eclass ? OR BOTH !? I prefer to avoid EAPI whenever possible, as it just makes things slower and more complex. Exactly the wrong approach; it winds up with master repositories/overlays cloning the functionality all over the damn place. Why are people cloning anything if it's in eutils.eclass in gentoo-x86? There are more repositories than just gentoo-x86, and overlay is *not* the only configuration in use. Who else besides you is using any other configuration? Should we really give a crap about the 0.001% population with some weird setup when we're trying to improve things for the 99.999% one? Specious argument; the point of controllable stacking was to avoid the issue of overlay's forcing their eclasses upon gentoo-x86 ebuilds (which may not support those modified eclasses) via the old PORTDIR_OVERLAY behaviour. This is why multiple repositories have layout.conf master definitions- to explicitly mark that they require/consume a seperate repo. What you're basically proposing is a variation of the push format definitions into a central tree, and require everyone to use that central tree. This discussion has come and gone; I say that being one of the folks who was *pushing for the repository solution*. The problem there is it fundamentally enables (more forces) fragmentation. Realistically I assume you're taking the stance EAPI gets in the way, lets do away with it- if so, well, out with it, and I'll dredge up the old logs/complaints that lead to EAPI. In the old days of the PM only handling a single overlay stack, what you're suggesting would be less heinous- heinous in detail, but pragmatic in reality. These days it's a regressive approach- requiring everyone to slave gentoo-x86 isn't sane, nor is avoiding eapi (resulting in people having to duplicate code into each repository stack). I don't know many people who aren't using gentoo-x86 or a repo that pulls in changes directly from it. rephrase please; either you're saying everyone uses gentoo-x86 which is sort of true (funtoo/chrome aren't necessarily riding HEAD/trunk however which means things can get ugly for derivative repository usage), but still ignores the situations where people have overlays w/ developmental eclasses that they need to selectively control where it overrides (which is where the notion of repo stacking comes into play). ~brian As I've mentioned in bug #380391 [1], a possible hybrid approach would be to distribute an eclass library that can be installed as a dependency of the package manager. This would provide benefits similar to those of using eclasses from gentoo-x86, while avoiding the introduction of a dependencies on either gentoo-x86 or package manager implementations. [1] https://bugs.gentoo.org/show_bug.cgi?id=380391#c3 -- Thanks, Zac
[gentoo-dev] FEATURES=stricter as a default in developer profile not the best idea
TLDR: Let's remove FEATURES=stricter from developer profile, I bet
most people have it disabled anyway and it doesn't seem useful.
I recently started more testing in one of my stable chroots, and I
switched it to the developer profile. During the update the following
error happened:
* QA Notice: Package has poor programming practices which may compile
*fine but exhibit random runtime failures.
* gam_listener.c:101: warning: implicit declaration of function
'gam_exclude_check'
* gam_server.c:624: warning: implicit declaration of function
'gam_error_init'
* gam_connection.c:402: warning: implicit declaration of function
'gam_exclude_check'
* gam_inotify.c:145: warning: implicit declaration of function
'gam_poll_basic_init'
* Please do not file a Gentoo bug and instead report the above QA
* issues directly to the upstream developers of this software.
* Homepage: http://www.gnome.org/~veillard/gamin/
* ERROR: app-admin/gam-server-0.1.10 failed:
* install aborted due to poor programming practices shown above
*
* Call stack:
* misc-functions.sh, line 992: Called install_qa_check
* misc-functions.sh, line 688: Called die
* The specific snippet of code:
* has stricter ${FEATURES} die install
aborted due to \
Which is of course for a package I'm not a developer of. It seems to me
that any people who use the developer profile productively have already
disabled FEATURES=stricter. In fact, I've done so in my development VM.
I suggest that we remove FEATURES=stricter from developer profile's
defaults.
/usr/portage/profiles/targets/developer/make.defaults:FEATURES=collision-protect
multilib-strict sign splitdebug stricter test test-fail-continue
userpriv usersandbox
An alternative solution would be to make misc-functions.sh not bail out
on errors beyond control of most people, but that's an order of
magnitude more complicated.
Thoughts?
signature.asc
Description: OpenPGP digital signature
[gentoo-dev] Last rites: www-plugins/weave
# Mounir Lamouri volk...@gentoo.org (18 Sep 2011) # Masked for removal in 30 days. Use Firefox 4 or higher instead. www-plugins/weave
Re: [gentoo-dev] Fwd: [gentoo-dev-announce] Call for items for September 13 council meeting
On 14:06 Fri 16 Sep , Zac Medico wrote: Bumping the EAPI of the root profiles/eapi file would be a different matter, since it applies to the whole repository. If you want to version bump that repository-level EAPI, then you need to wait until at least 6 months after supporting package managers have been available in stable. So in your opinion, it would be fine to bump profiles/eapi to EAPI=4 now? -- Thanks, Donnie Donnie Berkholz Council Member / Sr. Developer Gentoo Linux Blog: http://dberkholz.com pgpT74nMsKtpS.pgp Description: PGP signature
Re: [gentoo-dev] new `usex` helper
On 13:43 Fri 16 Sep , Brian Harring wrote: What I said from the getgo and you're missing is that pushing EAPI implementation into the tree and ignoring EAPI, or having this notion that every repository must automatically use gentoo-x86 (pushing the format into the tree) is /wrong/; I'm not necessarily requiring that every repository must automatically use gentoo-x86 — just the ones that want to use features in an eclass distributed with gentoo-x86. It sounds to me like the logical consequence of what you're saying is that every useful function that's now or could eventually be in an eclass must instead be incorporated into EAPI. I guess I just don't see where you're drawing the line. What I'm suggesting is that we should add useful stuff to eclasses by default. If people want to use that stuff, they can stack on the gentoo-x86 repo and inherit the eclass. I don't know why EAPI needs to come into it at all. aside from meaning that the format definition can now *vary*, which is great for wasting dev time and screwing up compatibility, it opens up tweaking/customizing that functionality- aka, fragmentation/divergence. People doing that outside gentoo-x86 should do it the same way as ones within it, by bumping the eclass to a new unique name. Ideally one that's not just a numeric value so it won't conflict with ours, in the same way as EAPI naming. If we did the sort of thing you're basically pushing for, how long do you think it would be till funtoo added support for a new archive format to unpack? That's a *simple*, and *likely* example of how this can diverge. So, what I'm getting out of this is that we should make it harder for derivative distributions to innovate? Why should I care if they want to do stuff with new archive formats? Further, doing as you propose means we're flat out, shit out of luck /ever/ distributing a usable cache for standalone repositories. If they're bound to the changes of another repository, distributing a cache in parallel is pointless (and not doable in current form since metadata/cache lacks necessary eclass validation data for overlay inheritance). Not much different from other cross-repository dependencies; you have to keep everything in lockstep because who knows what other people will do with their repos. Maybe people would want to distribute their own copies of forked dependent repositories too, I haven't thought much about it. -- Thanks, Donnie Donnie Berkholz Council Member / Sr. Developer Gentoo Linux Blog: http://dberkholz.com pgptazUS3EHSs.pgp Description: PGP signature
[gentoo-dev] glibc-2.13 stabilization
'bout that time again. if you peeps know of anything blocking glibc-2.13 from going stable, mark the relevant bug as blocking the tracker. tracker: https://bugs.gentoo.org/354107 stabilization: https://bugs.gentoo.org/382377 -mike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Fwd: [gentoo-dev-announce] Call for items for September 13 council meeting
On 09/17/2011 08:47 PM, Donnie Berkholz wrote: On 14:06 Fri 16 Sep , Zac Medico wrote: Bumping the EAPI of the root profiles/eapi file would be a different matter, since it applies to the whole repository. If you want to version bump that repository-level EAPI, then you need to wait until at least 6 months after supporting package managers have been available in stable. So in your opinion, it would be fine to bump profiles/eapi to EAPI=4 now? Yes, it's feasible. As a consequence, we may get some complaints from users who haven't upgraded during the last six months. For users like these, we could take a snapshot of the tree before the EAPI is bumped, and archive it so they can use it to update their package manager to a version that supports the new EAPI. -- Thanks, Zac
Re: [gentoo-dev] udev and /usr
On 9/15/11 1:33 PM, Joost Roeleveld wrote: On Thursday, September 15, 2011 09:31:45 PM Luca Barbato wrote: On 15/09/2011 16:33, Joost Roeleveld wrote: Hi Devs, Not sure if you are aware of the discussions on the gentoo-user list about the upcoming change where systemd and udev require /usr to be available prior to starting of udev. systemd seems more and more just a support burden for no gain... Myself and a lot of others on the gentoo-user list agree with this. Especially as there are plenty of installations where udev works without /usr mounted. Glad we agree. I think putting more pressure so systemd isn't given as granted would be more healthy for both those who are not using it (because, again, is an aberration for any kind of daemon not written for it) and those that want to use it (since maybe as desktop-only it might have some nice integrations). Probably just adding the dbus interfaces and thinning it down might be something useful if that integration might have use-cases. lu - not feeling a mindless lemming
