Bug#809705: general: let people use non-free software but opt-out of non-open software

[Niels Thykier]

Philippe Cerfon:
> On Sun, Jan 3, 2016 at 7:35 AM, Christian PERRIER  wrote:
>> Discussing infrastructure changes like what you're proposing (which I
>> have no advice about) should usually be done through our mailing
>> lists,
> Which one would be the appropriate list?
> 
> I thought general would fit, as it likely affects multiple packages
> and infrastructure systems form Debian.
> Anyway, I don't mind to forward this to some list as well.
> 
> Thanks,
> Philippe.
> 

Your second item has been brought up before with different
focus/rationale/purpose.  At least I remember there being an interest in
splitting "non-free" into "non-free/firmware" vs. various other non-free
sub components.

Mind you, its primary goal was not to address "source vs. no-source",
but it is the closest related idea I could think of.  Sadly, I don't
have a reference ready to backup my memory.


On your first item, I would have to agree with Christian.  It is not
actionable and much less by Debian.  At best we could stop packaging
such software or disabling such features, but both have their disadvantages:

 * Even if we stop shipping them, people will still download them.
   Except your average user will probably be worse off because most of
   them do not verify their downloads.
 * If we disable the functionality, we would "cripple" the software to
   many people.  If this annoys people, we will end up in a situation
   where people will advise /against/ using the Debian package because
   it is "crippled", which leads to the situation above.  Or we could
   get badly unpopular with upstream (see the "Debian vs. Ruby" issue
   from a couple of years ago).


Thanks,
~Niels

Bug#809705: general: let people use non-free software but opt-out of non-open software

[Philippe Cerfon]

On Sun, Jan 3, 2016 at 7:35 AM, Christian PERRIER  wrote:
> Discussing infrastructure changes like what you're proposing (which I
> have no advice about) should usually be done through our mailing
> lists,
Which one would be the appropriate list?

I thought general would fit, as it likely affects multiple packages
and infrastructure systems form Debian.
Anyway, I don't mind to forward this to some list as well.

Thanks,
Philippe.

Re: support for merged /usr in Debian

[Ben Hutchings]

On Mon, 2016-01-04 at 00:55 +0100, Christian Seiler wrote:
[...]
> Btw. initramfs-tools (via switch_root from util-linux) recursively
> deletes the files in the initramfs in a forked-off process right
> before exec()ing init. Is that really necessary? Shouldn't the
> kernel loose the data of the initramfs when the last process using
> the mount is replaced with the system's init? (If I look at
> /proc/self/mountinfo, there's no entry for the initramfs anymore
> after just doing what my PoC does.)

I hadn't noticed that, but it seems to be a recent change in the kernel
(somewhere between 3.16 and 4.3).

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Christian Seiler]

On 01/04/2016 12:39 AM, Ben Hutchings wrote:
> On Sun, 2016-01-03 at 21:35 +0100, Christian Seiler wrote:
> [...]
>>
>> Well, just for the heck of it I wrote a braindead-simple initrd
>> implementation in just 300 LOC:
>>
>> https://gist.github.com/chris-se/e0fbc073fcbd9ac2d7ae
> [...]
> 
> Neat.  It should probably implement 'ro', 'rw' and 'rootdelay', though.

Yeah, those are on my list of things to implement if enough people
are interested in this. My code was just a simple PoC.

Ideally, the goal would be to support:

 - get /usr information from /etc/fstab of rootfs
 - support ro, rw, rootdelay, rootwait, rootflags, rootfstype
 - better error reporting
 - support MAJ:MIN device id syntax

 - for Debian: integration into the hooks as initramfs-tools and
   dracut currently do

But not more, because I want the binary to remain *really* small,
and I don't want to replace initramfs-tools or dracut with this.

Btw. initramfs-tools (via switch_root from util-linux) recursively
deletes the files in the initramfs in a forked-off process right
before exec()ing init. Is that really necessary? Shouldn't the
kernel loose the data of the initramfs when the last process using
the mount is replaced with the system's init? (If I look at
/proc/self/mountinfo, there's no entry for the initramfs anymore
after just doing what my PoC does.)

Regards,
Christian



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 21:35 +0100, Christian Seiler wrote:
[...]
> 
> Well, just for the heck of it I wrote a braindead-simple initrd
> implementation in just 300 LOC:
> 
> https://gist.github.com/chris-se/e0fbc073fcbd9ac2d7ae
[...]

Neat.  It should probably implement 'ro', 'rw' and 'rootdelay', though.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 19:43 +0100, Marco d'Itri wrote:
> On Jan 03, Andreas Henriksson  wrote:
> 
> > First, it would be nice to have a preinst check if the system has any
> > running services that uses ProtectSystem and offer a choice to stop
> > (and restart) them in case having them running is really a problem...
> I will think about this, I need to check how much complex it would be to 
> lift the related debconf code from the glibc package. :-)
> But this is only useful if every service can actually be restarted: as 
> soon as any of them (what about dbus?) cannot be stopped and restarted 
> later without causing troubles then this would be a waste of time.
> 
> > Second, as you already noted in your TODO I think it's probably a
> > good idea to turn the initramfs preinst check into a debconf prompt.
> I just have not found yet how to determine if the system was booted 
> using an initramfs or not. Does anybody have any hint?
[...]

test -d /run/initramfs

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 18:10 +0100, Iustin Pop wrote:
> On 2016-01-03 17:03:02, Simon McVittie wrote:
> > […] For
> > instance, /bin -> /usr/bin is needed because otherwise #!/bin/sh would
> > stop working, […]
> 
> This brings to mind—I wonder if the performance impact of having /bin/sh
> be read through two indirections (/bin/sh → /usr/bin/sh →
> /usr/bin/{dash, bash, etc.}) is non-zero and if it could be reliably
> measured.

Trivial compared to the cost of running a single line of a shell
script.

Ben.

> This is not an argument against UsrMerge, I'm very much for it; I'm just
> curious.
> 
> iustin
-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 23:42 +0100, Simon Richter wrote:
> Hi,
> 
> On 03.01.2016 23:32, Ben Hutchings wrote:
> 
> > An initramfs is mandatory if using the standard kernel packages, as I
> > think most people do.
> 
> Not on embedded systems.

Sure.

> While we do have the advantage there that the
> people putting the system together are generally clueful, it should
> still remain possible to boot a system consisting of just a kernel and
> the output of debootstrap -- everything else is just needlessly
> complicated IMO.

What makes you think that doesn't or won't work?  I seem to recall that
working for me, using jessie on an ARM SoC development board.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: Re: support for merged /usr in Debian

[Eric Valette]

I'm confused why you think anything will break.  There would obviously be
symlinks, so anything that's currently in /bin will continue to work if
invoked with an absolute /bin path.


I consider linking across file system a very bad practice because if 
/usr gets errors all the symlinks may be broken until it is repaired. 
Currently / is 600 MB on my system /usr being over 12 GB nad merging 
will just make it bigger. So there are more chances to break /usr and it 
takes more time to repair it.


Aditionaly remount ro on error is hardly set in fstab and fs_passno is 
usually 2 not 1...


With what you propose, any /usr fs bug will end up inserting a CD or USB 
key for repair... For sure it is not the case at the moment!



Just as another reality check: I believe Red Hat has already done this.
Lots of people use Red Hat and derivatives, and there doesn't seem to have
been that much breakage.


Red hat is mainly for servers nowadays with paying support. So I wonder 
if you are really doing meaningful comparison. Likewise Systemd was used 
by various distrib before debian, and it did break a lot at the 
beginning when upgrading existing debian installs, most init script are 
still unconverted and they cannot really accommodate the /etc/defaut/pkg 
configurable options...


-- eric

Re: support for merged /usr in Debian

[Christian Seiler]

On 01/03/2016 04:23 AM, Martinx - ジェームズ wrote:
> This "UseMerge" just brings more insanity into Debian. What is wrong
> with you guys?! For God's sake...

Even if you disagree with something, please don't call it insanity.

> It violates the FHS 2.3 standards.

Sure, if you mean the sentence that "/" needs to have the tools to
mount the other filesystems. I can even one-up you: the same sentence
is present in FHS 3.0, so it also violates that. So what?

 - Debian already violates FHS when it comes to {,/usr}/lib: the
   Multi-Arch specification of Debian, which adds things like
   /usr/lib/x86_64-linux-gnu doesn't conform to FHS. But there is a
   very good technical reason why to use /usr/lib/ instead
   of /usr/lib{32,64}. (which are in FHS!)

   The question is always: is there a good technical reason for the
   violation? Or is it just arbitrary. In case of the UsrMerge there
   are good technical reasons for doing so (and I'm sorry, even if you
   disagree with the assessment that they outweigh the disatvantages,
   you should acknowledge that there _are_ advantages).

   So: a game storing its data in /highscores is a FHS violation where
   it makes sense to say "please don't do that", because there is NO
   advantage over "/var/lib/$game/highscores" or similar. But UsrMerge
   doesn't fall into that category.

   (Btw. if you consider what I've said in [1], you could argue that
   Debian already doesn't properly follow the FHS in all cases,
   because /usr on NFS in Jessie *only* works if mounted in initrd at
   the moment.)

 - Don't forget why the FHS was created: to unify Linux distributions.
   Most major Linux distributions (except Debian and Ubuntu) have
   switched to a shared /usr, for quite a while actually.

   So it could actually be the case that the next version of the FHS
   standard will say that /lib etc. should be symlinks to /usr/lib and
   that everything should be in /usr. If that were to be the case in
   FHS 4.0 or so - would you then say "oh, now Debian must switch and
   follow that"?

Sorry, but bringing the FHS into this is just not a good argument.

[1] https://lists.debian.org/debian-devel/2016/01/msg00089.html

> Also, if you guys (Debian) reeeally wants to move everything to
> /usr, so, there is no reason for any symbolic links at the root file
> system.

In the (very) long run - maybe. But in the short term, they have to be
there. Just two very trivial examples (there are *many* more):

 - /bin/sh is hard-coded in SO many places (and required by I don't
   know how many different standards to be present)
 - The kernel tries to execute /sbin/init by default if init= is not
   specified.

You can't just get rid of those.

> Again, If Debian wants to do that, then, do NOT create any symlink on
> the root file system. Do not bring CentOS shit to our shiny
> distribution.
> 
> Those symlinks:
> 
> "/bin -> /usr/bin"
> "/sbin -> /usr/sbin"
> "/lib -> /usr/lib"
> "/lib64 -> /usr/lib64"
> 
> Looks very, very unprofessional.

"Looks unprofessional"? Do you really want a response here?

Btw. symlinks in / are nothing new: on a default installation the
kernel packages symlink the most recent initrd.img and vmlinuz to /.

I've seen a lot of systems where network filesystems mounted via
autofs were symlinked to /, i.e. /var/autofs is a directory handled
by the autofs daemon and there are symlinks such as
/shared => /var/autofs/shared
/backup => /var/autofs/backup
in the root filesystem so people can easily access those things.

> It looks like a huge workaround,
> created to deal with something that you broke for no reason.

On Jessie:

$ find /usr/bin -type l | xargs readlink | grep -E '^/s?bin'
/sbin/on_ac_power
/bin/lesskey
/bin/nano
/bin/less
/bin/dumpkeys
/bin/lesspipe
/sbin/xtables-multi
/bin/chacl
/bin/lesspipe
/bin/lessecho
/bin/which
/bin/getfacl
/bin/setfacl
/bin/loadkeys
/bin/touch

15 compatibility symlinks on my system in /usr/bin because they were
moved to / from /usr.

After /usr-merge there would be 4 links in / for /lib, /lib64, /bin
and /sbin to their counterparts in /usr.

 => 15 - 4 = 11 fewer compatibility symlinks if /usr is merged.

So UsrMerge actually reduces the number of compatibility
"workarounds" as you call them.

Regards,
Christian



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 13:10 -0800, Russ Allbery wrote:
> Eric Valette  writes:
> 
> > But could you elaborate a bit on "mounting /usr early, something we
> > *already do*" if you do not implicitly refer to initramfs solution.
> 
> There are other people who know a lot more about the machinery here than I
> do, but my understanding is that we currently use a variety of hacks and
> workarounds to try to make sure that /usr is mounted very early in the
> boot process, before the various things that expect it to be there start
> breaking.  Moving it into the initramfs stage will make this more
> reliable, but I believe we're already taking special measures to try to
> mount /usr early, as opposed to treating it like any other file system.
[...]

Roger Leigh prepared the implementation of this for initramfs-tools; I
finished it off and was able to get it into jessie just before release.
Currently it's conditional on using systemd because initscripts didn't 
get updated to work with /usr already
mounted.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Simon Richter]

Hi,

On 03.01.2016 23:32, Ben Hutchings wrote:

> An initramfs is mandatory if using the standard kernel packages, as I
> think most people do.

Not on embedded systems. While we do have the advantage there that the
people putting the system together are generally clueful, it should
still remain possible to boot a system consisting of just a kernel and
the output of debootstrap -- everything else is just needlessly
complicated IMO.

   Simon




signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 21:21 +0100, Simon Richter wrote:
> Hi,
> 
> On 03.01.2016 19:15, Marco d'Itri wrote:
[...]
> > Anyway, if you think that the merged /usr scheme is about systemd then 
> > you are automatically disqualified from taking part in this discussion 
> > because you are not understanding the basic underlying issues.
> 
> It is very strongly related -- before systemd, we just did not start any
> services before all file systems were checked and present.
[...]

Any service that used to start in run levels 2-5 was not started before
filesystems were mounted.  systemd hasn't changed that behaviour,
except that it actually checks that filesystem mounting succeeded.

Services that start in run level S under sysvinit certainly can start
before all filesystems are mounted.  That's, er, how filesystems other
than root get mounted.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 21:34 +0100, Eric Valette wrote:
> > > > This is not true: you just need to use an initramfs.
> > > Ok, so it should warn that this setup will soon require to use an 
> > > initramfs.
> > It is the Debian default, there is no need to do this.
> 
> Being debian installer default does not mean any debian users
>   1) really has any benefit of using it as explained,
>   2) nor use it after initial installation,

An initramfs is mandatory if using the standard kernel packages, as I
think most people do.

> Others have expressed better than I did that initramfs by itself is evil 
> and just does what / was supposed to do originally:
[...]

'Evil', really?  Sure, it takes more time to load and more time to run.
The implementation isn't exactly pretty.  But many systems do require
it because the kernel doesn't have the code to set up every different
kind of device that the root filesystem might live on.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Christian Seiler]

On 01/02/2016 06:42 PM, Geert Stappers wrote:
> To me is this "TheUsrMerge" something like among 
> * "it is hard too to explain to have /sbin/fsck and not /usr/sbin/fsck"
> * "there was a question about /bin/kill and /usr/bin/killall being 
> inconsequent"
> * "we could not agree if p{erl,ython,hp} should in /bin or in /usr/bin"
> * "when calling `foo` we rely on $PATH. To avoid $PATH we call `/bin/foo`,
>to have a reason to rant it should be /usr/bin/foo"
> * "reverting a historic decission is much better then accepting a historic 
> decission"
> * "just because we can"
> * "others doing also"
> 
> In other words: I don't yet see a _good_ reason for "TheUsrMerge".

You need to separate two things:

 1. The recognition that the separation of / and /usr is fundamentally
broken. Not because it's technically impossible to have it
(obviously you could do that technically), but because it adds
such a large amount of complexity to the system that it becomes
increasingly unrealistic to maintain it properly.

See my other email in this thread for a detailed rationale as to
why it doesn't work in practice:

https://lists.debian.org/debian-devel/2016/01/msg00089.html

 2. The UsrMerge proposal itself.

Once you agree with point 1, that having / mounted without /usr just
doesn't work any more in practice, then the UsrMerge proposal becomes
interesting:

 - If we assume that /usr is always available anyway (and it should be,
   regardless of whether UsrMerge is done or not, see my other email),
   we don't need to separate binaries anymore, they can now reside in
   just one place.

 - Package maintenance burden is reduced in the long term (we just
   install everything to /usr, which both dh(7) and cdbs know how to
   pass the proper parameters for that automatically for most build
   systems, no special-casing for specific packages)

 - Increased consistency within Debian: all binaries can be found in
   /usr/bin or /usr/sbin. People don't need to guess anymore.

 - Increased consistency with other distributions: a lot of other
   distributions have already done this, so this would unify paths
   for binaries even more.
   (Note that this is not a _reason_ for doing so, because "other
   people did so" is never a good reason, but if you have other good
   reasons for a change in the first place, then the fact that other
   people have already done so does speak slightly in favor of it.)

 - Snapshotting all of the OS binaries without touching anything
   else (configuration, state) is now very simple if you just
   separate /usr. (Previously that was much more complicated.)

 - Maintaining shared operating system binaries is now much easier
   (think e.g. provisioning of containers, thin clients with / on
   ramdisk but /usr on network filesystem etc.).

 - If other software is improved a bit, things like stateless systems
   become really easily possible. (e.g. initrd creates an empty rootfs
   with just the necessary directories/symlinks in it, mounts /usr
   from somewhere readonly and then just starts init)

If you don't accept point 1, then the advantages of these things
become more arguable - but I have yet to see a good response to the
observation that mount /usr from / is currently broken - other than
"well, in principle it could work technically", which I think is
trivially true, but completely useless in practice, and "works for
me currently", which completely glosses over the amount of effort
that will have to be put into maintaining that state.

> And I think that it is ill-named,
> it should be named "PutAllExecutablesInRootFS"  :-)

No. It means that / doesn't carry any executables anymore and all
executables are in /usr now. /lib, /bin and /sbin would be symlinks
to their equivalents in /usr. And /usr doesn't have to be the rootfs.

> That makes it harder to split out executables in different file systems.
> (having a mechanisme for executables in different places,
> makes it easy to add another place)

No, that's simply not true. If you have software that is started by
users (and not during boot), such as GUI applications, then it's
perfectly fine to have that on something that's mounted later by the
system. Always has been. Doesn't change here. If you have software that
is started at boot, but in late boot (standard system service), ordered
after remote-fs.target (or $remote_fs and in a standard runlevel in
sysvinit/LSB) and it doesn't hook into udev, and it doesn't hook into
other things that are executed before all filesystems are mounted,
that's also perfectly fine and that was never the issue.

The problem isn't that the vast majority of software couldn't live in
/usr without that being mounted until a bit later at boot, if you think
of something like GNOME's nautilus, the problem is that it's really
hard to make sure that the right set of software is in / and that
software doesn't suddenly behave in subtly different ways if /usr is
present or not.

So unless you

Re: support for merged /usr in Debian

[Christian Seiler]

On 01/03/2016 11:06 PM, Henrique de Moraes Holschuh wrote:
> On Sun, Jan 3, 2016, at 19:59, Christian Seiler wrote:
>> On 01/03/2016 10:53 PM, Henrique de Moraes Holschuh wrote:
>>> On Sun, Jan 3, 2016, at 18:35, Christian Seiler wrote:
 My question would be: would those people here who have separate /usr
 and aren't using initrd be willing to put up with something like that?
>>>
>>> I don't know if they will, but the list of potential users for your hack
>>> is a bit larger:
>>>
>>> As far as I know, the Linux kernel cannot really support a
>>> firmware-update/override "early initramfs" without a "boot" initramfs,
>>> which is a really annoying limitation for anyone that doesn't want a
>>> "boot" initramfs for whatever reason.
>>
>> Oh, I didn't know that. I've always used initrds on my systems for the
>> last - I don't know - 15 years or so.
>>
>> Anyway, if I hear from enough people who'd actually use my piece of
>> code, I'll make a real project out of it and package it for Debian.
>> (Do you count yourself as one of those?)
> 
> No, but I've had to help a couple people that needed a microcode update
> and couldn't make it work because they didn't use an initramfs.

Good to know.

> Not all of them were Debian users, though.

While I don't have plans on packaging that for anything other than Debian:

Well, it's free software. :-)

Regards,
Christian



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Henrique de Moraes Holschuh]

On Sun, Jan 3, 2016, at 19:59, Christian Seiler wrote:
> On 01/03/2016 10:53 PM, Henrique de Moraes Holschuh wrote:
> > On Sun, Jan 3, 2016, at 18:35, Christian Seiler wrote:
> >> My question would be: would those people here who have separate /usr
> >> and aren't using initrd be willing to put up with something like that?
> > 
> > I don't know if they will, but the list of potential users for your hack
> > is a bit larger:
> > 
> > As far as I know, the Linux kernel cannot really support a
> > firmware-update/override "early initramfs" without a "boot" initramfs,
> > which is a really annoying limitation for anyone that doesn't want a
> > "boot" initramfs for whatever reason.
> 
> Oh, I didn't know that. I've always used initrds on my systems for the
> last - I don't know - 15 years or so.
> 
> Anyway, if I hear from enough people who'd actually use my piece of
> code, I'll make a real project out of it and package it for Debian.
> (Do you count yourself as one of those?)

No, but I've had to help a couple people that needed a microcode update
and couldn't make it work because they didn't use an initramfs.  Not all
of them were Debian users, though.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique de Moraes Holschuh 

Re: support for merged /usr in Debian

[Christian Seiler]

On 01/03/2016 10:53 PM, Henrique de Moraes Holschuh wrote:
> On Sun, Jan 3, 2016, at 18:35, Christian Seiler wrote:
>> My question would be: would those people here who have separate /usr
>> and aren't using initrd be willing to put up with something like that?
> 
> I don't know if they will, but the list of potential users for your hack
> is a bit larger:
> 
> As far as I know, the Linux kernel cannot really support a
> firmware-update/override "early initramfs" without a "boot" initramfs,
> which is a really annoying limitation for anyone that doesn't want a
> "boot" initramfs for whatever reason.

Oh, I didn't know that. I've always used initrds on my systems for the
last - I don't know - 15 years or so.

Anyway, if I hear from enough people who'd actually use my piece of
code, I'll make a real project out of it and package it for Debian.
(Do you count yourself as one of those?)

Regards,
Christian



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Henrique de Moraes Holschuh]

On Sun, Jan 3, 2016, at 18:35, Christian Seiler wrote:

> Well, just for the heck of it I wrote a braindead-simple initrd
> implementation in just 300 LOC:
> 
> https://gist.github.com/chris-se/e0fbc073fcbd9ac2d7ae

Oho!  Cool!

> My question would be: would those people here who have separate /usr
> and aren't using initrd be willing to put up with something like that?

I don't know if they will, but the list of potential users for your hack
is a bit larger:

As far as I know, the Linux kernel cannot really support a
firmware-update/override "early initramfs" without a "boot" initramfs,
which is a really annoying limitation for anyone that doesn't want a
"boot" initramfs for whatever reason.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique de Moraes Holschuh 

Re: support for merged /usr in Debian

[Vincent Bernat]

 ❦  3 janvier 2016 22:30 +0100, Eric Valette  :

>> The problem of getting /usr mounted before things start using it is mostly
>> separate from the question of whether we want to merge it with /bin and
>> /lib.  This thread is more about the latter than the former.  (Obviously,
>> mounting /usr early is a *prerequisite* for merging /bin and /lib with
>> /usr, which is why it comes up.)
>
> Well, I do not see the need for that (merging /bin and /lib with
> /usr) once /usr is mounted early enough as PATH and LD_LIBRARY_PATH
> should allow to find binaries and libraries wherever they are at the
> end when both fs are mounted...

If this was the case, we wouldn't have all those symbolic links in
/usr/bin. Many things are just execl().
-- 
Each module should do one thing well.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Henrique de Moraes Holschuh]

On Sun, Jan 3, 2016, at 16:43, Marco d'Itri wrote:
> I just have not found yet how to determine if the system was booted 
> using an initramfs or not. Does anybody have any hint?

I don't think there is any source for that information that one can
really trust ATM, AFAIK.

We could add it in userspace, but this is the kind of thing much better
done in the kernel (and it is trivial to do it there, too). That way, it
would not get in the way of creative ideas like some of the stuff posted
elsewhere in this thread, such as that 6KiB binary initramfs "init" stub
to keep working.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique de Moraes Holschuh 

Re: support for merged /usr in Debian

[Russ Allbery]

Eric Valette  writes:

> If it is just to close once for all the right location philosophical
> debate, I would say it will be over priced: changing executable PATH
> will just breaks million scripts people have written themselves on top
> of original debian install to maintain their system or do whatever their
> system is really needed for. System admins do like using absolute path
> for security reasons...

I'm confused why you think anything will break.  There would obviously be
symlinks, so anything that's currently in /bin will continue to work if
invoked with an absolute /bin path.

Just as another reality check: I believe Red Hat has already done this.
Lots of people use Red Hat and derivatives, and there doesn't seem to have
been that much breakage.

-- 
Russ Allbery (r...@debian.org)   

Re: support for merged /usr in Debian

[Iustin Pop]

On 2016-01-03 22:22:16, Tom H wrote:
> On Sun, Jan 3, 2016 at 6:17 PM, Iustin Pop  wrote:
> > On 2016-01-03 12:59:01, Tom H wrote:
> >>
> >> I don't like usr-merge because it goes against my historical
> >> expectation that "/{,s}bin" be separate from their /usr namesakes and
> >> contain binaries required for boot.
> >
> > OK, so adjust your historical expectation. It's not a technical issue,
> > it's simply a matter of expectations, which have no reason to stay the
> > same for ever.
> 
> Did you read the next para of my email?!

Yes. Instead of trying to think of initramfs as the new / or any other
way of thinking that keeps the distinction, I'm suggesting to simply
drop this distinction/expectation. You didn't give any reasons except
historical expectation, and I don't think that's a needed one (given
that you can't properly make a distinction between what-is-boot and what
isn't).

regards,
iustin


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Eric Valette]

On 03/01/2016 22:10, Russ Allbery wrote:

Eric Valette  writes:




The problem of getting /usr mounted before things start using it is mostly
separate from the question of whether we want to merge it with /bin and
/lib.  This thread is more about the latter than the former.  (Obviously,
mounting /usr early is a *prerequisite* for merging /bin and /lib with
/usr, which is why it comes up.)


Well, I do not see the need for that (merging /bin and /lib with
/usr) once /usr is mounted early enough as PATH and LD_LIBRARY_PATH 
should allow to find binaries and libraries wherever they are at the end 
when both fs are mounted...


If it is just to close once for all the right location philosophical 
debate, I would say it will be over priced: changing executable PATH 
will just breaks million scripts people have written themselves on top 
of original debian install to maintain their system or do whatever their 
system is really needed for. System admins do like using absolute path 
for security reasons...



-- eric

Re: support for merged /usr in Debian

[Russ Allbery]

Florian Lohoff  writes:
> On Sun, Jan 03, 2016 at 10:14:14AM -0800, Russ Allbery wrote:

>> No.  Debian has basically given up on this; there are way too many
>> packages and way too much stuff that would have to be moved to /bin and
>> /lib in order to preserve the traditional semantics that allow /usr to
>> be mounted very late.  I've poked a bit at this in the past, and the
>> amount of work that would be required is daunting, and benefits only a
>> few highly unusual edge cases.

> From my 25 year Unix experience i dont like the usr merge.

>From my 22 years of UNIX experience, I don't understand why you care.  :)
Maybe something in those other three years was particularly enlightening?

I do understand why people working in the embedded space care about some
unusual mount orderings, file system separations, and very light cores,
and I hope that we can accomodate and support all of their use cases
inside Debian.  I think that's the most productive part of this thread.

But I don't get why people who are using non-embedded UNIX systems
particularly care.  If you've used UNIX for a long time, you've seen
binaries in all sorts of bizarre and irritating locations.  This is minor
compared to the organizational differences between various commercial
UNIXes back in the day.  I don't particularly care if we do it, and don't
particularly care if we don't do it, since neither the current state nor
the desired destination state seems likely to impact my life in any way.
If people feel like it would provide lots of benefits to them, I'm dubious
that anything is actually benefiting from the split /bin and /lib at this
point, and it would slightly reduce the cognitive load for everyone
maintaining packages if no one has to think about whether something should
go in /bin again.

If we had a system where /bin plus /lib plus /etc was a fully-working base
UNIX system that was entirely functional without /usr, and then /usr was a
properly-segregated set of supplemental programs, I can vaguely see why
maintaining that state could be useful.  But, reality check: we *don't*,
and we haven't for *years* (if, in fact, we ever did).  I'm sure that you
could cobble together some special cases that mostly worked this way, but
Debian as a project has never gotten this right.

The reason why is pretty obvious if you look at it as a software
engineering problem.  You're asking all package maintainers (a huge set of
mostly-uncoordinated developers) to maintain a very subtle and careful
distinction (it's very inobvious to most people whether some specific
program should go into /bin or some specific library should go into /lib)
involving complex transitive dependencies (libraries you would never
expect to be part of "early boot" end up being runtime dependencies of
some tool that some init script needs to use).  And, worse, this very
subtle and complex distinction is *never tested* and *almost untestable*
since even getting mistakes to actually fail requires setting up a very
unusual partitioning and mounting scheme and then exercising it in unusual
ways.

This is, to say the least, not a recipe for success.

Insofar as I have any agenda in this thread, it's that, with my Policy
editor hat on, I don't think we should be claiming we're maintaining
invariants that we actually aren't maintaining, and that, given a
realistic look at our coordination and testing capabilities, we aren't
going to maintain.  That's just bad for everyone.  People take us at our
word and then get upset when things don't work, some maintainers will put
in tons of painful work to try to support this because we say that we
support it, and that work will be mostly useless because other people
won't even be aware of the problem and will accidentally break it.

> As you sum up very nicely and i agree on is that Debian has given up on
> being slim at this point.

Maintaining a separate /bin and /lib has nothing to do with being slim.
Truly.  They are completely unrelated.

Being slim is about keeping the essential and required set small and
tight.  It makes no difference whatsoever to disk or memory utilization
whether things are installed in /usr/lib or in /lib.

> There is no such thing as a single user mode boot with only the rootfs
> anymore.

Yes, there is.  The rootfs just includes /usr.

> PS: And i hate giving up on technical issues.

That's the whole thing, though.  Maintaining a meaningful /bin and /lib
vs. /usr split is not primarily a technical issue.  It's a coordination
issue.  The technical work for a single package is painful only because
moving things is really painful.  The problem is more that it affects
thousands of packages maintained by numerous different maintainers who are
never testing that configuration and may not even be aware that it exists.

You could possibly make it a technical issue if you developed
comprehensive test suites for the invariants that you feel we should
maintain that could report those results to the maintainers, similar 

Re: Debian name change

[Troy Benjegerdes]

We may never know what Ian the man actually wrote, or what
sort of influence he was under when he wrote it.


The only way we begin to know is if Ian's family, the executor
of his estate, the police, and administrators of Twitter come
forward and tell the story of what they have observed.


One might argue that in some way, all of us that use, contribute
and maintain the debian project are all executors of Ian's digital
estate. It is up to us what we choose to carry forward. It is 
probably Ian's greatest legacy that we have the debian social
contract, and if some group wants to change the name, they have
the freedom to do so. I wish the supporters of the Euphemia
project success in their endeavor, for whether it succeeds or
fails, we can learn from it.

Personally, I like the name Debian, so I'll keep using it, and
I want to focus on the amazing gifts this project has given to
the entire world, for no charge. This project is far bigger 
than the tragic end of one man's life.

If you must search for a foe to fight, an enemy to vanquish,
and evil to conqueor, then I would call you to action to look
very closely at who might profit from sowing dissent and 
confusion among this project. There are people who might stand
to make a lot of money if the project that bears Ian's name
falters.

But don't waste our time or energy on such things. Take it to
another forum, another mailing list, or start your own derivative
project, for you have the freedom do to so, and what you are
fighting desires to strip it from you.


On Thu, Dec 31, 2015 at 11:42:54PM +, envite wrote:
> Please do not get ANY conclusions before we _know_ for real if he wrote any 
> of that, or not, and if he was on his own mind or not when writing.
> 
> Just wait.
> 
> er Envite
> 
> 
> Enviado de Samsung Mobile
> 
>  Mensaje original 
> De: Jihadi Jermane  
> Fecha:31/12/2015  19:12  (GMT+00:00) 
> Para: debian-devel@lists.debian.org 
> Asunto: Debian name change 
> 
> As you may know you posted on you blog about the recent passing of our 
> beloved Ian Murdock, by completely slandering his name, and the debian image. 
> As a black American I find it thoroughly racist that you would even insist 
> that changing the name to Euphemia is a good option. Euphemia is in no way 
> relevant to the debian project no matter what excuse you come up with. Please 
> do not try to include blacks in a project that has nothing to do with them. 
> 
> 
> 
> 
>                                                                               
>                                     Signed 
>                                                                               
>                                          Jermane King.

-- 

Troy Benjegerdes 'da hozer'  ho...@hozed.org
7 elements  earth::water::air::fire::mind::spirit::soulgrid.coop

  Never pick a fight with someone who buys ink by the barrel,
 nor try buy a hacker who makes money by the megahash

Re: support for merged /usr in Debian

[Bernd Zeimetz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 01/03/2016 07:15 PM, Marco d'Itri wrote:
> On Jan 03, Simon Richter  wrote:
> 
>>> "I have always done this in a different way" is not a valid use
>>> case, sorry.
>> "Compatibility" is a very valid use case. Debian is famous for
>> backwards compatibility and trouble-free upgrades.
> Requiring to use an initramfs in some configurations does not break
>  compatibility nor causes significant troubles on upgrades.

Please don't forget that Debian supports a lot of unusual architectures
which come with various limitations. I know at least one platform where
adding an initramfs is something between difficult and impossible.
And I'm sure there is a lot of other (embedded) hardware out there
which needs a reliable upgrade path.


- -- 
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJWiZFtAAoJEOs2Fxpv+UNfchcP/3mVX8Tc/uuij5iETWaRuWsV
9twLLcYkk0MLMrYw4NmXM4hwCYmPaStBdf8RE6xrvU0EWrm2WKF+NtDJIvkgUc+/
6mMzqZbgb5I3cmFxkFS4R3jeFK9d5f0Y+UBMG8+ROHJAsY+AXYEDO/c2wkt1hcE7
BB6H8mbTpbp8C/Jq4ELt1oKs2JR72Pmvdcq8p5j+PSup/HinOmeCSPIn4Nr4dQ2B
ljFCD8Gx0HZYVR5ROAEvBA96B6EcZIBhv2yIURCr9eFz5r9u67Y/K5MbzM8tzBee
AovcOKcJ8+7vSVlL8W9p9yw7KXebMhnPP00FRflaV0a88Jh2lgIn4xzqRNRv9evv
EgDjmYhMDvgJJogVJ/Md08JbnYz4Rl2zkbHdJoYwvoeizx3NmPH7zDzlHefGhS+L
xsJIaUpugDbrQWTFQnCTZV8DiNU7blM8+GJs5aHs7COG3anDAmJzQk4ecH0q99ZJ
qppiCeyZUWOv/+/wtJik9aQhfJgeYPPRnd7gdPapACSE91+H8sVjFgiUjFP2muXD
3tVyMPBSBHHVZMmXkHW86G3ZlAJ0o8XuBvLrwOxCW+I6apeUzpG6ayX7E/gqbyHo
XI2qieiOfaBUYnwTEgAIQjC+ZhAPDZOjzvDWAd1CYC306UxFn5wQeVCCt0wRJbFk
TusJAPcdZymSvayikCCG
=sD5e
-END PGP SIGNATURE-

Re: support for merged /usr in Debian

[Tom H]

On Sun, Jan 3, 2016 at 6:17 PM, Iustin Pop  wrote:
> On 2016-01-03 12:59:01, Tom H wrote:
>>
>> I don't like usr-merge because it goes against my historical
>> expectation that "/{,s}bin" be separate from their /usr namesakes and
>> contain binaries required for boot.
>
> OK, so adjust your historical expectation. It's not a technical issue,
> it's simply a matter of expectations, which have no reason to stay the
> same for ever.

Did you read the next para of my email?!

Re: support for merged /usr in Debian

[Florian Lohoff]

On Sun, Jan 03, 2016 at 10:14:14AM -0800, Russ Allbery wrote:
> Daniel Reurich  writes:
> 
> > Ah, so it's actually packages that don't separate device configuration
> > logic from the application or daemons properly that has caused the
> > brokenness.  Can we identify and fix the packages that cause this issue?
> 
> No.  Debian has basically given up on this; there are way too many
> packages and way too much stuff that would have to be moved to /bin and
> /lib in order to preserve the traditional semantics that allow /usr to be
> mounted very late.  I've poked a bit at this in the past, and the amount
> of work that would be required is daunting, and benefits only a few highly
> unusual edge cases.

From my 25 year Unix experience i dont like the usr merge. As you sum
up very nicely and i agree on is that Debian has given up on being
slim at this point. There is no such thing as a single user mode boot
with only the rootfs anymore. 


For me it boils down to - "We have parallel startup so we need
all the little bells and whistles a lot faster and earlier in boot
than we used to have them. Sequential filesystem processing is
slow - lets reduce the number of filesystems we need to mount."


I was a opponent of the systemd issue because when it came to solve
the parallel/dependency based booting those were moot in my eyes. 
Nobody cares about booting anymore. On your Desktop/Notebook you
do it probably 10 times a year because of reliable suspend/resume
possibilities. Typical Virtual Machine setups even only do
it ONCE in their whole lifecycle - who cares if it takes some seconds
more? 


Making boot THE reason for the UsrMerge is simply overrating it.

Flo
PS: And i hate giving up on technical issues.
-- 
Florian Lohoff f...@zz.de
  We need to self-defend - GnuPG/PGP enable your email today!


signature.asc
Description: Digital signature

Re: support for merged /usr in Debian

[Russ Allbery]

Eric Valette  writes:

> But could you elaborate a bit on "mounting /usr early, something we
> *already do*" if you do not implicitly refer to initramfs solution.

There are other people who know a lot more about the machinery here than I
do, but my understanding is that we currently use a variety of hacks and
workarounds to try to make sure that /usr is mounted very early in the
boot process, before the various things that expect it to be there start
breaking.  Moving it into the initramfs stage will make this more
reliable, but I believe we're already taking special measures to try to
mount /usr early, as opposed to treating it like any other file system.

> I understand that for complex fs setup (lvm/raid/mounted networked
> fs...)  this may require to much work to be done realistically at first
> install (like the solutions above by the way), but for most common cases
> (two ext4fs on a same disk, or separate disk but same drivers sets), I
> do not get what really prevents to mount /usr really early and propose
> that as a viable alternative to the mess (3) we have been proposed.

The problem of getting /usr mounted before things start using it is mostly
separate from the question of whether we want to merge it with /bin and
/lib.  This thread is more about the latter than the former.  (Obviously,
mounting /usr early is a *prerequisite* for merging /bin and /lib with
/usr, which is why it comes up.)

-- 
Russ Allbery (r...@debian.org)   

Re: Re: support for merged /usr in Debian

[Eric Valette]

Note that mounting /usr early, something we *already do*, is separate from
actually merging /usr with /bin and /lib.  Once you mount /usr early, it's
rather less important whether you actually merge the file systems.  While
it does let you do some interesting things, I see it as more of a cleanup.


Thanks for summarizing things so clearly : the root of the problem is 
indeed to mount /usr early enough at init time. Then :
	1) initramfs can be seen as a well known solution that I personally 
dislike as explained elsewhere in this thread. But I do agree upon the 
ROI of using such a mechanism provided it is not imposed,
	2) no separate partition for / and /usr being another one. I will 
probably end up changing the way I install new debian system to this one 
now that even SSD disks are so huge compared to system requirements,

3) merged /usr proposal being a kind of UFO...

But could you elaborate a bit on "mounting /usr early, something we 
*already do*" if you do not implicitly refer to initramfs solution.


I understand that for complex fs setup (lvm/raid/mounted networked 
fs...) this may require to much work to be done realistically at first 
install (like the solutions above by the way), but for most common cases 
(two ext4fs on a same disk, or separate disk but same drivers sets), I 
do not get what really prevents to mount /usr really early and propose 
that as a viable alternative to the mess (3) we have been proposed.


--eric

Re: support for merged /usr in Debian

[Christian Seiler]

On 01/03/2016 12:03 PM, Daniel Reurich wrote:
> On 03/01/16 23:18, Ben Hutchings wrote:
> 
>>> Then why is it that since the introduction of systemd is having /usr on
>>> a separate partition suddenly considered evil and systemd complains
>>> loudly about it.  It always has worked and does work fine for me with
>>> sysvinit
>>
>> systemd complains if it has to mount /usr itself.  This is because
>> mounting of local filesystems generally depends on various services and
>> udev hooks that may themselves depend on /usr.  This is also true when
>> using sysvinit.  Some services go through contortions to work before
>> /usr is mounted; others may behave subtly differently if it's a
>> separate filesystem.  We really need a simplified code path for
>> mounting /usr early on, and that is now provided by the initramfs.
> 
> Ah, so it's actually packages that don't separate device configuration
> logic from the application or daemons properly that has caused the
> brokenness.  Can we identify and fix the packages that cause this issue?

Because that doesn't work in practice. Case in point: #777547:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777547

I found this bug nearly a year ago (Jessie was not released yet!) while
testing something completely differently (I'm not using that setup, I
just stumbled upon it): you're using sysvinit as init system (or
potentially systemd without initrd, but I haven't tested that), and
have /usr on NFS, but / locally. This is _exactly_ the kind of corner
case that I would have expected from reading this thread that people
are interested in - but apparently that is not the case because not
only do I appear to be the only one to have found this problem (present
since August 2nd, 2014!) but nobody thought this was important enough
to fix, even after the release of Jessie.

And this is just one example. There are lots of other packages that
could in principle be required for mounting /usr in some setups but
aren't installed in the rootfs directly - even in Jessie today.

> Is this also something to do with the inherent lack of determinism and
> parallelization in systemd's startup as well (just out of interest)?

Well, it's not systemd's fault here, but it's not completely unrelated
to parallelization.

If you look at it historically, the earliest systemd versions assumed
that / and /usr could be separate and that was what the systemd
developers assumed would remain the case - up to systemd v19. The
warning that /usr is separate was introduced in the commit
80758717a6359cbe6048f43a17c2b53a3ca8c2fa between v19 and v20. The
problem was that setups started to break, especially on other distros
that adopted systemd earlier. Why? Because systemd will start
everything in parallel that isn't explicitly ordered (which I would
argue is a good thing, because it makes boot faster) - but a lot of
ordering on sysvinit systems, especially w.r.t. the presence of /usr,
was implicit, not explicit. Debian was historically generally better at
ensuring explicitness even before the systemd adoption (via proper LSB
headers) as compared to other distributions, but it wasn't perfect
either (far from it).

Of course, at this point, you can argue that you _should_ fix all the
broken packages (systemd itself at that point _wasn't_ broken on split
/usr systems, just the things it executed - and as Debian Jessie shows
systemd itself still works on those types of systems), but:

 - there are a *lot* of packages that are affected by this, because
   you have to consider all the library dependencies of a package. This
   cascades down: say you have software A required in early boot that
   requires library B. Library B in version 42 now has a new dependency
   on library C - library C will now need to be moved to / from /usr;
   even though the maintainer of library C might never have expected
   there to be an early-boot use case for that library. This is a huge
   maintenance burden on a lot of people.

   It's not just filesystems either - anything that installs a udev
   rule is affected; (udev itself is fine w/o /usr mounted) because if
   there's hardware already plugged in that requires udev rule
   processing with things in /usr, this currently either fails
   completely, OR there needs to be some hack that defers the execution
   of this until later in the boot process (OR the package needs to go
   in /).

 - the rootfs will be quite large, making the split from /usr less and
   less attractive: all library dependencies of all filesystems which
   could *possibly* carry /usr need to be in / now - and I've seen FUSE
   filsystems written in C++ and Python. Ok, not all FUSE filesystems
   are useful for /usr, but there is probably some filesystem out there
   that requires either of those dependencies.

   The problem here is that the cost for the larger rootfs is not
   incurred only for those people using weird filesystems for their
   setups, but for everybody else that has software installed that u

Re: Re: support for merged /usr in Debian

[Eric Valette]

>This is not true: you just need to use an initramfs.
Ok, so it should warn that this setup will soon require to use an initramfs.

It is the Debian default, there is no need to do this.


Being debian installer default does not mean any debian users
1) really has any benefit of using it as explained,
2) nor use it after initial installation,

Others have expressed better than I did that initramfs by itself is evil 
and just does what / was supposed to do originally:
	1) because of binary firmware blob mainly allthough as explained this 
can be solved by putting the blob in kernel itself or using modules if 
things can be started late enough
	2) because systemd requires many things that are traditionally located 
in /usr


I for sure hate to have things twice on a system and make sure they are 
kept coherent by black magic (and using busybox rather than original 
utilities makes it even more evil...)




Same for your proposal : nothing really sound except systemd wants it...

Again, this is not related to systemd.
I am not interested in continuing this discussion with people who are
not understanding the basic facts.


Sure. And I'm not interested to discuss with someone that feels so 
superior to normal human either and does not answer various points that 
have been detailed in this thread.


I'm still 100% against your proposal.

-- eric

Re: support for merged /usr in Debian

[Simon Richter]

Hi,

On 03.01.2016 19:15, Marco d'Itri wrote:

>> However, this also means that systemd can never fully replace sysvinit,
>> except on desktops, laptops and servers that follow a standard layout.

> I see no reason why this would be true.

Because the alternative is to bloat the scope of the systemd project so
it covers all previously existing use cases, which is a monumental
undertaking and would require design decisions that run counter to the
project's goals.

> Anyway, if you think that the merged /usr scheme is about systemd then 
> you are automatically disqualified from taking part in this discussion 
> because you are not understanding the basic underlying issues.

It is very strongly related -- before systemd, we just did not start any
services before all file systems were checked and present. This means a
boot time penalty, but avoids problems with undeclared dependencies.

Dependency based booting has always been a difficult problem, and that
the systemd maintainers are often pushing to narrow the scope rather
than bloat systemd is the right choice from their perspective, but this
should not narrow the scope for Debian as a whole.

The current debate is about cementing a previously introduced
restriction in order to get rid of a workaround that was necessary in
the first place because design decisions taken earlier meant losing
support for mounting /usr through the regular channels.

That workaround is not too painful to keep around regardless of whether
the system is booting into systemd or sysvinit later on, because any
system with enough resources to use a standard initramfs is likely to
have a local /usr anyway, and any smaller devices cannot run systemd
sensibly anyway. So no one cared.

Changing the file system hierarchy to accommodate one project can be
debated, but it should not be assumed that just because new
installations by end users look a particular way, all other deployments
necessarily do so as well.

I'm in no particular hurry to merge / and /usr, as all it does is get
rid of a workaround that has already been deployed to millions of
systems with very little adverse effects. It would not even allow
booting systemd based systems without an initramfs, even if there was
sufficient interest in doing so, so frankly I don't see the point.

All I see is a major headache as I need to build upgrade paths for
embedded devices.

   Simon



signature.asc
Description: OpenPGP digital signature

Re: [Fwd: Re: [DNG] FW: support for merged /usr in Debian]

[Marc Haber]

On Sun, 3 Jan 2016 16:08:53 +0100, Iustin Pop 
wrote:
>Wow, here I was thinking this would be some informed oppinion, but:
>
>"Oh, there are tools with which you can periscope into
>initramfs, but have you ever really looked at everything in an
>initramfs."
>
>Wait, what? Yes, I have many times unpacked the simple cpio archive that
>an iniramfs is, and I have looked at its entire contents. It's not black
>magic.

I fully agree. Initramfs is not quite easy to understand, but it is
possible to understand, it is easy to debug, and I have been able in
the past to find the root cause for any initramfs issues I had in the
past.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: support for merged /usr in Debian

[Marc Haber]

On Sun, 3 Jan 2016 13:44:36 +0100, Simon Richter 
wrote:
>"Compatibility" is a very valid use case. Debian is famous for backwards
>compatibility and trouble-free upgrades.

We have been able to do this in the past, yes.

I doubt we will be able to keep this promise in the future.

But I have been wrong in the past, the last time when I said that the
wheezy => jessie update would be painful. It isn't.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: support for merged /usr in Debian

[Paul Gevers]

Hi Marco,

On 03-01-16 19:43, Marco d'Itri wrote:
> Also, I am not really up to date on the modern debconf best practices: 
> can somebody recommend a package that I can use foor an example of 
> a debconf question that can abort the package installation?

Not sure if it is really what you are looking for, but the error
handling in dbconfig-common allows you to abort the package installation.

Paul



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Marco d'Itri]

On Jan 03, Andreas Henriksson  wrote:

> First, it would be nice to have a preinst check if the system has any
> running services that uses ProtectSystem and offer a choice to stop
> (and restart) them in case having them running is really a problem...
I will think about this, I need to check how much complex it would be to 
lift the related debconf code from the glibc package. :-)
But this is only useful if every service can actually be restarted: as 
soon as any of them (what about dbus?) cannot be stopped and restarted 
later without causing troubles then this would be a waste of time.

> Second, as you already noted in your TODO I think it's probably a
> good idea to turn the initramfs preinst check into a debconf prompt.
I just have not found yet how to determine if the system was booted 
using an initramfs or not. Does anybody have any hint?
Also, I am not really up to date on the modern debconf best practices: 
can somebody recommend a package that I can use foor an example of 
a debconf question that can abort the package installation?

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Marco d'Itri]

On Jan 03, Simon Richter  wrote:

> > "I have always done this in a different way" is not a valid use case, 
> > sorry.
> "Compatibility" is a very valid use case. Debian is famous for backwards
> compatibility and trouble-free upgrades.
Requiring to use an initramfs in some configurations does not break 
compatibility nor causes significant troubles on upgrades.

> However, this also means that systemd can never fully replace sysvinit,
> except on desktops, laptops and servers that follow a standard layout.
I see no reason why this would be true.
Anyway, if you think that the merged /usr scheme is about systemd then 
you are automatically disqualified from taking part in this discussion 
because you are not understanding the basic underlying issues.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Russ Allbery]

Daniel Reurich  writes:

> Ah, so it's actually packages that don't separate device configuration
> logic from the application or daemons properly that has caused the
> brokenness.  Can we identify and fix the packages that cause this issue?

No.  Debian has basically given up on this; there are way too many
packages and way too much stuff that would have to be moved to /bin and
/lib in order to preserve the traditional semantics that allow /usr to be
mounted very late.  I've poked a bit at this in the past, and the amount
of work that would be required is daunting, and benefits only a few highly
unusual edge cases.

I'm pretty sure we have not released a version of Debian that did this
fully properly for at least four or five years.  There have been various
unexpected dependencies on /usr in at least some packages for at least
that long (such as sysvinit scripts that use binaries in /usr without
depending on $remote_fs).  I'm pretty sure of this because I looked at
trying to write Lintian checks for it, once upon a time.

Instead, everyone just takes the approach of mounting /usr very early so
that stuff in /usr can be used during boot.  I suspect you have not even
noticed that this has happened, because it basically just works for the
average system, although my understanding is that it's not particularly
clean (yet) under the hood.

> Is this also something to do with the inherent lack of determinism and
> parallelization in systemd's startup as well (just out of interest)?

No, it has nothing to do with systemd parallelism.

It does have a lot to do with udev (specifically with udev rules that want
to run commands), but udev is definitely not the only culprit in wanting
/usr to be available.

> Having to do all this contortionism and particularly mounting /usr in
> initramfs sounds like a horribly insane way to do things!

And yet, it works, and it means that we don't have to try to harass a
thousand package maintainers into doing essentially untestable busy-work
to try to move things around between /usr, /bin, and /lib to support a
tiny handful of systems for which other approaches are available.

The cost/benefit tradeoff is, dare I say, *extremely* obvious.

Note that mounting /usr early, something we *already do*, is separate from
actually merging /usr with /bin and /lib.  Once you mount /usr early, it's
rather less important whether you actually merge the file systems.  While
it does let you do some interesting things, I see it as more of a cleanup.

-- 
Russ Allbery (r...@debian.org)   

Re: [Fwd: Re: [DNG] FW: support for merged /usr in Debian]

[Andrey Rahmatullin]

On Sun, Jan 03, 2016 at 03:59:37PM +0100, Svante Signell wrote:
> This message was not intended to be sent to a debian-* mailing list by
> the author. However, since it is (in my opinion) of large interest I
> got the permission to forward it to debian-devel. Hopefully, also some
> of the debian-devel subscribers will appreciate it too.
It's a good thing that people who do appreciate such examples of hate
speech and conspiracy theories are mostly at the place the message was
intended to be sent to. I personally would like for such messages to not
be forwarded here.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Iustin Pop]

On 2016-01-03 12:59:01, Tom H wrote:
> On Sat, Jan 2, 2016 at 6:42 PM, Geert Stappers  wrote:
> I don't like usr-merge because it goes against my historical
> expectation that "/{,s}bin" be separate from their /usr namesakes and
> contain binaries required for boot.

OK, so adjust your historical expectation. It's not a technical issue,
it's simply a matter of expectations, which have no reason to stay the
same for ever.

regards,
iustin


signature.asc
Description: PGP signature

Re: [Fwd: Re: [DNG] FW: support for merged /usr in Debian]

[Guido Günther]

Hi,
On Sun, Jan 03, 2016 at 04:08:53PM +0100, Iustin Pop wrote:
> On 2016-01-03 15:59:37, Svante Signell wrote:
> > Hi, 
> > 
> > This message was not intended to be sent to a debian-* mailing list by
> > the author. However, since it is (in my opinion) of large interest I
> > got the permission to forward it to debian-devel. Hopefully, also some
> > of the debian-devel subscribers will appreciate it too.
> 
> Wow, here I was thinking this would be some informed oppinion, but:
> 
> "Oh, there are tools with which you can periscope into
> initramfs, but have you ever really looked at everything in an
> initramfs."
> 
> Wait, what? Yes, I have many times unpacked the simple cpio archive that
> an iniramfs is, and I have looked at its entire contents. It's not black
> magic.

It is a bit more complicated if you use microcode updates:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=807759

but the tools to handle this are all there and in Debian.
Cheers,
 -- Guido

   

Re: support for merged /usr in Debian

[Iustin Pop]

On 2016-01-03 17:03:02, Simon McVittie wrote:
> […] For
> instance, /bin -> /usr/bin is needed because otherwise #!/bin/sh would
> stop working, […]

This brings to mind—I wonder if the performance impact of having /bin/sh
be read through two indirections (/bin/sh → /usr/bin/sh →
/usr/bin/{dash, bash, etc.}) is non-zero and if it could be reliably
measured.

This is not an argument against UsrMerge, I'm very much for it; I'm just
curious.

iustin


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Simon McVittie]

On 03/01/16 13:23, Eduard Bloch wrote:
> I didn't say or assume that. It doesn't matter how you see it, in the
> end all /usr/*foo* would be come /*foo*, sooner or later.

Not in the UsrMerge design. Because UsrMerge deprecates
/{bin,lib,sbin...} and treats /usr/* as the canonical location for those
directories, the only symlinks in the root directory are those that are
necessary for compatibility with current or historical practice. For
instance, /bin -> /usr/bin is needed because otherwise #!/bin/sh would
stop working, and /lib64 -> /usr/lib64 is needed because the x86-64 ABI
says the runtime linker is canonically named
/lib64/ld-linux-x86-64.so.2, but /share -> /usr/share is not needed (and
so is not created) because there is no existing software in Debian that
refers to paths in /share.

S

Re: support for merged /usr in Debian

[Josh Triplett]

Marc Haber wrote:
> See the discussion about EnvironmentFile which "should never have been
> implemented" and "should be removed" on systemd-devel which resulted
> in one side of the discussion being put on moderation (guess which one
> got silenced).

The ones posting messages like
http://lists.freedesktop.org/archives/systemd-devel/2015-December/035323.html
and
http://lists.freedesktop.org/archives/systemd-devel/2015-December/035354.html
, I would guess.

I see a thread full of people having a reasonable discussion, including
many people expressing various opinions about EnvironmentFile and
related configuration, but most of them managed to do so without
devolving into hatefulness.  Do you expect to go to the development
mailing list of a project and post rants about that project and its
developers, and retain the ability to post there?  And then come here
and rant about it further and expect sympathy?

I would also note that the mail informing you of the moderation of your
mails, at
http://lists.freedesktop.org/archives/systemd-devel/2015-December/035343.html
, explicitly said your technical mails would be passed onto the list,
just filtering out the hateful mails.  That seems rather generous.

Re: support for merged /usr in Debian

[Andreas Henriksson]

Hello Marco d'Itri.

On Thu, Dec 31, 2015 at 01:51:45AM +0100, Marco d'Itri wrote:
> We have a reasonably tested usrmerge package which can be used to 
[...]
> I welcome your comments, but if you have any questions then please read 
> the FAQ first:
> https://wiki.debian.org/UsrMerge
> https://anonscm.debian.org/cgit/users/md/usrmerge.git/plain/debian/README.Debian
> 
> If you want to help then please have a look at the open bugs linked on 
> wiki.d.o about lintian and policy work.

Thanks for your work on this! (even though I'd personally would be happy if
this went one step further with "TheOneAndOnlyTrueBin" from Arch Linux fame so
I'd never have to waste time being involved in another discussion about where
the right place to put an executable is. As I learned at debconf, "technical
solution to social issues"...)

I just found some time to read through the sources of usrmerge and
ended up with a couple of comments that I'm not sure is worth
implementing but mentioning them anyway in case anyone is interested...

First, it would be nice to have a preinst check if the system has any
running services that uses ProtectSystem and offer a choice to stop
(and restart) them in case having them running is really a problem...
(Similar to how glibc upgrades (used to?) offer restarting of services.)
While codesearch tells me these seem to be pretty uncommon in the
archive right now, AFAIK it's pretty much a recommended setting
for services that works with it enabled and with the ever growing
number of services in the archive I can only guess that also enabling
this setting will become more common An easier way for the users to
deal with this might be worth some developer time.

Second, as you already noted in your TODO I think it's probably a
good idea to turn the initramfs preinst check into a debconf prompt.
And to answer your followup question in TODO, no I do not think
you should always prompt on /usr on separate fs. Don't bother the
user unless needed. This way the messages should properly show
up in all package management frontends and can also be translated.

Last but not least thanks for so clearly documenting the tradeoffs you
have had to make and mark the XXX spot where to look closer. I don't
feel like I'm in a position to give any recommendations on which side of
the remaining tradeoffs to lean towards. (I'm also very curious that
noone from the critical crowd has picked up on your documented issues,
but OTOH it seems many didn't even read the FAQ)

Regards,
Andreas Henriksson

Re: [Fwd: Re: [DNG] FW: support for merged /usr in Debian]

[Iustin Pop]

On 2016-01-03 15:59:37, Svante Signell wrote:
> Hi, 
> 
> This message was not intended to be sent to a debian-* mailing list by
> the author. However, since it is (in my opinion) of large interest I
> got the permission to forward it to debian-devel. Hopefully, also some
> of the debian-devel subscribers will appreciate it too.

Wow, here I was thinking this would be some informed oppinion, but:

"Oh, there are tools with which you can periscope into
initramfs, but have you ever really looked at everything in an
initramfs."

Wait, what? Yes, I have many times unpacked the simple cpio archive that
an iniramfs is, and I have looked at its entire contents. It's not black
magic.

This is just more FUD being spread by somebody who doesn't want to
change their ways, at all.

iustin, who wasted 1 minute on this


signature.asc
Description: PGP signature

[Fwd: Re: [DNG] FW: support for merged /usr in Debian]

[Svante Signell]

Hi, 

This message was not intended to be sent to a debian-* mailing list by
the author. However, since it is (in my opinion) of large interest I
got the permission to forward it to debian-devel. Hopefully, also some
of the debian-devel subscribers will appreciate it too.

Thanks!
 Forwarded Message 
From: Steve Litt 
To: d...@lists.dyne.org
Subject: Re: [DNG] FW: support for merged /usr in Debian
Date: Fri, 1 Jan 2016 12:07:34 -0500

On Fri, 01 Jan 2016 15:45:49 +0100
Micky Del Favero  wrote:

> If I remember well Solaris has /bin linked to /usr/bin since many
> years, so linking /bin to /usr/bin is not a poetteringisation, or
> almost it's not an original idea of poettering.
> 
> Ciao, Micky

Well, OK, if we're really going to discuss this... 

This *is* poetterization, regardless of what Sun or anyone else did
before. It's supported by Freedesktop.org, and I think everyone here
can agree that anything Freedesktop supports is anti-init choice,
anti-simplicity, anti-modularity, and pro-systemd.

http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/

Those of you who have tried to lay down an alternate init system, to
replace systemd, without the aid of a package manager, will probably
agree with me that the toughest obstacle isn't udev, it isn't dbus,
it's initramfs. I looked up the word "black box" in the dictionary last
night, and they had a picture of initramfs.

Hey, I'll be the first to admit that sometimes you need an initramfs.
Maybe you have LUKS plus LVM plus software raid. Merge or not, you'll
need to compile yourself one heck of a kernel to avoid needing
initramfs. But for the very prevalent use case of Ext4, no raid, no
LVM, no LUKS, no silly merge, and a few partitions, initramfs is as
useful as udders on a snake. I mean seriously, in such a use case, you
forego initramfs: boot to the root partition, run /sbin/mount -a, and
bang, you have all resources available to you. But nooo.

Initramfs does have one big benefit for the Poetterists: It provides a
dark, safe place for them to start up their megacomplexities and call
it magic. Oh, there are tools with which you can periscope into
initramfs, but have you ever really looked at everything in an
initramfs? It's a jungle in there. Just right for the Poetterists to
incubate their plague.

Now, the Freedesktop.Org to which I referred earlier in this email has
a link to the following Rob Landley page explaining what they call the
"historical reasons" for separate directories:

http://lists.busybox.net/pipermail/busybox/2010-December/074114.html

Note that Landley's #1 reason for merging is the existance of
initramfs. Now I'm not stupid enough to call the author of Busybox a
Poetterist. He wrote this in 2010, before anyone really knew the
Napoleonistic aspirations of systemd, back in the days when a complex
and opaque "early boot" wasn't a big deal.

But now it's 5 years later, and that early boot black box is exactly
where the Poetterists fester most virulently.

In summary, if you accept the merge and /usr on a separate partition,
you need initramfs. And if you have initramfs, you've just made it
three times as hard to lay down Runit or Epoch or s6 or Suckless Init
plus daemontools-encore plus Littkit.

We all have to pick our own battles, and I'm not sure how much effort
I'd make to roll back the merge. It may indeed be a good thing that
only 3 changes are required to patch up Devuan for the merge. But make
no mistake about it: regardless of its initial motivation, today the
merge's primary beneficiaries are Red Hat and their proxies,
Freedesktop.org and Lennart Poettering.

SteveT

Steve Litt 
November 2015 featured book: Troubleshooting Techniques
 of the Successful Technologist
http://www.troubleshooters.com/techniques

Bug#809735: ITP: foma -- Xerox-compatible finite-state compiler

[Timo Jyrinki]

Package: wnpp
Severity: wishlist
Owner: Timo Jyrinki 

* Package name: foma
  Version : 0.9.18
  Upstream Author : Mans Hulden 
* URL : https://bitbucket.org/mhulden/foma/
http://code.google.com/p/foma/
* License : GPL
  Programming Lang: C
  Description : Xerox-compatible finite-state compiler

 Foma is a compiler, programming language, and C library for constructing
 finite-state automata and transducers for various uses. It has specific
 support for many natural language processing applications such as producing
 morphological analyzers. Although NLP applications are probably the main use
 of foma, it is sufficiently generic to use for a large number of purposes.

The package would be required by suomi-malaga / voikko-fi 2.0, which 
drops the malaga backend dependency and switches to using VFST backend that 
requires foma to build.

I'm currently planning to maintain it on my own as part of the other 
Voikko packages. Voikko upstream however helps here, as also the initial 
packaging is based on upstream's packaging for Debian.

I don't need a sponsor, but a co-maintainer would always be welcome.
Especially for libvoikko's optional HFST requirements for other languages 
than Finnish.

Re: support for merged /usr in Debian

[Eduard Bloch]

Hallo,
* Samuel Thibault [Sun, Jan 03 2016, 02:15:31PM]:
> Eduard Bloch, on Sun 03 Jan 2016 14:06:22 +0100, wrote:
> > OTOH UsrMerge breaks some habits, i.e. going to /share/doc/foo feels
> 
> UsrMerge proposed by Marco is the converse.
> 
> It is *not* putting a /usr -> / symlink.

I didn't say or assume that. It doesn't matter how you see it, in the
end all /usr/*foo* would be come /*foo*, sooner or later.

Regards,
Eduard.

-- 
error compiling committee.c: too many arguments to function

Re: support for merged /usr in Debian

[Eduard Bloch]

Hallo,
* Daniel Reurich [Mon, Jan 04 2016, 12:46:46AM]:
> On 04/01/16 00:25, Marco d'Itri wrote:
> > On Jan 03, Eric Valette  wrote:
> > 
> >> The debian installer should first loudly warn that having a separated / and
> >> /usr may break things in the future but not forbid it. With that in place,
> > This is not true: you just need to use an initramfs.
> > "I have always done this in a different way" is not a valid use case, 
> > sorry.
> 
> Really, just because the use case isn't one you understand doesn't mean
> it should be entirely ignored.  Sure there are corner cases that will

What's the use case here? Something with /usr as read-only remote-fs
with a local mutable / ? Why cannot you just identify the few spots
which need to be writable and do some bind-mounting on top of them?

> always be fragile, but that's not the issue here.  The issue at hand is
> the provision of tooling that fundamentally violates the FHS by changing
> the location of where binaries and libraries are located, and only
> because of sloppy workmanship that has allowed packages in that will
> break systems if we don't adopt these massive changes.

For me, sloppy workmanship is building a special setup in a crude way
while not being able to adapt it to a slightly modified base
configuration, considering all tools we have nowadays. It's not like we
just entered Y2K...

Regards,
Eduard.

-- 
Schalten Sie auch morgen wieder irc.openprojects.net ein,
wenn es heißt: boot-floppies - Freud und Leid des Debuggings.
-- Karsten Merker

Re: support for merged /usr in Debian

[Samuel Thibault]

Eduard Bloch, on Sun 03 Jan 2016 14:06:22 +0100, wrote:
> OTOH UsrMerge breaks some habits, i.e. going to /share/doc/foo feels

UsrMerge proposed by Marco is the converse.

It is *not* putting a /usr -> / symlink.

Samuel

Re: support for merged /usr in Debian

[Eduard Bloch]

Hallo,
* Michael Biebl [Sun, Jan 03 2016, 03:22:03AM]:
> Am 02.01.2016 um 22:08 schrieb Marc Haber:
> > On Sat, 2 Jan 2016 18:42:14 +0100, Geert Stappers
> >  wrote:
> >> To me is this "TheUsrMerge" something like among 
> >> * "it is hard too to explain to have /sbin/fsck and not /usr/sbin/fsck"
> >> * "there was a question about /bin/kill and /usr/bin/killall being 
> >> inconsequent"
> >> * "we could not agree if p{erl,ython,hp} should in /bin or in /usr/bin"
> >> * "when calling `foo` we rely on $PATH. To avoid $PATH we call `/bin/foo`,
> >>   to have a reason to rant it should be /usr/bin/foo"
> >> * "reverting a historic decission is much better then accepting a historic 
> >> decission"
> >> * "just because we can"
> >> * "others doing also"
> > 
> > Amen.
> > 
> >> In other words: I don't yet see a _good_ reason for "TheUsrMerge".
> > 
> > Seconded.
> > 
> 
> I see a lot of good reasons for "TheUsrMerge".
> So thanks for pushing for this.

And I, for one, don't really care. The last time I configured something
dependent on the usr split was about a decade ago, and even then it was
a kludge, for a problem solvable with a couple of minutes of extra
work.

The only thing {/bin,/sbin,/lib} are good for ATM is being and
information source, to see what's the really important s..t without
using extra tools.

OTOH UsrMerge breaks some habits, i.e. going to /share/doc/foo feels
weird. Which is a human issue, already spreading in the crowd of
anti-systemd lunatics.

Regards,
Eduard.

-- 
Sicherheitshinweis:
Achtung: Die grundlegenden Partikel dieses Gerätes werden durch eine
Gluonenkraft zusammengehalten, über die wenig bekannt ist. Ihr
fortdauernder Zusammenhalt unterliegt daher nicht der Garantie.


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Simon Richter]

Hi,

On 03.01.2016 12:25, Marco d'Itri wrote:

> "I have always done this in a different way" is not a valid use case, 
> sorry.

"Compatibility" is a very valid use case. Debian is famous for backwards
compatibility and trouble-free upgrades.

I can certainly see the allure of a tightly integrated system and that
certain trade-offs have to be made for that. It is also good that the
systemd maintainers are drawing clear boundaries of the scope of their
project, which is sensible project management.

However, this also means that systemd can never fully replace sysvinit,
except on desktops, laptops and servers that follow a standard layout.

However, we do have a huge installation base outside of that. In most of
my embedded systems projects, Debian has been the starting point for the
customized installation, simply because before jessie, you could simply
call "debootstrap --foreign" and get a working root filesystem, where
all you need to add is a kernel that matches your hardware.

This mechanism is already broken, and Debian's reputation has suffered
for it. We can bootstrap an oldstable system and upgrade from there, but
that is a cumbersome hack.

Honestly, I am starting to believe that forking is a good choice, into a
Debian that provides excellent supports for PCs, and an "universal
operating system", because we obviously cannot have both in the same
project.

   Simon



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Marco d'Itri]

On Jan 03, Eric Valette  wrote:

> >This is not true: you just need to use an initramfs.
> Ok, so it should warn that this setup will soon require to use an initramfs.
It is the Debian default, there is no need to do this.

> Same for your proposal : nothing really sound except systemd wants it...
Again, this is not related to systemd.
I am not interested in continuing this discussion with people who are 
not understanding the basic facts.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Tom H]

On Sun, Jan 3, 2016 at 11:04 AM, Daniel Reurich  wrote:
> On 03/01/16 22:33, Philip Hands wrote:
>> Daniel Reurich  writes:


>>> Because systemd doesn't work without /usr on the root partition isn't a
>>> good reason either.
>>
>> You are right ... it is a poor reason, because it is pure fantasy.
>
> Then why is it that since the introduction of systemd is having /usr on
> a separate partition suddenly considered evil and systemd complains
> loudly about it. It always has worked and does work fine for me with
> sysvinit

It's udev (already pre-systemd) that needs "/usr" early, with sysvinit
or any other init. Whether this need breaks split-usr can be a matter
of opinion because there'll always be some people saying "it's OK for
me." But does this mean that it's OK for all? It also doesn't mean
that udev doesn't have to go through some workarounds in order to make
it work.


>>> That just means systemd is broken by design and needs to be fixed.
>>
>> If what you claimed were true, then I'd agree with you, but since all
>> the systems I've upgraded to systemd have a separate /usr, and are
>> working without any issue whatsoever, this drivel can be safely ignored.
>
> Then what's the problem and why are we even having this conversation
> about merged /usr???

usr-merge isn't correcting the broken-ness of split-usr. Mounting a
separate "/usr" via the initramfs is the fix to that problem.

Re: support for merged /usr in Debian

[Tom H]

On Sat, Jan 2, 2016 at 6:42 PM, Geert Stappers  wrote:
> On Fri, Jan 01, 2016 at 03:53:03PM +0100, Marco d'Itri wrote:
>> On Jan 01, Ian Jackson wrote:


>> With a merged /usr you would be able to serve the whole OS over NFS (and
>> even share it among multiple systems without the constant threat of
>> having / and /usr diverge) and only configuration + data from the local
>> disk, which makes this kind of setup much more useful.
>
> "whole OS over NFS" is the same as "whole OS on /usr"
>
> A design with "whole OS over NFS" breaks the good pratice of having
> A design with "whole OS on /usr" breaks the good pratice of having
> tools like /bin/mount and /sbin/ifconfig available when /usr is unavailable.

I don't like usr-merge because it goes against my historical
expectation that "/{,s}bin" be separate from their /usr namesakes and
contain binaries required for boot.

On RHEL and RHEL clone systems that I manage, I reconcile this
expectation with RHEL's usr-merge by thinking of the initramfs as the
new "/" (it's easier in RHEL than in Debian because the initramfs has
"real" binaries rather than their busybox equivalents).


> To me is this "TheUsrMerge" something like among
> * "it is hard too to explain to have /sbin/fsck and not /usr/sbin/fsck"
> * "there was a question about /bin/kill and /usr/bin/killall being 
> inconsequent"

Your mentioning of "kill" reminds me of a samba-devel@ thread where
three RH samba developers insisted that the upstream-supplied systemd
units should use "/usr/bin/kill" in spite of the fact that a fellow
samba developer pointed out that the "/bin" symlink obviated the need
for "/usr" in "ExecStart" and that this change voided the systemd
intent to have distros use upstream units as-is.

So, even though Marco's proposing a non-compulsory usr-merge, I expect
that it'll become at some point the unique and default Debian setup,
if only to reduce the delta with the various RH-dominated upstreams.


> * "we could not agree if p{erl,ython,hp} should in /bin or in /usr/bin"
> * "when calling `foo` we rely on $PATH. To avoid $PATH we call `/bin/foo`,
> to have a reason to rant it should be /usr/bin/foo"
> * "reverting a historic decission is much better then accepting a historic 
> decission"
> * "just because we can"
> * "others doing also"
>
> In other words: I don't yet see a _good_ reason for "TheUsrMerge".
>
> And I think that it is ill-named,
> it should be named "PutAllExecutablesInRootFS" :-)
>
> And the "PutAllExecutablesInRootFS" is
> in fact "put all executables in a single file system".

"put all executables in a single file system" is the main/only benefit
of usr-merge.


To Simon Richter:

You mentioned earlier in the thread that you had resource-constrained
systems possibly incompatible with an initramfs.

Gentoo mandated at least two years ago that systems with a separate
"/usr" have an initramfs. A (I suspect unhappy) developer created a
busybox-based binary called "ginit" to mount "/usr" early without an
initramfs. IIRC, you have to set "init=/ginit" at the kernel cmdline
to use it - and it hands over boot to "/sbin/init" once it's done its
job.

Should you be interested, the source must be
"sys-apps/busybox/files/ginit.c" in the Gentoo distfiles tree.

Re: Re: support for merged /usr in Debian

[Eric Valette]

The debian installer should first loudly warn that having a separated / and
/usr may break things in the future but not forbid it. With that in place,

This is not true: you just need to use an initramfs.


Ok, so it should warn that this setup will soon require to use an 
initramfs. I just pointed out that this implies a slower boot and 
consumes times nearly at each package install (plus initrd tools seldom 
guess right everything that should be in the initrd to function).



"I have always done this in a different way" is not a valid use case,
sorry.


Same for your proposal : nothing really sound except systemd wants it...


And again, this is not related to supporting a merged /usr scheme.


I think I gave you other reasons for not using an initrd that you have 
not answered (other did also in the discussion) and other gave you 
reasons to have /usr separated and not mounted at boot time that you 
have not answered either.


-- eric

Re: support for merged /usr in Debian

[Daniel Reurich]

On 04/01/16 00:25, Marco d'Itri wrote:
> On Jan 03, Eric Valette  wrote:
> 
>> The debian installer should first loudly warn that having a separated / and
>> /usr may break things in the future but not forbid it. With that in place,
> This is not true: you just need to use an initramfs.
> "I have always done this in a different way" is not a valid use case, 
> sorry.

Really, just because the use case isn't one you understand doesn't mean
it should be entirely ignored.  Sure there are corner cases that will
always be fragile, but that's not the issue here.  The issue at hand is
the provision of tooling that fundamentally violates the FHS by changing
the location of where binaries and libraries are located, and only
because of sloppy workmanship that has allowed packages in that will
break systems if we don't adopt these massive changes.

> 
> And again, this is not related to supporting a merged /usr scheme.
> 
That doesn't make sense given the subject you started this whole thread
with...


-- 
Daniel Reurich
Centurion Computer Technology (2005) Ltd.
021 797 722



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Vincent Bernat]

 ❦  4 janvier 2016 00:03 +1300, Daniel Reurich  :

>>> Then why is it that since the introduction of systemd is having /usr on
>>> a separate partition suddenly considered evil and systemd complains
>>> loudly about it.  It always has worked and does work fine for me with
>>> sysvinit
>> 
>> systemd complains if it has to mount /usr itself.  This is because
>> mounting of local filesystems generally depends on various services and
>> udev hooks that may themselves depend on /usr.  This is also true when
>> using sysvinit.  Some services go through contortions to work before
>> /usr is mounted; others may behave subtly differently if it's a
>> separate filesystem.  We really need a simplified code path for
>> mounting /usr early on, and that is now provided by the initramfs.
>
> Ah, so it's actually packages that don't separate device configuration
> logic from the application or daemons properly that has caused the
> brokenness.  Can we identify and fix the packages that cause this
> issue?

The merge is also here to not have to fix those subtle issues and reduce
the amount of code needed to boot (more reliability, less
bugs). Manpower is limited.

The amount of brokenness of a separate /usr can also be witnessed by the
number of executables that had to migrate from /usr/bin to /bin. 115 on
my system (counting the symlinks).

Also, what other distributions do matter. Hard-coded path for
executables are a reality and once other distributions will have merged
/usr, people may start invoking /bin/wget instead of /usr/bin/wget and
this won't work on Debian. For the end user, this will just be something
that works on Fedora, but not on Debian and the blame will be put on
Debian.
-- 
10.0 times 0.1 is hardly ever 1.0.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Marco d'Itri]

On Jan 03, Eric Valette  wrote:

> The debian installer should first loudly warn that having a separated / and
> /usr may break things in the future but not forbid it. With that in place,
This is not true: you just need to use an initramfs.
"I have always done this in a different way" is not a valid use case, 
sorry.

And again, this is not related to supporting a merged /usr scheme.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Daniel Reurich]

On 03/01/16 23:18, Ben Hutchings wrote:

>> Then why is it that since the introduction of systemd is having /usr on
>> a separate partition suddenly considered evil and systemd complains
>> loudly about it.  It always has worked and does work fine for me with
>> sysvinit
> 
> systemd complains if it has to mount /usr itself.  This is because
> mounting of local filesystems generally depends on various services and
> udev hooks that may themselves depend on /usr.  This is also true when
> using sysvinit.  Some services go through contortions to work before
> /usr is mounted; others may behave subtly differently if it's a
> separate filesystem.  We really need a simplified code path for
> mounting /usr early on, and that is now provided by the initramfs.

Ah, so it's actually packages that don't separate device configuration
logic from the application or daemons properly that has caused the
brokenness.  Can we identify and fix the packages that cause this issue?

Is this also something to do with the inherent lack of determinism and
parallelization in systemd's startup as well (just out of interest)?

Having to do all this contortionism and particularly mounting /usr in
initramfs sounds like a horribly insane way to do things!
> 
 That just means systemd is broken by design and needs to be fixed.
>>>
>>> If what you claimed were true, then I'd agree with you, but since all
>>> the systems I've upgraded to systemd have a separate /usr, and are
>>> working without any issue whatsoever, this drivel can be safely ignored.
>>>
>> Then what's the problem and why are we even having this conversation
>> about merged /usr???
> 
> As I understand it, merging /bin and /lib into /usr allows us to
> support certain uses cases more easily - e.g. package installation in a
> filesystem transaction, or sharing an NFS /usr filesystem.
> 
Package installation in a filesystem transaction?  Is that in relation
to btrfs and filesystem versioning to support easy rollbacks?  For that
you'd need to keep everything except /home and /srv to be able to easily
and consistently roll back to a particular state.

TBH the only argument I've heard so far that holds some water is the nfs
shared /usr and even that seems largely a convenience issue rather then
a hard problem.


-- 
Daniel Reurich
Centurion Computer Technology (2005) Ltd.
021 797 722



signature.asc
Description: OpenPGP digital signature

Re: Re: support for merged /usr in Debian

[Andrey Rahmatullin]

On Sun, Jan 03, 2016 at 11:40:34AM +0100, Eric Valette wrote:
> The debian installer should first loudly warn that having a separated / and
> /usr may break things in the future
ITYM "already breaks things"

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Re: support for merged /usr in Debian

[Eric Valette]

What is the "upgrade path" for an older system that has /usr split
off? Will it just stop being bootable after upgrading?

It just needs to use an initramfs.
A standalone /usr without an initramfs IS ALREADY NOT SUPPORTED by
systemd.
This is not relevant for merged /usr.



What is the "upgrade path" for an older system that has /usr split
off? Will it just stop being bootable after upgrading?

It just needs to use an initramfs.
A standalone /usr without an initramfs IS ALREADY NOT SUPPORTED by
systemd.
This is not relevant for merged /usr.



It is not supported *BUT* just works as expected on more than 20 
machines I have access to,  especially now that the debian systemd 
maintainers make sure systemd do not depend on libraries installed in 
/usr... Systemd shall make linux boot easier, faster, secure, not render 
things impossible because it does not know how to handle it because it 
wants to do too much.


For years (actually 20), I've been installing Linux on various machine 
using various distro to end up using exclusively debian because, I was 
able to tune the system as I want and not because other have decided I 
MUST do it in whatever way.


For machines I really manage, I have a separate /usr, no initrd and self 
build kernel:
   	1) Why should I build modules when I always load them and 
hardware almost never changes. Its slower to start, does not bring any 
benefit,
	2) I can even add binary blob in the kernel nowadays, so I do not need 
an initrd at all and the packages needed to build them are not even 
installed

3) There is not always places to copy / in /usr
	4) My grub setup does not enable initrd, nor UUID for rot file system 
because without initrd you just can't,
	5) I have machine with a byte of bad memory, and specific grub setup. I 
do not think you will ever be able to guess the correct  grub config
	6) I have machine that do network boots, and do not mount /usr the 
usual way,...


The debian installer should first loudly warn that having a separated / 
and /usr may break things in the future but not forbid it. With that in 
place, new debian installations that have no good reason for different 
filesystem for / and /usr will be installed the preferred new way. Old 
folks that have been doing this for years on hundred of machines will 
eventually learn new tricks. People that are using this setup for 
reasons will still be able to do so.


So please, do not make that kind of proposition that you will never be 
able to transition gracefully...


-- eric

Re: support for merged /usr in Debian

[Marc Haber]

On Sun, 03 Jan 2016 23:04:20 +1300, Daniel Reurich
 wrote:
>On 03/01/16 22:33, Philip Hands wrote:
>> Daniel Reurich  writes:
>> 
>> ...
>>> Because systemd doesn't work without /usr on the root partition isn't a
>>> good reason either.
>> 
>> You are right ... it is a poor reason, because it is pure fantasy.
>
>Then why is it that since the introduction of systemd is having /usr on
>a separate partition suddenly considered evil and systemd complains
>loudly about it.  It always has worked and does work fine for me with
>sysvinit

Within the systemd world, one can never know whether something that
has been working previously will not be removed at a whim in the next
major release.

See the discussion about EnvironmentFile which "should never have been
implemented" and "should be removed" on systemd-devel which resulted
in one side of the discussion being put on moderation (guess which one
got silenced).

systemd needs to be forked by some party with enough personpower that
removed features can get patched in again to not break backwards
compatiblity. Unfortunately, this will most probably lead to a wave of
code refactoring upstream-side so that maintaining such patches will
be harder.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: support for merged /usr in Debian

[Ben Hutchings]

On Sun, 2016-01-03 at 23:04 +1300, Daniel Reurich wrote:
> On 03/01/16 22:33, Philip Hands wrote:
> > Daniel Reurich  writes:
> > 
> > ...
> > > Because systemd doesn't work without /usr on the root partition isn't a
> > > good reason either.
> > 
> > You are right ... it is a poor reason, because it is pure fantasy.
> 
> Then why is it that since the introduction of systemd is having /usr on
> a separate partition suddenly considered evil and systemd complains
> loudly about it.  It always has worked and does work fine for me with
> sysvinit

systemd complains if it has to mount /usr itself.  This is because
mounting of local filesystems generally depends on various services and
udev hooks that may themselves depend on /usr.  This is also true when
using sysvinit.  Some services go through contortions to work before
/usr is mounted; others may behave subtly differently if it's a
separate filesystem.  We really need a simplified code path for
mounting /usr early on, and that is now provided by the initramfs.

> > > That just means systemd is broken by design and needs to be fixed.
> > 
> > If what you claimed were true, then I'd agree with you, but since all
> > the systems I've upgraded to systemd have a separate /usr, and are
> > working without any issue whatsoever, this drivel can be safely ignored.
> > 
> Then what's the problem and why are we even having this conversation
> about merged /usr???

As I understand it, merging /bin and /lib into /usr allows us to
support certain uses cases more easily - e.g. package installation in a
filesystem transaction, or sharing an NFS /usr filesystem.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

signature.asc
Description: This is a digitally signed message part

Re: support for merged /usr in Debian

[Andrey Rahmatullin]

On Sun, Jan 03, 2016 at 11:04:20PM +1300, Daniel Reurich wrote:
> >> Because systemd doesn't work without /usr on the root partition isn't a
> >> good reason either.
> > 
> > You are right ... it is a poor reason, because it is pure fantasy.
> 
> Then why is it that since the introduction of systemd is having /usr on
> a separate partition suddenly considered evil and systemd complains
> loudly about it.  It always has worked and does work fine for me with
> sysvinit
You either didn't check your information or are deliberately trolling, in
either case you should consider stopping posting here.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Daniel Reurich]

On 03/01/16 22:33, Philip Hands wrote:
> Daniel Reurich  writes:
> 
> ...
>> Because systemd doesn't work without /usr on the root partition isn't a
>> good reason either.
> 
> You are right ... it is a poor reason, because it is pure fantasy.

Then why is it that since the introduction of systemd is having /usr on
a separate partition suddenly considered evil and systemd complains
loudly about it.  It always has worked and does work fine for me with
sysvinit

>> That just means systemd is broken by design and needs to be fixed.
> 
> If what you claimed were true, then I'd agree with you, but since all
> the systems I've upgraded to systemd have a separate /usr, and are
> working without any issue whatsoever, this drivel can be safely ignored.
> 
Then what's the problem and why are we even having this conversation
about merged /usr???

-- 
Daniel Reurich
Centurion Computer Technology (2005) Ltd.
021 797 722



signature.asc
Description: OpenPGP digital signature

Re: support for merged /usr in Debian

[Philip Hands]

Daniel Reurich  writes:

...
> Because systemd doesn't work without /usr on the root partition isn't a
> good reason either.

You are right ... it is a poor reason, because it is pure fantasy.

> That just means systemd is broken by design and needs to be fixed.

If what you claimed were true, then I'd agree with you, but since all
the systems I've upgraded to systemd have a separate /usr, and are
working without any issue whatsoever, this drivel can be safely ignored.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: support for merged /usr in Debian

[Ole Streicher]

Martinx - ジェームズ  writes:
> Just tell the users that "/usr" isn't supported on a separated
> partition anymore (tell users what will happen if they do this /
> limitations), and do not touch anything else. During install, while
> partitioning storage, print a message about this, if user tries to
> separate "/usr".

As a strawman, I don't understand your doubts here. Just curious: if
/usr/ has to be on the same partition as /bin/ and /lib/, what is the
difference between /bin/ and /usr/bin/ resp. /lib/ and /usr/lib/? Why
should one *not* unite what has the same purpose?

Best regards

Ole

Re: support for merged /usr in Debian

[Martinx - ジェームズ]

On 3 January 2016 at 03:52, ChangZhuo Chen (陳昌倬)  wrote:
> On Sun, Jan 03, 2016 at 01:23:14AM -0200, Martinx - ジェームズ wrote:
>> It violates the FHS 2.3 standards.
>>
>> http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html
>
> Can you cite the requirement in FHS 2.3 which is violated by usrmerge. I
> only found the requirements that allow us to do usrmerge via symbolic
> link. For example:
>
> http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#Requirements
>
> The following directories, or symbolic links to directories, are
> required in /.
>
>> Also, if you guys (Debian) reeeally wants to move everything to
>> /usr, so, there is no reason for any symbolic links at the root file
>> system.
>
> Symbolic links at the root file system is required by FHS 2.3. Removing
> them will violate it.
>
> --
> ChangZhuo Chen (陳昌倬) 
> Debian Developer (https://nm.debian.org/public/person/czchen)
> Key fingerprint = EC9F 905D 866D BE46 A896  C827 BE0C 9242 03F4 552D

Thank you for your time and explanation! I really appreciate it...

I just don't get why you guys are pushing this unneeded change.

Debian doesn't need this.

Systemd doesn't need this either!

So, why? Why!?

I'm not convinced, even after reading this:


https://wiki.freedesktop.org/www/Software/systemd/separate-usr-is-broken/


I don't see any reason to move /bin/bash, to /usr/bin/bash!   LOL

Some udev rules will silently fail to work if "/usr" is split off and
not pre-mounted? So what?

This is NO reason to UsrMerge. Just keep "/usr" on the same block
device of "/", period.

No ugly symlinks on root file system, no Bash at /usr/bin/bash!

The problem is that "/usr" is not supported anymore, on a separated
partition! Okay, okay... But, moving /bin/* to /usr/bin/* is NOT a
solution to this problem, it is a wrong approach.

Just tell the users that "/usr" isn't supported on a separated
partition anymore (tell users what will happen if they do this /
limitations), and do not touch anything else. During install, while
partitioning storage, print a message about this, if user tries to
separate "/usr".

Please, do not bring this UsrMerge thing to Debian.

Regards,
Thiago