Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-05 Thread Andrew Savchenko
Hello,

On Thu, 4 Dec 2014 12:28:59 -0500 Rich Freeman wrote:
 On Thu, Dec 4, 2014 at 11:12 AM, Andrew Savchenko birc...@gentoo.org wrote:
  On Thu, 4 Dec 2014 08:59:22 -0500 Rich Freeman wrote:
  On Thu, Dec 4, 2014 at 7:54 AM, Andrew Savchenko birc...@gentoo.org 
  wrote:
  
   1. There are multiple services having after $all statement (an
   analog in Gentoo is after *, which is currently used only by
   local init.d script).
  
 
  Seems to me that the solution to this is to ban this sort of syntax
  entirely, and use proper deps.
 
  There is no serious problem in Gentoo here, because local is _the
  only_ service that uses this kind of dependency. But in Debian
  after $all syntax is in common use and we can't change that.
 
 
 But Debian CAN change that.  Why don't they?  Do they want to use OpenRC or 
 not?

That's a complicated issue. As far as I understand current
situation, majority of Debian devs are happy with systemd and
absolutely don't care about any other possible alternative. But
there are a few people who do care.

Under these conditions it is highly unlikely that anything in
Debian dependencies approach will change for the sake of OpenRC if
it works fine for them with their LBS and systemd right now
irregardless of is it right or wrong way for things to be done.

If we can tune OpenRC for their demands without damaging default
OpenRC setup, why not?

 I didn't ask why we need local.d.  I asked why we need to run it LAST,
 and why we need to run all of that other stuff LAST?  Of course, the
 reality is that we aren't running all of that stuff last since exactly
 one script can REALLY be run last.
 
 If your answer is that something in local.d might need to use the
 network, then specify that it needs the network.  If the answer is
 that it needs to use nfs, then specify that it needs to use nfs.  If
 it needs to happen after a bunch of random things but you can't be
 sure which things they are, then just create a virtual service and
 make it need that.
 
 The solution is the same for both local.d and all that other stuff
 that needs to be last.  Just specify what they actually need.  This
 is a dependency-based service manager.

This will require every sysadmin to know well how openrc works and
how to specify appropriate dependencies for custom out-of-tree
daemons. While I support the idea that sysadmins should be well
prepared to tweak internals of systems they are working with — and
to know this internals well in order to do such tweaks —, in real
life people often don't have time to dig such deep into the system
and will prefer other solutions which keeping stuff simple.

local.d where people can just put their scripts without handling
any deps is a simple way. That's why we need it. (Of course if
someone dislikes this approach, it not hard to disable local.d
at all.)

And if local.d needs dependencies to be specified how does it
differs from just writing plain new service and putting it into
init.d? Frankly, I do such things when I need custom complicated
services for my needs.

  It seems to be a statement of fact that OpenRC ISN'T compatible with
  LSB dependencies.  What it should be is anything but a statement of
  fact, which is what the word should means...
 
  Yes, it is not compatible now. And if OpenRC wants to step out of
  Gentoo scope to other distros, it should be. But as one can see,
  Gentoo will also benefit from proposed fixes, which will made OpenRC
  init system more robust and error-prone.
 
 I think you meant to say something different.  :)
 
 This honestly seems like the general trend of ignoring compiler
 warnings.  I'm not suggesting that the solution is -Werror.  However,
 what you propose basically makes the service startup process less
 deterministic, and everytime I read one of those threads on service
 managers it seems like everybody is railing about how that is a bad
 thing.

What I basically propose is to turn soft loops from errors to
warnings. Behaviour will be still deterministic (because early loop
solver uses deterministic and not random approach to broke loops),
but more sophisticated and slightly less transparent of course.

  I don't get why Gentoo gets by just fine with things as they are, but
  nobody else apparently can. Just fix your dependencies.
 
  I don't have broken dependencies on my systems right now. And this
  is not a discussion of my personal issues at all. What I'm trying
  to do is to make OpenRC robust and resilient to errors which may
  occur as well as expand its scope outside of Gentoo as a mature
  init system.
 
 There is a difference between tolerating a random error, and building
 a system that tries to do the right thing (defined as something other
 than error, fix your mess) when you throw a heap of garbage at it.
 On the one hand you talk about error resilience, and on the other you
 talk about this just being the way Debian does things and there isn't
 any error in the first place.
 
 Just fix the dependencies.


Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-05 Thread Andrew Savchenko
On Thu, 4 Dec 2014 12:55:44 -0600 William Hubbs wrote:
 Several issues not related to the original have been brought up, which I
 will briefly respond to, but let's try to move back to the original
 issue I brought up, which is whether the early loop solver should break
 loops or just output messages about them.

Just make it configurable and let users to decide what they want.
People who want their loops to be broken will use it. Algorithm
itself looks quite reasonable and safe. If someone hesitates and
prefers handle such cases manually, loop breaker may be disabled.
 
   It seems to be a statement of fact that OpenRC ISN'T compatible with
   LSB dependencies.  What it should be is anything but a statement of
   fact, which is what the word should means...
  
  Yes, it is not compatible now. And if OpenRC wants to step out of
  Gentoo scope to other distros, it should be. But as one can see,
  Gentoo will also benefit from proposed fixes, which will made OpenRC
  init system more robust and error-prone.
 
 We are talking about two issues; I did not mean for the lsb support to
 get into this thread. I was just wanting to discuss the early loop solver.

Issues are connected, that's why there were put into consideration
also. Anyway discussion seems to be quite productive :)
 
   I don't get why Gentoo gets by just fine with things as they are, but
   nobody else apparently can. Just fix your dependencies.
  
  I don't have broken dependencies on my systems right now. And this
  is not a discussion of my personal issues at all. What I'm trying
  to do is to make OpenRC robust and resilient to errors which may
  occur as well as expand its scope outside of Gentoo as a mature
  init system.
 
 Being error-resistant is a good thing, but the issue with breaking
 dependency loops without human intervention is we can't be sure we are
 breaking them in a sane way.

We can discuss early loop solver algorithm then.

1. It respects need  after  use priority and never breaks need
deps. After deps will be broken only if there is no use deps to
broke.

2. In case we have not a simple loop but branches, inner loops and
so on algorithm uses heuristics to break as little dependencies as
possible, keeping after upon use.

On algorithm details you should talk directly to xai (dyokunev),
since he wrote them. But from my analysis they look safe, though
some details may be optimized in future.

Warnings for users about loops is a good idea for Gentoo, but will
produce a lot of not always wanted output on Debian, that's why
this option should be configurable.
 
 I'm not completely opposed to it being configurable. However, I am
 opposed to the configurability being implemented the way it is in the
 patch, because it breaks the public API for what I see as a minor gain,
 if it is a gain at all.

Then we should either look for a way to add it without API breaking
or update the public API, if current version doesn't allow required
features to be implemented.

 For more ways to control how services stop/start/deal with the timeout
 issue, you might want to look up the -stop and -timeout keywords in man
 openrc-run.

I'm aware of this option, but this is not exactly what I need. I'm
interested in some kind of trigger which can change global timeout
value to something like 1 second, so that a wrapper may be put into
SHUDOWNCMD of upsmon.conf or OpenRC should support some hook to
check for /etc/killpower and adjust timeout accordingly. Maybe I'm
asking too much...

Best regards,
Andrew Savchenko


pgpULlZJKlfmv.pgp
Description: PGP signature


Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-05 Thread Rich Freeman
On Fri, Dec 5, 2014 at 3:25 AM, Andrew Savchenko birc...@gentoo.org wrote:

 I didn't ask why we need local.d.  I asked why we need to run it LAST,
 and why we need to run all of that other stuff LAST?  Of course, the
 reality is that we aren't running all of that stuff last since exactly
 one script can REALLY be run last.

 If your answer is that something in local.d might need to use the
 network, then specify that it needs the network.  If the answer is
 that it needs to use nfs, then specify that it needs to use nfs.  If
 it needs to happen after a bunch of random things but you can't be
 sure which things they are, then just create a virtual service and
 make it need that.

 The solution is the same for both local.d and all that other stuff
 that needs to be last.  Just specify what they actually need.  This
 is a dependency-based service manager.

 This will require every sysadmin to know well how openrc works and
 how to specify appropriate dependencies for custom out-of-tree
 daemons. While I support the idea that sysadmins should be well
 prepared to tweak internals of systems they are working with — and
 to know this internals well in order to do such tweaks —, in real
 life people often don't have time to dig such deep into the system
 and will prefer other solutions which keeping stuff simple.

 local.d where people can just put their scripts without handling
 any deps is a simple way. That's why we need it. (Of course if
 someone dislikes this approach, it not hard to disable local.d
 at all.)

 And if local.d needs dependencies to be specified how does it
 differs from just writing plain new service and putting it into
 init.d? Frankly, I do such things when I need custom complicated
 services for my needs.


There is mention here that there are many scripts that are set to run
last, not just local.d.  You haven't addressed those.

For local.d it seems reasonable to just have a default config that
makes it run after network and mounts.  Or maybe make local.d and
allow that alone to run last.  You can have one thing run last.

--
Rich



Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Andrew Savchenko
Hello,

On Wed, 3 Dec 2014 12:39:12 -0600 William Hubbs wrote:
 All,
 
 we have a pull request on OpenRC for a dependency checker [1].
 
 The author of this patch believes that we should not only scan for
 circular deps, but break some of them automatically.

Situation is more complicated than that. There are two loop
solvers, see below.

 I, and several other team members I have spoken with on IRC, disagree
 with this and think that we should just warn about the circular deps
 since users can break them by modifying files in /etc/conf.d, and the
 service script writers should be told about these kinds of issues so
 they can determine whether they neeed to adjust the dependencies in
 their scripts.
 
 I wanted to post a question here to see what people think, so feel free
 to comment.
 
 My opinion is the less automatic adjustment we do the better.

Short opinion for those not interested in reading a long text below.

1. Warn users about loops.
2. Break loops.
3. Make both options above configurable (on/off).
Gentoo is all about choise, right? 

Now the long story follows.

Why do loops occur? This depends on distribution being used. While
in Gentoo loops in deps are usually errors, in Debian loops are
unavoidable and must be handled, otherwise they will never accept
OpenRC as an alternative to systemd and other init systems.

Why there are loops in Debian? Here are few cases (though full list
of reasons is not limited to these cases):

1. There are multiple services having after $all statement (an
analog in Gentoo is after *, which is currently used only by
local init.d script).

2. LSB dependencies are allowed to be asymmetrical relative to start
and stop, while in OpenRC they are symmetrical. This yields to
loops in OpenRC while in LSB the same services work fine. Example
follows:

cryptdisks -a umountfs -u hwclock.sh -a checkroot -n cryptdisks
where X -a Y means Y after X;
X -u Y is for Y uses X; and
X -n Y is for Y needs X.

Actually cryptdisks needs checkroot only on start and hwclock.sh
uses umountfs only on stop (shutdown), so there are no issues for
LSB, but OpenRC have a loop here, which can be broken between 
umountfs -u hwclocs, because use is the weakest type of
dependecy.

While it was suggested on #openrc IRC channel that Debian may switch
to runscript-format dependencies, this may be possible only in
distant future (and I doubt even this) and this step is not
acceptable right now. So it is a statement of fact that OpenRC
should be compatible with LSB dependencies. Probably zigo and
heroxbd may give you more insight on this issue.

Warnings for users about loops is a good idea for Gentoo, but will
produce a lot of not always wanted output on Debian, that's why
this option should be configurable.

As for loop breaker, its presence depends on setup and user needs.
It is definitely needed in Debian for the reasons described above.

As for Gentoo it is desirable too, becase it is better to boot
system somehow instead of not booting it at all (or with long
delays due to 60-seconds timeout on service startup). This is
crucial for remote servers, e.g. admin needs to reboot machine due
to critical security kernel update ASAP and having it hang during
boot is really a very bad idea. Another example from my experience
is emergency shutdown due to power failure and low battery signal
from sys-power/nut. I had several nasty cases when system failed to
shutdown properly due to 60-second timeouts for services failed to
shutdown — battery just ran out of charge while OpenRC was try to
do thing right way.

Of course you mileage may vary, that's why loop breaker option
should be configurable.

But I see of no reason why we have right to force users to do what
we believe is right, instead of letting them choose what they need
based on their profile, preferences, setups, workflow and so on.

For those interested in more details:

There are two loop solvers. The first one, early-loop-solver which
is currently discussed:
https://github.com/openrc/openrc/pull/12
https://github.com/xaionaro/openrc/tree/earlyloopdetector

It solves all loops where there is at least one weak dependency
(after or use). And this is done during dependency cache
generation process, so there is no run-time penalty for system
startup or shutdown.

But if one have a hard dependency where all graph edges are
need, e.g.:
A -need B -need C -need A
then there is no way to break this dependency during cache
generation and it should be broken on run-time, that's why
later-loop-detector exists:
https://github.com/xaionaro/openrc/tree/laterloopdetector
And it does its dirty job :)

If someone is enterested in loop detection and solver algorithms,
there are well described in the following presentation:
https://github.com/xaionaro/documentation/blob/master/openrc/earlyloopdetector/early-loop-detection.pdf

I tested both loop detectors on several Gentoo hosts for about 9
months now and they work fine for me. In Debian people also tested
them for a 

Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Rich Freeman
On Thu, Dec 4, 2014 at 7:54 AM, Andrew Savchenko birc...@gentoo.org wrote:

 1. There are multiple services having after $all statement (an
 analog in Gentoo is after *, which is currently used only by
 local init.d script).


Seems to me that the solution to this is to ban this sort of syntax
entirely, and use proper deps.  WHY do we have to run local.d last?
Why do we have to run any of that other stuff last?

 2. LSB dependencies are allowed to be asymmetrical relative to start
 and stop, while in OpenRC they are symmetrical. This yields to
 loops in OpenRC while in LSB the same services work fine. Example
 follows:

Seems like the solution to this is to allow a different syntax for
start vs stop dependencies then, with the default being to keep them
the same if the current syntax is used.

 So it is a statement of fact that OpenRC
 should be compatible with LSB dependencies.

It seems to be a statement of fact that OpenRC ISN'T compatible with
LSB dependencies.  What it should be is anything but a statement of
fact, which is what the word should means...

It should be a nice warm day today is not a statement of fact, as much
as I'd like it to be.

I don't get why Gentoo gets by just fine with things as they are, but
nobody else apparently can. Just fix your dependencies.  I also don't
get why being as compatible as possible with LSB means that it is OK
to specify non-nonsensical dependencies like A and B must both be last
at the same time.


 Warnings for users about loops is a good idea for Gentoo, but will
 produce a lot of not always wanted output on Debian, that's why
 this option should be configurable.

I'd call these situations errors - a warning should certainly be used.

 As for Gentoo it is desirable too, becase it is better to boot
 system somehow instead of not booting it at all (or with long
 delays due to 60-seconds timeout on service startup). This is
 crucial for remote servers, e.g. admin needs to reboot machine due
 to critical security kernel update ASAP and having it hang during
 boot is really a very bad idea.

So, I'm fine with some kind of emergency mode, though expecting it to
work on a remote server might be asking a bit much.  Dropping to shell
is certainly preferable to just hanging.  You certainly couldn't
guarantee a successful boot, since the configuration contains errors.

 Another example from my experience
 is emergency shutdown due to power failure and low battery signal
 from sys-power/nut. I had several nasty cases when system failed to
 shutdown properly due to 60-second timeouts for services failed to
 shutdown — battery just ran out of charge while OpenRC was try to
 do thing right way.

Seems like the solution to this is to let you configure the timeout
per-service and globally, and maybe even give you the ability to
override it at time of shutdown.  During a routine shutdown you don't
want the service manager randomly killing stuff.

--
Rich



Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Dmitry Yu Okunev
Hello, everybody!

I'm the author of discussed patches and let me put my 2¢. I want to
clarify some things and explain my position… Right away sorry for my
English skills. Also I wrote the patches year ago and may remember
something incorrectly.


1. The Later Loop Detector

There are really two approaches which can complement each other. But
personally I as the author recommend only the early loop detector
(below - ELD) to be approved into the upstream. This is due to next facts:
 - I'm not sure in correctness of the later loop detector (below — LLD).
 - LLD can be easily replaced in most of cases by modifying 2 lines in
the ELD. So it's just an extra code.

However I can imagine few situations where ELD will be unable to handle
a loop situation, while the LLD will be able to. Nevertheless even if
this situations really exists, they should appears only in quite exotic
conditions (like read-only dependencies cache file). Some time is
required to make tests in order to verify this aspect, which can be done
if somebody here thinks that this is really important.

So I recommend to discuss only the ELD.


2. The Early Loop Detector, summary

As Andrew Savchenko already mentioned the algorithm is represented in
PDF-presentation [1], and ELD works while building dep tree cache.

  [1]
https://github.com/xaionaro/documentation/blob/master/openrc/earlyloopdetector/early-loop-detection.pdf

Actually ELD is not only a detector but also a solver, so it can be
considered as two separate components:
 - Loop detector (below — Detector). It detects a dependency loop chains.
 - Loop solver (below — Solver). Analyzes different variants of loops
solutions and solves them (if possible).

I think almost everybody here agree that Detector is really required in
OpenRC. It's quite definitely that sysadmin should be able to see any
error situation on his/her machine. So let's talk only about the Solver.
And I think the Solver should be enabled by default. But I even don't
hope that somebody will support this position in this mail-list and IMHO
it's quite useless to argue on this issue. But I'm very sure that the
Solver should be saved at least as an option.


3. Using ELD Solver as an option

Sorry if I remember something wrong. I need time I don't have right now
to recheck and refresh my memory. But IIRC currently without the Solver,
OpenRC in parallel:
 - Hangs with timeout (and can hang multiple times, as it was in may
case with ~60 services the same time).
 - Just doesn't run all looped services.

So any extra use dependency can break startup of great lot of
services. As it already happened on Debian. Here's a quote from Gentoo
Handbook [2]:

 The *use* settings informs the init system that this script uses
 functionality offered by the selected script, but does not directly
 depend on it. A good example would be *use logger* or *use dns*. If
 those services are available, they will be put in good use, but if
 you do not have a logger or DNS server the services will still work.
 If the services exist, then they are started before the script that
 *use*'s them.

  [2]
https://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=2chap=4#doc_chap3

Moreover the system boot can be delayed for few minutes or even for few
hours (as in my case, IIRC).

Almost anyone tells me that the Solver is even not an option for OpenRC.
But I disagree. It has heuristic algorithm to minimize the detriment. If
you boot looping system without the Solver the detriment will be much
more significant. For example network and sshd wouldn't start due to one
extra use dependency and system will boot a long time. IIRC, in case
of Debian the whole system is not running (inc. mounting, network and so
on) due to extra use dependencies only. I can understand that you
don't care about Debian and I'm prepared to the
unfortunate fact that we will have to maintain extra patch for OpenRC
package in Debian. So returning to Gentoo.

Let's just compare behavior in loop situation with the Solver and without.

Without the Solver (sorry for repeating this):
 - System will hang until timeout will be reached and so for each loop.
 - All looping services won't start. Likely enough, system will be
unreachable.

With the Solver:
 - System won't hang, of course.
 - Some low-cost dependency will be removed and likely all looping
services will start.


Also what sysadmin should do to make the Solver harm him/her. He should:
 - Enable the Solver manually in /etc/rc.conf (if it's not enabled by
default).
 - Use need instead of use and use instead of need (or something
like this).
 - Create the loop situation.
 - Ignore warnings and reboot.

I can imagine this only if this is done willfully.


I very need this option in Debian and I'd prefer to use it on Gentoo. So
I recommend to save the Solver at least as an option.


4. Few comments

 My opinion is the less automatic adjustment we do the better.

Don't enable the option. :)

On 12/04/2014 04:59 PM, Rich Freeman wrote:
 I 

Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Christopher Head
On December 4, 2014 8:12:58 AM PST, Andrew Savchenko birc...@gentoo.org wrote:

Yes. But booting as much services as possible is even more
preferable, especially when box is remote.

Are you sure booting most, but not all, services in a loop is always better 
than booting none of them at all? What if I have an insecure dæmon listening on 
TCP, I need it running, but I want to ensure only local processes can connect 
to it? Obviously, I would make it “need iptables”, assuming the dæmon doesn’t 
have its own bind address config knob.

What if now, by some accident, iptables ends up in a loop (maybe not even a 
loop including $insecure_service, but some other loop entirely), and it’s the 
randomly chosen victim? Is it still good to boot as many services as possible? 
I think not.

-- 
Christopher Head



Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Wyatt Epp
On Thu, Dec 4, 2014 at 12:37 PM, Christopher Head ch...@chead.ca wrote:

 What if now, by some accident, iptables ends up in a loop (maybe not even a 
 loop including $insecure_service, but some other loop entirely), and it’s the 
 randomly chosen victim? Is it still good to boot as many services as 
 possible? I think not.

My understanding of the algorithm is that it explicitly does not break
on need boundaries and cycle breaking doesn't affect the rest of the
graph.  So in that scenario, if iptables isn't started, your
hypothetical insecure service won't be started either.  It's rather
conservative and sane, IMO.

-Wyatt



Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Dmitry Yu Okunev
One more ¢…

On 12/04/2014 08:37 PM, Christopher Head wrote:
 On December 4, 2014 8:12:58 AM PST, Andrew Savchenko
 birc...@gentoo.org wrote:
 
 Yes. But booting as much services as possible is even more 
 preferable, especially when box is remote.
 
 Are you sure booting most, but not all, services in a loop is always
 better than booting none of them at all? What if I have an insecure
 dæmon listening on TCP, I need it running, but I want to ensure only
 local processes can connect to it? Obviously, I would make it “need
 iptables”, assuming the dæmon doesn’t have its own bind address
 config knob.
 
 What if now, by some accident, iptables ends up in a loop (maybe not
 even a loop including $insecure_service, but some other loop
 entirely), and it’s the randomly chosen victim? Is it still good to
 boot as many services as possible? I think not.

 I would make it “need iptables”

Firstly, the loop solver doesn't remove need dependencies [1]. There
will be no problem.

  [1]
https://github.com/xaionaro/documentation/blob/master/openrc/earlyloopdetector/early-loop-detection.pdf

But there are few ways to bypass such problems. For example:
 - Don't enable the option in this case. You should understand
consequences of enabling any non-default option. Also for example
sysadmin shouldn't setup public sshd with pass test on root. Here's
the same. It's just required to understand what are you doing.
 - Use network namespaces for insecure processes without ability to
setup the bind address. And use iptables to redirect to the real
listening port (in the namespace).


Best regards, Dmitry.




signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread Andrew Savchenko
On Thu, 04 Dec 2014 09:37:24 -0800 Christopher Head wrote:
 On December 4, 2014 8:12:58 AM PST, Andrew Savchenko birc...@gentoo.org 
 wrote:
 
 Yes. But booting as much services as possible is even more
 preferable, especially when box is remote.
 
 Are you sure booting most, but not all, services in a loop is
 always better than booting none of them at all?

If we're talking about early loop solver, then yes. Because this
solver never breaks need dependencies.

 What if I have an insecure dæmon listening on TCP, I need it
 running, but I want to ensure only local processes can connect to
 it? Obviously, I would make it “need iptables”, assuming the dæmon
 doesn’t have its own bind address config knob.

And need iptables will do the job. Either weaker part in the loop
will be broken, or your loop will be left unsolved (e.g. if
iptables directly or indirectly _needs_ your daemon).

 What if now, by some accident, iptables ends up in a loop (maybe
 not even a loop including $insecure_service, but some other loop
 entirely), and it’s the randomly chosen victim? Is it still good to
 boot as many services as possible?

Yes, it is, because only weak dependencies like after and use
may be broken (and after is considered stronger than use IIRC).

As for later loop detector, it may break need dependency. Current
need dependency for iptables is fsck - localmount - iptables, so
it is still unlikely that your daemon will be caught in such
need-only loop. Though on author's request later loop solver is out
of scope of this discussion now...

Best regards,
Andrew Savchenko


pgpqurCNF2A_f.pgp
Description: PGP signature


Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-04 Thread William Hubbs
Several issues not related to the original have been brought up, which I
will briefly respond to, but let's try to move back to the original
issue I brought up, which is whether the early loop solver should break
loops or just output messages about them.

On Thu, Dec 04, 2014 at 07:12:58PM +0300, Andrew Savchenko wrote:
 Hello,
 
 On Thu, 4 Dec 2014 08:59:22 -0500 Rich Freeman wrote:
  On Thu, Dec 4, 2014 at 7:54 AM, Andrew Savchenko birc...@gentoo.org wrote:
  
   1. There are multiple services having after $all statement (an
   analog in Gentoo is after *, which is currently used only by
   local init.d script).
  
  
  Seems to me that the solution to this is to ban this sort of syntax
  entirely, and use proper deps. 

I agree with this. As soon as more than one service tries to use
before/after *, things are broken, because more than one service can't
possibly meet those dependencies.

 
 There is no serious problem in Gentoo here, because local is _the
 only_ service that uses this kind of dependency. But in Debian
 after $all syntax is in common use and we can't change that.

I am chatting with qnikst on irc as I write this, and he says this might
be able to be solved by stacked runlevels. I haven't looked at this
myself yet so can't verify.

 The prime cause of these patches was to bring OpenRC as usable init
 system for other distributions and not to limit its scope only to
 Gentoo. And Debian has requirement of additional features in order
 to allow OpenRC to be used as a reliable replacement of other init
 systems.

A dependency checker is something we have needed for some time,
regardless of the distro; you are talking about bug 391945 [1].

The lsb support is a temporary measure until
Debian can move away from lsb init scripts, which is their plan, so I
don't think we need to worry about it upstream.

  WHY do we have to run local.d last?
  Why do we have to run any of that other stuff last?
 
 I doubt this is in POSIX standard or whatever, but it is a quite
 common custom among distributions to have some rc.local service
 where admins may put their custom daemons to be started in quick and
 dirty way, so that admins without prior knowledge of how particular
 init system works may put their stuff to that rc.local script/
 service/whatever. This is mostly needed when out-of-tree packages
 are used or in-tree packages lack init script. The former is quite
 common in production, unfortunately.
 
 I am not planning on changing this; I want to keep local running last.

   2. LSB dependencies are allowed to be asymmetrical relative to start
   and stop, while in OpenRC they are symmetrical. This yields to
   loops in OpenRC while in LSB the same services work fine. Example
   follows:
  
  Seems like the solution to this is to allow a different syntax for
  start vs stop dependencies then, with the default being to keep them
  the same if the current syntax is used.
 
 Yes, this particular case can be solved with optionally separated
 start and stop dependencies.

This is a separate thread; I would like to know more about this before I
attempt to implement it, including what the use cases are for it. Let's
table this one for now.

 What proposed solutions do?
 
 1a. For the first issue early loop solver allows to start all
 services from a *soft* loop gracefully, so that all services should
 work, because all mandatory need dependencies will be satisfied.
 
 1b. If loop is *hard*, then later loop solver will intelligently
 break loop at some point, so that as much services as possible will
 have their need dependencies satisfied. Of course, some services
 may fail in this case, but this is still better than fail _all_ of
 them.
 
 2. No time will be lost in case of *soft* loop and time loss will
 be minimized in case of *hard* loop.

Later loop solver is not even under consideration. The author made it
very clear that it shouldn't be; read his comments about it not being a
good solution in the pull request [2]. In fact, I think he meant to
close it (see comment 2) so I am going to do that today.

 
   So it is a statement of fact that OpenRC
   should be compatible with LSB dependencies.
  
  It seems to be a statement of fact that OpenRC ISN'T compatible with
  LSB dependencies.  What it should be is anything but a statement of
  fact, which is what the word should means...
 
 Yes, it is not compatible now. And if OpenRC wants to step out of
 Gentoo scope to other distros, it should be. But as one can see,
 Gentoo will also benefit from proposed fixes, which will made OpenRC
 init system more robust and error-prone.

We are talking about two issues; I did not mean for the lsb support to
get into this thread. I was just wanting to discuss the early loop solver.

  I don't get why Gentoo gets by just fine with things as they are, but
  nobody else apparently can. Just fix your dependencies.
 
 I don't have broken dependencies on my systems right now. And this
 is not a discussion of my personal 

[gentoo-dev] rfc: openrc service script dependency checker

2014-12-03 Thread William Hubbs
All,

we have a pull request on OpenRC for a dependency checker [1].

The author of this patch believes that we should not only scan for
circular deps, but break some of them automatically.

I, and several other team members I have spoken with on IRC, disagree
with this and think that we should just warn about the circular deps
since users can break them by modifying files in /etc/conf.d, and the
service script writers should be told about these kinds of issues so
they can determine whether they neeed to adjust the dependencies in
their scripts.

I wanted to post a question here to see what people think, so feel free
to comment.

My opinion is the less automatic adjustment we do the better.

Thanks,

William

[1] https://github.com/openrc/openrc/pull/12


signature.asc
Description: Digital signature


Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-03 Thread Alexander V Vershilov
Let me state my idea here.

At first I want to mention that author provided 2 different approaches to
the solution, simple dependency loop checker and another more complicated
algorithm that is a loop breaker.

I think that on a boot phase in case of parallel boot rc should try to
check if loop exists and it is then print a warning and switch to a
sequential boot. The only problem here is if its possible to switch to a
sequential boot during the boot process. So with this approach we will have
a simple solution that will definitely solve the problem, opposed to the
loop breaker that afaiu doesn't give such guarantees.

A loop breaker can live as a standalone application or as a part of
rc-update.

Just my 2¢.

--
Alexander (qnikst)

 On Dec 3, 2014 9:39 PM, William Hubbs willi...@gentoo.org wrote:

 All,

 we have a pull request on OpenRC for a dependency checker [1].

 The author of this patch believes that we should not only scan for
 circular deps, but break some of them automatically.

 I, and several other team members I have spoken with on IRC, disagree
 with this and think that we should just warn about the circular deps
 since users can break them by modifying files in /etc/conf.d, and the
 service script writers should be told about these kinds of issues so
 they can determine whether they neeed to adjust the dependencies in
 their scripts.

 I wanted to post a question here to see what people think, so feel free
 to comment.

 My opinion is the less automatic adjustment we do the better.

 Thanks,

 William

 [1] https://github.com/openrc/openrc/pull/12


Re: [gentoo-dev] rfc: openrc service script dependency checker

2014-12-03 Thread William Hubbs
On Wed, Dec 03, 2014 at 10:53:38PM +0400, Alexander V Vershilov wrote:
 Let me state my idea here.
 
 At first I want to mention that author provided 2 different approaches to
 the solution, simple dependency loop checker and another more complicated
 algorithm that is a loop breaker.

The other approach being referred to is here [1]. The author clearly
states that this shouldn't be used in preference to the other method.

William

[1] https://github.com/openrc/openrc/pull/13


signature.asc
Description: Digital signature