Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Steven Chamberlain 
On 13/01/14 20:57, Bdale Garbee wrote:
> Ian Jackson  writes:
>> I'm coming round to the view that we should be planning to support
>> multiple systems indefinitely.
> 
> This has been my opinion all along.  Various assertions that it's
> somehow just too hard really haven't swayed me.  The tricky bit, I
> think, is to define just what "support" means in the context of
> non-default init systems.  

IIRC, when kfreebsd became a release goal for squeeze, there was some
policy that certain fixes were allowed to be handled as RC bugs,
especially during the freeze.  That allowed porters to potentially get
something NMUd or unblocked if it was important to getting things
working on that system.

Could each proposed/approved init system for jessie be handled like
this, generally?  The respective teams would contribute the necessary
work to make sure things work with that system.  Maintainers would need
to accommodate reasonable changes (mostly adding native init scripts) if
they haven't already.

That to me sounds enough like 'supporting' an init system.  After
committing to such goals, once the work really gets underway, any
specific disagreements between teams over how to do things or what's
reasonable, could be handled separately as they arise, and escalated to
the ctte where necessary?

It wouldn't matter to me which is ultimately chosen as the default in
that case, if I only knew I wouldn't be wasting my time working on a
particular init system.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Bdale Garbee 
Ian Jackson  writes:

> I'm coming round to the view that we should be planning to support
> multiple systems indefinitely.

This has been my opinion all along.  Various assertions that it's
somehow just too hard really haven't swayed me.  The tricky bit, I
think, is to define just what "support" means in the context of
non-default init systems.  

Bdale


pgpVSTN_p1_Qt.pgp
Description: PGP signature

Bug#727708: Status of the init system discussion

2014-01-13 Thread Russ Allbery 
Several people have asked me about this in private, so I thought it was
worth sending a public message about my understanding of the current
status.

I think those of us who have been active on the discussion have mostly
stopped writing more because we had started repeating ourselves, or were
in danger of doing so, and were getting off into minutia.  Ian's and my
discussion of wording for one of the ballot options was at the point where
it needed input from other people, so I was holding off on discussing it
pending that (and I think Ian was as well).  I also don't know if Keith,
Don, or Andreas wanted to weigh in to the general discussion.

Remaining work that I'm aware of, beyond any further discussion that might
be sparked by the people who haven't yet participated, is mostly around
hashing out what the ballot options will be.  I think we have most of the
framework set up in response to Ian's draft language (thank you again,
Ian!), but there are details that are uncertain and would benefit from
more discussion.

We will also need wording for a ballot option to throw the question to a
GR if we want to have that on the ballot.  I think the general feeling of
those who have commented on that approach is that it's basically an
alternative to voting further discussion rather than anyone's primary
choice.

Speaking only personally, one of my occasional problems with Debian's
voting system is that it's very unclear in some cases what "futher
discussion" *means*.  It clearly means "we're not going to make a
decision," but in cases where we have to make a decision, it's sometimes a
weird way of phrasing the status quo.  I like having the "have a GR"
option on the ballot next to further discussion because it provides a
clear path forward for the project to make a decision, whereas in this
case further discussion doesn't really.  That effectively reserves further
discussion as the voting option for people who really think we should not
make a decision right now, as opposed to thinking we're in the wrong
decision-making process or feeling like the project should make a decision
now even if the technical committee can't reach consensus.

On a personal note, I've also gone much more quiet because I was spending
four to eight hours a day on the discussion, something that I could only
do because I was on vacation.  Since I'm now back to work, I have to
drastically reduce the amount of time I can spend on the discussion.
Thankfully, the major time investment portion seems past, but I thought it
was worth mentioning in case anyone was worried about slow responses.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/878uuj38iz@windlord.stanford.edu

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Russ Allbery 
Thorsten Glaser  writes:
> Steven Chamberlain dixit:

>> Actually, even if they forked in the same order every time, they might
>> not be *ready* in the same order.  That would be the rationale for
>> readiness protocols and other features of the more complex init
>> systems.

> Strong disagree. This actually is a bug in the initscripts in question
> that they return before the service is operational. Fixing this does not
> require exchanging PID 1 at all. In fact, there was some posting on
> Planet Debian where someone used
> #!/path/to/some-initscriptwriting-helper instead of shell for them.

If the daemon itself is ill-behaved, you generally have to do some
additional work to make this happen beyond just writing a better init
script.  Well-behaved daemons will provide some synchronization point (not
exiting in the parent until the daemon is started and ready, ideally, but
this requires a pipe or some other method to coordinate, so more common is
to have the child not write out its PID file until it's ready).  If they
don't, then it requires upstream patching, and can be tricky.

This doesn't change across the various init systems, although upstart and
systemd provide less tricky and racy ways of synchronizing than writing
PID files.  That said, most of the race conditions involved in the PID
file approach are rare.  Mostly you get into trouble if two copies of the
process are started at the same time, particularly if they're racing each
other to write the PID file.

Synchronizing on the PID file is my personal preference for old-style
daemons since it's so much easier to implement than controlling when the
parent process exits.

I *think* that start-stop-daemon properly implements PID-file-based
synchronization based on the description of the --pidfile option, but I'm
not positive and haven't checked the source.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87a9ez4sjk@windlord.stanford.edu

Re: Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Thorsten Glaser 
Steven Chamberlain dixit:

>Actually, even if they forked in the same order every time, they might
>not be *ready* in the same order.  That would be the rationale for
>readiness protocols and other features of the more complex init systems.

Strong disagree. This actually is a bug in the initscripts in question
that they return before the service is operational. Fixing this does
not require exchanging PID 1 at all. In fact, there was some posting on
Planet Debian where someone used #!/path/to/some-initscriptwriting-helper
instead of shell for them.

bye,
//mirabilos
-- 
Support mksh as /bin/sh and RoQA dash NOW!
‣ src:bash (268 (291) bugs: 0 RC, 188 (204) I&N, 80 (87) M&W, 0 F&P)
‣ src:dash (89 (106) bugs: 2 RC, 43 (49) I&N, 44 (55) M&W, 0 F&P)
‣ src:mksh (2 bugs: 0 RC, 0 I&N, 2 M&W, 0 F&P, 1 gift)
http://qa.debian.org/data/bts/graphs/d/dash.png is pretty red, innit?


--
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/pine.bsm.4.64l.1401131414090.25...@herc.mirbsd.org

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Thorsten Glaser 
Vincent Lefevre dixit:

>Well, the scripts may be started sequentially, but this doesn't mean
>that the daemons will be and always in the same order. And they don't,
>as shows in the following log:

That’s due to the (now default with sysv-rc) use of parallel boot
in combination with insserv. It used to be possible to still use
sequential boot even with insserv, and file-rc enabled that auto‐
matically (which is why I kept using it even with insserv) but I
don’t recall where the switch for sysv-rc is.

If you disable “makefile-style concurrent whatever”, sysv-rc is
deterministic even with insserv.

bye,
//mira“tys”bilos
-- 
 Beware of ritual lest you forget the meaning behind it.
 yeah but it means if you really care about something, don't
ritualise it, or you will lose it. don't fetishise it, don't
obsess. or you'll forget why you love it in the first place.


--
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/pine.bsm.4.64l.1401131411340.25...@herc.mirbsd.org

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Steven Chamberlain 
On 13/01/14 13:48, Vincent Lefevre wrote:
> On 2014-01-13 13:15:07 +, Steven Chamberlain wrote:
>> In the slides[0] 13 to 15, he summarises init systems something like:
>> * SysV - simple, familiar and deterministic
> 
> Deterministic?

Only the traditional SysV.  Debian since squeeze uses startpar so will
start *some* things concurrently (same Sxx number).  And where that
happens, it's much simpler to see/control it as files in /etc/rc2.d,
than e.g. events being triggered and such.

> Well, the scripts may be started sequentially, but this doesn't mean
> that the daemons will be and always in the same order.

Actually, even if they forked in the same order every time, they might
not be *ready* in the same order.  That would be the rationale for
readiness protocols and other features of the more complex init systems.

> In (1):
>   spamd start
>   wicd start/OK
>   sshd start/OK
>   spamd OK
>   postfix start/OK
> 
> In (2):
>   spamd start
>   sshd start/OK
>   wicd start/OK
>   spamd OK
>   postfix start/OK
> 
> This isn't deterministic at all.

I think that's just because insserv+startpar was being used here, not
the traditional SysV.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/52d3f298.6030...@pyro.eu.org

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Vincent Lefevre 
On 2014-01-13 13:15:07 +, Steven Chamberlain wrote:
> In the slides[0] 13 to 15, he summarises init systems something like:
> * SysV - simple, familiar and deterministic

Deterministic?

Well, the scripts may be started sequentially, but this doesn't mean
that the daemons will be and always in the same order. And they don't,
as shows in the following log:

[...]
Sat Dec 24 17:09:45 2011: Starting SpamAssassin Mail Filter Daemon: Starting 
Network connection manager: wicd.
Sat Dec 24 17:09:50 2011: Starting OpenBSD Secure Shell server: sshd.
Sat Dec 24 17:09:52 2011: spamd.
[...]

and still now (but less readable):

(1)
[...]
Thu Aug  1 10:24:51 2013: Starting SpamAssassin Mail Filter Daemon: [] 
Starting Network connection manager: wicd^[[?25l^[[?1c^[7^[[1G[^[[32m ok 
^[[39;49m^[8^[[?25h^[[?0c.
Thu Aug  1 10:24:59 2013: [] Starting OpenBSD Secure Shell server: 
sshd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.
Thu Aug  1 10:25:08 2013: spamd.
Thu Aug  1 10:25:08 2013: [] Starting Postfix Mail Transport Agent: 
postfix^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.
[...]

(2)
[...]
Sun Aug 18 12:18:21 2013: Starting SpamAssassin Mail Filter Daemon: [] 
Starting OpenBSD Secure Shell server: sshd^[[?25l^[[?1c^[7^[[1G[^[[32m ok 
^[[39;49m^[8^[[?25h^[[?0c.
Sun Aug 18 12:18:26 2013: [] Starting Network connection manager: 
wicd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.
Sun Aug 18 12:18:34 2013: spamd.
Sun Aug 18 12:18:34 2013: [] Starting Postfix Mail Transport Agent: 
postfix^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.
[...]

In (1):
  spamd start
  wicd start/OK
  sshd start/OK
  spamd OK
  postfix start/OK

In (2):
  spamd start
  sshd start/OK
  wicd start/OK
  spamd OK
  postfix start/OK

This isn't deterministic at all.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20140113134859.gg4...@xvii.vinc17.org

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Ian Jackson 
Steven Chamberlain writes ("Bug#727708: Bits from linux.conf.au"):
> Then he gives a preference for Debian's own insserv and startpar.  It
> allows for boot order to be fixed (after running insserv once, the same
> /etc/rc2.d/Sxx numbering may be rsync'd out to many machines).  startpar
> allows for some limited/controlled amount of concurrency to happen, for
> extra speed.

I think that what this shows is how valuable it is for our downstreams
that Debian is flexible and doesn't impose a particular approach.

I'm coming round to the view that we should be planning to support
multiple systems indefinitely.

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21203.59708.160009.777...@chiark.greenend.org.uk

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Steven Chamberlain 
On 13/01/14 12:15, Thorsten Glaser wrote:
> Алексей Шилин dixit:
>> In his talk [2] at 13:50 Marc briefly touched the init system choice 
>> question.
> 
> Can you please provide a transcript, for those among us who
> do not watch any video?

In the slides[0] 13 to 15, he summarises init systems something like:
* SysV - simple, familiar and deterministic
* Upstart - fast boot, makes sense on laptops, but inherently racey
* systemd - interesting concept, but too disruptive/complex to buy into

Then he gives a preference for Debian's own insserv and startpar.  It
allows for boot order to be fixed (after running insserv once, the same
/etc/rc2.d/Sxx numbering may be rsync'd out to many machines).  startpar
allows for some limited/controlled amount of concurrency to happen, for
extra speed.

For servers, their priority is in reliability/reproducibility of boot
(especially for pre-deployment testing), as the machines are so rarely
rebooted, and engineer time to debug any boot problem is so costly.

It's worth mentioning their boot is customised already for their
environment.  Before the root filesystem is mounted, there seems to be
some centralised logging, and an sshd started in the initrd, for human
or automated intervention in case the machine doesn't finish booting.
That may have pushed them in favour of a simpler init system.

[0]: http://marc.merlins.org/linux/talks/ProdNG-LCA2014/ProdNG.odp

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/52d3e6db.9040...@pyro.eu.org

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Sergey B Kirpichev 
On Mon, Jan 13, 2014 at 12:15:02PM +, Thorsten Glaser wrote:
> Алексей Шилин dixit:
> 
> > In his talk [2] at 13:50 Marc briefly touched the init system choice 
> > question.
> 
> Can you please provide a transcript, for those among us who
> do not watch any video?

This talk in article format:
http://marc.merlins.org/linux/talks/ProdNG-LCA2014/Paper/


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20140113124350.ga21...@darkstar.order.hcn-strela.ru

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Dmitry Yu Okunev 
Hello.

On 01/13/2014 03:57 PM, Алексей Шилин wrote:
> In his talk [2] at 13:50 Marc briefly touched the init system choice 
> question. Perhaps it's worth taking
> into account.
> 
> [2] 
> http://mirror.linux.org.au/linux.conf.au/2014/Wednesday/71-Live_upgrading_many_thousands_of_servers_from_an_ancient_RedHat_7.1_to_a_10_year_newer_Debian_-_Marc_MERLIN.mp4

[2] is placed in Australia, so I've mirrored it to [3].

[3]
http://mirror.mephi.ru/other/2014/71-Live_upgrading_many_thousands_of_servers_from_an_ancient_RedHat_7.1_to_a_10_year_newer_Debian_-_Marc_MERLIN.mp4

I hope this will be useful for somebody.

-- 
Best regards, Dmitry,
head of UNIX-tech department NRNU MEPhI,
tel. 8 (495) 788-56-99, add. 8255



signature.asc
Description: OpenPGP digital signature

Bug#727708: Bits from linux.conf.au

2014-01-13 Thread Алексей Шилин 
There was a talk on January 08, 2014 at linux.conf.au Linux conference by Marc 
Merlin, Google server
sysadmin and software engineer.

Quoting the description [1]:

> This talk will look at how we upgraded our ancient linux distribution on all 
> the Google production
> servers to a more modern one based on debian stripped down and built from 
> source.

In his talk [2] at 13:50 Marc briefly touched the init system choice question. 
Perhaps it's worth taking
into account.

[1] http://linux.conf.au/schedule/30014/view_talk?day=wednesday
[2] 
http://mirror.linux.org.au/linux.conf.au/2014/Wednesday/71-Live_upgrading_many_thousands_of_servers_from_an_ancient_RedHat_7.1_to_a_10_year_newer_Debian_-_Marc_MERLIN.mp4

-- 
Алексей Шилин