Hi Isaac !
On 12.03.2015 02:05, Isaac Dunham wrote:
I just don't think you're quite thinking through exactly what this means.
In which sense? Let me know, what I got wrong. I'm a human, making mistakes,
not a perfect machine.
It seems like you want to force everyone who uses mdev to use a multi-part
setup.
Whops, you got that definitely wrong! Who told you, I want to force you
to use a different setup. My clear intention, was adding some extra
flexibility, without breaking existing usage.
... but criticism arose, which poked for more clarity. Only due to this
and the wish to allow fitting as much preferences of people as possible,
there might result a slight change: One extra operation may be required
(when not not combined / hidden in the mdev system for clarity). At the
location you setup the hotplug handler or do "mdev -s" you either need a
slight change of the command parameters or need to insert a single extra
command (details on this have not been discussed yet). But that's it.
Are you really concerned about inserting a single extra line in your
startup scripts or having a modification to one of those lines? That way
you would block any innovation, *and* the needs of other people.
Specifically, you are proposing to *replace* the standard hotplugger design
(where the kernel spawns one program that then sets up the device itself)
with a new design where there's a hotplugger that submits each event to
a longer-lived program, which in turn submits the events to a program
that creates/sets up the devices.
You say I propose a change of the device management system?
The mdev system suffer on event bursts and this has been grumbled about
several times in the past. What I'm now trying is to build a solution
for this into Busybox. Not reinventing the wheel, but implementing known
solutions.
I am saying, don't force this design on people who want the hotplug helper
that the kernel spawns to finish things up.
The only way to solve this, would be to let mdev as it is, and create an
additional netlink-mdev, which brings us as at the situation where it
gets complicated (or at least complex) to share code and work between
both of them. Selection between both (when you don't want to include
both hunks), needs to be done with BB configuration, which make all
thinks like pre-build binaries a mess (how many different versions shall
be maintained? Only yours? Only mine?).
To solve this conflict and to also give the current device management
system some speed improvement on burst, I tried to find a solution. This
solution centers on the most important problem, the parallelism and
parsing the conf for each single hotplug event.
So I *propose* to split the function of "mdev" into two separate parts.
Part one will contain the kernel hotplug helper part (and should be as
small and fast as possible), and part two the parsing of the conf and
device node operations ... with the requirement of communication between
any number of running part ones and the single part two. To reduce cost
on those "part one" hunks (remember they shall be as fast as possible),
and the availability of failure management (which BB currently
completely lack), the communication between the parts need to be watched
by a small helper daemon. A very minimalist daemon, which doesn't touch
any data, it just fire up the device operation stuff, when it's required
and wait until this process dies. Then goes back to wait for more events
to arrive (this not like udev).
Do I wish to overcome the suffering of Busybox device management? Yes.
Do this need some changes? Yes, I propose as one or two line change in
your startup files, with the benefit of speed improvement.
Do I otherwise propose to change your system setup or flip to using
netlink operation? *NO*
Hence, I do not force anybody to change from using the kernel hotplug
feature. I improve, that mechanism ... with the ability to add further
mechanisms, without duplicating code. One shall be a netlink reader in
BB, the other may be external programs of others, with a better
interface to use device management functions from there.
Are your really complaining against any kind of innovation? Any step to
overcome long problems, discussed several times ... but dropped, mostly
due to the amount of required work?
Do I *propose* some innovation? Yes.
Do I propose to force someone changing to different mechanism? No.
Agreed.
But I would include "hotplug daemons" under "etc."
I used "etc." so add any similar type of program you like, including
"hotplug daemons" ... but stop, what are "hotplug daemons"? Do you mean
daemons like udev? udev uses netlink reading. Otherwise I know about hotplug
helper programs (current mdev operation), but not about "hotplug daemons".
That's the best description I can come up with for the FIFO reader that
you proposed that would read from a fifo and write to a pipe, monitoring
the state of "mdev -p"
Wow? You got it wrong!
It won't be a FIFO reader, it will be a FIFO watcher, which is a big
difference. That is in functionality comparable to the operation of
tcpsvd, which opens a network socket waiting for incoming connections,
accept them, fork a handler program and pass the socket of the incoming
connection. This is a well known and proven practice on Unix like
systems. And what is wrong with this?
The gathering parts need to acquire the device information, sanitize this
information, and serialize the event operations to the right order. The
device node handling part shall receive the information from the gathering
part(s) (whichever is used) and call the required operations, but shall
avoid reparsing the conf on every event (speed up) *and* drop as much memory
usage as possible, when the event system is idle.
That *shall* is where you're forgetting about the first principle you
mentioned (assuming that you mean "shall" in the normative sense used
in RFCs).
??? Sorry, may be it's I'm not a native English speaker. Can you explain me
what is wrong? How should it be?
The idea that the hotplug helper is *required* to fit into a redesign
where the parser runs as a different process from the backend and handles
multiple hotplug events is what I'm objecting to.
So you are complaining at the wrong location. The problem lies at the
principle of the kernel hotplug feature, where the kernel spawns a
separate process for each new event. I should only spawn *one* soch
hotplug process and then send this process the events.
I didn't create that brain damaged method, but I have to live with it.
So I'm trying to overcome some of the problems ... still with the
possibility of using the hotplug mechanism (for those who dislike or
can't use netlink), beside additional over all speed improvement.
... and what does you complain about "hotplug helper required"? How else
other than starting a program, can you gather the information from the
kernel when using the hotplug helper mechanism? The hotplug helper is
nothing new, it is only the first part of code from current mdev,
avoiding duplicating the second part for any further parallel arriving
hotplug event.
roughly, architecture = design.
Not how the scripts/config files are written, but the fact that you're
changing "kernel starts hotplug helper, hotplug helper sets up device"
into "kernel starts hotplug helper, hotplug helper writes to fifo,
fifo reader supervises and writes message to tool that sets up device"
So you prefer pure parallelism with known problems, over splitting the
operation into separate threads, using a communication method?
And see above, it is not a FIFO reader. The extra daemon is just a
process who fires up the device handling "thread" when that thread is
not active, and may catch failures of that "thread".
Do you really like forking a separate conf parser for each hotplug event,
even if they tend to arrive in bursts?
Yes, depending on the circumstances (I assume that by "forking a separate
conf parser", you mean "letting the kernel spawn a new hotplug helper that
parses a config file").
Then this means blocking any innovation, and forcing other people to use
different software or do it the same way you like, if you are not
willing do accept some slight modifications / big improvements.
*I* certainly like it better than spawning a queue submitter on each
hotplug event, which will then submit it to a supervisor, which will
then submit it to a device creator.
One submit to much: The supervisor fire up a device creator if required,
which processes the incoming events until no more to do. Splitting of
that queue submitter also add some buffering, which allows the hotplug
helper process to exit early (freeing system resources).
In short, don't expect people to buy into your design before it gets merged,
and don't expect it to replace mdev before everyone buys into your design.
Come up with an extra applet that lets people test it without replacing
a known-working mdev, and you might see a lot more people test it.
And I highly dislike that, as others too. I asked ahead to gather as
much preference wishes and ideas as possible, the usual way of
programming, not hacking. No one asked you or others to use untested
stuff. The first step is planning, the second hacking, the third testing
... then a patch can be offered for those who like to test ... and if no
one more complains on those tests, it may be merged into the main stream.
... but I neglect having different versions of mdev, not talking about
you are forcing me to change the name of a well known operation.
Won't you like to get a faster system startup with mdev, without changing
system setup / configuration?
Absolutely NOT without changing system setup/configuration.
This is where I see no room for negotiation.
???
Think about this:
Would it be nice if someone changed "mount" so that instead of parsing
/etc/fstab and mounting drives, it just submitted a job request to a
service that then supervised another service that uses a pre-parsed
fstab to determine which drive to mount?
Beside your mistake about forwarding data a step to much, what would be
wrong, if the overall operation of the mount system does not change
(neither the mount call nor the fstab format change)?
What if modprobe and insmod did that?
This is Natanaels idea and request, and could be a good practice to
speed up module loading on system startup and device changes.
Wouldn't it be a nightmare if you upgraded, mount or modprobe broke,
and you went to strace it and found that it's just writing messages
and you need to figure out where those are going/failing to go?
You are tracing mount or modprobe operation???
Even if it *did* work better, you *don't* make changes that large as a
transparent mandatory change, where one upgrades and all of a sudden,
it works completely different behind the scenes.
Your new code *will* fail at some point when dealing with a system
configuration that you never had access to or heard of, probably sooner
than later. That's the way Murphy works.
And if you make this happen without changing configuration, you make it
much harder to troubleshoot, because the person who first troubleshoots
it will likely try to use their knowledge of how things have always
worked.
Now, if you can do it faster by adopting this approach, people will
be willing to change their configurations to use it. It would be one word
to change to another hotplug helper, so they can trivially switch between
your proposal and mdev.
That is: New automobiles may be dangerous, don't even think about
creating a newer one ... sorry, but I'm not able to understand your
point of view :(
So, in order to respect that preference, it would be nice if you could
let the hotplug part of mdev keep doing what it does.
What expect you to be the hotplug part? The full hotplug handler including
conf file parser, spawned in parallel for each event?
Yes, at least for mdev.
Then I expect you, staying at current versions of Busybox, saving the
current version of mdev, and replacing this version in may be newer
versions ... that's more or less just the opposite of what you expect,
that I do all that on a private basis.
My main point is that what you propose for the default action of mdev
should be in a separate applet;
Which mean doubling of code, as it can't share the code ...
(provocative) why not just copying mdev ahead, renaming into
mdev-hp-old, for those who are really picky about that usage. Giving the
average users an improved, but compatible mdev implementation?
This would become the FIFO-reading daemon if need be, and dump its
environment into the fifo or (as daemon) read from the fifo, spawn
mdev -p, and write to a pipe.
I'd assume it needs a *simple* config file, which could be just a line
or two:
/sbin/mdev -p
timeout <n>
??? <confused> :(
The first line specifies the hotplug parser to use (so if someone wants
to write a parser for udev rules or something else, they're able to use
the fifo daemon.
Rather than using mdev -h to set things up, it sets itself up from the
(extremely minimal) config file as soon as possible.
This is significant, since there are at least four ways to set the
hotplugger:
-In the kernel .config
-On the kernel command line
-Via /proc (also accessible via sysctl)
-Via /sys
So someone might not realize that they need to start the fifo-watching
daemon/set the hotplugger.
You completely misunderstand, on how this will work!
--
Harald
_______________________________________________
busybox mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/busybox
