Re: [systemd-devel] The Linux Way or Some ideas to make systemd better
On Mon, Jul 18, 2011 at 15:38, Sergey wrote: > It's not, if one of these two processes actually never starts (we're talking > about separate dbus service here, right?). Instead of providing two interfaces > (unix sockets and dbus) systemd could provide just one. Dbus interface for > systemd is mostly unused and splitting it to a separate application that > does NOT start by default (dbus-daemon will start it if needed) we can get > a lot of benefits. First we're getting the ability to not install dbus on > servers where it has no use and free a little bit of resources. That all systemd IPC is the D-Bus _protocol_ but no the D-Bus _daemon_. The D-Bus daemon is not needed at all if you want to build a system that works that way. > Also systemd > init daemon don't have to link against libdbus any more. Fewer libraries > linked - faster startup and smaller boot time which was the initial goal. > These are advantages, are there any disadvantages to that? What do I miss? sytemctl uses libdbus to talk to systemd. Now what, invent your own protocol? > But then implementing it in a separate daemon works in both directions. It > not only makes this feature optional, but also makes it compatible with other > init systems. Which means that more users would be able to install and use > it. Which means that it will spread faster. Right? Wrong. The main goal is to have almost no dependencies to bring up the basic services by default. If you rely on a bunch of external service again to manage the service startup, you have all of them back, with all the problems we have solved. You can also not really mange these services race-free with a unified interface anymore. >> I see no value in the portability to other Unixes. I do see value however >> in our platform becoming stronger by removing the big variable of the OS >> we build on. > > Even if you don't need it (which is pretty usual unless you're using other > unixes), portability is still a benefit. Such benefit may bring more users > and more developers to the project. And this is definitely good. Is there > anything bad? Reality showed us that the balance of maintaining the code of other OSs in the main repository is much more work than the work the few lines of useful code the few people of the other OSs contribute. We tried with many projects in the past, and decided against it. The needed abstractions are just hard to manage and get into our way all the time. The only really thinkable solution for the niche OSs is to port the needed Linux interfaces to their kernels. But I guess that will never happen, and so systemd, udev, ... will probably never happen for them. People can try to port systemd to other OSs, we can even work together on defining and sharing the same interfaces, but they should use git and rebase their patches, instead of merging the code into the Linux git repository. > Just curious, what's wrong with SysV scripts? And how is it possible to > request status of arbitrary service without them? I mean, you can tell only > whether it was started or not, but only SysV script knows i.e. is it alive > or frozen. They are _programs_, not _descriptions_ what a service should do. We need machine-readable and user-interface-manageable service descriptions, and not free-text, sometimes insane programs to bring up services. >> Upstream developers generally do not write init scripts, package >> maintainers do, simple because you have tow write different ones for >> different distros. Which coincidentally is something systemd fixes. > > LSB standard does the same thing. If people don't want to conform to standard > it's not the problem of standard. But if they don't want to follow LSB > standard what makes you sure they will follow yours? ;) We are not even thinking about creation any new 'standard'. :) Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] The Linux Way or Some ideas to make systemd better
2011/7/17 Lennart Poettering wrote: > sorry, but I am not interested. Unix is an inspiration but it is not > dogma. True. All those words "Unix philosophy", "flexibility", "portability" really don't mean anything. Same as words "duplication" and "integration". They're... just words. They either give a hint about the best solution for problems, or they don't. Problems are the only real things that must be adressed. > Modularization for the only sake of modularization is however not what > what we buy into. Of course, and this applies to any development. Integration for the sake of integration is no better. That's why I was trying to explain every idea and every suggestion in detais, to show advantages and disadvantages of each solution. For example one of benefits of modular structure for systemd is that other developers may work on it, i.e. write some runner plugins for systemd that they need. You won't have to worry about them, you won't even have to know about them. And more developers interested in the project is good, right? > The more you modularize the bigger the overhead of communication becomes Only if modules are in separate applications. If modules are just plugins loaded with dlopen() there's not much difference in communication there. > Also note that there's a reason why sockets, services, mount points and > so on, are all handled in the same concept of units: they have complex > interdependencies There's no need for dependencies if everything is started in parallel. :) Really, I was wondering about that. I can think of dependencies between two services, for example [iptables] and [sysinetd] from the previous email (reason - sockets must not be created before iptables applies all the rules). But can you name any real case where dependency is needed between a service and a mountpoint? > In addition, you are very much overestimating the code size of the socket > handling, and similar code. Splitting short code like tht into separate > processes is wastage. It's not, if one of these two processes actually never starts (we're talking about separate dbus service here, right?). Instead of providing two interfaces (unix sockets and dbus) systemd could provide just one. Dbus interface for systemd is mostly unused and splitting it to a separate application that does NOT start by default (dbus-daemon will start it if needed) we can get a lot of benefits. First we're getting the ability to not install dbus on servers where it has no use and free a little bit of resources. Also systemd init daemon don't have to link against libdbus any more. Fewer libraries linked - faster startup and smaller boot time which was the initial goal. These are advantages, are there any disadvantages to that? What do I miss? > That means I want to allow developers to rely that certain features in > the OS are available, for example socket activation -- so that they > actually use it. Hence making that optional would be diametrically against > my goals. I assume what you're calling "socket activation" here is the ability to create a socket outside of the application to allow parallel startup - the thing that were done by `sysunixd` and `sysinetd` in my example. Then, yes, I agree, it's a good idea and implemented by mainstream developers it would be better than patched by distribution maintainers. But then implementing it in a separate daemon works in both directions. It not only makes this feature optional, but also makes it compatible with other init systems. Which means that more users would be able to install and use it. Which means that it will spread faster. Right? > I see no value in the portability to other Unixes. I do see value however > in our platform becoming stronger by removing the big variable of the OS > we build on. Even if you don't need it (which is pretty usual unless you're using other unixes), portability is still a benefit. Such benefit may bring more users and more developers to the project. And this is definitely good. Is there anything bad? > Well, it's neither more flexible nor simpler or easier to support. And > portabality to other Unixes or being the "unix way" I don't consider > much of an advantage. A "unix way" was just an inspiration for the ideas I wrote about. > Not using D-Bus wastes memory, since programs have to reinvent their IPC > systems each and every time. 1 good IPC instead of 20 fucked up ones > means using less resources, having more security, more simplicity, and > more transparency. DBus is good in certain cases of course, but it's not a panacea either. > The idiocy of people who believe they could just remove the IPC from > their system and everything would still be able to communicate as before > but using fewer resources is amazing. The matter is not in communication between each other without IPC, but in the fact that there may be nothing to communicate. Whom can init-daemon communicate with on ssh/ftp/http-server? (and servers market is where Linux is
Re: [systemd-devel] The Linux Way or Some ideas to make systemd better
On Sun, 17.07.11 06:32, Sergey (sergem...@gmail.com) wrote: Heya, sorry, but I am not interested. Unix is an inspiration but it is not dogma. Integration avoids duplication, integration hence avoids bloat. Note that systemd is not monolothic anyway. For example, you can just remove the bus mechanisms, and the early-boot components and everything will still work fine. Modularization for the only sake of modularization is however not what what we buy into. The more you modularize the bigger the overhead of communication becomes, and that again ... is bloat. Also note that there's a reason why sockets, services, mount points and so on, are all handled in the same concept of units: they have complex interdependencies, and managing those if the managers supervising them are separate would mean complex synchronozation logic. In addition, you are very much overestimating the code size of the socket handling, and similar code. Splitting short code like tht into separate processes is wastage. Finally, I am also not interested to encourage too much mix and matching, have too many variables in the installation. I want to help streamline and standardize our platform. That means I want to allow developers to rely that certain features in the OS are available, for example socket activation -- so that they actually use it. Hence making that optional would be diametrically against my goals. I see no value in the portability to other Unixes. I do see value however in our platform becoming stronger by removing the big variable of the OS we build on. > "Why is this better?" > = > Because it's flexible, portable, simple, easy to support and it's > unix-way. Well, it's neither more flexible nor simpler or easier to support. And portabality to other Unixes or being the "unix way" I don't consider much of an advantage. (Also, I think you misunderstand Unix a bit.) > It does not have any extra dependencies. So if someone needs to build a > compact > system for netbook with 128MB RAM he can install just `systemd` and save a few > MB of RAM without dbus (Xorg+IceWM don't need dbus) and other systemd > services. > There's also no need in dbus on ssh/dns/http-servers. Not using D-Bus wastes memory, since programs have to reinvent their IPC systems each and every time. 1 good IPC instead of 20 fucked up ones means using less resources, having more security, more simplicity, and more transparency. I am not sure what the folks who claim dbus was bloat are smoking, but it certainly interferes with their ability to think logically. The idiocy of people who believe they could just remove the IPC from their system and everything would still be able to communicate as before but using fewer resources is amazing. You are welcome to question the implementation of D-Bus, but if you question the existance and need for it then this says more about your nescience than about D-Bus. > For example, this structure forces developers (or maintainers) of every > service > to write two startup scripts - one for systemd-based script and one sysv-init > script for non-systemd-based systems. Why not use a single script for that? > For example instead of writing second startup script we just add the line: > # TCPListen: 22 > to the LSB Header of /etc/init.d/sshd. That will make transition easy and > remains fully backward compatible. Services that don't have this line will > be treated as usual sysv services. I am not interested in that. If people want to keep using SysV scripts, it's their own burden. > That also makes developers' work easier - they won't have to write two > startup scripts and worry about compatibility. Upstream developers generally do not write init scripts, package maintainers do, simple because you have tow write different ones for different distros. Which coincidentally is something systemd fixes. Sorry if this reply is harsh, Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] The Linux Way or Some ideas to make systemd better
On Sat, Jul 16, 2011 at 8:32 PM, Sergey wrote: > The End > === > I'm writing these ideas here because I'm not experienced enough to implement > them myself. So I just hope that people in this list, that are much more > experienced, will like them. Thanks for reading. I've seen this type of post a dozen times. It's almost impossible to explain to you why your idea is probably not going to work, or why we don't want it, or All because, as you already said yourself, you're not experienced enough to do the work yourself. You can probably fix it by fixing exactly that: Get the skills needed to do projects like this. Once you do, you can either prove that we're wrong, or you'll understand why. Regards, Auke ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] The Linux Way or Some ideas to make systemd better
Op 17 jul 2011, om 05:32 heeft Sergey het volgende geschreven: > "Why is this better?" > = > Because it's flexible, portable, simple, easy to support and it's unix-way. > > Such structure would work under any circumstances on almost any configuration. > Users of other Linux distributions can install `sysunixd`/`sysinetd` and it > will work out of box, together with any existing init system. > > If someone, having a tightly integrated bunch of native upstart scripts, > cannot > switch entire init system, he can still install `sysunixd` and `sysinetd` and > get some speed improvements. Even if he cannot use `sysunixd` and `sysinetd` > for starting services he can still benefit from faster disks mounting using > `sysmountd`. > > It's extremely portable. Users of other operating systems can install almost > every component (except systemd-cgroups). Such structure can be ported even > to the systems that don't support UNIX sockets (just disable `sysunixd` > and patch `libsystemd` to use different communication method). > > It does not have any extra dependencies. So if someone needs to build a > compact > system for netbook with 128MB RAM he can install just `systemd` and save a few > MB of RAM without dbus (Xorg+IceWM don't need dbus) and other systemd > services. > There's also no need in dbus on ssh/dns/http-servers. FWIW, I run systemd on a 300MHz arm926 with 64MB ram :) ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] The Linux Way or Some ideas to make systemd better
Sounds like a UNIX way, but really no.
On 2011年07月17日 11:32, Sergey wrote:
> Hello.
>
> I was thinking about it for some time, and got a few ideas I wanted to
> share. Sorry for the long email, I'm just trying to explain in details
> what the ideas are about, what benefit they bring and why I've chosen
> that way, i.e. why these ideas cannot be done using existing tools.
>
> If you don't want to read the entire email, just scroll down to words
> "systemd packages".
>
>
> "What's the problem?"
> =
> There's such thing as Linux. I often tell people how great it is - it's so
> flexible, you can always modify it the way you want.
>
> But what makes it great? Programs? Features? Price? No, it's Unix philosophy
> behind it. The best part of it is:
> "Write programs that do one thing and do it well"
> What's so good in it?
>
> The matter is that there's no ideal program. Of course every program has
> bugs, but that's not the main reason. The main reason is that people don't
> like when program does not do what they expect AND different people expect
> different things. This is where the philosophy hits.
>
> On the windows-like systems, when something does not work like you want,
> it's still a single highly integrated system, you can't do much about it.
> But in Linux, where the system is a number of separated components, you
> can replace any part of it and everything else will still work. You don't
> like KDE - there's GNOME. You like KDE apps, but don't like KDE WM - easy,
> replace it with compiz and use the rest of KDE apps as usual. You need some
> light WM to save memory - IceWM is for you, all the browsers and office
> programs will still work there. You're setting up a server and don't want
> Xorg to eat your resources - no problem, just remove it. If you need, you
> can replace every part of the system, DM (KDM, GDM, XDM, SLIM), glibc
> (uclibc, eglibc), init-daemon (sysvinit, initng, upstart), even kernel...
> It's so cool, not many systems in the world give you such a great flexibility.
> All this works because in Linux all programs "do one thing and do it well".
>
> And there comes systemd, which aims to do every thing in the world and a
> few more. The question one can think when he sees it - is it really a
> linux program?
>
>
> "Bla-bla-bla, what do you suggest?"
> ===
> You can split it up! Make it the Unix way. Instead of a large and highly
> integrated bunch of programs make it a number of small separated components
> with weak dependencies between each other.
>
> That would give many benefits to the project. For developers it will make
> the structure a lot cleaner, easy to understand, easy to extend and support.
> Developers usually don't like bloated software that nobody-knows-how-it-works.
> For users it will give the ability to make it look like they want it to be.
> Any program is perfect for user if it does exactly what he wants. But it's
> impossible to write the program that suits everybody out of the box, so in
> the end the most flexible programs win.
>
> Trying to add too many features into a single program ("Let's write a builtin
> tetris game, that shows up when user presses C-A-D! Why? Why not?!") you may
> end up with none of them implemented good enough. It's rather pointless to
> make the change just to change something. I.e. you can fight with bash
> scripts, but what for? For speed? Bash can spawn ~700 scripts per second and
> can parse about 5 megabytes of code during the second. How much can you
> possibly win for 20 scripts a few kilobytes each? Instead of guessing what
> features can be implemented let's look at real tasks that should be solved,
> and implement each task in a separate application.
>
> What are those tasks? What exactly are those problems that should be fixed
> with new init system, that could not be fixed before? If the main goal -
> a faster boot process (I'm actually following a video presentation of
> systemd here), then let's look at the usual boot process:
> [udev] -> [filesystems] -> [iptables] -> [network] -> [syslog] ->
> [avahi] -> ...
> and see what we can do here.
>
> First task - start services in parallel. The key idea is to create all the
> sockets and then start all the services... well, you already know that. :)
> Let's remember this task.
>
> Second task - check and mount filesystems in parallel with starting services.
> Let's remember this task too.
>
> [do I miss anything else related to faster boot?]
>
> So, to get the first task done we need a single daemon, let's call it
> `sysunixd`, that does one simple thing - creates a lot of unix sockets and
> then starts a lot of applications. That's all. :)
> Can we do it using existing applications, like `inetd`? Well, not really. The
> key difference between `inetd` and `sysunixd` is that every inets service is
> either started on-demand (when someone accesses the socket) or is not started
> at all. But services in
[systemd-devel] The Linux Way or Some ideas to make systemd better
Hello.
I was thinking about it for some time, and got a few ideas I wanted to
share. Sorry for the long email, I'm just trying to explain in details
what the ideas are about, what benefit they bring and why I've chosen
that way, i.e. why these ideas cannot be done using existing tools.
If you don't want to read the entire email, just scroll down to words
"systemd packages".
"What's the problem?"
=
There's such thing as Linux. I often tell people how great it is - it's so
flexible, you can always modify it the way you want.
But what makes it great? Programs? Features? Price? No, it's Unix philosophy
behind it. The best part of it is:
"Write programs that do one thing and do it well"
What's so good in it?
The matter is that there's no ideal program. Of course every program has
bugs, but that's not the main reason. The main reason is that people don't
like when program does not do what they expect AND different people expect
different things. This is where the philosophy hits.
On the windows-like systems, when something does not work like you want,
it's still a single highly integrated system, you can't do much about it.
But in Linux, where the system is a number of separated components, you
can replace any part of it and everything else will still work. You don't
like KDE - there's GNOME. You like KDE apps, but don't like KDE WM - easy,
replace it with compiz and use the rest of KDE apps as usual. You need some
light WM to save memory - IceWM is for you, all the browsers and office
programs will still work there. You're setting up a server and don't want
Xorg to eat your resources - no problem, just remove it. If you need, you
can replace every part of the system, DM (KDM, GDM, XDM, SLIM), glibc
(uclibc, eglibc), init-daemon (sysvinit, initng, upstart), even kernel...
It's so cool, not many systems in the world give you such a great flexibility.
All this works because in Linux all programs "do one thing and do it well".
And there comes systemd, which aims to do every thing in the world and a
few more. The question one can think when he sees it - is it really a
linux program?
"Bla-bla-bla, what do you suggest?"
===
You can split it up! Make it the Unix way. Instead of a large and highly
integrated bunch of programs make it a number of small separated components
with weak dependencies between each other.
That would give many benefits to the project. For developers it will make
the structure a lot cleaner, easy to understand, easy to extend and support.
Developers usually don't like bloated software that nobody-knows-how-it-works.
For users it will give the ability to make it look like they want it to be.
Any program is perfect for user if it does exactly what he wants. But it's
impossible to write the program that suits everybody out of the box, so in
the end the most flexible programs win.
Trying to add too many features into a single program ("Let's write a builtin
tetris game, that shows up when user presses C-A-D! Why? Why not?!") you may
end up with none of them implemented good enough. It's rather pointless to
make the change just to change something. I.e. you can fight with bash
scripts, but what for? For speed? Bash can spawn ~700 scripts per second and
can parse about 5 megabytes of code during the second. How much can you
possibly win for 20 scripts a few kilobytes each? Instead of guessing what
features can be implemented let's look at real tasks that should be solved,
and implement each task in a separate application.
What are those tasks? What exactly are those problems that should be fixed
with new init system, that could not be fixed before? If the main goal -
a faster boot process (I'm actually following a video presentation of
systemd here), then let's look at the usual boot process:
[udev] -> [filesystems] -> [iptables] -> [network] -> [syslog] ->
[avahi] -> ...
and see what we can do here.
First task - start services in parallel. The key idea is to create all the
sockets and then start all the services... well, you already know that. :)
Let's remember this task.
Second task - check and mount filesystems in parallel with starting services.
Let's remember this task too.
[do I miss anything else related to faster boot?]
So, to get the first task done we need a single daemon, let's call it
`sysunixd`, that does one simple thing - creates a lot of unix sockets and
then starts a lot of applications. That's all. :)
Can we do it using existing applications, like `inetd`? Well, not really. The
key difference between `inetd` and `sysunixd` is that every inets service is
either started on-demand (when someone accesses the socket) or is not started
at all. But services in `sysunixd` should be either started always or not
started at all. "On-demand" startup in `sysunixd` may be a nice option, but
not a requirement, because we want to make boot faster, while on-demand start
makes it slower.
Another difference between `sysunixd` and
