Re: [Server-devel] Yummy incron
On Wed, Jun 25, 2008 at 2:50 PM, Martin Langhoff [EMAIL PROTECTED] wrote: On Wed, Jun 25, 2008 at 2:36 PM, Bill Bogstad [EMAIL PROTECTED] wrote: I'm confused. Won't the XS servers always have swap/paging space? Paging is tolerable for batch processes, or for interactive stuff that is not subject to timeouts (let the user wait while we bring OpenOffice back from swap). Network services and swapping don't mix well at all - they just timeout, which leads to users perceiving it as crashed, hitting refresh if available, and in our case perhaps volunteering a fix in the form of a hard reboot. Sure but any process that is waiting for a file to appear in the filesystem seem more like a batch process to me. There is no way to know how long it will take (and thus your timeouts). Ouch. If you look at apache and other web-services tuning strategies, the *first* thing they do is ensure that the working set _never_ touches swap. As soon as any network service gets swapped out, it's unavailable for all intents and purposes. Yes, you want your working set to fit in memory. Still not clear to me how a process which is waiting (I'm assuming for a while) for another process to do something can in any sense be considered part of the current working set. In fact the usage model of incrond is that it won't even be a process (so clearly not in the working set). A paged out process is also clearly not in the current working set. Still not seeing the problem. OTOH, I do see the problem of using cron to startup a process just to check for existence of a file. Some other rendezvous is preferable. You don't seem to like DBUS (I'm guessing because of the complexity of retrofitting it into preexisting programs or scripts). Perhaps using a named pipe with a write to it from the source process to wake up the destination process might be preferable. This can easily be done from any language, doesn't require an additional permanent daemon process (incrond), and allows the destination process to have gotten past its startup cost and be ready for immediate action. Paging the destination process back from disk should not be slower then starting a new process (via incrond). The only advantage I see of incrond is the destination process can remain almost oblivious to the signaling mechanism. Named pipe rendezvous would require a small amount of additional code in both the source and destination process. Processes sitting idly in memory are wasteful unless they have a ton of state in them to make them valuable. And even then, in many cases they can just marshall that data, and pick it up later (lots of exceptions for this - connection poolers for example). Processes that are truly 'idle' (waiting for a socket event for example) should be paged out as the system needs their memory. I can see lots of methods to avoid filesystem polling which don't involve adding a new daemon to the set of programs that OLPC is already supporting. Maybe the tradeoff is such that incrond is still the right choice, but I don't see why you seem to be objecting to a paged out process (as opposed to one which hasn't even been started yet). If the memory taken up by the process slot and other kernel bookkeeping is that much of a problem, then things are already so close to desperate that you are probably going to fall over soon anyway. Bill Bogstad ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] Yummy incron
On Thu, Jun 26, 2008 at 12:45 AM, Bill Bogstad [EMAIL PROTECTED] wrote: Sure but any process that is waiting for a file to appear in the filesystem seem more like a batch process to me. There is no way to know how long it will take (and thus your timeouts). Bill, everything you say makes sense, except that perhaps a key concept is missing. Most of our scripts and processes are in Python. If I have 20 such scripts idling in memory, waiting for something to happen (a dbus event?), the footprint is huge - clearly this does not scale. Even if they get paged out. Incron, otoh, is a single process, weighing 600K, and can have a config file listing 20 scripts that might be triggered by a (inotify) event. Make that 200 scripts. The memory footprint doesn't change. You don't seem to like DBUS Why do you repeat that? I have no problem with DBus where it makes sense: processes that are guaranteed to be running in memory all the time. The key problem with paging out is that if you have a dozen idle processes in memory that are network daemons, and a dozen that are batch processes waiting for a dbus signal, the kernel will page them indiscriminately - the batch ones will be able to do their job ok when called, the network ones will timeout on their users. The kernel has no way of knowing. This isn't theoretical stuff -- rather very practical concerns of offering network services in real life. Memory usage matters. OTOH, I'm open to seeing facts that contradict my analysis - if you or anyone has a strategy that is better than incron - say, way to keep 20 to 200 python scripts in memory in 600K of (safely swappable) memory - I'd love to see it. cheers, m -- [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] Yummy incron
On Tue, Jun 24, 2008 at 8:54 PM, Martin Langhoff [EMAIL PROTECTED] wrote: Looking for a (memory, cpu, power) efficient way to trigger events/scripts on the XS, I came across incrond. It weights ~600KB according to ps_mem.py, and it looks like the kind of tool we want to be using. There are a few processes I have on the XS that signal completion by touching a file in a tmpdir, so that a different process (with different privileges) can perform other steps. Using incron rather than poll frequently from an in-memory process or crond seems much smarter. rant Yes, this a bit like DBus, but DBus was designed for a desktop with Gobs Of Mighty RAM, and it requires that your process must be running in memory, and that it'd be forking/threaded if you want to process stuff in parallel. This means our (mostly python) processes are sitting idly on memory we don't have the luxury to spare. And that little or nothing happens in parallel. I'm confused. Won't the XS servers always have swap/paging space? Idle processes should take hardly any RAM at all. Not being familar with python's threading model, it may be that idle threads (in a single process) will still take up memory. Even then though, partial process/thread paging should be active. Does DBUS wake up processes it shouldn't? (creating RAM thrashing problems) That seems to me to be more of a problem with DBUS message types not being specific enough. OTOH, If you were talking about the XO platform (paging to flash probably being a bad idea), I would agree with you. Bill Bogstad ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] Yummy incron
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Langhoff wrote: | Looking for a (memory, cpu, power) efficient way to trigger | events/scripts on the XS, I came across incrond. It weights ~600KB | according to ps_mem.py, and it looks like the kind of tool we want to | be using. There are a few processes I have on the XS that signal | completion by touching a file in a tmpdir, so that a different process | (with different privileges) can perform other steps. Using incron | rather than poll frequently from an in-memory process or crond seems | much smarter. | | rant | Yes, this a bit like DBus What it sounds even more like, to me, is any modern init system. For example: Upstart (http://upstart.ubuntu.com/) OpenRC (http://roy.marples.name/openrc) InitNG (http://www.initng.org/) etc. This new generation of init are designed to provide proper inter-service dependency management and post-startup event response. It appears that both Fedora and Ubuntu are using Upstart. Given your opposition to DBUS in this context, you might enjoy http://upstart.ubuntu.com/faq.html#replace-dbus - --Ben -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkhhnAQACgkQUJT6e6HFtqRJRgCfY/+oj/iKJ076oyeT41HlA9QE NegAnjQqaHTYBHCSu4F5+nWJuG6ck2sv =XMIJ -END PGP SIGNATURE- ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] Yummy incron
On Tue, Jun 24, 2008 at 9:14 PM, Benjamin M. Schwartz [EMAIL PROTECTED] wrote: What it sounds even more like, to me, is any modern init system. For example: I'm very keen on new init systems, but this is not one. Given your opposition to DBUS in this context, you might enjoy :-) I'm not really opposed to DBus, but it encourages a model that only makes sense for a subset of what we are doing. It's great to tell the main Sugar shell something, as you surely expect it to be there, up all the time. Other things, well, they have no business being in memory - http://upstart.ubuntu.com/faq.html#replace-dbus And that tells you exactly what I've been saying - DBus != Upstart, and I'll add that both of those are != incrond. cheers, m -- [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
