Re: [DNG] /etc/debian_version
On Sat, Apr 15, 2017 at 01:06:53PM +0200, Jaromil wrote: > Yet I believe we should file this as a bug as /etc/debian_version is > not there on new installs. The /etc/debian_version file is checked by > an enormous quantity of scripts in all sorts of deployements, we have > encountered this problem also with Vagrant, that I remember. So we > shall keep that file around and fill it with the Debian version we use > to fallback packages in each release, that is so far: Thanks. File this as a bug where, debian or devuan, and against what specifically? Greg -- web site: http://www.gregn.net gpg public key: http://www.gregn.net/pubkey.asc skype: gregn1 (authorization required, add me to your contacts list first) If we haven't been in touch before, e-mail me before adding me to your contacts. -- Free domains: http://www.eu.org/ or mail dns-mana...@eu.org ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On Sat, 15 Apr 2017 22:50:20 +0200 marcwrote: > > Concerning the fork-parent.c and fork-parent.h revealed at > > http://welz.org.za/notes/fork-parent.tar.gz , I can't envision how > > you'd bind the fork-parent() function to the stuff your daemon > > actually does, in order to have your daemon properly fork and then > > terminate the parent when ready for business. > > > > I look at the code in fork-parent.c, and it looks to me like all the > > actual tasks of the daemon should be placed immediately above the > > child's closing p[1], so p[1] is closed when the daemon is ready. > > But in fact, fork-parent() has no interface to the real work of the > > daemon, as far as I can see. > > So fork_parent() can be seen as a minor extension of the > daemon(3) library call: In both cases after you have called it > your function is now running in the child process. The parent > process stays "stuck" in the fork_parent() or daemon() > functions. > > Note that there are "extra" close() calls in the fork_parent() > code, as after the fork() both ends of the pipe are visible > in both processes. Without closing the "other" end in each > process a read would not return EOF and the parent would stay > up even after the child has closed its end. > > Here pseudocode, commented differently: > > pipe() /* done in fork_parent */ > if(fork() > 0){ /* in fork_parent */ > close() /* in fork_parent, not the stderr of the child > */ read()/* in fork_parent */ > exit()/* in fork_parent */ > /* parent no longer exists beyond this point */ > /* this point in code, not in time */ > } > > /* here we are in the child, the parent is still running */ > /* here do all the complicated setup routes, eg */ > /* check databases, bind ports */ > ... Wait. Are you saying that the way fork_parent() is used is to modify the source code of fork_parent() itself? If that's the case, I understand. I was under the impression that fork-parent() was a library call that you made to properly background your source code, which exists outside of function fork_parent(). And therefore I was wondering why fork_parent() didn't take a function address as an argument, and call that callback function's address where you have the elipses. > /* when everything is ready, you do: */ > > close(STDERR_FILENO); Yes. It makes sense. My question was where you put the code represented by your elipses above. Is fork_parent() a template (in the general sense, not the C++ sense) that you modify, or is it a tool you use, as is, to background code you put in a separate function? SteveT Steve Litt April 2017 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
> Concerning the fork-parent.c and fork-parent.h revealed at > http://welz.org.za/notes/fork-parent.tar.gz , I can't envision how > you'd bind the fork-parent() function to the stuff your daemon actually > does, in order to have your daemon properly fork and then terminate the > parent when ready for business. > > I look at the code in fork-parent.c, and it looks to me like all the > actual tasks of the daemon should be placed immediately above the > child's closing p[1], so p[1] is closed when the daemon is ready. But > in fact, fork-parent() has no interface to the real work of the daemon, > as far as I can see. So fork_parent() can be seen as a minor extension of the daemon(3) library call: In both cases after you have called it your function is now running in the child process. The parent process stays "stuck" in the fork_parent() or daemon() functions. Note that there are "extra" close() calls in the fork_parent() code, as after the fork() both ends of the pipe are visible in both processes. Without closing the "other" end in each process a read would not return EOF and the parent would stay up even after the child has closed its end. Here pseudocode, commented differently: pipe() /* done in fork_parent */ if(fork() > 0){ /* in fork_parent */ close() /* in fork_parent, not the stderr of the child */ read()/* in fork_parent */ exit()/* in fork_parent */ /* parent no longer exists beyond this point */ /* this point in code, not in time */ } /* here we are in the child, the parent is still running */ /* here do all the complicated setup routes, eg */ /* check databases, bind ports */ ... /* when everything is ready, you do: */ close(STDERR_FILENO); /* at this point the parent exits (see above) */ /* and the rcS scripts know to start the dependencies */ It turns out that the same basic effect can be achieved with a pause() and kill(getppid()), but that makes it tricky to relay errors like "-x requires a parameter" to stderr, where usage errors should be reported. regards marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On Sat, 15 Apr 2017 08:14:48 +0200 marcwrote: > I have written up the details at > http://welz.org.za/notes/on-starting-daemons.html Concerning the fork-parent.c and fork-parent.h revealed at http://welz.org.za/notes/fork-parent.tar.gz , I can't envision how you'd bind the fork-parent() function to the stuff your daemon actually does, in order to have your daemon properly fork and then terminate the parent when ready for business. I look at the code in fork-parent.c, and it looks to me like all the actual tasks of the daemon should be placed immediately above the child's closing p[1], so p[1] is closed when the daemon is ready. But in fact, fork-parent() has no interface to the real work of the daemon, as far as I can see. So how does one bolt fork-parent() on to his or her daemon code to enable it with proper backgrounding and timing? Thanks, SteveT Steve Litt April 2017 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On Sat, 15 Apr 2017 13:50:22 -0400 Steve Littwrote: Oops. Concerning my statement: > What I > wrote was based on "why would you not simply use a process supervisor > like systemd?" s/systemd/daemontools/ SteveT Steve Litt April 2017 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On Sat, 15 Apr 2017 19:12:31 +0200 marcwrote: > I appreciate that Steve likes the daemontools approach where the > server process stays in the foreground, and I have no quarrel with > the advice of including a -f foreground option... The daemontools > approach makes a completely different set of tradeoffs - some > people really like them, others consider them to be the lunatic > fringe. To each their own. > > But for a classic sysvinit startup path, the above logic > will do a decent job of signalling that a process is ready to > serve requests. And has been an appropriate mechanism for > decades. No need for yet another API. I'll admit this: For the person not wanting to use either a daemontools-inspired process supervisor, nor systemd (which I understand has the same capability as daemontools to background a foreground process), the fork-early, parent blocks, kid does the work, kid messages (and in this case the message is just closing the pipe), parent unblocks and exits is an excellent dependency tool, assuming all daemons do it, do it right, and do it accurately. About my characterizations: "Baroque" is a relative thing. What I wrote was based on "why would you not simply use a process supervisor like systemd?" If a person has a reason not to use such a supervisor, and in fact the whole OpenRC init system seems to be based on such objections to supervisors, then the six system call solution you outline transitions from "a whole bunch of needless stuff" to "hey, that's a pretty darn kool solution." So your solution is baroque only so far as one enjoys using daemontools or similar. LOL, as far as subtle, I'm just not seeing subtle in your solution. The implementation is elegant, but is the solution subtle? Not in my opinion. Then again, I don't think any software I ever wrote was subtle. I have some technical questions about your software, but that's for another email. SteveT Steve Litt April 2017 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
Hello again Steve > > The TLDR is that "A good daemon should fork soon, but have the > > parent process exit only once the child is ready to service requests" > > > > Sadly it seems too subtle for many people and the concept goes > > un-noticed ... > > It's as subtle as a 10 megaton nuclear device, and goes un-noticed > because most of us who know of it hope it will be forgotten, so we > don't speak of it. > > Read http://welz.org.za/notes/on-starting-daemons.html. According to > this document, quite apart from the daemon's real function, the daemon > must fork itself early, form a pipeline between parent and child, and > when the child believes it's ready to function completely, it sends a > message to the parent, who upon receipt of the message exits. That's > not subtle: It's quite baroque. Your idea of canning it into a neat > little tool is quite elegant, but it's an elegant implementation of > something baroque. Baroque, 10 megaton nuclear device, eh ? It is (error checking/reporting elided) an effort of 6 system calls: pipe() if(fork() > 0){ /* if we are in parent */ close() /* close the child end of the pipe */ read()/* wait for child */ exit()/* exit the parent process */ } ... /* do whatever is need to bring up the daemon */ close()/dup2() /* no message required, Steve misunderstood that */ If these system calls make you hide in a nuclear fallout shelter with your powdered white wig, one worries what will happen if somebody tells you what gethostbyname() does. Or even worse have you straced python recently ? It does 800+ system calls before it even gets going ... and it only goes downhill from there. I appreciate that Steve likes the daemontools approach where the server process stays in the foreground, and I have no quarrel with the advice of including a -f foreground option... The daemontools approach makes a completely different set of tradeoffs - some people really like them, others consider them to be the lunatic fringe. To each their own. But for a classic sysvinit startup path, the above logic will do a decent job of signalling that a process is ready to serve requests. And has been an appropriate mechanism for decades. No need for yet another API. regards marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On 15.04.2017 00:04, Daniel Abrecht wrote: > I still think this isn't a problem the service manager should attempt to > solve. This is a situation where the database is temporary unavailable, > for which there are many possible reasons. The services which relay on > the database should be able to handle such a situation gracefully, or > they are just not yet stable enough. In theory, you're right. But the actual practise tends to be very different. Probably, databases aren't the prime usecase for that. There could be other scenarios, where a service has to do some preparations for others to work (maybe certain filesystems mounted, etc). I remember, eg., problems w/ nfs, when one of the dozens servers isn't up yet, so they needed to be started in specific order. Years went by since I last touched that, no idea that's still the case. --mtx ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On 15.04.2017 03:17, Steve Litt wrote: > Sounds like a good idea, but it's not, for the following reasons: > > 1) There will be endless arguments about HOW each and every daemon will >let its init system know "I'm now ready for business". The number of >different ways will make the number of incompatible Unixes of 1985 >seem trivial to reconcile. Exactly the reason why I'd put it an separate library - so it can be easily replaced (even on binary level), w/o ever having to touch the application itself. It will be the operator's choice, which one shall be used. The whole problem arised w/ upstreams wanting such an mechanism, and nobody there who gagged up Lennart quick enough ... So, what do we make out of that now ? a) patch behind all the upstreams all that thoughless upstreams for all eternity b) maintain an dummy drop-in replacement (and keep up w/ everyting that the Lennartists come up) for all eternity c) provide a simple and stable API for exactly the root problem, (and convince upstreams to use that instead of Lennartware) and everybody can go its own way in piece. > 1.5) The resource daemon's opinion of being "ready for business" might > be erroneous, or not reflect how the user daemon will use the > resource daemon. Such things must be defined (on per service level) anyways - we'll never get around that. All we can do is try to standardize certain scenarios. For start, I'd define "READY" as fully operational. If some daemon provides several services, which be up/down independently (which I'd consider bad design), we could introduce sub-services (eg. apache might serve http as well as ftp) - would just add another const char* parameter. > 2) The connected and powerful will unilaterally declare one method of >"phoning home" is the one true way, thereby driving compatability >wedges between various software, and facilitating the defacto >banning of some daemons from a distro. That something we do not want. Actually the (de)potterization is one aspect of that. > 3) If the "one true way" is complex, for instance, if it has to >intimately mesh with dbus (sy what?), small daemon projects >will be ostricized out of the community. There doesn't need to be "one true way". It's only for a specific range of scenario: let the daemon just shout out it's overall status (what's required for startup ordering, and similar stuff, eg. some *trivial* monitoring) to anybody who's interested in it - the exact definition of that "anybody" would be confined to the specific implementation of the library - the application has no idea about that, and shouldn't even care whether somebody's listening. So, again, in the trivial case that would just be an dummy. If the library also maintains pidfiles (which I'd recommend, so we now have that stuff confined in one place), the lib of course has to implement that. But whether anybody actually reacts on the service notifies should be completely optional. As an application developer, I can just call that library, and dont have to care what happens then (okay, I'll have to consider that it might a few cycles in there). > 4) The problem this backward communication purports to solve was solved >around the time the Western Roman Empire was conquored by the >Barbarians. You mean, back then when the fascists just killed everybody who didn't submit to their rule and destroyed freedom (we still suffer from the roman law) ? ... >Regardless of your init system, run your daemons from >daemontools, and have your daemontools run scripts refuse to start >unless a test of the needed resource indicates it's working. >Whatever init system you're using, it's easy to get Daemontools, or >preferably Daemontools-Encore, running.[1] How exactly does it get that notification from the application ? Or does it go the other way round, having - per application - monitoring programs, that try to figure out whether the service is up and signals that to the rest ? Again: application developers want an interface for sending such notifies to the init system / service monitor. Whether you, or me, or anybody else, is actually using that, isn't in their scope - they just want that interface, therefore they're using the only thing they know, which happens to be Lenntart's proprietary one. So, let's just give them what they want, in a way that doesn't cause us any pain. > A word about the source of this thread: A guy, whom I /dev/nulled over a > year ago for making very disturbing political posts in response to > technical discussions, magically woke up a few days ago and started > resurrecting all sorts of dead threads. You're not talking about me, don't you ? (you wouldn't reply my mails then, right ?) > My /dev/null log tells me he's > sent 20 or 30 emails. I don't begin to know what causes this person to > act this way, but he's not a friend of our community, and we might want > to think twice about
[DNG] Devuan 1.0 RC and DistroWatch
How can you advertise to Devuan 1.0 RC in DistroWatch? Best Regrards | ISMAEL | ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On 15.04.2017 11:42, Steve Litt wrote: > Once upon a time, daemon self-backgrounding was necessary, and this nohup (or something similar) was not sufficient ? I don't know when start-stop-daemon was incarnated (related to daemontools ?), but IMHO it seems to handle the daemonizing quite well. IMHO, a daemon should not daemonize itself, but always leave that to the some supervisor or startup tool. And if it can handle one connection per process, it should even leave socket listening to something inted-alike. Even, for multi-connection clients, opening the listener socket should be left to it. (people can still optionally provide wrappers that glue everything together to an completely-standalone server machinery, for convenience). --mtx ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] pbuilder w/ docker
On 15.04.2017 10:42, Adam Borowski wrote: > Nope, this doesn't work unless you either: > * have a multiarch mixed i386+amd64 on the host -- you need at least the > kernel to be amd64 (well, if the kernel is self-compiled you don't even > need multiarch) -- but why would you run i386 on an amd64 box? Yeah, that's right. I'm running an amd64 kernel, but 32bit userland on Trusty on my notebook, but also some jails for special applications like skype, but also for testing. > * use qemu (tcg) to emulate amd64, that works but is not pretty on any > machine ancient enough to need i386 that would be *SLOOOW*. OTOH, a complete crosscompiling environment would be quite nice. But then we'd need a clear separation of libs vs tools dependencies. --mtx ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] /etc/debian_version
On Fri, 14 Apr 2017, Gregory Nowak wrote: > cat: /etc/debian_version: no such file or directory. > > I can probably put that file back, and all would be good. However, > my question is what is still looking for that file during apt-get > dist-upgrade, and more to the point, how can I make this message > during upgrades stop without bringing back /etc/debian_version? Devuan does have already its own /etc/devuan_version which can be symlinked from /etc/debian_version especially since the correct value its jessie for both. Yet I believe we should file this as a bug as /etc/debian_version is not there on new installs. The /etc/debian_version file is checked by an enormous quantity of scripts in all sorts of deployements, we have encountered this problem also with Vagrant, that I remember. So we shall keep that file around and fill it with the Debian version we use to fallback packages in each release, that is so far: Jessie -> Jessie Ascii -> Stretch After Ascii is yet to be seen if we'll keep relying on Debian. ciao ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
On Sat, 15 Apr 2017 08:14:48 +0200 marcwrote: > > > Go on down your path, but I suspect not many people would cheer > > > at you in this camp... > > > > I can see the merit - on two points, technical and political. > > > > > On the technical side, I can see the usefulness of a system > > wide standardised service status reporting - making it easy > > for one process to see if a service it depends on is actually > > running and ready (as opposed to, "has started"). I have a > > customer system I've inherited where it regularly fails to > > startup properly because Asterisk starts before MySQL has > > finished starting up. > > So it turns out that there exists an subtle yet elegant mechanism > for a process to report that it is ready. It has been in use > for decades - daemons as old as sysklog from the previous century > use it. > > I have written up the details at > http://welz.org.za/notes/on-starting-daemons.html Danger Will Robinson! If you write a daemon that backgrounds itself, whether early or late, whether parent and child communicate via signals or pipes or, well, what the heck, network sockets, be ABSOLUTELY sure to include a command line option to tell your daemon NOT to background itself, but to stay in the foreground all the time. If your daemon cannot stay in the foreground, your daemon is incompatible with runit, s6, perp and daemontools. And this foreground option should be separate from any "debugging" options, because nobody wants production logs to have debugging output. > > The TLDR is that "A good daemon should fork soon, but have the > parent process exit only once the child is ready to service requests" > > Sadly it seems too subtle for many people and the concept goes > un-noticed ... It's as subtle as a 10 megaton nuclear device, and goes un-noticed because most of us who know of it hope it will be forgotten, so we don't speak of it. Read http://welz.org.za/notes/on-starting-daemons.html. According to this document, quite apart from the daemon's real function, the daemon must fork itself early, form a pipeline between parent and child, and when the child believes it's ready to function completely, it sends a message to the parent, who upon receipt of the message exits. That's not subtle: It's quite baroque. Your idea of canning it into a neat little tool is quite elegant, but it's an elegant implementation of something baroque. It's also not a good solution. It won't work unless: 1) ALL daemons act this way 2) All daemons accurately time parent destruction to full readiness 3) Nobody uses daemontools, runit, s6, perp, etc. Once upon a time, daemon self-backgrounding was necessary, and this fork-early, parent terminate on readiness was state of the art and the Cadillac of the industry. But then, in 2001, daemontools was invented, making self-backgrounding unnecessary. From 2001 on, a daemon author could write a daemon as a normal foreground program, confident that daemontools, or one of its descendents (runit, s6, perp, daemontools-encore) would handle all aspects of getting it into the background while still supervising it, without the need of a PID file. Because daemontools, daemontools-encore, runit, s6 (and I would guess) perp can run as mere process supervisors on any arbitrary init system, the daemontools method of supervising foreground daemons is available to anyone willing to install daemontools, and installing daemontools is trivial. Anyway, by far the most important thing I said in this email is this: If you insist on making your daemon self-background, for gosh sakes give it a command line option to stay in the foreground, and make that command line option quite apart from any option that increases debugging output. SteveT Steve Litt April 2017 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] pbuilder w/ docker
On Sat, Apr 15, 2017 at 10:24:58AM +0200, aitor_czr wrote: > If you are in i386, you can build for amd64 build a chroot jail by the > following way: > > cowbuilder --create --distribution="jessie" --architecture="amd64" Nope, this doesn't work unless you either: * have a multiarch mixed i386+amd64 on the host -- you need at least the kernel to be amd64 (well, if the kernel is self-compiled you don't even need multiarch) -- but why would you run i386 on an amd64 box? * use qemu (tcg) to emulate amd64, that works but is not pretty on any machine ancient enough to need i386 -- ⢀⣴⠾⠻⢶⣦⠀ Meow! ⣾⠁⢠⠒⠀⣿⡁ ⢿⡄⠘⠷⠚⠋⠀ Collisions shmolisions, let's see them find a collision or second ⠈⠳⣄ preimage for double rot13! ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] pbuilder w/ docker
Hi, On 04/14/2017 07:19 PM, "Enrico Weigelt, metux IT consult"wrote: Hi folks, anyone here already used pbuilder w/ docker ? (instead of, eg. cowbuilder) Or is there anything else that can do that ? --mtx If you are in i386, you can build for amd64 build a chroot jail by the following way: cowbuilder --create --distribution="jessie" --architecture="amd64" and using: --git-pbuilder as an argument for git-buildpackage. All the build dependencies will be installed automatically within the jail, and removed after the packaging proccess. However, i've never got it working on Devuan. Cheers, Aitor. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] We need to speak up
Hi, On 04/14/2017 01:13 PM, Jaromilwrote: What is GUIX ? distro-agnostic packaging system done right (aka using a functional language) https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-package.html ciao Somebody talked about GUIX in the past (here or in the IRC Channel), one of the libre distros endorsed by the FSF. It uses Lisp in its packaging system, i seem to remember. Cheers, Aitor. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] tiny service state api [WAS: Fwd: init system agnosticism]
> > Go on down your path, but I suspect not many people would cheer at you > > in this camp... > > I can see the merit - on two points, technical and political. > > > On the technical side, I can see the usefulness of a system > wide standardised service status reporting - making it easy > for one process to see if a service it depends on is actually > running and ready (as opposed to, "has started"). I have a > customer system I've inherited where it regularly fails to > startup properly because Asterisk starts before MySQL has > finished starting up. So it turns out that there exists an subtle yet elegant mechanism for a process to report that it is ready. It has been in use for decades - daemons as old as sysklog from the previous century use it. I have written up the details at http://welz.org.za/notes/on-starting-daemons.html The TLDR is that "A good daemon should fork soon, but have the parent process exit only once the child is ready to service requests" Sadly it seems too subtle for many people and the concept goes un-noticed ... regards marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng