Re: Documenting library promises.
HI, Remco wrote on Sat, Apr 07, 2018 at 09:23:53AM +0200: > What about having some sort of function yourlib_pledge_set_promises() > that sets up these requirements ? I think this also has the advantage of > automatically separating pledge specific code (and a man page) from > generic code as well as lessening the chance of users of your library > setting up incorrect promises for your library. Or in light of updates > to your library, users may not need to change their program's pledges > since the updated yourlib_pledge_set_promises function will hopefully be > sufficient to take care of that. Obviously a very stupid idea for several reasons: 1. You need one pledge(2) call per application program, not one pledge(2) call per library used. That call must include the promises required for all functionality the program uses, i.e. the union of promises needed for syscalls done directly from the program itself and syscalls done through each of the libraries. Obviously, that cannot be achieved by doing one call for each library used. 2. Even if a program uses no other libraries, you still have to add the promises required for directly invoked functionality. 3. Even if you require no promises for directly invoked syscalls, a program using a library doesn't necessarily need all the promises potentially required by the library. It depends on which functions from the library are called, and as Kristaps explained, it even makes a difference which arguments are passed to these functions. Just like for programs using nothing but the C library, by the way. Your suggestion is obviously completely contrary to the basic way pledge(2) works and cannot be implemented. Using pledge(2) requires understanding which functionality a program uses. Your proposal boils down to: "You do not need to understand your own program if you simply call a function that performs some unspecified magic which is impossible to perform." Yours, Ingo
Re: Documenting library promises.
Op 04/06/18 om 16:57 schreef Kristaps Dzonsons: Hi folks, Short: what do you recommend for documenting an external library's pledge(2) requirements? Longer: https://bsd.network/@florian/99802355448571943 The question raised in this... um... toot?... is which promises are required by an external library call, in this case khttp_parse(3) in kcgi. Sure, we can always just run the program, look in /var/log/messages for failure, and edit our promises. But just... no. In this particular case, I've documented this function's requirements unofficially here and there---tutorials and such. But it's not canonical. What I'd like is to put these directly into the manpages. Something like: .Sh SANDBOXING On .Ox , the .Fn khttp_parse function requires the .Qq stdio proc promises to .Xr pledge 2 . This encourages developers to use the tightest possible promises. And as mdoc(7) is meant not to be system-specific, this might also include information on, say, .Fx's Capsicum, or maybe whatever Linux uses this week. It already has "SECURITY CONSIDERATIONS", but that just doesn't seem quite right. Thoughts? Kristaps What about having some sort of function yourlib_pledge_set_promises() that sets up these requirements ? I think this also has the advantage of automatically separating pledge specific code (and a man page) from generic code as well as lessening the chance of users of your library setting up incorrect promises for your library. Or in light of updates to your library, users may not need to change their program's pledges since the updated yourlib_pledge_set_promises function will hopefully be sufficient to take care of that.
Re: Documenting library promises.
Ingo Schwarze wrote: > Hi Kristaps, > > Kristaps Dzonsons BSD.LV wrote on Sat, Apr 07, 2018 at 01:37:32AM +0700: > > > The only reason I suggest a standalone section is that it's easier > > to standardise across manpages. > > For that goal, using ".Ss Pledge promises" > at the end of the DESCRIPTION might work. > > For now, such consistency would be local to the ksql package (and > maybe some related packages like kcgi?), and i don't anticipate a > large number of other non-base libraries that want to integrate > with pledge(2) in the near future. But even if some other libraries > pick it up from there, that wouldn't seem undesirable to me. BTW, there is another approach: Describe what your function does properly. For instance, use terminology that a libc-level programmer will recognize as implying use of particular libc interfaces, which they can mentally maps back pledge concepts. (This reminds me of the warts in libc we got rid of -- to allow the creation of pledge. syslog() doesn't use sockets. DNS is a special socket, and regular sockets can't do DNS. isatty() is used extensively in libc, so it abstracted out to ensure ioctl is taken away. Various things like this were taken away inside libc, so that programmers don't need to be shocked "That operation seems so simple, why does it do something so complicated which I never expected?"
Re: Documenting library promises.
Hi Kristaps, Kristaps Dzonsons BSD.LV wrote on Sat, Apr 07, 2018 at 01:37:32AM +0700: > The only reason I suggest a standalone section is that it's easier > to standardise across manpages. For that goal, using ".Ss Pledge promises" at the end of the DESCRIPTION might work. For now, such consistency would be local to the ksql package (and maybe some related packages like kcgi?), and i don't anticipate a large number of other non-base libraries that want to integrate with pledge(2) in the near future. But even if some other libraries pick it up from there, that wouldn't seem undesirable to me. > How about .Sh RESOURCES? I don't like that because the term "resources" is so broad that it is hard for users to guess what they might find there. Maybe something related to X11, or to setrlimit(2), or books and mailing lists on the subject? Also, trying to standardize a section requires that the section is widely needed and that the subject matter is very mature. Premature attempts to standardize new top-level sections, mostly in other operating systems, already left us with various half-dead sections that are not being used consistently and of doubtful utility (LIBRARY, IMPLEMENTATION NOTES, SECURITY CONSIDERATIONS, ..., not even talking about what OpenSolaris / illumos did). > This would also fit other systems like Capsicum. > Though in practice, I think we'll find only pledge documented there. Then i'd suggest to not try to plan ahead for theoretical ideas that are not likely to get done at all. Besides, as Theo said, Capsicum is so different from pledge(2) that what is possible with pledge(2) likely cannot be done with Capsicum - regarding documentation just like regarding consistent application to the complete codebase of a whole operating system. Yours, Ingo
Re: Documenting library promises.
Kristaps Dzonsons wrote: > The only reason I suggest a standalone section is that it's easier to > standardise across manpages. I do not see a way to do this in libc. So standardise isn't really required. You are talking about doing this in a port library, not a base library. I don't know how we would do this in a base library, without being overly specific and screwing ourselves in the future somehow. It feels too early to give pledge a formal specification. > How about .Sh RESOURCES? This would also fit other systems like > Capsicum. Though in practice, I think we'll find only pledge documented > there. You really think you'll find common ground?? I'm extremely sceptical. The underlying concepts couldn't be different. pledge disables "grouped actions" taken against objects, whereas capsicum reduces ability to gain new objects but generally leaves all style of actions intact. A pledge context can be described in 2-3 words, but capsicum is a heavy environment one cannot describe so easily. Also, don't want capsicum or any other non-OpenBSD described in OpenBSD manual pages. What you do in your own pages is your own concern, but then premature standardization doesn't matter.
Re: Documenting library promises.
>> .Sh SANDBOXING >> On >> .Ox , >> the >> .Fn khttp_parse >> function requires the >> .Qq stdio proc >> promises to >> .Xr pledge 2 . > > As long as it is only a single sentence, that could easily go right > after the description of the individual function in the DESCRIPTION, > or alternatively at the end of the DESCRIPTION section. Custom > sections are not very nice, in particular when they are so short > that they hardly justify opening a new section in the first place. > > If you really want to describe multiple different sandboxing > systems, then i guess ".Sh SANDBOXING" after the DESCRIPTION > might make sense. I'm not all that sure that describing any other > system will be possible without going overboard; they are typically > much too complicated to be summarized without excessive verbosity. > But you shall no doubt see whether or not it works... Ingo, The only reason I suggest a standalone section is that it's easier to standardise across manpages. Saying "it should go after the function" is too open to interpretation. Moreover, some functions may have varying promises depending upon function parameters, which will rapidly eat into space better left to the functions themselves. How about .Sh RESOURCES? This would also fit other systems like Capsicum. Though in practice, I think we'll find only pledge documented there. Best, Kristaps
Re: Documenting library promises.
> .Sh SANDBOXING And please stop using that word. It has been misused so many times, by now it is misleads. pledge is not a sandbox (whatever the hell a sandbox is)
Re: Documenting library promises.
Hi Kristaps, Kristaps Dzonsons BSD.LV wrote on Fri, Apr 06, 2018 at 09:57:09PM +0700: > Short: what do you recommend for documenting an external library's > pledge(2) requirements? That is an interesting question indeed. I never considered it before, so i will think about it in some detail. For a bit of context, let me first explain how documentation of pledge(2) requirements inside the OpenBSD C library itself works, and why the documentation was designed that way. On first sight, it might seem most user-friendly if each and every section 3 manual page for the C library would state the pledge(2) requirements for each individual library function. But that would have three important downsides: (1) It would be hard to do. There are very many manual pages, so it would require much work for initial setup. (2) It would be unmaintainable, in particular while pledge(2) is still in very active development. Pledge evolved through incremental tweaking. Usage patterns were studied and modeled, the resulting pledge promises applied across the whole three, and the resulting experience allowed to refine the analysis and improve the way pledge(2) itself worked, based on real-world usage. Rinse and repeat, literally dozens of times, over years of careful work. Keeping many hundred manual pages in sync with that gradual evolution would have been flat-out impossible. (3) It would have put the focus in the wrong place. One among the chief ideas of pledge(2) is to avoid the splintered character of traditional syscall filters, where you are forced to filter each syscall, or even worse each of the arguments of each syscall, individually, and consequently completely lose the overview of what you are really doing before you even get started. Instead, pleadge promises enable high-level logical feature groups. The essential job of pledge(2) documentation is not to say that the FOO promise enables the BAR argument of the BAZ syscall, but that pledging FOO is required when you want to do operations of the BAR kind, no matter with which calls or arguments. Of course, once pledge(2) is completely stable, there would be nothing wrong with *also* documenting the effect in individual library functions manuals for user convenience, even though that would imply some redundancy. But given that pledge(2) is still quite actively worked on, i doubt it will happen soon, if ever at all. That said, for external libraries, putting your stuff directly into the pledge(2) manual is obviously not an option. Also, i agree that documenting pledge(2) requirements for external libraries is very useful, because these requirements are often quite non-obvious and hard to guess by just looking at the pledge(2) manual page itself, because many library manuals intentionally, and sensibly, do not document how exactly the library operates internally. Note that for external libraries, the above arguments do not apply: (1) Well-designed external libraries have small numbers of functions, so problem (1) is not a concern. (2) The logical functionality of functions in external libraries is hopefully mostly stable, and the required promises will rarely be affected by changes in pledge(2) itself. Also, pledge(2) has already stabilized considerably, so problem (2) is no longer a major concern - though a bit of maintenace may be required now and then if you put such information into your manual pages. (3) The pledge(2) manual explains the high-level logic, your manuals add the specifics for your library, without duplication of information. That is a clean division of tasks, so problem (3) is not a concern. > Longer: https://bsd.network/@florian/99802355448571943 > > The question raised in this... um... toot?... is which promises are > required by an external library call, in this case khttp_parse(3) in > kcgi. Sure, we can always just run the program, look in > /var/log/messages for failure, and edit our promises. But just... no. > > In this particular case, I've documented this function's requirements > unofficially here and there---tutorials and such. But it's not > canonical. What I'd like is to put these directly into the manpages. Absolutely agreed. "Complete functional description in one canonical place, do not scatter reference-manual-type information across multiple sources" is among the most important requirements for good documentation. > .Sh SANDBOXING > On > .Ox , > the > .Fn khttp_parse > function requires the > .Qq stdio proc > promises to > .Xr pledge 2 . As long as it is only a single sentence, that could easily go right after the description of the individual function in the DESCRIPTION, or alternatively at the end of the DESCRIPTION section. Custom sections are not very nice, in particular when they are so short that they hardly justi