Re: Documenting library promises.

[Ingo Schwarze]

HI,

Remco wrote on Sat, Apr 07, 2018 at 09:23:53AM +0200:

> What about having some sort of function yourlib_pledge_set_promises() 
> that sets up these requirements ? I think this also has the advantage of 
> automatically separating pledge specific code (and a man page) from 
> generic code as well as lessening the chance of users of your library 
> setting up incorrect promises for your library. Or in light of updates 
> to your library, users may not need to change their program's pledges 
> since the updated yourlib_pledge_set_promises function will hopefully be 
> sufficient to take care of that.

Obviously a very stupid idea for several reasons:

 1. You need one pledge(2) call per application program, not one
pledge(2) call per library used.  That call must include the
promises required for all functionality the program uses, i.e.
the union of promises needed for syscalls done directly
from the program itself and syscalls done through each of the
libraries.  Obviously, that cannot be achieved by doing one
call for each library used.

 2. Even if a program uses no other libraries, you still have to
add the promises required for directly invoked functionality.

 3. Even if you require no promises for directly invoked syscalls,
a program using a library doesn't necessarily need all the
promises potentially required by the library.  It depends
on which functions from the library are called, and as
Kristaps explained, it even makes a difference which
arguments are passed to these functions.  Just like for
programs using nothing but the C library, by the way.

Your suggestion is obviously completely contrary to the basic way
pledge(2) works and cannot be implemented.

Using pledge(2) requires understanding which functionality a program
uses.  Your proposal boils down to:  "You do not need to understand
your own program if you simply call a function that performs some
unspecified magic which is impossible to perform."

Yours,
  Ingo

Re: Documenting library promises.

[Remco]

Op 04/06/18 om 16:57 schreef Kristaps Dzonsons:

Hi folks,

Short: what do you recommend for documenting an external library's
pledge(2) requirements?

Longer: https://bsd.network/@florian/99802355448571943

The question raised in this... um... toot?... is which promises are
required by an external library call, in this case khttp_parse(3) in
kcgi.  Sure, we can always just run the program, look in
/var/log/messages for failure, and edit our promises.  But just... no.

In this particular case, I've documented this function's requirements
unofficially here and there---tutorials and such.  But it's not
canonical.  What I'd like is to put these directly into the manpages.

Something like:

.Sh SANDBOXING
On
.Ox ,
the
.Fn khttp_parse
function requires the
.Qq stdio proc
promises to
.Xr pledge 2 .

This encourages developers to use the tightest possible promises.  And
as mdoc(7) is meant not to be system-specific, this might also include
information on, say, .Fx's Capsicum, or maybe whatever Linux uses this
week.  It already has "SECURITY CONSIDERATIONS", but that just doesn't
seem quite right.

Thoughts?

Kristaps




What about having some sort of function yourlib_pledge_set_promises() 
that sets up these requirements ? I think this also has the advantage of 
automatically separating pledge specific code (and a man page) from 
generic code as well as lessening the chance of users of your library 
setting up incorrect promises for your library. Or in light of updates 
to your library, users may not need to change their program's pledges 
since the updated yourlib_pledge_set_promises function will hopefully be 
sufficient to take care of that.

Re: Documenting library promises.

[Theo de Raadt]

Ingo Schwarze  wrote:

> Hi Kristaps,
> 
> Kristaps Dzonsons BSD.LV wrote on Sat, Apr 07, 2018 at 01:37:32AM +0700:
> 
> > The only reason I suggest a standalone section is that it's easier
> > to standardise across manpages.
> 
> For that goal, using ".Ss Pledge promises"
> at the end of the DESCRIPTION might work.
> 
> For now, such consistency would be local to the ksql package (and
> maybe some related packages like kcgi?), and i don't anticipate a
> large number of other non-base libraries that want to integrate
> with pledge(2) in the near future.  But even if some other libraries
> pick it up from there, that wouldn't seem undesirable to me.

BTW, there is another approach:

Describe what your function does properly.

For instance, use terminology that a libc-level programmer will
recognize as implying use of particular libc interfaces, which they can
mentally maps back pledge concepts.

(This reminds me of the warts in libc we got rid of -- to allow the
creation of pledge.  syslog() doesn't use sockets.  DNS is a special
socket, and regular sockets can't do DNS.  isatty() is used extensively
in libc, so it abstracted out to ensure ioctl is taken away.  Various
things like this were taken away inside libc, so that programmers don't
need to be shocked "That operation seems so simple, why does it do
something so complicated which I never expected?"

Re: Documenting library promises.

[Ingo Schwarze]

Hi Kristaps,

Kristaps Dzonsons BSD.LV wrote on Sat, Apr 07, 2018 at 01:37:32AM +0700:

> The only reason I suggest a standalone section is that it's easier
> to standardise across manpages.

For that goal, using ".Ss Pledge promises"
at the end of the DESCRIPTION might work.

For now, such consistency would be local to the ksql package (and
maybe some related packages like kcgi?), and i don't anticipate a
large number of other non-base libraries that want to integrate
with pledge(2) in the near future.  But even if some other libraries
pick it up from there, that wouldn't seem undesirable to me.

> How about .Sh RESOURCES?

I don't like that because the term "resources" is so broad that it
is hard for users to guess what they might find there.  Maybe
something related to X11, or to setrlimit(2), or books and mailing
lists on the subject?  Also, trying to standardize a section requires
that the section is widely needed and that the subject matter is
very mature.  Premature attempts to standardize new top-level
sections, mostly in other operating systems, already left us with
various half-dead sections that are not being used consistently and
of doubtful utility (LIBRARY, IMPLEMENTATION NOTES, SECURITY
CONSIDERATIONS, ..., not even talking about what OpenSolaris /
illumos did).

> This would also fit other systems like Capsicum.
> Though in practice, I think we'll find only pledge documented there.

Then i'd suggest to not try to plan ahead for theoretical ideas
that are not likely to get done at all.  Besides, as Theo said,
Capsicum is so different from pledge(2) that what is possible with
pledge(2) likely cannot be done with Capsicum - regarding documentation
just like regarding consistent application to the complete codebase
of a whole operating system.

Yours,
  Ingo

Re: Documenting library promises.

[Theo de Raadt]

Kristaps Dzonsons  wrote:

> The only reason I suggest a standalone section is that it's easier to
> standardise across manpages.

I do not see a way to do this in libc.  So standardise isn't really
required.  You are talking about doing this in a port library, not a
base library.

I don't know how we would do this in a base library, without being
overly specific and screwing ourselves in the future somehow.  It feels
too early to give pledge a formal specification.


> How about .Sh RESOURCES?  This would also fit other systems like
> Capsicum.  Though in practice, I think we'll find only pledge documented
> there.

You really think you'll find common ground??  I'm extremely sceptical.
The underlying concepts couldn't be different.  pledge disables "grouped
actions" taken against objects, whereas capsicum reduces ability to gain
new objects but generally leaves all style of actions intact.  A pledge
context can be described in 2-3 words, but capsicum is a heavy
environment one cannot describe so easily.

Also, don't want capsicum or any other non-OpenBSD described in OpenBSD
manual pages.  What you do in your own pages is your own concern, but
then premature standardization doesn't matter.

Re: Documenting library promises.

[Kristaps Dzonsons]

>> .Sh SANDBOXING
>> On
>> .Ox ,
>> the
>> .Fn khttp_parse
>> function requires the
>> .Qq stdio proc
>> promises to
>> .Xr pledge 2 .
> 
> As long as it is only a single sentence, that could easily go right
> after the description of the individual function in the DESCRIPTION,
> or alternatively at the end of the DESCRIPTION section.  Custom
> sections are not very nice, in particular when they are so short
> that they hardly justify opening a new section in the first place.
> 
> If you really want to describe multiple different sandboxing
> systems, then i guess ".Sh SANDBOXING" after the DESCRIPTION
> might make sense.  I'm not all that sure that describing any other
> system will be possible without going overboard; they are typically
> much too complicated to be summarized without excessive verbosity.
> But you shall no doubt see whether or not it works...

Ingo,

The only reason I suggest a standalone section is that it's easier to
standardise across manpages.  Saying "it should go after the function"
is too open to interpretation.  Moreover, some functions may have
varying promises depending upon function parameters, which will rapidly
eat into space better left to the functions themselves.

How about .Sh RESOURCES?  This would also fit other systems like
Capsicum.  Though in practice, I think we'll find only pledge documented
there.

Best,

Kristaps

Re: Documenting library promises.

[Theo de Raadt]

> .Sh SANDBOXING

And please stop using that word.  It has been misused so many
times, by now it is misleads.

pledge is not a sandbox (whatever the hell a sandbox is)

Re: Documenting library promises.

[Ingo Schwarze]

Hi Kristaps,

Kristaps Dzonsons BSD.LV wrote on Fri, Apr 06, 2018 at 09:57:09PM +0700:

> Short: what do you recommend for documenting an external library's
> pledge(2) requirements?

That is an interesting question indeed.  I never considered it
before, so i will think about it in some detail.

For a bit of context, let me first explain how documentation of
pledge(2) requirements inside the OpenBSD C library itself works,
and why the documentation was designed that way.

On first sight, it might seem most user-friendly if each and every
section 3 manual page for the C library would state the pledge(2)
requirements for each individual library function.

But that would have three important downsides:

 (1) It would be hard to do.  There are very many manual pages,
 so it would require much work for initial setup.

 (2) It would be unmaintainable, in particular while pledge(2)
 is still in very active development.  Pledge evolved through
 incremental tweaking.  Usage patterns were studied and modeled,
 the resulting pledge promises applied across the whole three,
 and the resulting experience allowed to refine the analysis
 and improve the way pledge(2) itself worked, based on real-world
 usage.  Rinse and repeat, literally dozens of times, over years
 of careful work.  Keeping many hundred manual pages in sync
 with that gradual evolution would have been flat-out impossible.

 (3) It would have put the focus in the wrong place.
 One among the chief ideas of pledge(2) is to avoid the splintered
 character of traditional syscall filters, where you are forced
 to filter each syscall, or even worse each of the arguments
 of each syscall, individually, and consequently completely lose
 the overview of what you are really doing before you even get
 started.  Instead, pleadge promises enable high-level logical
 feature groups.  The essential job of pledge(2) documentation
 is not to say that the FOO promise enables the BAR argument
 of the BAZ syscall, but that pledging FOO is required when you
 want to do operations of the BAR kind, no matter with which
 calls or arguments.

 Of course, once pledge(2) is completely stable, there would
 be nothing wrong with *also* documenting the effect in individual
 library functions manuals for user convenience, even though
 that would imply some redundancy.  But given that pledge(2)
 is still quite actively worked on, i doubt it will happen soon,
 if ever at all.

That said, for external libraries, putting your stuff directly into
the pledge(2) manual is obviously not an option.  Also, i agree
that documenting pledge(2) requirements for external libraries
is very useful, because these requirements are often quite
non-obvious and hard to guess by just looking at the pledge(2)
manual page itself, because many library manuals intentionally,
and sensibly, do not document how exactly the library operates
internally.

Note that for external libraries, the above arguments do not
apply:

 (1) Well-designed external libraries have small numbers of
 functions, so problem (1) is not a concern.

 (2) The logical functionality of functions in external libraries
 is hopefully mostly stable, and the required promises will
 rarely be affected by changes in pledge(2) itself.  Also,
 pledge(2) has already stabilized considerably, so problem (2)
 is no longer a major concern - though a bit of maintenace
 may be required now and then if you put such information
 into your manual pages.

 (3) The pledge(2) manual explains the high-level logic, your
 manuals add the specifics for your library, without
 duplication of information.  That is a clean division
 of tasks, so problem (3) is not a concern.

> Longer: https://bsd.network/@florian/99802355448571943
> 
> The question raised in this... um... toot?... is which promises are
> required by an external library call, in this case khttp_parse(3) in
> kcgi.  Sure, we can always just run the program, look in
> /var/log/messages for failure, and edit our promises.  But just... no.
> 
> In this particular case, I've documented this function's requirements
> unofficially here and there---tutorials and such.  But it's not
> canonical.  What I'd like is to put these directly into the manpages.

Absolutely agreed.  "Complete functional description in one
canonical place, do not scatter reference-manual-type information
across multiple sources" is among the most important requirements
for good documentation.

> .Sh SANDBOXING
> On
> .Ox ,
> the
> .Fn khttp_parse
> function requires the
> .Qq stdio proc
> promises to
> .Xr pledge 2 .

As long as it is only a single sentence, that could easily go right
after the description of the individual function in the DESCRIPTION,
or alternatively at the end of the DESCRIPTION section.  Custom
sections are not very nice, in particular when they are so short
that they hardly 

Documenting library promises.

[Kristaps Dzonsons]

Hi folks,

Short: what do you recommend for documenting an external library's
pledge(2) requirements?

Longer: https://bsd.network/@florian/99802355448571943

The question raised in this... um... toot?... is which promises are
required by an external library call, in this case khttp_parse(3) in
kcgi.  Sure, we can always just run the program, look in
/var/log/messages for failure, and edit our promises.  But just... no.

In this particular case, I've documented this function's requirements
unofficially here and there---tutorials and such.  But it's not
canonical.  What I'd like is to put these directly into the manpages.

Something like:

.Sh SANDBOXING
On
.Ox ,
the
.Fn khttp_parse
function requires the
.Qq stdio proc
promises to
.Xr pledge 2 .

This encourages developers to use the tightest possible promises.  And
as mdoc(7) is meant not to be system-specific, this might also include
information on, say, .Fx's Capsicum, or maybe whatever Linux uses this
week.  It already has "SECURITY CONSIDERATIONS", but that just doesn't
seem quite right.

Thoughts?

Kristaps