Re: [rust-dev] Danger of throwing exceptions through Rust code

[Thad Guidry]

In awe,

Niko does a fantastic job of breaking things down.  Give that guy a raise
already. ;)

And he brings the most important point of all in this discussion.
 "Sometimes authors care and sometimes they don't"

Maybe the important distinction is "how" the author wraps the library, to
bring about better failure semantics, regardless if the author needs them
or not, at least they are available to care about or not ?

-- 
-Thad
+ThadGuidry 
Thad on LinkedIn 
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Florian Weimer]

* Daniel Micay:

> It's undefined behaviour for a C++ function to throw an exception past an
> `extern "C"` boundary

I don't think this is true.  Certainly GCC supports throwing from C as
an extension (if the C side has been compiled with -fexceptions), and
requires that non-throwing functions are explicitly annotated, even if
they have C linkage.

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Niko Matsakis]

The interplay between exceptions and foreign code is a complicated
topic. Here are some miscellaneous thoughts.

1. I don't believe there is any memory safety reason we could not
   support catchable exceptions in Rust. The main concern is
   more one of good design; it's difficult to recover meaningfully
   and correctly from an exception. Generally the best thing to do,
   particularly in an interactive application, is to catch the
   exception up at a high level (an event loop, say) and try to
   throw away all the affected state and start over. Tasks provide
   a wonderful abstraction for doing this, of course.

2. Now, that said, some exceptions are expected and should be caught.
   File not found and so forth. Java recognized this distinction and
   called those checked exceptions; for various reasons, this is a
   pain in the neck to work with. But erasing the distinction, as many
   other languages chose to do, has its downsides. Currently the Rust
   path seems to be the same as Haskell -- use the return type,
   specifically `Result<>` or something similar, to indicate common
   error conditions. If you think about it, this is very analogous to
   what Java does, but because we are using types, we support
   parameteric error handling -- that is, you can have generic
   wrappers that return a `Result` iff the thing they are wrapping
   returns a `Result` (which Java can't do).

3. Presuming we want to keep the above philosophy (which I personally
   like, I find exception handlers to be awkward at the best of
   times), there is still the question of what to do when a C library
   fails. The C library will (typically) indicate this by returning
   an error code in some fashion. Ideally, the Rust task would observe
   this error code and then fail in response. OK, so far so good.

 (If the callee does not signal errors by returning a result code,
  then Rust probably won't be able to respond to it. We can't
  generate cleanup code for every possible exception scheme. So I
  think the best thing here would be for the author to insert a
  shim that intercepts exceptions and translates them to a return
  code.)

4. Problems start to occur though when you have interleaved C and Rust
   code, such as Rust1 -> C -> Rust2. Now what happens if the Rust task
   fails? A well-designed C callback API will include a way for the
   Rust task to indicate failure, probably a return code. So, in an
   ideal world, the Rust code would return this error code when
   failing.  Of course, we can't know in advance what error code would
   be needed, so that implies that Rust2 must have some way
   to catch the failure and translate it into the C error code.
   Hopefully this will cause the C code to unwind and return an error
   to Rust1, which can then fail! again. This process can be repeated
   as needed.

5. But what do we do if some C library doens't include error return
   codes, or doesn't (necessarily) unwind in response to an error?
   This is a hard question and I think there is no single right
   answer, instead the person wrapping the library will have to find
   the right answer for their particular case. For example, Rust1 and
   Rust2 could communicate using TLS, so that an exception occurring
   in Rust2 can (eventually) be re-raised in Rust1, even if the C code
   doesn't consider the possibility of failure. Or maybe the author
   just doesn't really care if Rust2 fails and they just swallow the
   failure, violating Rust's failure semantics (because they know
   better than us for their particular case anyhow).

6. Some would argue that based on that last point, we might as well
   just permit catch anywhere, and discourage its use. I am
   sympathetic to this as well though I think Rust's general approach
   to error handling (which we stole from Erlang) is really the
   more robust one: when things go wrong, tear it all up, then build
   it back up again. Still, perhaps community norms are enough here.



Niko



On Tue, Nov 12, 2013 at 11:07:50AM -0800, Kevin Ballard wrote:
> Right now, Rust does not support catching task failure from within a task, it 
> only supports preventing task failure from cascading into other tasks. My 
> understanding is that this limitation is done because of safety; if a task 
> unwinds through a few frames of code, and then stops unwinding, data 
> structure invariants may have been broken by the unwinding, leaving the task 
> in an unsafe state. Is this correct?
> 
> Given this assumption, my worry now is about task unwinding outside of the 
> control of Rust. Namely, if I’m using Rust to write a library with extern “C” 
> functions, or I’m providing callbacks to C code from within Rust, (and my 
> Rust code calls back into C at some point), then it’s very possible for the 
> called C code to throw an exception that is then caught in the calling C code 
> a few frames up. The net effect is that the thread will unwind through my 
> Rust code, b

Re: [rust-dev] Danger of throwing exceptions through Rust code

[SiegeLord]

On 11/12/2013 10:18 PM, Patrick Walton wrote:

On 11/13/13 12:06 PM, Daniel Micay wrote:

If a library takes a callback, writing safe Rust bindings isn't going to
turn out well. Rust functions can fail, so the Rust API can't pass an
arbitrary function to the callback.


Yeah, this has been a concern for a while (years). Maybe we should have
an unsafe "catch" just for this case, to allow turning a Rust failure
into a C-style error code.

Patrick


Is that not, essentially, just spawning a task inside the Rust callback 
and catching the result (via task::try)?


-SL

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Patrick Walton]

On 11/13/13 12:06 PM, Daniel Micay wrote:

It's completely possible to write safe bindings to a C++ library. The
process involves wrapping the whole thing with `extern "C"` functions
using catch blocks for any function possibly throwing an exception. Keep
in mind that libraries already have to do this to support usage from
most other languages, so it's a sunken cost.

If a library takes a callback, writing safe Rust bindings isn't going to
turn out well. Rust functions can fail, so the Rust API can't pass an
arbitrary function to the callback.


Yeah, this has been a concern for a while (years). Maybe we should have 
an unsafe "catch" just for this case, to allow turning a Rust failure 
into a C-style error code.


Patrick

___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Daniel Micay]

On Tue, Nov 12, 2013 at 9:42 PM, Kevin Ballard  wrote:

> I guess I was being too vague when I said “C exceptions”, because you’re
> right, that’s not actually a specific thing. More concretely, I was
> thinking of either C++ exceptions, or Obj-C exceptions.
>
> One situation I was thinking of would be a rust library that exposes
> `extern “C”` functions. Let’s say one of these functions, frob(), takes an
> `extern “C”` callback and calls it. Now let’s say I’m writing an Obj-C app,
> and I do something like the following:
>
> static void callback() {
> [NSException raise:@“Foo Exception” format:@“I just felt like
> unwinding”];
> }
>
> @try {
> frob(callback)
> }
> @catch (NSException *e) {
> NSLog(@“I caught my exception!”);
> }
>
> This will unwind through the Rust code, before being caught again. It’s
> hard to come up with a scenario where this actually results in unsafe state
> within the rust library, but let’s say `frob()` uses some sort of shared
> state (e.g. a RWArc<>), and is in the middle of modifying the shared state
> in a way that causes invariants to be broken temporarily. If unwinding
> happens when the invariants are in the broken state, then they’ll be left
> in the broken state and the next bit of Rust code that tries to access this
> shared state will likely blow up.
>
> This is obviously pretty contrived, but the question is, is this legal to
> do, and if so, do we need to do anything?
>
> Daniel’s response says that it’s undefined for a C++ function to throw an
> exception past an `extern “C”` boundary. This is something I did not know.
> But what about Obj-C exceptions? I haven’t heard about any undefined
> behavior regarding Obj-C exceptions. It is generally recognized that
> throwing an Obj-C exception past a framework boundary is unsafe (i.e.
> throwing an exception in user code that unwinds through Foundation code),
> but this is because Obj-C code rarely bothers with @try/@finally and
> doesn’t have stack objects, so unwinding through code that doesn’t expect
> it will often leave data structures in invalid states.
>
> If all forms of unwinding are considered to be undefined when passing
> through an `extern “C”` boundary, then I guess we can consider this issue
> to be covered by undefined behavior. Although this doesn’t make me
> particularly happy, because it means that it may be impossible to truly
> contain the unsafety in FFI functions. One possible way to mitigate this
> would be to provide a way to wrap an FFI function with a stub that catches
> C++/Obj-C exceptions (I *think* Obj-C exceptions are unified with the C++
> exception machinery on all modern platforms (i.e. not 32-bit PPC, and I’m
> not sure about 32-bit x86)) and triggers task failure. This would mean any
> attempt to unwind through a Rust function (using C++ or Obj-C exceptions)
> would be contained, after a fashion.
>
> Though this does raise another question. What if a C++ function calls a
> Rust function, and the Rust function triggers task failure? Is it possible
> for the C++ function to catch the task failure using a catch(…) block?
> Furthermore, since Daniel said it’s undefined for a C++ exception to be
> thrown past an `extern “C”` boundary, does this mean that it’s technically
> undefined for Rust to trigger task failure in any function that’s rooted in
> an `extern “C”` function?
>
> -Kevin
>

It's completely possible to write safe bindings to a C++ library. The
process involves wrapping the whole thing with `extern "C"` functions using
catch blocks for any function possibly throwing an exception. Keep in mind
that libraries already have to do this to support usage from most other
languages, so it's a sunken cost.

If a library takes a callback, writing safe Rust bindings isn't going to
turn out well. Rust functions can fail, so the Rust API can't pass an
arbitrary function to the callback.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Kevin Ballard]

I guess I was being too vague when I said “C exceptions”, because you’re right, 
that’s not actually a specific thing. More concretely, I was thinking of either 
C++ exceptions, or Obj-C exceptions.

One situation I was thinking of would be a rust library that exposes `extern 
“C”` functions. Let’s say one of these functions, frob(), takes an `extern “C”` 
callback and calls it. Now let’s say I’m writing an Obj-C app, and I do 
something like the following:

static void callback() {
[NSException raise:@“Foo Exception” format:@“I just felt like unwinding”];
}

@try {
frob(callback)
}
@catch (NSException *e) {
NSLog(@“I caught my exception!”);
}

This will unwind through the Rust code, before being caught again. It’s hard to 
come up with a scenario where this actually results in unsafe state within the 
rust library, but let’s say `frob()` uses some sort of shared state (e.g. a 
RWArc<>), and is in the middle of modifying the shared state in a way that 
causes invariants to be broken temporarily. If unwinding happens when the 
invariants are in the broken state, then they’ll be left in the broken state 
and the next bit of Rust code that tries to access this shared state will 
likely blow up.

This is obviously pretty contrived, but the question is, is this legal to do, 
and if so, do we need to do anything?

Daniel’s response says that it’s undefined for a C++ function to throw an 
exception past an `extern “C”` boundary. This is something I did not know. But 
what about Obj-C exceptions? I haven’t heard about any undefined behavior 
regarding Obj-C exceptions. It is generally recognized that throwing an Obj-C 
exception past a framework boundary is unsafe (i.e. throwing an exception in 
user code that unwinds through Foundation code), but this is because Obj-C code 
rarely bothers with @try/@finally and doesn’t have stack objects, so unwinding 
through code that doesn’t expect it will often leave data structures in invalid 
states.

If all forms of unwinding are considered to be undefined when passing through 
an `extern “C”` boundary, then I guess we can consider this issue to be covered 
by undefined behavior. Although this doesn’t make me particularly happy, 
because it means that it may be impossible to truly contain the unsafety in FFI 
functions. One possible way to mitigate this would be to provide a way to wrap 
an FFI function with a stub that catches C++/Obj-C exceptions (I think Obj-C 
exceptions are unified with the C++ exception machinery on all modern platforms 
(i.e. not 32-bit PPC, and I’m not sure about 32-bit x86)) and triggers task 
failure. This would mean any attempt to unwind through a Rust function (using 
C++ or Obj-C exceptions) would be contained, after a fashion.

Though this does raise another question. What if a C++ function calls a Rust 
function, and the Rust function triggers task failure? Is it possible for the 
C++ function to catch the task failure using a catch(…) block? Furthermore, 
since Daniel said it’s undefined for a C++ exception to be thrown past an 
`extern “C”` boundary, does this mean that it’s technically undefined for Rust 
to trigger task failure in any function that’s rooted in an `extern “C”` 
function?

-Kevin

On Nov 12, 2013, at 11:35 AM, Alex Crichton  wrote:

> You're correct about the safeness of catching failure at a task
> boundary. Rust's invariants about spawning a task involve knowing a
> fair bit about what's allowable to share between a task boundary, and
> that allows us to reason about the failure unwinding to the task
> boundary being a safe operation (as opposed to stopping unwinding at
> an arbitrary location).
> 
> I'm not entirely sure what you mean by throwing an exception from C (I
> think there are many flavors of doing this). Right now we implement
> unwinding via C++ exceptions. When a task unwinds, it actually throws
> a magical uint token which then triggers all the C++ machinery for
> unwinding the stack. When compiling with LLVM, we using LLVM's invoke
> instruction + landing pads to generate our "cleanup locations", and
> LLVM will codegen the right code such that all the landing pads are
> invoked during unwinding. What this means is that the C++ exception
> throwing infrastructure will probably fly right past all C code
> because none of it is hooked into the exception handling stuff of C++.
> This may mean, however, that intermediate C++ code may have landing
> pads run (not entirely sure).
> 
> All that being said, that's just how it's currently implemented today.
> I don't think that we're guaranteeing this sort of behavior to always
> happen. It will probably always be the case that C stack frames are
> always sailed past during unwinding, but we may implement unwinding
> via precise stack tables and manual stack unwinding at some point
> which wouldn't trigger C++ landing pads (or use LLVM's landing pad
> infrastructure the same way that we're using it today).
> 
> Right now it's basically the case that in

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Daniel Micay]

On Tue, Nov 12, 2013 at 2:35 PM, Alex Crichton  wrote:

> You're correct about the safeness of catching failure at a task
> boundary. Rust's invariants about spawning a task involve knowing a
> fair bit about what's allowable to share between a task boundary, and
> that allows us to reason about the failure unwinding to the task
> boundary being a safe operation (as opposed to stopping unwinding at
> an arbitrary location).
>
> I'm not entirely sure what you mean by throwing an exception from C (I
> think there are many flavors of doing this). Right now we implement
> unwinding via C++ exceptions. When a task unwinds, it actually throws
> a magical uint token which then triggers all the C++ machinery for
> unwinding the stack. When compiling with LLVM, we using LLVM's invoke
> instruction + landing pads to generate our "cleanup locations", and
> LLVM will codegen the right code such that all the landing pads are
> invoked during unwinding. What this means is that the C++ exception
> throwing infrastructure will probably fly right past all C code
> because none of it is hooked into the exception handling stuff of C++.
> This may mean, however, that intermediate C++ code may have landing
> pads run (not entirely sure).
>
> All that being said, that's just how it's currently implemented today.
> I don't think that we're guaranteeing this sort of behavior to always
> happen. It will probably always be the case that C stack frames are
> always sailed past during unwinding, but we may implement unwinding
> via precise stack tables and manual stack unwinding at some point
> which wouldn't trigger C++ landing pads (or use LLVM's landing pad
> infrastructure the same way that we're using it today).
>
> Right now it's basically the case that intermingling C with Rust stack
> frames and then triggering failure will only trigger unwinding in rust
> functions (what does it mean to unwind in C?), and I'm not sure I'd
> recommend that as a safe strategy for implementing bindings to a C
> function (all intermediate C allocations are leaked).
>
> Does that make sense? It may not quite answer your question, but
> hopefully that clears up at least a little bit about how it's
> implemented today.
>

It's undefined behaviour for a C++ function to throw an exception past an
`extern "C"` boundary so Rust doesn't need to worry about a need to handle
exceptions from foreign libraries.

In practice, compilers will often build code with support for passing
through exceptions but it's not required and C cannot maintain the
invariants required for safety in the face of unwinding. It's simply not
safe to pass Rust functions directly as callbacks into C if they aren't
known to never throw.

Rust isn't committed to using the C++ exception personality so
interoperability with unwrapped C++ libraries just isn't on the table at
this point. It actually *doesn't* even use the C++ exception personality
because there are hooks to reset the used stack space.
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Alex Crichton]

You're correct about the safeness of catching failure at a task
boundary. Rust's invariants about spawning a task involve knowing a
fair bit about what's allowable to share between a task boundary, and
that allows us to reason about the failure unwinding to the task
boundary being a safe operation (as opposed to stopping unwinding at
an arbitrary location).

I'm not entirely sure what you mean by throwing an exception from C (I
think there are many flavors of doing this). Right now we implement
unwinding via C++ exceptions. When a task unwinds, it actually throws
a magical uint token which then triggers all the C++ machinery for
unwinding the stack. When compiling with LLVM, we using LLVM's invoke
instruction + landing pads to generate our "cleanup locations", and
LLVM will codegen the right code such that all the landing pads are
invoked during unwinding. What this means is that the C++ exception
throwing infrastructure will probably fly right past all C code
because none of it is hooked into the exception handling stuff of C++.
This may mean, however, that intermediate C++ code may have landing
pads run (not entirely sure).

All that being said, that's just how it's currently implemented today.
I don't think that we're guaranteeing this sort of behavior to always
happen. It will probably always be the case that C stack frames are
always sailed past during unwinding, but we may implement unwinding
via precise stack tables and manual stack unwinding at some point
which wouldn't trigger C++ landing pads (or use LLVM's landing pad
infrastructure the same way that we're using it today).

Right now it's basically the case that intermingling C with Rust stack
frames and then triggering failure will only trigger unwinding in rust
functions (what does it mean to unwind in C?), and I'm not sure I'd
recommend that as a safe strategy for implementing bindings to a C
function (all intermediate C allocations are leaked).

Does that make sense? It may not quite answer your question, but
hopefully that clears up at least a little bit about how it's
implemented today.

On Tue, Nov 12, 2013 at 11:07 AM, Kevin Ballard  wrote:
> Right now, Rust does not support catching task failure from within a task, it 
> only supports preventing task failure from cascading into other tasks. My 
> understanding is that this limitation is done because of safety; if a task 
> unwinds through a few frames of code, and then stops unwinding, data 
> structure invariants may have been broken by the unwinding, leaving the task 
> in an unsafe state. Is this correct?
>
> Given this assumption, my worry now is about task unwinding outside of the 
> control of Rust. Namely, if I’m using Rust to write a library with extern “C” 
> functions, or I’m providing callbacks to C code from within Rust, (and my 
> Rust code calls back into C at some point), then it’s very possible for the 
> called C code to throw an exception that is then caught in the calling C code 
> a few frames up. The net effect is that the thread will unwind through my 
> Rust code, but it will then be caught before unwinding any further, 
> potentially leaving any data structures in an invalid state (assuming that 
> there’s still Rust code higher up on this same stack that cares).
>
> Has this been considered before? Is this actually a danger or am I just being 
> paranoid?
>
> -Kevin
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Re: [rust-dev] Danger of throwing exceptions through Rust code

[Corey Richardson]

Is it possible for data low in the stack to propagate upwards through
the stack before function return? It seems like if this were to be an
issue, you would need to move into something parent gives you, at
which point you no longer have ownership and unwinding won't destroy
it.

It seems like the lifetime system prevents this from being a problem.

I think Rust *could* have catchable exceptions, you just wouldn't be
allowed to use anything that can throw when constructing a value. That
way, everything is fully constructed when an exception happens, so
destructors can never run on inconsistent state and resources won't
leak.

Maybe there's something I'm missing, though.

On Tue, Nov 12, 2013 at 2:07 PM, Kevin Ballard  wrote:
> Right now, Rust does not support catching task failure from within a task, it 
> only supports preventing task failure from cascading into other tasks. My 
> understanding is that this limitation is done because of safety; if a task 
> unwinds through a few frames of code, and then stops unwinding, data 
> structure invariants may have been broken by the unwinding, leaving the task 
> in an unsafe state. Is this correct?
>
> Given this assumption, my worry now is about task unwinding outside of the 
> control of Rust. Namely, if I’m using Rust to write a library with extern “C” 
> functions, or I’m providing callbacks to C code from within Rust, (and my 
> Rust code calls back into C at some point), then it’s very possible for the 
> called C code to throw an exception that is then caught in the calling C code 
> a few frames up. The net effect is that the thread will unwind through my 
> Rust code, but it will then be caught before unwinding any further, 
> potentially leaving any data structures in an invalid state (assuming that 
> there’s still Rust code higher up on this same stack that cares).
>
> Has this been considered before? Is this actually a danger or am I just being 
> paranoid?
>
> -Kevin
> ___
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

[rust-dev] Danger of throwing exceptions through Rust code

[Kevin Ballard]

Right now, Rust does not support catching task failure from within a task, it 
only supports preventing task failure from cascading into other tasks. My 
understanding is that this limitation is done because of safety; if a task 
unwinds through a few frames of code, and then stops unwinding, data structure 
invariants may have been broken by the unwinding, leaving the task in an unsafe 
state. Is this correct?

Given this assumption, my worry now is about task unwinding outside of the 
control of Rust. Namely, if I’m using Rust to write a library with extern “C” 
functions, or I’m providing callbacks to C code from within Rust, (and my Rust 
code calls back into C at some point), then it’s very possible for the called C 
code to throw an exception that is then caught in the calling C code a few 
frames up. The net effect is that the thread will unwind through my Rust code, 
but it will then be caught before unwinding any further, potentially leaving 
any data structures in an invalid state (assuming that there’s still Rust code 
higher up on this same stack that cares).

Has this been considered before? Is this actually a danger or am I just being 
paranoid?

-Kevin
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev