Re: Minutes from July 18 Mobile Tech Leads meeting

[Michael Comella]

Hey everyone.

Nick and I clarified our understanding of coroutines via IRC. Between this
email and IRC, it seems Nick's primary concern is that it may be difficult
to make the FFI libraries internally threadsafe [1]. By default, coroutines
dispatch to the `CommonPool`, where 1) coroutines can be dispatched to
arbitrary common pool threads and 2) an already dispatched coroutine, after
suspension, can resume on an arbitrary thread in the common pool.

Unfortunately, I don't understand the FFI thread safety in Rust issue well
enough to provide a single solution, however, some varied solutions to
thread safety are:
- Coroutines optionally suspend when they call a function with the
`suspend` keyword. afaik, if you never call one of these functions, you'll
never suspend and you'll remain on a single thread. This is hard to
enforce, however.
- Use a single dedicated thread for all FFI calls by running on a dedicated
CoroutineContext: see `newSingleThreadContext

`
- (may not be possible) Implement a new
CoroutineContext/CoroutineDispatcher that dispatches to a thread pool but
will resume only on the same thread. CoroutineContexts can be built on top
of Executor, if that's helpful.
- If your problem is sequential calls rather than which actual thread the
coroutines run on, you can use a coroutine `mutex
`
which will suspend until the resource can be acquired.

For our raw conversation, see the IRC logs
.

Hope it helps! Please let me know if you have more questions!
- Mike

[1]: https://mozilla.logbot.info/mobile/20180724#c15058851 (and previous
day)

On Fri, Jul 20, 2018 at 9:44 AM Nicholas Alexander 
wrote:

> Mike,
>
> Thanks so much for teaching us some things.
>
> On Thu, Jul 19, 2018 at 1:20 PM, Michael Comella 
> wrote:
>
>> Thanks for the updates, Nick – it's interesting to hear what other teams
>> are doing and how it may affect me. :) One quick thought:
>>
>>
>>> Rust code should be synchronous
>>
>>
>> I haven't thought about this enough to have an opinion on whether this is
>> the right decision or not. However, this may encourage the Rust library
>> consumers to make a mistake I once made with Kotlin coroutines: *I want
>> to share so others don't make the same mistake.*
>>
>> While you can create an obscenely large number of coroutines, there is a
>> limit to the number that can run concurrently: this limit is defined by the
>> thread pool they run on. The default built-in `CommonPool` has a fixed
>> number of threads. If you run blocking code (e.g. potentially synchronous
>> Rust code ^) on this thread pool, i*t prevents other coroutines from
>> beginning to run* until the blocking code completes. Here's a
>> demonstration using Android.
>> 
>>
>
> I was concerned about this, for slightly different reasons: I care about
> the same thread of execution servicing the next FFI call, i.e., that FFI
> calls don't rotate between threads non-deterministically.  But I think it's
> very similar to the situation you warn about.  I took an action item to
> read more about Kotlin co-routines, and summarized (in the notes) like this:
>
>
>-
>
>  AI: nalexander to read more about how Kotlin coroutines are
>  implemented
>  -
>
> [nalexander] OK, I’ve done this.  Based on the Kotlin 1.1+
> coroutines documentation
> ,
> Kotlin coroutines are implemented as a CPS transformation and an 
> internal
> state machine.  That state machine is pumped from a single 
> thread. That’s
> very similar to clojure/core.async
>  and means that
> there’s no thread pool servicing coroutines!  Any additional 
> thread must be
> the result of the application (or a library).  That means
> that all my hypothetical concerns about coroutines invoking FFI 
> methods on
> multiple threads are not justified.
>
> Am I incorrect?  Please correct me if so!
>
> Why is this a problem? *If you're a library (e.g. android-components)
>> that uses the same fixed thread pool as the application, you may block the
>> application's coroutines from starting quickly!* For example, if the
>> library makes several calls to read configuration files from disk, each on
>> a new coroutine on the CommonPool, these may block all of the CommonPool
>> threads. If all the threads are blocked and the application spawns a
>> coroutine on the CommonPool to do background processing before rendering
>> the results in the UI, the application's coroutine (and thus 

Re: Minutes from July 18 Mobile Tech Leads meeting

[Nicholas Alexander]

Mike,

Thanks so much for teaching us some things.

On Thu, Jul 19, 2018 at 1:20 PM, Michael Comella 
wrote:

> Thanks for the updates, Nick – it's interesting to hear what other teams
> are doing and how it may affect me. :) One quick thought:
>
>
>> Rust code should be synchronous
>
>
> I haven't thought about this enough to have an opinion on whether this is
> the right decision or not. However, this may encourage the Rust library
> consumers to make a mistake I once made with Kotlin coroutines: *I want
> to share so others don't make the same mistake.*
>
> While you can create an obscenely large number of coroutines, there is a
> limit to the number that can run concurrently: this limit is defined by the
> thread pool they run on. The default built-in `CommonPool` has a fixed
> number of threads. If you run blocking code (e.g. potentially synchronous
> Rust code ^) on this thread pool, i*t prevents other coroutines from
> beginning to run* until the blocking code completes. Here's a
> demonstration using Android.
> 
>

I was concerned about this, for slightly different reasons: I care about
the same thread of execution servicing the next FFI call, i.e., that FFI
calls don't rotate between threads non-deterministically.  But I think it's
very similar to the situation you warn about.  I took an action item to
read more about Kotlin co-routines, and summarized (in the notes) like this:


   -

 AI: nalexander to read more about how Kotlin coroutines are
 implemented
 -

[nalexander] OK, I’ve done this.  Based on the Kotlin 1.1+
coroutines documentation
, Kotlin
coroutines are implemented as a CPS transformation and an
internal state
machine.  That state machine is pumped from a single
thread. That’s very
similar to clojure/core.async
 and means that there’s
no thread pool servicing coroutines!  Any additional
thread must be the
result of the application (or a library).  That means that all
my hypothetical concerns about coroutines invoking FFI
methods on multiple
threads are not justified.

Am I incorrect?  Please correct me if so!

Why is this a problem? *If you're a library (e.g. android-components) that
> uses the same fixed thread pool as the application, you may block the
> application's coroutines from starting quickly!* For example, if the
> library makes several calls to read configuration files from disk, each on
> a new coroutine on the CommonPool, these may block all of the CommonPool
> threads. If all the threads are blocked and the application spawns a
> coroutine on the CommonPool to do background processing before rendering
> the results in the UI, the application's coroutine (and thus the UI update)
> will wait until the library's coroutines finish blocking before it runs.
>
> ---
>
> Some possible solutions to this are:
> - Use a separate thread pool for blocking calls (here's an open ticket
> for a system-wide IO thread pool
> )
> - Use non-blocking IO (supported in Java 7+ with nio
> )
>

I agree that `CommonPool` can cause issues.  (Just for the record, I had
not heard of `CommonPool`.)  But to me, those issues are clearly _above_
the Rust FFI layer, and the _common_ part of the name is pretty clear that
applications and libraries need to coordinate in order to prevent bad
things (in this case, limiting throughput due to resource starvation, or,
in the worst case, livelocking).

Coming back to my summary of how Kotlin coroutines are implemented,
something above the Rust library layer has to handle adding asynchrony to
the synchronous Rust -- either by spawning threads (with a pool, common or
otherwise, if desired); or by doing non-blocking things where possible; or
...

Are we in accord?
Nick
___
Dev-fxacct mailing list
Dev-fxacct@mozilla.org
https://mail.mozilla.org/listinfo/dev-fxacct

Re: Minutes from July 18 Mobile Tech Leads meeting

[Michael Comella]

Thanks for the updates, Nick – it's interesting to hear what other teams
are doing and how it may affect me. :) One quick thought:


> Rust code should be synchronous


I haven't thought about this enough to have an opinion on whether this is
the right decision or not. However, this may encourage the Rust library
consumers to make a mistake I once made with Kotlin coroutines: *I want to
share so others don't make the same mistake.*

While you can create an obscenely large number of coroutines, there is a
limit to the number that can run concurrently: this limit is defined by the
thread pool they run on. The default built-in `CommonPool` has a fixed
number of threads. If you run blocking code (e.g. potentially synchronous
Rust code ^) on this thread pool, i*t prevents other coroutines from
beginning to run* until the blocking code completes. Here's a demonstration
using Android. 

Why is this a problem? *If you're a library (e.g. android-components) that
uses the same fixed thread pool as the application, you may block the
application's coroutines from starting quickly!* For example, if the
library makes several calls to read configuration files from disk, each on
a new coroutine on the CommonPool, these may block all of the CommonPool
threads. If all the threads are blocked and the application spawns a
coroutine on the CommonPool to do background processing before rendering
the results in the UI, the application's coroutine (and thus the UI update)
will wait until the library's coroutines finish blocking before it runs.

---

Some possible solutions to this are:
- Use a separate thread pool for blocking calls (here's an open ticket for
a system-wide IO thread pool
)
- Use non-blocking IO (supported in Java 7+ with nio
)

Hope it helps!
- Mike

On Wed, Jul 18, 2018 at 8:28 PM Nicholas Alexander 
wrote:

> We had a Mobile Tech Leads Meeting earlier today (NA/AU timezones).  Due
> to PTO, it was a smaller group than usual.
>
> A very brief and opinionated summary of our discussion (from this doc
> 
> ):
>
>1. *How do we manage threading* between consumers (primarily Android
>invoking Rust from Java/Kotlin, but also iOS invoking Rust from Swift) and
>Rust libraries?
>- Discussion focused on JVM thread pools and Kotlin coroutines.
>   - Strong desire to not expose concurrency in the API surface
>   - Initial proposal, subject to much further discussion:
>   -
> *Rust code should be synchronous *
>  - *Rust code should expect to be invoked from multiple threads*
>  - *Rust code should expect to be serialized (via Java
>  `synchronize` keyword or equivalent)*
>  2. There was some discussion of dependency injection for logging
>and telemetry.
>   - markh will be leading that discussion for telemetry.
>
> There's more in the notes but that's the gist.
>
> Your interlocutor,
> Nick
>
> ___
> android-components mailing list
> android-compone...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/android-components
>
___
Dev-fxacct mailing list
Dev-fxacct@mozilla.org
https://mail.mozilla.org/listinfo/dev-fxacct