Re: How mature is async/threading in Nim?
Speaking from my user experience with nim (~1 yr), you don't have to worry about the stability of async related things, they are quite robust. The things that is not very mature is the GC. I am not talking about the arc one, but the current default one refc. I have a single threaded project around 10k~20k line. When I run tests with defaut gc, 3 to 4 times out of 10, I will run into illegal storage access error. After I moved to boehm, I have never seen the same error again. There are chance that could be my fault, but I believe this is GC bug. For inter-thread communication, I have made a library [https://github.com/jackhftang/threadproxy.nim](https://github.com/jackhftang/threadproxy.nim) to simplify ITC programming. You can take a look at it~ Again, in my practical experience, the little trick to be stable for ITC is to use JSON as data exchange =] When I was developing multi-threaded program, I found that the deep copy of channels do not handle well with null pointer. It seems it will run into problem when there are nil somewhere in data structure. And with JSON you would easily have nil and cyclic structures.
Re: How mature is async/threading in Nim?
> Not really. What you can achieve is something similar to what I created with > httpbeast: each thread running its own Async event loop and allowing the > system to load balance the socket connections. For clients that will likely > be trickier. This _event-loop per thread_ is only required on Linux where epoll is inherently single threaded. Using IOCP you simply spawn N threads and make them spin on GetQueuedCompletionStatus. All balancing is done by the kernel. ARC should make this work more natural.
Re: How mature is async/threading in Nim?
I didn't use ARC because when I started evaluating multithreading options (July
2019) and implementing Weave (November 2019) it was not ready at all.
Now at the moment, I'm ready to run my batteries of tests on arc:
[https://github.com/mratsim/weave/blob/33a446ca/weave.nimble#L99-L167](https://github.com/mratsim/weave/blob/33a446ca/weave.nimble#L99-L167)
But I need to be able to at least have custom channels working:
[https://github.com/nim-lang/Nim/issues/13936](https://github.com/nim-lang/Nim/issues/13936)
I do provide already the primitives necessary to have `awaitable` (in the async
sens) Weave jobs via `isReady`, you only need to replace the `sleep()` by
`poll()` here in my own
[waitFor](https://github.com/mratsim/weave/blob/33a446ca4ac6294e664d26693702e3eb1d9af326/weave/datatypes/flowvars.nim#L234-L249):
proc waitFor*[T](p: Pending[T]): T {.inline.} =
## Wait for a pending value
## This blocks the thread until the value is ready
## and then returns it.
preCondition: onSubmitterThread
var backoff = 1
while not p.isReady:
sleep(backoff)
backoff *= 2
if backoff > 16:
backoff = 16
let ok = p.fv.tryComplete(result)
ascertain: ok
cleanup(p.fv)
Run
See writeup/RFC:
[https://github.com/mratsim/weave/issues/132](https://github.com/mratsim/weave/issues/132)
The main issue right now that the GC cannot solve is if we ever want to
implement an `executor` API/concept or at least common `Task` abstraction to
could be dispatch on either:
* Async executors (async dispatch/chronos/libuv)
* Parallel executors (Nim threadpool / Weave)
* An simple executor for blocking tasks like `readline()` because just as you
shouldn't block asyncdispatch, you shouldn't block Weave with non-computational
tasks.
The simple reason why is that tasks in a parallel runtime are threadsafe
closures + metadata, and the GC being thread-local prevents closures from being
sent across threads.
This brings us to why we would want a common Task abstraction
To be honest I'm not sure, but I'm sure we would want a threadsafe async
executor, potentially with task stealing as well, it's not even hard to write
(at least the task stealing side, ~2k lines):
*
[https://github.com/stjepang/async-task](https://github.com/stjepang/async-task)
* [https://github.com/stjepang/smol](https://github.com/stjepang/smol)
but this would require threadsafe closures.
i.e. threadsafe /sendable closures are the core primitive that ARC brings us.
It also gives us `owned` and destructors give use sink which we can use to
enforce ownership when we send tasks across threads. This significantly reduce
the need of Atomic Refcounting schemes as we can just deep-copy the pointers
instead.
For example, we need to enforce ownership of futures. (Hence why I hope that
when a proc is tagged with `sink` parameters it also prevents the caller from
ever reusing that variable again)
Re: How mature is async/threading in Nim?
> Read mratsim's post from the same thread then, > [https://forum.nim-lang.org/t/6352#39200](https://forum.nim-lang.org/t/6352#39200) @mratsim shows no examples of what ARC can do, but I assume that ARC can help with the third scenario that he describes... > You need a shared state, for example a shared hash table. Now you have a > problem :P, if it has infinite lifetime, you can allocate it on the heap and > pass a pointer to it, if not you need to implement atomic refcounting which > is not too difficult with destructors, see for example my refcounted events > in Weave You will note that I wrote "The fact is that I have so far seen no evidence that --gc:arc makes mixing concurrency and parallelism in Nim easier". Yes, ARC does offer a shared heap and makes sharing memory easier, but the interoperability still isn't there between concurrency and parallelism. A shared heap does not solve this. As a side note, I would like to see an example of that third scenario implemented using ARC, why hasn't @mratsim used it by default? :)
Re: How mature is async/threading in Nim?
Adding to that [https://github.com/nim-lang/Nim/issues/14429](https://github.com/nim-lang/Nim/issues/14429) :)
Re: How mature is async/threading in Nim?
> I may very well be missing something, ... Read mratsim's post from the same thread then, [https://forum.nim-lang.org/t/6352#39200](https://forum.nim-lang.org/t/6352#39200)
Re: How mature is async/threading in Nim?
> I have limited experience with nim mobile apps. But knowing what I know don't > think I would use async on the client. Async is great if you are doing tons > of http style requests. But really a mobile client? Just regular threads are > probably better if you are just writing and reading from a single websocket > connection. I would try both methods to see which one fits you better. @treeform nothing wrong with using async on the client. Not sure why you find it so weird?
Re: How mature is async/threading in Nim?
All this talk of `--gc:arc` makes me a little bit frustrated, I feel like @andrea has a similar reaction. The fact is that I have so far seen no evidence that `--gc:arc` makes mixing concurrency and parallelism in Nim easier (I may very well be missing something, so please educate me with examples if so). So yes, as things stand in Nim, you cannot easily mix parallelism and concurrency right now, but to me there is a fairly simple solution to this: we need to implement the ability to `await` a FlowVar (what `spawn` returns) and/or `channel.recv`. Either a member of the core dev team needs to be convinced to make this work or someone who's passionate about it needs to get it past the finish line (there were [some](https://github.com/nim-lang/Nim/pull/12232) [efforts](https://github.com/nim-lang/Nim/pull/12372) which made great progress, so I would say there is just a "little" push needed to get it to work). Of course, Nim's threading model is a little different than most languages. Threads are somewhat "heavy", with each having their own heap, so for use cases that require a lot of communication between threads you are better off not holding out hope for my suggestion above. There are alternatives, and I have been pretty successful in creating a very performant HTTP server for example: [https://github.com/dom96/httpbeast](https://github.com/dom96/httpbeast) (which this forum runs on). Now to answer some of your questions... > The asyncnet module documentation has a couple of caveats about Windows, like > "on Windows it only supports select()" and "In theory you should be able to > work with any of these layers interchangeably (as long as you only care about > non-Windows platforms)." I'm not clear on how this would impact clients, and > whether these caveats apply to using asyncnet or just the lower-level modules. We might need to rewrite that paragraph, but it is specifically only talking about the `selectors` module. We have a custom IOCP implementation in asyncdispatch which asyncnet uses. So you will get the greatest API support on Windows too. > The async examples I've seen run on a single thread. Is there any support for > distributing async calls across a thread pool? Not really. What you can achieve is something similar to what I created with httpbeast: each thread running its own Async event loop and allowing the system to load balance the socket connections. For clients that will likely be trickier. > The spawn function's thread safety seems to be based on segregating heap > objects per thread (Erlang-style.) This can involve a lot of copying, > especially when passing data buffers around. Is this something baked into the > language or is it specific to the spawn function? Are there alternatives to > this, like the more Rust-style model using move semantics? Yep, this is unavoidable right now. But I believe arc will indeed make move semantics possible, that's really the main advantage it will bring to threading in Nim. I could be wrong though.
Re: How mature is async/threading in Nim?
I've tried to summarize the state of Nim's concurrency and parallelism in this blog post: [https://onlinetechinfo.com/concurrency-and-parallelism-in-nim/](https://onlinetechinfo.com/concurrency-and-parallelism-in-nim/). Some of it was based on this discussion.
Re: How mature is async/threading in Nim?
Regarding multithreading it really depends on your workload but here are 2
kinds of code architectures that would allow mixing async on threads:
1\. If the part of your code that you want threaded is stateless and only
allows the following types to crossover threads:
* plain old datatypes/variants
* Ref, seq, strings types but only if they are created and destroyed within
the task and are not sent across threads
* pointer to buffers (raw or Nim sequences) that outlive the task (for
example pointers to matrices)
You can use a threadpool and just spawn myFunctionCall(a, b, c) (or Weave if
dynamic load balancing, parallel for or producer-consumer task dependencies is
needed).
2\. Your threads are long-lived, maintain (independent) state and work as a
service that needs to communicate with other services. Then use channels to
communicate between them. Nim channels support sending seq and strings via deep
copy.
3\. You need a shared state, for example a shared hash table. Now you have a
problem :P, if it has infinite lifetime, you can allocate it on the heap and
pass a pointer to it, if not you need to implement atomic refcounting which is
not too difficult with destructors, see for example my [refcounted events in
Weave](https://github.com/mratsim/weave/blob/17257c2f95594b566abaa7b5c1a875f1a77f3536/weave/cross_thread_com/flow_events.nim#L176-L201)
type
FlowEvent* = object
e: EventPtr
EventPtr = ptr object
refCount: Atomic[int32]
kind: EventKind
union: EventUnion
# Internal
#
# Refcounting is started from 0 and we avoid fetchSub with release semantics
# in the common case of only one reference being live.
proc `=destroy`*(event: var FlowEvent) =
if event.e.isNil:
return
let count = event.e.refCount.load(moRelaxed)
fence(moAcquire)
if count == 0:
# We have the last reference
if not event.e.isNil:
if event.e.kind == Iteration:
wv_free(event.e.union.iter.singles)
# Return memory to the memory pool
recycle(event.e)
else:
discard fetchSub(event.e.refCount, 1, moRelease)
event.e = nil
proc `=sink`*(dst: var FlowEvent, src: FlowEvent) {.inline.} =
# Don't pay for atomic refcounting when compiler can prove there is no
ref change
`=destroy`(dst)
system.`=sink`(dst.e, src.e)
proc `=`*(dst: var FlowEvent, src: FlowEvent) {.inline.} =
`=destroy`(dst)
discard fetchAdd(src.e.refCount, 1, moRelaxed)
dst.e = src.e
Run
So for the first 2 architectures, it's easy to "mix", threading and async live
in separate domains.
I've also added facilities this weekend for Weave to run as a background
service so that [long-lived threads can also submit jobs to
Weave](https://github.com/mratsim/weave#foreign-thread--background-service-experimental)
to ease interaction with async.
Re: How mature is async/threading in Nim?
> which GC is the default The _\--gc:refc_ (deferred reference counting/heap per thread) is the default right now. The _\--gc:arc_ (immediate reference counting/shared heap) or _\--gc:orc_ (immediate reference counting/shared heap + cycle detector) is set to replace it. The _async_ stuff works well with _\--gc:refc_ but buggy with _\--gc:arc_ because it creates cycles for which you need _\--gc:orc_. Both _\--gc:arc_ and _\--gc:orc_ are pretty new. The _async_ stuff works badly with threads because it kind of requires _\--gc:refc_ right now and passing stuff between thread requires copying between heaps. Soon _\--gc:arc_ and _\--gc:orc_ will work with _async_ and will replace _\--gc:refc_. Then _async_ will work with threads better.
Re: How mature is async/threading in Nim?
> how does your C++ architecture share data between threads? Carefully ;-) The Actor library doesn't restrict what parameters you can pass, but by convention we're careful to limit it to primitives, pass-by-copy objects (like std::string), immutable ref-counted objects, and Actor references. Or else we use rvalue references to enforce 'move' semantics. But mistakes happen, so I'm looking forward to a system that will have better enforcement.
Re: How mature is async/threading in Nim?
> Again, I'm considering porting a large existing project, not writing > something from scratch. I already know the requirements and the architecture. > Nim is different from C++ but that doesn't mean I'm going to change the > design to single-threaded. I definitely need multiple threads because the > WebSocket messages invoke tasks that may be CPU-intensive or just block a > while (database calls.) It seems like the threading support is ok but you just need to be careful when sharing data between threads. I'm coming from mostly Elixir programming the last couple of years, so the "actor" model and copying messages between them makes sense to me. Out of curiosity, how does your C++ architecture share data between threads?
Re: How mature is async/threading in Nim?
> Threads are just kind of hard to use with gc:refc, that is why gc:arc is > getting worked on. I'm still confused as to which GC is the default. I thought it was ARC, but what I'm reading recently makes it sound like ARC isn't quite stable enough yet. > Just regular threads are probably better if you are just writing and reading > from a single websocket connection. Async and threads are orthogonal, not opposite choices. What interests me about async/await is how it cleans up the control flow in source code, avoiding "callback hell". Again, I'm considering porting a large existing project, not writing something from scratch. I already know the requirements and the architecture. Nim is different from C++ but that doesn't mean I'm going to change the design to single-threaded. I definitely need multiple threads because the WebSocket messages invoke tasks that may be CPU-intensive or just block a while (database calls.)
Re: How mature is async/threading in Nim?
> Could you explain why not? Current current gc:refc does not allow refs object to be passed between threads willy nilly. Async creates a reactor ref object per thread. So you can't really share the async corutines between threads. Threads are just kind of hard to use with gc:refc, that is why gc:arc is getting worked on. > This project is a library for (primarily) mobile apps. I have limited experience with nim mobile apps. But knowing what I know don't think I would use async on the client. Async is great if you are doing tons of http style requests. But really a mobile client? Just regular threads are probably better if you are just writing and reading from a single websocket connection. I would try both methods to see which one fits you better. I am also a huge fan of UDP, packed based protocol instead of streaming http/tcp. You don't even need threads to have a good UDP system. Here is what I use [https://github.com/treeform/netty](https://github.com/treeform/netty) if I control both ends. But some times you just have to go with WebSockets, I wrote a library for that too: [https://github.com/treeform/ws](https://github.com/treeform/ws) though its async based.
Re: How mature is async/threading in Nim?
Are you using Jester or another web framework?
Re: How mature is async/threading in Nim?
> it requires an introduction that explains to users what ARC is, how to make > use of it +1 The existing documentation is **great** (I’ve read the tutorial, manual, and “Nim In Action” cover to cover), but in some areas seems to lag behind. Which is understandable since the language is evolving quickly. I’m one of those weird people who likes writing documentation, so maybe when/if I get up to speed on this stuff I can help out.
Re: How mature is async/threading in Nim?
> Async does not mesh well with threds. Could you explain why not? My understanding is that it’s thread-agnostic; an unfinished async call is just a sort of lightweight continuation that can be restarted in any context. > Multiprocessing is more scalable anyways. This project is a library for (primarily) mobile apps, so that’s not an option!
Re: How mature is async/threading in Nim?
> for new projects I wouldn't use anything else because the tooling is so much > better That would be great, but it requires an introduction that explains to users what ARC is, how to make use of it, how it impacts multithreading, the new sync and lent parameters, how to design collections and libraries without a GC and much more.
Re: How mature is async/threading in Nim?
> At this point I'm unclear on how much of this stuff is solid and > enabled-by-default (in particular, what's the difference between "arc" and > "orc"?) ARC is in version 1.2 with significant stability improvements around the corner in 1.2.2. Many Nimble packages already work with `--gc:arc`. While the stability is still not good enough for Nim compiler bootstrapping, for new projects I wouldn't use anything else because the tooling is so much better. All the sanitizers from C++ simply work, compile your code with `nim c --gc:arc --debuginfo -d:useMalloc y.nim && valgrind ./y` and you can be assured the remaining ARC bugs (sorry!) don't affect you. You can also slowly move from the C++ code to Nim, the interop between Nim and C++ is superb and only getting better with ARC.
Re: How mature is async/threading in Nim?
Parallelism/concurrency and async are some of the few pain points of the language. ARC/ORC together with [https://github.com/mratsim/weave](https://github.com/mratsim/weave) might be very promising.
Re: How mature is async/threading in Nim?
I use async/await in production, but only server side on Linux. It's pretty mature for a simple web application. My app runs on multiple servers instead of threds. Async does not mesh well with threds. Multiprocessing is more scalable anyways.
Re: How mature is async/threading in Nim?
ORC is ARC with a cycle collector to deal with cycles (and collect them so they won't leak)
Re: How mature is async/threading in Nim?
Right after posting that (how often this happens!) I came across [the big ARC thread](https://forum.nim-lang.org/t/5734) here from last December. It sounds like ARC means a lot of (positive) changes to the things I've read earlier, like: > The heap is now shared as it's done in C++, C#, Rust, etc etc. A shared heap > allows us to move subgraphs between threads without the deep copies but the > subgraph must be "isolated" ensuring the freedom of data races and at the > same time allowing us to use non-atomic reference counting operations. How to > ensure this "isolation" at compile-time was pioneered by Pony and we can do > it too via our owned ref syntax I've been interested in the Pony language for several years and adopting its memory model would be amazing! I'm still reading through this long thread. At this point I'm unclear on how much of this stuff is solid and enabled-by-default (in particular, what's the difference between "arc" and "orc"?)
How mature is async/threading in Nim?
The project for which I'm evaluating Nim does a lot of concurrent network I/O (over WebSockets) and database access. It needs to run on most mainstream platforms (iOS, Android, Mac, Windows, Linux.) The current implementation is in C++17, using a homemade Actor library. The other language I'm seriously considering is Rust. Nim seems to have the necessary pieces, like the `async` macro and the `asyncnet` module. But are they mature enough to use in production code? I have a few specific questions: 1. The asyncnet module documentation has a couple of caveats about Windows, like "on Windows it only supports select()" and "In theory you should be able to work with any of these layers interchangeably (as long as you only care about non-Windows platforms)." I'm not clear on how this would impact clients, and whether these caveats apply to using asyncnet or just the lower-level modules. 2. The async examples I've seen run on a single thread. Is there any support for distributing async calls across a thread pool? 3. The `spawn` function's thread safety seems to be based on segregating heap objects per thread (Erlang-style.) This can involve a lot of copying, especially when passing data buffers around. Is this something baked into the language or is it specific to the `spawn` function? Are there alternatives to this, like the more Rust-style model using move semantics? We found [an interesting quote on r/nim](https://www.reddit.com/r/nim/comments/giaeev/what_are_the_biggest_weaknesses_of_nim_in_your/fqdlmhf/) from a few days ago: > What ... caused my last team to abandon Nim in favor of Haskell ... was the > weak concurrency story. There is _a_ story there (thread-local heaps) but we > found it far too easy to get yourself into very confusing situations where > behavior wasn't as expected (especially since you can pass around pointers > which breaks the thread safety checker). To be fair, our C++ code has almost no built-in thread safety at all; you have to be careful what types you pass to Actor methods. But that's one of the things I'd like to improve on! I've written some prototype code in both languages (C API bindings, not any async or net code yet) and I have to say I really enjoyed Nim a lot. Rust I found frustrating, and ugly due to lack of function overloading and the need to convert between equivalent types like str/String. Nim also builds faster, and seems to generate much smaller code. But Rust does have huge momentum behind it so it feels safer for that reason, and the ironclad memory safety is a good thing to have. Any comments or perspective from those who've been using Nim a lot? \--Jens| ---|---
