> I don't care about direct comparisons with Golang...
I don't really want to make comparisons to GoLang either as I really don't like
it, but there is just this one feature I would like Nim to have: how easy it is
to make a function run on a separate "green thread" and communicate via
channels, which for us seems to mean on a thread pool as our `Thread[T]`'s are
just wrappers for OS threads which can be slowish to do context switches.
**In fact, the current `spawn` used calling a function without a return value
is essentially like calling a "go routine" except **it is currently recommended
that we shouldn't use `Channel[T]` with `spawn`** unlike GoLang where this is
the recommended way. Currently, Channels have these major limitations as per
the documentation:
1. Note: Channels are designed for the Thread type. They are unstable when
used with spawn
2. Note: The current implementation of message passing does not work with
cyclic data structures.
3. Note: Channels cannot be passed between threads. Use globals or pass them
by ptr.
Okay, item 2 maybe no longer needs to be a problem if we use `--gc:orc`, and
item 3 is only a bit ugly in requiring a global channel variable or a
(self-managed) raw pointer, but item 1 is a major limitation: implementation of
channels must work with the implementation of the thread pool else the
performance will suffer in creating OS threads or the user needs to implement
their own thread pool (which is what I have done up to now with the instability
of the `threadpool` library) The difference is major: typically OS threads
can't be created more than something like one a few milliseconds or performance
suffers, whereas thread pool "green threads" can be "spawned" at a rate of
millions a second.
**I don 't think you answered whether the {.thread.} pragma actually does
anything once there is no distinction between shared and non-shared memory so
that if threads share memory they need to protect the accesses themselves, not
through "compiler magic"?**
> The plan is to have more light-weight channels that are based on
> `Isolated[T]`. We don't need `FlowVar[T]` anymore...
If we don't have `FlowVar[T]` anymore then `spawn` which can optionally return
a `FlowVar[T[` has a more restricted API that may break some existing code, and
if it is decided that `parallel` is a good idea that didn't work out, then all
of the `threadpool` library is depreciated although it has always been marked
as an Unstable API; **The only thing recoverable out of `threadpool` is a
partially compatible `spawn` and `spawn` support functions; does that summarize
it?**
However, if we are going to have threads servicing a "light weight channels
based on `Isolated[T]`", then we still need something to replace the
`threadpool` library that implements thread pools (or `threadpools` needs a
complete rewrite), and it still needs an API to launch those "green threads";
**Is that a new library, or do we break `threadpool` since the changes seem to
break the current API?**
If we consider `Channel[T]` or Channel[Isolated[T]` to be lightweight message
passing mechanisms, **Doesn 't there need to be a `threadpool` mechanism is
capable of using these new-type channels in receiving messages, processing
them, and possibly sending them on via another channel as is the usual way of
using them?**
**Also, does this imply some changing the the `channels` library as to API?**
Since these are library changes, I am likely capable of making these changes
for you, with maybe a little help from others such as @mratsim, but I need to
know your thoughts and ideas before proceeding.
> > Guess I had better file an issue on that?
>
> The bug is that the compiler doesn't prevent ptr Channel[sink
> Isolated[string]] which is not a valid type, sink is a type modifier that can
> only be applied to parameters directly.
I removed the `sink`'s characterizing the contents of the `Isolated[T]` and
still get the same error; ** Does that mean this is a more serious issue now?**
However, it is an non-issue if we are going to change `spawn` so it doesn't
have a return value, as that seems to be the problem area.
> > As to parallel, do you see any way of stabilizing it and getting it off of
> > the experimental list, at least for use with --gc:arc/orc, for version 2.0?
>
> No. parallel is a nice idea but hasn't seen enough testing. I also have not
> yet figured out how to map it to GPU programming which was one of my goals.
> And as you said yourself, maybe its use cases are too narrow so that the
> complexity of a "disjoint checker" doesn't pay off.
**So, what 's the plan forward?** In the interests of making Nim a language
capable of doing everything, without a `parallel` then we can't do what HPC
languages such as Fortran does (kind of) and Cray's Chapel can definitely do
with their `forall`/`coforall` built-in's (although Chapel currently has other
problems being entirely OOP with no closures and limited type
reductions/promotions). Perhaps those language implementations aren't very
sophisticated in "disjoint" checking either?
I suppose one way of handling it simply is to just force every input on the rhs
to be an immutable type for the duration of the `spawn` so it doesn't matter if
the inputs are disjoint or not? That would imply every `spawn` function can't
take var or ref parameters unless they are `sink` or `Isolated[T]` which means
the `spawn` thread owns them, no matter whether they got there by copying or
moving. Then, rather than return them, the `spawn` thread returns any results
through a channel.
But I guess that could as well describe the new vision of "`spawn` without
`FlowVar[T]` that is expected to communicate via the new vision of
"light-weight channels"?
This doesn't really affect Nim version 2.0, but something we may want to look
at in the future. **Is it on the back burner for now?**
