Below is part of a generic ordered fan-in for data pipelines. It works by
reading data from multiple channels using one goroutine for each channel,
sending that data element to another goroutine that does the actual
ordering, but in the mean time, pauses the pipeline stage until all
out-of-order data elements are sent.
func orderedFanIn[T sequenced](done <-chan struct{}, channels ...<-chan T)
<-chan T {
fanInCh := make(chan fanInRecord[T])
wg := sync.WaitGroup{}
for i := range channels {
pauseCh := make(chan struct{})
wg.Add(1)
// Create one goroutine for each incoming channel
go func(index int, pause chan struct{}) {
defer wg.Done()
for {
var ok bool
var data T
select {
case data, ok = <-channels[index]:
if !ok {
return
}
// Send data to the ordering goroutine with the pause channel
fanInCh <- fanInRecord[T]{
index: index,
data: data,
pause: pause,
}
case <-done:
return
}
// Pause the pipeline stage until the ordering goroutine writes to the
pause channel
select {
case <-pause:
case <-done:
return
}
}
}(i, pauseCh)
}
On Sun, Apr 2, 2023 at 9:44 PM Nigel Tao <nigel...@golang.org> wrote:
> I'm working on a multi-threaded C++ project. We have the equivalent of
> Go's channels, and are considering whether we also need to implement
> the equivalent of Go's select.
>
> Does anyone have interesting, non-trivial examples of a Go select
> statement in real code?
>
> By non-trivial, I mean that a lot of the selects that I've seen have
> exactly two cases, one of them doing "real work" and the other being
> either (1) "default" or (2) a timeout/cancel channel (e.g.
> ctx.Done()).
>
> In our C++ API, our channel send/recv methods already have
> try_send/try_recv equivalents for (1) and a timeout/cancel mechanism
> for (2).
>
> bcmills' "Rethinking Classical
> Concurrency Patterns"
> (https://drive.google.com/file/d/1nPdvhB0PutEJzdCq5ms6UI58dp50fcAN/view)
> uses select to implement higher level ResourcePool / WorkerPool APIs
> but select is arguably a private implementation detail. While it might
> not be as beautiful under the hood, I think we can already present
> similar APIs using C++'s std::counting_semaphore.
>
> r's "A Concurrent Window System"
> (https://swtch.com/~rsc/thread/cws.pdf) discusses select'ing from
> separate window, keyboard and mouse channels but this could arguably
> instead be a single channel of heterogenous elements (e.g. in C++, a
> std::variant).
>
> It's more interesting to select over both input and output channels,
> and output channels may become "ready to communicate" without new
> input. But again, it may be possible to work around that by downstream
> actors sending "I'm ready to receive" events onto the upstream actor's
> heterogenous input channel.
>
> The most interesting selects I have so far is the
> golang.org/x/net/http2 source code, whose internals have a bit of a
> learning curve. If anyone has other examples, please share.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOeFMNWBtuEci9oPUFNa0v0gDC%3DV6Xb0N05Jyxo%3DxN2ywJALGA%40mail.gmail.com
> .
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqq-_Xg_npzoJk89V2bbuo-oDH5kXQnGqPGixBGnQhjMmg%40mail.gmail.com.
