Hi Caleb

Your approach seems to me as a reasonable use case for channels and I'm 
sure it would be a lot complexer if you use a bunch of mutexes.

One part in the article looks a little bit weird to me:

>   // ...
>   s.mtx.Lock()
>   // ...
>   s.ch <- val // might block!
>   s.mtx.Unlock()
>   // ...
My experience with go is limited but until now I have never seen that kind 
of code, I don't think it is something advanced go users would do or I'm 

Cheers snmed

Am Mittwoch, 9. August 2017 14:30:38 UTC+2 schrieb Caleb Doxsey:
> Channels are very useful and necessary to really get your program using 
> all the resources at your disposal. Doing that with locks or callbacks is 
> error prone and makes solving some problems all but impossible. Let me give 
> you an example of a pattern we used a few days ago:
> We were processing message from a queue, writing some data to Cassandra, 
> and then committing offsets when the work was completed. It was really 
> important for the commits to come out in the same order they came in, so 
> that we never commit an offset when we haven't actually written all the 
> data. (on crash or restart its ok to process the same messages again 
> because the writing is idempotent)
> Modeled as a pipeline we had a few stages:
> intake -> decode -> encode -> write to cassandra -> commit
> The naive approach would be to spin up a bunch of writer goroutines, but 
> then the order of messages to commit would be unpredictable. Instead we 
> created two slices of channels:
> ins := make([]chan intakeMessage, options.workers)
> outs := make([]chan commitMessage, options.workers)
> And spun up goroutines that read from one of the input channels and wrote 
> to the corresponding out channel:
> for i := 0; i < options.workers; i++ {
>                 ins[i] = make(chan intakeMessage, 1)
>                 outs[i] := make(chan commitMessage, 1)
>                 go worker(ins[i], outs[i])
> }
> We then use these slices as a circular buffer and keep track of two 
> pointers, one for the next available in and another for the next remaining 
> out. There are 3 cases:
>    1. The slice is empty, in which case we insert at 0 and increment the 
>    in counter
>    2. The slice is full (all workers are busy), in which case we wait 
>    until a result is pushed into the channel at the out counter
>    3. We're somewhere in between, in which case we use a select on either 
>    the next available in channel or the next remaining out counter
> So all the commits come out in the order they came in, we get nice 
> parallelism and there's no blocking or polling.
> There is overhead to channels but if you send messages of adequate size 
> you'll barely notice it in real programs. Batch and send slices.
> The thing I find remarkable about channels and goroutines is how often the 
> solutions for improving the performance of a Go program are almost exactly 
> the same as the solutions for distributing that program across multiple 
> machines. The patterns really map well to these kinds of problems.
> On Tuesday, August 8, 2017 at 2:01:12 AM UTC-4, snmed wrote:
>> Hi Gophers
>> I stumbled over a nice and very interesting Blog entry "Go channels are 
>> bad and you should feel bad 
>> <http://www.jtolds.com/writing/2016/03/go-channels-are-bad-and-you-should-feel-bad/>"
>> , I would like to hear some opinions about that article
>> from seasoned go developers. Because I just used go for a couple of 
>> months for my private web projects and rarely get in touch with channels.
>> By the way, that article is not a rant and the author likes go very much 
>> as far as I can conclude from the article.
>> Cheers snmed

You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to