On Mon, Oct 16, 2017 at 1:01 PM, Caleb Spare <cesp...@gmail.com> wrote: > > I have a server which is doing log processing and I'm trying to > improve throughput. The bottleneck is access to a shared resource > protected by a sync.Mutex. The issue is that even though there are > nearly always worker goroutines blocked on Lock, only about 10% of the > time (in aggregate) is spent running the code inside the critical > section. > > To be concrete, I have 300 worker goroutines, and they collectively > spend about 280 seconds / second blocked on the mutex, and about 100 > ms / second inside (as measured from Lock returning to Unlock > returning). > > To investigate this, I added some instrumentation around sync.Mutex. I > log events when a Lock is called, when Lock returns (acquire lock), > and when Unlock returns. This shows me that, typically, my code takes > around 5-15 μs between acquire and unlock, but it takes tens or > hundreds of μs between unlock and some other goroutine acquiring the > lock.
One thing to do would be to find out whether the mutex is in starvation mode. From your logs, it looks like it should be. You can probably just fmt.Print it and look at the state value: if state & (1 << 2) != 0, it is in starvation mode. When a mutex is in starvation mode, then when one goroutine unlocks the first waiting goroutine is made ready directly. However, that goroutine will be put on the run queue of the current goroutine's P. That means that by default the waiting goroutine won't start running until the current goroutine blocks or is preempted. So: what do your goroutines do after they call Unlock? If they keep running doing computations without making any system calls, that will tend to block the waiting goroutine from running. The waiting goroutine will run either when the running goroutine blocks or when some other P steals it. This is often the right thing to do, but may not be ideal for your particular case. If this does seem to describe your program, you might try calling runtime.Gosched immediately after the mutex unlock. Ian -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
