Re: [go-nuts] Re: locks and "happens before" withing a goroutine

[Daniel Fanjul]

> Within a single goroutine, the happens-before order is the order 
expressed by the program. 
If this is true, why is the order of 'a = 1; b = 2;' not ensured?
I think you may be forgetting that "compilers and processors may reorder 
the reads and writes executed within a single goroutine"

> Inside a single goroutine the code proceeds in statement order. 
The code generated by the compiler, right? Of course, it is typically the 
processor who reorders. But what is the limit of the reorder? Is 'a = 1; b 
= 2;' so simple that we worry about this case, but is 'a = 1; x.Lock(); b = 
2;' so complex that we assume that it will never be reordered? Now I see it 
is because we assume it is a memory barrier though it is undocumented. 
Thanks, Ian, I am sure now it is undocumented.

> - a = "hello, world" happens-before l.Unlock() 
That is false unless l.Unlock() is memory barrier or some other 
restriction. It happens it is a barrier, so you can say it truly happens 
before. The spec or the memory model does not mention it, so technically we 
cannot state it because of the spec or the memory model.

> If it didn't chaos would ensue, it would be as if statements in a 
function were executed in random order.
In theory, the statements are allowed to run in any order but only "when 
the reordering does not change the behavior within that goroutine as 
defined by the language specification", that is "when reads and writes must 
behave as if they executed in the order specified by the program". But I 
agree with you, Dave, in practice the deviation is small.

> I agree that this fact is not documented, but it is true nonetheless. 
Cool, thank you. I think it is quite important and really necessary to 
understand the memory model. Otherwise its examples do not make sense at 
all.

Thank you all, I finally and completely understood the memory model.


On Thursday, December 1, 2016 at 7:15:45 AM UTC+1, Ian Lance Taylor wrote:
>
> On Wed, Nov 30, 2016 at 5:44 PM, Daniel Fanjul 
> <daniel.fan...@gmail.com > wrote: 
> > I cannot tell if there is any memory barrier in the code of 
> > sync.RWMutex.Lock(), I don't understand it completely. I just think 
> there is 
> > not. 
> > https://golang.org/src/sync/mutex.go?s=1143:1165#L34 
>
> The sync/atomic routines provides memory barriers. 
>
> I agree that this fact is not documented, but it is true nonetheless. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: locks and "happens before" withing a goroutine

[Daniel Fanjul]

I cannot tell if there is any memory barrier in the code of 
sync.RWMutex.Lock(), I don't understand it completely. I just think there 
is not.
https://golang.org/src/sync/mutex.go?s=1143:1165#L34


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: locks and "happens before" withing a goroutine

[Daniel Fanjul]

I think the current implementations of the methods of sync.RWMutex happen 
to be actual memory barriers  
and this is why everything works just fine.

But I don't think the spec or the memory model or the doc of sync mentions 
this.

If this is not described, the example in the memory model for the locks is 
wrong:

   // https://play.golang.org/p/pmhbeyn_wZ

var l sync.Mutex
var a string

func f() {
a = "hello, world"
l.Unlock()
}

func main() {
l.Lock()
go f()
l.Lock()
print(a)
}

 
The statements in f() might be reordered to run l.Unlock() first and then 
the assignment. The code would not guarantee to print "hello, world" at the 
end.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] locks and "happens before" withing a goroutine

[Daniel Fanjul]

Hi gophers,

The code "a = 1; b = 2;" within a goroutine does not ensure any "happens 
before" relation between these assignments for an external observer.

My doubt is with the code "a = 1; x.Lock(); b = 2;" where x is a 
sync.RWMutex. Does the spec (or the memory model or anything else) ensure 
that "a = 1;" happens before "x.Lock();" and that "x.Lock();" happens 
before "b = 2"? Does it ensure it for any method of x?

The memory model ensures "happens before" relations between accesses to the 
same variables and between calls to the same mutexes. This example uses 
different variables: a != x, a != b and x != b, so it does not seem to be 
included by the model.

Daniel.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Understanding go routine heap

[Daniel Fanjul]

Err... I think you understood the opposite. 

A no answer would mean this is that issue. A yes answer means the mutex is 
still not enough. I said yes.


On Wednesday, November 30, 2016 at 11:23:49 PM UTC+1, Ian Lance Taylor 
wrote:
>
> On Wed, Nov 30, 2016 at 2:01 PM, Daniel Fanjul 
> <daniel.fan...@gmail.com > wrote: 
> > I was going to answer no, but after a careful review of the go memory 
> model 
> > section regarding locks, I am replying: yes, I would. 
> > 
> > Because the memory model only specifies "happens before" restrictions 
> > between calls to Lock() and Unlock() and that is not this case. 
> > "For any sync.Mutex or sync.RWMutex variable l and n < m, call n of 
> > l.Unlock() happens before call m of l.Lock() returns." 
> > "For any call to l.RLock on a sync.RWMutex variable l, there is an n 
> such 
> > that the l.RLock happens (returns) after call n to l.Unlock and the 
> matching 
> > l.RUnlock happens before call n+1 to l.Lock." 
> > 
> > The inner most goroutine would run the assignment and a call to l.Lock() 
> or 
> > l.Unlock(). You can still swap the actions and "the [resulting] 
> reordering 
> > does not change the behavior within that goroutine as defined by the 
> > language specification". 
>
> OK, I think this is https://golang.org/issue/7948. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Understanding go routine heap

[Daniel Fanjul]

I was going to answer no, but after a careful review of the go memory model 
section regarding locks, I am replying: yes, I would.

Because the memory model only specifies "happens before" restrictions 
between calls to Lock() and Unlock() and that is not this case.
"For any sync.Mutex or sync.RWMutex variable l and n < m, call n of 
l.Unlock() happens before call m of l.Lock() returns."
"For any call to l.RLock on a sync.RWMutex variable l, there is an n such 
that the l.RLock happens (returns) after call n to l.Unlock and the 
matching l.RUnlock happens before call n+1 to l.Lock."

The inner most goroutine would run the assignment and a call to l.Lock() or 
l.Unlock(). You can still swap the actions and "the [resulting] reordering 
does not change the behavior within that goroutine as defined by the 
language specification".


On Wednesday, November 30, 2016 at 10:40:34 PM UTC+1, Ian Lance Taylor 
wrote:
>
> Would you apply the same argument to calls to the Lock and Unlock 
> methods of a sync.Mutex?  After all, the original code could be 
> rewritten to use Lock and Unlock calls, and presumably that would be 
> OK. 
>
> If you agree with that, then I think it is sufficient to say that 
> WaitGroup is implemented using sync.Mutex. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Understanding go routine heap

[Daniel Fanjul]

Case A: because there is a write and then a method call that does not touch 
that variable. The go memory model apply only to reads and writes of the 
same variables. So any possible reordering in this scenario fulfills "the 
reordering does not change the behavior within that goroutine as defined by 
the language specification". 

Case C: because there is a read and a method call that does not touch that 
variable. Same corollary.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Understanding go routine heap

[Daniel Fanjul]

Yes, I saw your assert and I trust it but I don't see how that answers my 
question.

There are 4 things that we expect to happen in order:
1) assignment to *result,*
2) call to *wg.Done()*,
3) call to *wg.Wait()*,
4) read of *result*.

So there is no race condition if we can prove:
A) that (1) happens before (2),
B) that (2) happens before (3),
C) and that (3) happens before (4).

I understand that B) is true, because of the WaitGroup spec and because of 
your assertion. I don't see how it affects the other two points.

I fail to prove A). How does the spec ensure that (1) happens before (2)? 
The spec would ensure it if both actions accessed the same variable, but 
they actually do not.
I would expect the spec to say something like "all statements of the 
function/method happens before the call to the deferred functions/methods, 
regardless the variables they may access"

Now I realize that I also fail to prove C). I am probably missing some 
obvious detail now. I am sorry if that is the case.
I would expect something like "all statements of the function/method happen 
before the read of the variable that holds the value to be returned, 
regardless the variables they may access"




On Wednesday, November 30, 2016 at 8:42:59 PM UTC+1, Ian Lance Taylor wrote:
>
> On Wed, Nov 30, 2016 at 10:04 AM, Daniel Fanjul 
> <daniel.fan...@gmail.com > wrote: 
> > 
> > I still don't understand this issue completely. 
> > 
> > Sure, the call to wg.Done() will "happen before" wg.Wait() finishes, but 
> the 
> > assignment "result = {1}" might still happen concurrently. 
> > 
> > I think the problem is the semantics of the defer and the discussion is 
> > reduced completely to the question: is the assignment "result = {1}" 
> > going to *happen before" the deferred called to "wg.Done()"? 
>
> I am asserting that the wg.Done is a store-release and the wg.Wait is 
> a load-acquire. 
>
>
> > I don't see any occurrence of 'defer' in https://golang.org/ref/mem nor 
> a 
> > proper description in https://golang.org/ref/spec#Defer_statements, so 
> I 
> > feel inclined to conclude this behavior is undefined by the spec. 
>
> I don't see why the spec needs to say anything beyond what it already 
> says about defer statements. 
>
> What kind of statement do you think needs to be added? 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Understanding go routine heap

[Daniel Fanjul]

That quote of the spec does not apply here, because there are no multiple 
reads and writes. There is one write to 'result' and one method call that 
will not read or write that 'result'.


On Wednesday, November 30, 2016 at 7:34:12 PM UTC+1, Jan Mercl wrote:
>
> On Wed, Nov 30, 2016 at 7:04 PM Daniel Fanjul <daniel.fan...@gmail.com 
> > wrote:
>
> > Sure, the call to wg.Done() will "happen before" wg.Wait() finishes, but 
> the assignment "result = {1}" might still happen concurrently.
> >
> > I think the problem is the semantics of the defer and the discussion is 
> reduced completely to the question: is the assignment "result = {1}" 
> going to *happen before" the deferred called to "wg.Done()"?
> > 
> > I don't see any occurrence of 'defer' in https://golang.org/ref/mem nor 
> a proper description in https://golang.org/ref/spec#Defer_statements, so 
> I feel inclined to conclude this behavior is undefined by the spec.
>
> The specs say that the defered function executes on return from the 
> function where the defer statement is executed(1). The memory model 
> specifies that "Within a single goroutine, reads and writes must behave as 
> if they executed in the order specified by the program.". From (1) the 
> order specified is the assignment followed by the defered function 
> invocation (now actually calling wg.Done()), so the proper non-race 
> condition is fully covered.
>
> -- 
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Understanding go routine heap

[Daniel Fanjul]

Hi,

I still don't understand this issue completely.

Sure, the call to *wg.Done()* will "happen before" *wg.Wait()* finishes, 
but the assignment *"result = {1}"* might still happen concurrently.

I think the problem is the semantics of the *defer* and the discussion is 
reduced completely to the question: is the assignment *"result = {1}"* 
going 
to *happen before" the deferred called to *"wg.Done()"*?

I don't see any occurrence of 'defer' in https://golang.org/ref/mem nor a 
proper description in https://golang.org/ref/spec#Defer_statements, so I 
feel inclined to conclude this behavior is undefined by the spec.

Daniel.


On Tuesday, August 23, 2016 at 3:23:27 PM UTC+2, Ian Lance Taylor wrote:
>
> On Tue, Aug 23, 2016 at 3:14 AM, Yulrizka  
> wrote: 
> > 
> > But the case here is indeed as Marvin explained, There is race 
> condition. 
> > But in my example, I make sure that wg done is to protect result pointer 
> > before calling it to caller. 
> > 
> > with this structure some one could easily introduce a race condition if 
> he 
> > tried to access the result pointer. 
> > 
> > What I would like to understand if there is case that my snippet causes 
> race 
> > condition. 
> > 
> > This is my understanding. 
> > 1. Heap memory is shared between go rountine in the same process 
> > 2. main routine create a pointer to some struct on the heap (initially 
> nil) 
> > 3. the go routine allocate memory on the heap for the `foo` struct 
> > 4. the go routine assign main's result pointer 
> > 5. wg.Done() ensure that step 4 is done before it released 
> > 6. process will always returns value generated by the go routine. 
> > 
> > And I would like to make sure that my understanding of heap is correct. 
> That 
> > is shared per process and there are no heap copying between go routine. 
>
> That is correct. 
>
> There is no race in the original program you sent because the wg.Done 
> acts as a store-release and the wg.Wait acts as a load-acquire. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.