After some analysis and discussion, the sample I provided earlier has a data
race by definition - although in many cases these are benign and do no affect
the correctness of the program. E.g. the program may be “sampling” and so any
valid value read is “ok”. In detail, because of the loop, the writer could
mutate the y value multiple times before the reader observed the change and
read the updated value - thus the “race".
As to the “all accesses must be atomic” contention for “happens before” - this
is not correct.The following is a valid and common use case of atomics even
though multiple routines are modifying a variable non-atomically. The race
detector does not report any issues. The CAS below creates a spin lock, but the
technique can be more advanced with similar purpose - synchronization without
locks/blocking - depending on the data structure.
The neat thing is how the race detector is able to detect that the following is
“ok” - I look forward to understanding more on how that is accomplished.
(As a mea culpa - it was me forgetting that slices are a struct not a pointer -
and so atomically manipulating them is not possible - at least not directly).
package main
import (
"fmt"
"sync"
"sync/atomic"
)
func main() {
var x int32
var y int32
w := sync.WaitGroup{}
w.Add(3)
go func() {
for {
if atomic.CompareAndSwapInt32(&x, 0, 1) {
y = y + 1
atomic.StoreInt32(&x, 0)
}
}
w.Done()
}()
go func() {
for {
if atomic.CompareAndSwapInt32(&x, 0, 1) {
y = y + 1
atomic.StoreInt32(&x, 0)
}
}
w.Done()
}()
go func() {
for {
if atomic.CompareAndSwapInt32(&x, 0, 1) {
if y%1000000 == 0 {
fmt.Print("matched")
}
atomic.StoreInt32(&x, 0)
}
}
w.Done()
}()
w.Wait()
}
> On Sep 15, 2022, at 10:59 AM, Robert Engels <reng...@ix.netcom.com> wrote:
>
> I have not misunderstood it. Nor have I misunderstood the semantics with
> Javas volatile. One aspect of Go is broken - tbd - but most likely it is
> simply the race detector cannot support it (as my concurrent code functions
> as expected given the memory model).
>
>> On Sep 15, 2022, at 10:22 AM, Thomas Bushnell BSG <tbushn...@google.com>
>> wrote:
>>
>>
>> The GCC Wiki you reference is speaking only of a case where all the
>> variables concerned are in a class that specifies sequentially consistent
>> ordering. You have also misunderstood the C++ semantics, which is
>> understandable since the language is a disaster of bad specification.
>>
>> Thomas
>>
>> On Thu, Sep 15, 2022 at 11:20 AM robert engels <reng...@ix.netcom.com
>> <mailto:reng...@ix.netcom.com>> wrote:
>> Finally, you left out the most important clause from the “memory model”
>> document:
>>
>> "The preceding definition has the same semantics as C++’s sequentially
>> consistent atomics and Java’s volatile variables.”
>>
>> Both of these require the “happens before” semantics I describe - not what
>> you are describing.
>>
>>
>>> On Sep 15, 2022, at 10:16 AM, robert engels <reng...@ix.netcom.com
>>> <mailto:reng...@ix.netcom.com>> wrote:
>>>
>>> This is simply incorrect. The ‘issue’ about clarifying the memory has been
>>> about “happens before” since the beginning. The model was clarified. The
>>> race detector cannot cope with it.
>>>
>>> I don’t think you fully understand what “happens before” means. Please read
>>> the article I referred to.
>>>
>>>> On Sep 15, 2022, at 10:11 AM, Thomas Bushnell BSG <tbushn...@google.com
>>>> <mailto:tbushn...@google.com>> wrote:
>>>>
>>>> I cannot speak to "other accepted concurrency designs" here. Simply that
>>>> Go does not guarantee the operation you want it to, the memory model does
>>>> not actually imply that it does, and the last sentence of the memory model
>>>> is the most important one here: don't be clever.
>>>>
>>>> On Thu, Sep 15, 2022 at 11:08 AM robert engels <reng...@ix.netcom.com
>>>> <mailto:reng...@ix.netcom.com>> wrote:
>>>> This is not what “happens before” means - at least not in any other
>>>> accepted concurrency designs.
>>>>
>>>> See my second example as to why.
>>>>
>>>>> On Sep 15, 2022, at 10:02 AM, Thomas Bushnell BSG <tbushn...@google.com
>>>>> <mailto:tbushn...@google.com>> wrote:
>>>>>
>>>>> Happens before works just fine with atomics. But in your example, x is
>>>>> not an atomic.
>>>>>
>>>>> Thomas
>>>>>
>>>>> On Thu, Sep 15, 2022 at 10:51 AM robert engels <reng...@ix.netcom.com
>>>>> <mailto:reng...@ix.netcom.com>> wrote:
>>>>> Yea, the race detector is broken… it fails on the following code:
>>>>>
>>>>> package main
>>>>>
>>>>> import (
>>>>> "sync"
>>>>> "sync/atomic"
>>>>> )
>>>>>
>>>>> func main() {
>>>>>
>>>>> var x int32
>>>>> var y int32
>>>>>
>>>>> w := sync.WaitGroup{}
>>>>> w.Add(2)
>>>>>
>>>>> go func() {
>>>>> for {
>>>>> x = 1
>>>>> atomic.StoreInt32(&y, 1)
>>>>> }
>>>>> w.Done()
>>>>> }()
>>>>> go func() {
>>>>> for {
>>>>> if atomic.LoadInt32(&y) == 1 {
>>>>> if x != 1 {
>>>>> panic("should not happen")
>>>>> }
>>>>> }
>>>>> }
>>>>> w.Done()
>>>>> }()
>>>>> w.Wait()
>>>>>
>>>>> }
>>>>>
>>>>> The above code does not have a race, or Go doesn’t have “happens before”
>>>>> semantics with its atomics.
>>>>>
>>>>>
>>>>>> On Sep 15, 2022, at 9:41 AM, Robert Engels <reng...@ix.netcom.com
>>>>>> <mailto:reng...@ix.netcom.com>> wrote:
>>>>>>
>>>>>> To clarify, if the atomic read of Y sees the updated Y then a subsequent
>>>>>> non-atomic read of X must see the updated X. This is a happens before
>>>>>> relationship.
>>>>>>
>>>>>> The question was if the race detector understands this - I know - why
>>>>>> not try it out…
>>>>>>
>>>>>>> On Sep 15, 2022, at 9:39 AM, Robert Engels <reng...@ix.netcom.com
>>>>>>> <mailto:reng...@ix.netcom.com>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> I think it needs to see the updated X - which agrees with burak.
>>>>>>>
>>>>>>> Reading Z is race.
>>>>>>>
>>>>>>>> On Sep 15, 2022, at 9:24 AM, burak serdar <bser...@computer.org
>>>>>>>> <mailto:bser...@computer.org>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2022 at 8:03 AM 'Thomas Bushnell BSG' via golang-nuts
>>>>>>>> <golang-nuts@googlegroups.com <mailto:golang-nuts@googlegroups.com>>
>>>>>>>> wrote:
>>>>>>>> You cannot make that assumption. It's not about what the race detector
>>>>>>>> can detect.
>>>>>>>>
>>>>>>>> Goroutine one:
>>>>>>>> Writes non-synchronized X
>>>>>>>> Writes atomic Y
>>>>>>>> Writes non-synchronized Z with the value of X+Y
>>>>>>>>
>>>>>>>> Goroutine two
>>>>>>>> Reads atomic Y and sees the new value
>>>>>>>>
>>>>>>>> The way I read the Go memory model, if Goroutine two sees the new
>>>>>>>> value of Y, non-synchronizes writes to X by Goroutine 2 happened
>>>>>>>> before Y, and thus, anything that happens after Y. This is based on:
>>>>>>>>
>>>>>>>> "If a synchronizing read-like memory operation r observes a
>>>>>>>> synchronizing write-like memory operation w (that is, if W(r) = w),
>>>>>>>> then w is synchronized before r."
>>>>>>>>
>>>>>>>> And:
>>>>>>>>
>>>>>>>> "The happens before relation is defined as the transitive closure of
>>>>>>>> the union of the sequenced before and synchronized before relations."
>>>>>>>>
>>>>>>>> Because:
>>>>>>>> * The writes to non-synchronized X are sequenced before the atomic
>>>>>>>> write to Y
>>>>>>>> * The atomic read Y happened after atomic write to Y if it sees the
>>>>>>>> new value
>>>>>>>> * non-synchronized reads from X happen after that
>>>>>>>>
>>>>>>>> So that should not be a race.
>>>>>>>>
>>>>>>>> Am I reading this correctly?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Can goroutine two now read non-synchronized X and assume it sees the
>>>>>>>> new value written by one? No, it cannot. There is no "happens before"
>>>>>>>> relation connecting the two writes performed by goroutine one.
>>>>>>>> Requirement one does not establish such a relationship. It only
>>>>>>>> establishes that Z will be written with the correct sum of X and Y.
>>>>>>>> There must be some sequential order within the context of goroutine
>>>>>>>> one that sees the correct value; the compiler is free to swap the
>>>>>>>> order of the writes X and Y.
>>>>>>>>
>>>>>>>> If X were an atomic, then Requirement two would come into play. But
>>>>>>>> because X and Z are not atomic, they play no role in Requirement two.
>>>>>>>> Note that the description of atomic in the model says that writes to
>>>>>>>> atomic values have the property you want. And since there is no before
>>>>>>>> relationship established by any of the following text, this
>>>>>>>> synchronization cannot be relied on.
>>>>>>>>
>>>>>>>> Now you're asking whether the race detector ensures the
>>>>>>>> synchronization property you're suggesting? The race detector doesn't
>>>>>>>> ensure any synchronization properties; it detects bugs.
>>>>>>>>
>>>>>>>> I think it is capable of detecting this one.
>>>>>>>>
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 14, 2022 at 11:01 PM robert engels <reng...@ix.netcom.com
>>>>>>>> <mailto:reng...@ix.netcom.com>> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am working on a new project, and the race detector is reporting a
>>>>>>>> race.
>>>>>>>>
>>>>>>>> Essentially, the code is
>>>>>>>>
>>>>>>>> var S []int
>>>>>>>>
>>>>>>>> several go routines write new S values using a mutex
>>>>>>>>
>>>>>>>> go routine Y reads S without grabbing a lock (it reads it initially
>>>>>>>> under lock)
>>>>>>>>
>>>>>>>> The semantics are such that Y can operate successfully with any valid
>>>>>>>> value of S (e.g. could be stale). (essentially S is used with copy on
>>>>>>>> write semantics)
>>>>>>>>
>>>>>>>> The race detector reports this as a race.
>>>>>>>>
>>>>>>>> I could change all reads of Y to use an atomic load, but I don’t think
>>>>>>>> it should be necessary.
>>>>>>>>
>>>>>>>> Is there any way to perform “lazy loads” in Go?
>>>>>>>>
>>>>>>>> And a follow-up:
>>>>>>>>
>>>>>>>> Is the race detector smart enough so that if a routines write to
>>>>>>>> several vars (v1…n) and performs an atomic store to X, and another
>>>>>>>> routine atomically reads X it can also non atomically read v1…n and it
>>>>>>>> will see the stored values?
>>>>>>>>
>>>>>>>> This has been the long standing issue with the Go memory model and
>>>>>>>> “happens before”… but how does the race detector report this?
>>>>>>>>
>>>>>>>> (Some background, the library functions fine under heavy concurrent
>>>>>>>> stress tests - but the race detector says it is broken).
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "golang-nuts" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>>>> an email to golang-nuts+unsubscr...@googlegroups.com
>>>>>>>> <mailto:golang-nuts%2bunsubscr...@googlegroups.com>.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/8EC74417-C4AD-4490-9231-6E869EE72D93%40ix.netcom.com
>>>>>>>>
>>>>>>>> <https://groups.google.com/d/msgid/golang-nuts/8EC74417-C4AD-4490-9231-6E869EE72D93%40ix.netcom.com>.
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "golang-nuts" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>>>> an email to golang-nuts+unsubscr...@googlegroups.com
>>>>>>>> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/CA%2BYjuxtd%2BpaU_BNxXDrMAN9v71r-Qhm9LcXcN2fTtjD_6oWw-Q%40mail.gmail.com
>>>>>>>>
>>>>>>>> <https://groups.google.com/d/msgid/golang-nuts/CA%2BYjuxtd%2BpaU_BNxXDrMAN9v71r-Qhm9LcXcN2fTtjD_6oWw-Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "golang-nuts" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>>>> an email to golang-nuts+unsubscr...@googlegroups.com
>>>>>>>> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqr4vggPOWjiQg6qN0tJjhhXncKHLMCDwkqZTHBJJ7%3Dmug%40mail.gmail.com
>>>>>>>>
>>>>>>>> <https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqr4vggPOWjiQg6qN0tJjhhXncKHLMCDwkqZTHBJJ7%3Dmug%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "golang-nuts" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>> an email to golang-nuts+unsubscr...@googlegroups.com
>>>>>> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/golang-nuts/93E2826E-1D85-48BE-BBD3-ED17F70C74F2%40ix.netcom.com
>>>>>>
>>>>>> <https://groups.google.com/d/msgid/golang-nuts/93E2826E-1D85-48BE-BBD3-ED17F70C74F2%40ix.netcom.com?utm_medium=email&utm_source=footer>.
>>>>>
>>>>
>>>
>>
> On Sep 15, 2022, at 10:23 AM, burak serdar <bser...@computer.org> wrote:
>
>
>
> On Thu, Sep 15, 2022 at 9:21 AM Thomas Bushnell BSG <tbushn...@google.com
> <mailto:tbushn...@google.com>> wrote:
> On Thu, Sep 15, 2022 at 11:19 AM burak serdar <bser...@computer.org
> <mailto:bser...@computer.org>> wrote:
> On Thu, Sep 15, 2022 at 9:11 AM Thomas Bushnell BSG <tbushn...@google.com
> <mailto:tbushn...@google.com>> wrote:
> I cannot speak to "other accepted concurrency designs" here. Simply that Go
> does not guarantee the operation you want it to, the memory model does not
> actually imply that it does, and the last sentence of the memory model is the
> most important one here: don't be clever.
>
> I believe that's what we are discussing here. The way I read it, the memory
> model does imply that a non-synchronized operation x sequenced before an
> operation y that happened before z implies that all operations sequenced
> after z happens after x.
>
> You may be misreading requirement one as implying that no reordering within a
> goroutine is ever permitted. It does not say that. What it says is that,
> within one goroutine, any reordering is permissible provided the actual value
> read and written by that goroutine are the correct ones.
>
> I believe what the memory model implies is that no reordering is permitted to
> cross a memory barrier. An atomic operation has a memory barrier.
>
>
> Thomas
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com
> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoZ2J8hHY6mD_rNdJu7W4p0osW0E%2BUxx8pYnh815%3DFmTw%40mail.gmail.com
>
> <https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoZ2J8hHY6mD_rNdJu7W4p0osW0E%2BUxx8pYnh815%3DFmTw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/69E5B95A-E293-4859-A2D1-155B08416B5B%40ix.netcom.com.
