Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
* Robert Engels [191112 13:44]: > Sorry, at work so I need to use the 'web mail' interface and it > doesn't appear to be including them when I reply. Understood. I'll just try to follow the threading as best as I can. ...Marvin -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191112184729.bvvaenhww6dqp45g%40basil.wdw.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
Sorry, at work so I need to use the 'web mail' interface and it doesn't appear to be including them when I reply. -Original Message- >From: Marvin Renich >Sent: Nov 12, 2019 12:41 PM >To: golang-nuts >Subject: Re: [go-nuts] What is the correct way to access/modify slice elements >concurrently > >[You keep sending replies that are not connected to the message to which >you are replying. RFC 822 (and subsequent RFCs) defines the In-Reply-To >header, which has been in use for more than 30 years. Most MUAs will >add this automatically. Failure to add the header really messes up MUAs >that display messages from the same thread together. Does your MUA not >add this header???] > >* Robert Engels [191112 13:00]: >> This is not the issue I am referring to, I am referring to >> https://github.com/golang/go/issues/5045 >> >> -Original Message- >> >From: Marvin Renich >> >Sent: Nov 12, 2019 11:30 AM >> >To: golang-nuts >> >Subject: Re: [go-nuts] What is the correct way to access/modify slice >> >elements concurrently >> > >> >[I almost missed this post because you did not reply to the thread.] >> > >> >* Robert Engels [191108 11:41]: >> >> See https://github.com/golang/go/issues/10958 for using atomics in >> >> tight/busy-wait loops. >> > >> >This doesn't have anything to do with atomics. Atomic operations are >> >just one of many, many things you can do in a busy loop. There is >> >nothing about atomics that is any more relevant to this issue than the >> >increment statement, i++. >> > >> >...Marvin > >Hmm. I am really confused. You sent the link I quoted in reply to this >message: > >>From: burak serdar >>Sent: Nov 8, 2019 9:19 AM >>To: golang-nuts >>Subject: Re: [go-nuts] What is the correct way to access/modify slice >>elements concurrently > >which has > >Message-ID: > > >What were you trying to say with your link to issue 10958? > >...Marvin > >-- >You received this message because you are subscribed to the Google Groups >"golang-nuts" group. >To unsubscribe from this group and stop receiving emails from it, send an >email to golang-nuts+unsubscr...@googlegroups.com. >To view this discussion on the web visit >https://groups.google.com/d/msgid/golang-nuts/20191112184159.2yep4f66wutu776l%40basil.wdw. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1756801388.6075.1573584225040%40wamui-abby.atl.sa.earthlink.net.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
[You keep sending replies that are not connected to the message to which you are replying. RFC 822 (and subsequent RFCs) defines the In-Reply-To header, which has been in use for more than 30 years. Most MUAs will add this automatically. Failure to add the header really messes up MUAs that display messages from the same thread together. Does your MUA not add this header???] * Robert Engels [191112 13:00]: > This is not the issue I am referring to, I am referring to > https://github.com/golang/go/issues/5045 > > -Original Message- > >From: Marvin Renich > >Sent: Nov 12, 2019 11:30 AM > >To: golang-nuts > >Subject: Re: [go-nuts] What is the correct way to access/modify slice > >elements concurrently > > > >[I almost missed this post because you did not reply to the thread.] > > > >* Robert Engels [191108 11:41]: > >> See https://github.com/golang/go/issues/10958 for using atomics in > >> tight/busy-wait loops. > > > >This doesn't have anything to do with atomics. Atomic operations are > >just one of many, many things you can do in a busy loop. There is > >nothing about atomics that is any more relevant to this issue than the > >increment statement, i++. > > > >...Marvin Hmm. I am really confused. You sent the link I quoted in reply to this message: >From: burak serdar >Sent: Nov 8, 2019 9:19 AM >To: golang-nuts >Subject: Re: [go-nuts] What is the correct way to access/modify slice elements >concurrently which has Message-ID: What were you trying to say with your link to issue 10958? ...Marvin -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191112184159.2yep4f66wutu776l%40basil.wdw.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
This is not the issue I am referring to, I am referring to https://github.com/golang/go/issues/5045 -Original Message- >From: Marvin Renich >Sent: Nov 12, 2019 11:30 AM >To: golang-nuts >Subject: Re: [go-nuts] What is the correct way to access/modify slice elements >concurrently > >[I almost missed this post because you did not reply to the thread.] > >* Robert Engels [191108 11:41]: >> See https://github.com/golang/go/issues/10958 for using atomics in >> tight/busy-wait loops. > >This doesn't have anything to do with atomics. Atomic operations are >just one of many, many things you can do in a busy loop. There is >nothing about atomics that is any more relevant to this issue than the >increment statement, i++. > >...Marvin > >-- >You received this message because you are subscribed to the Google Groups >"golang-nuts" group. >To unsubscribe from this group and stop receiving emails from it, send an >email to golang-nuts+unsubscr...@googlegroups.com. >To view this discussion on the web visit >https://groups.google.com/d/msgid/golang-nuts/20191112173043.6mfa7iejihhkasos%40basil.wdw. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/921647355.5105.1573581609714%40wamui-abby.atl.sa.earthlink.net.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
[I almost missed this post because you did not reply to the thread.] * Robert Engels [191108 11:41]: > See https://github.com/golang/go/issues/10958 for using atomics in > tight/busy-wait loops. This doesn't have anything to do with atomics. Atomic operations are just one of many, many things you can do in a busy loop. There is nothing about atomics that is any more relevant to this issue than the increment statement, i++. ...Marvin -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191112173043.6mfa7iejihhkasos%40basil.wdw.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
-Original Message- >From: burak serdar >Sent: Nov 8, 2019 9:19 AM >To: golang-nuts >Subject: Re: [go-nuts] What is the correct way to access/modify slice elements >concurrently > >On Fri, Nov 8, 2019 at 7:55 AM Marvin Renich wrote: >> >> * burak serdar [191108 08:42]: >> > I was thinking about this. In general, it should be safe to replace >> > >> > Lock() >> > read/write one value >> > Unlock() >> > >> > with >> > >> > AtomicRead/Write >> > >> > Is that correct? The go memory model does not say anything about this. >> >> Yes. >> >> While others have argued that the GMM is not specific enough about this >> case, I disagree. The atomic package says it is "useful for >> implementing synchronization algorithms". This wording is, in my >> opinion, sufficient to make two guarantees: >> >> 1. Values read and written with the atomic package will read and >> write whole values without corruption. >> 2. If a memory write from one goroutine is observed by another >> goroutine, this establishes a "happens before" relationship. >> >> If these are not true, than the documentation in the atomic package is a >> blatant lie. Thus, the memory model does not need any additional >> verbiage to clarify this. >> >> And while the next paragraph in the atomic documentation discourages >> using it for synchronization, I think the doc'n is overly pessimistic. >> Certainly for the simple case of avoiding races due to concurrent >> write/read of a single value, the atomic operations are a very good >> solution. If you can guarantee that all writes to a particular value >> are done on a single goroutine, and all reads and writes are done using >> the atomic package, than it is safe to have many other goroutines >> reading that value, and using a mutex is overkill. >> >> I do agree that channels and the sync package should be preferred over >> implementing something similar with atomic, but don't shy away from >> atomic for the simple cases that it does well. > > >That's what I figured as well. Atomic read/writes have memory >barriers, so this should work. Afaik, an atomic read/write does not >yield, so busy-waiting using an atomic will probably not work. Maybe >some of these guarantees (and non-guarantees) can be explicitly stated >in the docs. > > > >> >> ...Marvin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/20191108145435.3tdii3v3zgx3jv22%40basil.wdw. > >-- >You received this message because you are subscribed to the Google Groups >"golang-nuts" group. >To unsubscribe from this group and stop receiving emails from it, send an >email to golang-nuts+unsubscr...@googlegroups.com. >To view this discussion on the web visit >https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoyJMC3XXTBKdcAitKUyvyzbykvrfZPgQcLw5%3D%2BY%2B0%2BCQ%40mail.gmail.com. See https://github.com/golang/go/issues/10958 for using atomics in tight/busy-wait loops. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1843585103.1669.1573231278267%40wamui-cheeto.atl.sa.earthlink.net.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
On Fri, Nov 8, 2019 at 7:55 AM Marvin Renich wrote: > > * burak serdar [191108 08:42]: > > I was thinking about this. In general, it should be safe to replace > > > > Lock() > > read/write one value > > Unlock() > > > > with > > > > AtomicRead/Write > > > > Is that correct? The go memory model does not say anything about this. > > Yes. > > While others have argued that the GMM is not specific enough about this > case, I disagree. The atomic package says it is "useful for > implementing synchronization algorithms". This wording is, in my > opinion, sufficient to make two guarantees: > > 1. Values read and written with the atomic package will read and > write whole values without corruption. > 2. If a memory write from one goroutine is observed by another > goroutine, this establishes a "happens before" relationship. > > If these are not true, than the documentation in the atomic package is a > blatant lie. Thus, the memory model does not need any additional > verbiage to clarify this. > > And while the next paragraph in the atomic documentation discourages > using it for synchronization, I think the doc'n is overly pessimistic. > Certainly for the simple case of avoiding races due to concurrent > write/read of a single value, the atomic operations are a very good > solution. If you can guarantee that all writes to a particular value > are done on a single goroutine, and all reads and writes are done using > the atomic package, than it is safe to have many other goroutines > reading that value, and using a mutex is overkill. > > I do agree that channels and the sync package should be preferred over > implementing something similar with atomic, but don't shy away from > atomic for the simple cases that it does well. That's what I figured as well. Atomic read/writes have memory barriers, so this should work. Afaik, an atomic read/write does not yield, so busy-waiting using an atomic will probably not work. Maybe some of these guarantees (and non-guarantees) can be explicitly stated in the docs. > > ...Marvin > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/20191108145435.3tdii3v3zgx3jv22%40basil.wdw. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoyJMC3XXTBKdcAitKUyvyzbykvrfZPgQcLw5%3D%2BY%2B0%2BCQ%40mail.gmail.com.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
* burak serdar [191108 08:42]: > I was thinking about this. In general, it should be safe to replace > > Lock() > read/write one value > Unlock() > > with > > AtomicRead/Write > > Is that correct? The go memory model does not say anything about this. Yes. While others have argued that the GMM is not specific enough about this case, I disagree. The atomic package says it is "useful for implementing synchronization algorithms". This wording is, in my opinion, sufficient to make two guarantees: 1. Values read and written with the atomic package will read and write whole values without corruption. 2. If a memory write from one goroutine is observed by another goroutine, this establishes a "happens before" relationship. If these are not true, than the documentation in the atomic package is a blatant lie. Thus, the memory model does not need any additional verbiage to clarify this. And while the next paragraph in the atomic documentation discourages using it for synchronization, I think the doc'n is overly pessimistic. Certainly for the simple case of avoiding races due to concurrent write/read of a single value, the atomic operations are a very good solution. If you can guarantee that all writes to a particular value are done on a single goroutine, and all reads and writes are done using the atomic package, than it is safe to have many other goroutines reading that value, and using a mutex is overkill. I do agree that channels and the sync package should be preferred over implementing something similar with atomic, but don't shy away from atomic for the simple cases that it does well. ...Marvin -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191108145435.3tdii3v3zgx3jv22%40basil.wdw.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
Almost always, but it is very difficult as most times you are not really
working with a single value (there are implied dependencies). These sort of
solutions fall under “lock free” structures/algorithms - and they are
fascinating to learn about.
> On Nov 8, 2019, at 7:42 AM, burak serdar wrote:
>
>> On Fri, Nov 8, 2019 at 6:25 AM Marvin Renich wrote:
>>
>> * Kasun Vithanage [191107 23:47]:
>>> type Foo struct {
>>> Alive bool
>>> }
>>>
>>> type Bar struct {
>>> Foos []*Foo
>>> }
>>>
>>> func (b *Bar) CheckFoosAlive() {
>>> for i := 0; i < len(b.Foos); i++ {
>>>if b.Foos[i].Alive {
>>> fmt.Println("Alive")
>>>}
>>> }
>>> }
>>>
>>> func (b* Bar) MarkFooStatus(index int, status bool){
>>> // after bound checking
>>> b.Foos[index].Alive = status
>>> }
>>
>> Volker's answer is very good, but for the simple case where Alive is (or
>> can be) a type that is handled by the atomic package, that is almost
>> certainly the best approach.
>
> I was thinking about this. In general, it should be safe to replace
>
> Lock()
> read/write one value
> Unlock()
>
> with
>
> AtomicRead/Write
>
> Is that correct? The go memory model does not say anything about this.
>
>
>>
>> If the Foo struct is complicated, and you have lots of non-overlapping
>> access to b.Foos (with the occasional overlapping access), I strongly
>> suspect that putting a mutex in Foo and using that will give you the
>> best results.
>>
>> As Volker said, try several different approaches, and measure with loads
>> approximating your real-world scenario. Alternatively, implement the
>> approach that you think is easiest to maintain (from a source code POV),
>> and test to see if the performance is acceptable under load. If it is,
>> don't bother trying to optimize.
>>
>> ...Marvin
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/20191108132521.bkrbxj2lv7wpinuo%40basil.wdw.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoGKG8FAqxn-M_E9hN3%2BzqX1UABWmdHu9w6rneKCu%2BsuA%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/58E8A687-9E28-4F39-9EF2-B095D1D80265%40ix.netcom.com.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
On Fri, Nov 8, 2019 at 6:25 AM Marvin Renich wrote:
>
> * Kasun Vithanage [191107 23:47]:
> > type Foo struct {
> > Alive bool
> > }
> >
> > type Bar struct {
> > Foos []*Foo
> > }
> >
> > func (b *Bar) CheckFoosAlive() {
> > for i := 0; i < len(b.Foos); i++ {
> > if b.Foos[i].Alive {
> > fmt.Println("Alive")
> > }
> > }
> > }
> >
> > func (b* Bar) MarkFooStatus(index int, status bool){
> > // after bound checking
> > b.Foos[index].Alive = status
> > }
>
> Volker's answer is very good, but for the simple case where Alive is (or
> can be) a type that is handled by the atomic package, that is almost
> certainly the best approach.
I was thinking about this. In general, it should be safe to replace
Lock()
read/write one value
Unlock()
with
AtomicRead/Write
Is that correct? The go memory model does not say anything about this.
>
> If the Foo struct is complicated, and you have lots of non-overlapping
> access to b.Foos (with the occasional overlapping access), I strongly
> suspect that putting a mutex in Foo and using that will give you the
> best results.
>
> As Volker said, try several different approaches, and measure with loads
> approximating your real-world scenario. Alternatively, implement the
> approach that you think is easiest to maintain (from a source code POV),
> and test to see if the performance is acceptable under load. If it is,
> don't bother trying to optimize.
>
> ...Marvin
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/20191108132521.bkrbxj2lv7wpinuo%40basil.wdw.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoGKG8FAqxn-M_E9hN3%2BzqX1UABWmdHu9w6rneKCu%2BsuA%40mail.gmail.com.
Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
* Kasun Vithanage [191107 23:47]:
> type Foo struct {
> Alive bool
> }
>
> type Bar struct {
> Foos []*Foo
> }
>
> func (b *Bar) CheckFoosAlive() {
> for i := 0; i < len(b.Foos); i++ {
> if b.Foos[i].Alive {
> fmt.Println("Alive")
> }
> }
> }
>
> func (b* Bar) MarkFooStatus(index int, status bool){
> // after bound checking
> b.Foos[index].Alive = status
> }
Volker's answer is very good, but for the simple case where Alive is (or
can be) a type that is handled by the atomic package, that is almost
certainly the best approach.
If the Foo struct is complicated, and you have lots of non-overlapping
access to b.Foos (with the occasional overlapping access), I strongly
suspect that putting a mutex in Foo and using that will give you the
best results.
As Volker said, try several different approaches, and measure with loads
approximating your real-world scenario. Alternatively, implement the
approach that you think is easiest to maintain (from a source code POV),
and test to see if the performance is acceptable under load. If it is,
don't bother trying to optimize.
...Marvin
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/20191108132521.bkrbxj2lv7wpinuo%40basil.wdw.
[go-nuts] What is the correct way to access/modify slice elements concurrently
Assume we have a type like follows
type Foo struct {
Alive bool
}
type Bar struct {
Foos []*Foo
}
In two goroutines we are supposed to read and write the Alive variable. For
example
func (b *Bar) CheckFoosAlive() {
for i := 0; i < len(b.Foos); i++ {
if b.Foos[i].Alive {
fmt.Println("Alive")
}
}
}
func (b* Bar) MarkFooStatus(index int, status bool){
// after bound checking
b.Foos[index].Alive = status
}
We would run both concurrently.
As we can clearly see putting a mutex for the entire Bar struct is seems a
bit overhead for me. For example, if CheckFoosAlive is blocking and takes a
lot of time, we don't want to lock until it's completed.
The whole idea is to reduce locking as much as possible to get access to an
underlying element.
What is the best approach? (I replaced bools with uint32 and used atomic
ops as for the current solution)
Would like to know your opinions
Regards,
Kasun
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/5cfe5a3c-40a5-4cd6-a6ed-f5de2730b767%40googlegroups.com.
