[go-nuts] Re: All goroutines are parked during bgsweep() running. Is it a issue?

[nealhoo]

Hi,

Please using `xz -d bgsweep.trace.xz 
`
 decompress 
this file before running `go tool trace bgsweep.trace`.


 
在 2019年8月16日星期五 UTC+8下午5:19:03，gerald...@gmail.com写道：
>
> How about uploading the svg, so people can zoom into?
>
> On Wednesday, August 14, 2019 at 11:34:58 AM UTC-4, nea...@gmail.com 
> wrote:
>>
>> [image: issue.png]
>>
>>
>> 在 2019年8月14日星期三 UTC+8下午7:35:48，neal hu写道：
>>>
>>> Hi All
>>>
>>> I found my application has a strange issue by go trace.  All goroutines 
>>> are parked during bgsweep() running.  Is it a issue?
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f1b38ff4-4409-441e-97da-a55169b82070%40googlegroups.com.

[go-nuts] All goroutines are parked during bgsweep() running. Is it a issue?

[nealhoo]

Hi All

I found my application has a strange issue by go trace.  All goroutines are 
parked during bgsweep() running.  Is it a issue?


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b858cd05-9f3a-4fae-a9e4-8e6c3f030309%40googlegroups.com.

[go-nuts] Re: Does runtime.RaceDisable not work in non-std library ?

[nealhoo]

According 
to 
https://github.com/golang/go/blob/7a6ccece877232fc58872c85fdf0718629b83d91/src/cmd/compile/internal/gc/racewalk.go#L45-L47
 
, the compile does not insert data race detecting code in package "sync".

在 2018年8月22日星期三 UTC+8下午6:25:24，nea...@gmail.com写道：
>
> Hi,
>
> When I copy the sync.Pool's source code in a repo, and using `
> //go:linkname` link those runtime functions manually.
> When I run `go test -race`, it reports `DATA RACE`.
> But the sync.Pool with the same test case does not report  `DATA RACE`.
>
> Does runtime.RaceDisable not work in non-std library ?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Does runtime.RaceDisable not work in non-std library ?

[nealhoo]

Thanks a lot.

在 2018年8月25日星期六 UTC+8上午1:32:12，Ian Lance Taylor写道：
>
> On Fri, Aug 24, 2018 at 12:28 AM,  > 
> wrote: 
> > 
> >> go test -v -race -count=5 . 
> > === RUN   TestSyncAPI 
> > == 
> > WARNING: DATA RACE 
> > Read at 0x00c4200ca080 by goroutine 8: 
> >   _/Users/yanqing11/test/go/sync1.(*Pool).Put() 
> >   /Users/yanqing11/test/go/sync1/pool.go:108 +0xe8 
> >   _/Users/yanqing11/test/go/sync1.TestSyncAPI.func2() 
> >   /Users/yanqing11/test/go/sync1/pool_test.go:18 +0x50 
> > 
> > Previous write at 0x00c4200ca080 by goroutine 7: 
> >   _/Users/yanqing11/test/go/sync1.(*Pool).Put() 
> >   /Users/yanqing11/test/go/sync1/pool.go:109 +0x2ad 
> >   _/Users/yanqing11/test/go/sync1.TestSyncAPI.func1() 
> >   /Users/yanqing11/test/go/sync1/pool_test.go:14 +0x5e 
>
> These race reports are data races on memory accesses, and as such are 
> not affect by runtime.RaceDisable. 
>
> I don't know what the problem is here but I don't think it's related 
> to RaceDisable. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Does runtime.RaceDisable not work in non-std library ?

[nealhoo]

Hi Dave,

1. the report:

> go version
go version go1.10 darwin/amd64

> go test -v -race -count=5 .
=== RUN   TestSyncAPI
==
WARNING: DATA RACE
Read at 0x00c4200ca080 by goroutine 8:
  _/Users/yanqing11/test/go/sync1.(*Pool).Put()
  /Users/yanqing11/test/go/sync1/pool.go:108 +0xe8
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func2()
  /Users/yanqing11/test/go/sync1/pool_test.go:18 +0x50

Previous write at 0x00c4200ca080 by goroutine 7:
  _/Users/yanqing11/test/go/sync1.(*Pool).Put()
  /Users/yanqing11/test/go/sync1/pool.go:109 +0x2ad
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func1()
  /Users/yanqing11/test/go/sync1/pool_test.go:14 +0x5e

Goroutine 8 (running) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:17 +0x129
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d

Goroutine 7 (finished) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:12 +0xfd
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d
==
==
WARNING: DATA RACE
Write at 0x00c4200ca090 by goroutine 8:
  _/Users/yanqing11/test/go/sync1.(*Pool).Put()
  /Users/yanqing11/test/go/sync1/pool.go:115 +0x206
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func2()
  /Users/yanqing11/test/go/sync1/pool_test.go:18 +0x50

Previous read at 0x00c4200ca090 by goroutine 7:
  _/Users/yanqing11/test/go/sync1.(*Pool).Get()
  /Users/yanqing11/test/go/sync1/pool.go:141 +0x1ee
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func1()
  /Users/yanqing11/test/go/sync1/pool_test.go:13 +0x38

Goroutine 8 (running) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:17 +0x129
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d

Goroutine 7 (finished) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:12 +0xfd
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d
==
==
WARNING: DATA RACE
Read at 0x00c4200ce000 by goroutine 9:
  _/Users/yanqing11/test/go/sync1.(*Pool).getSlow()
  /Users/yanqing11/test/go/sync1/pool.go:175 +0x17a
  _/Users/yanqing11/test/go/sync1.(*Pool).Get()
  /Users/yanqing11/test/go/sync1/pool.go:148 +0x243
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func3()
  /Users/yanqing11/test/go/sync1/pool_test.go:23 +0x38

Previous write at 0x00c4200ce000 by goroutine 8:
  _/Users/yanqing11/test/go/sync1.(*Pool).Put()
  /Users/yanqing11/test/go/sync1/pool.go:115 +0x1c6
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func2()
  /Users/yanqing11/test/go/sync1/pool_test.go:18 +0x50

Goroutine 9 (running) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:22 +0x155
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d

Goroutine 8 (finished) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:17 +0x129
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d
==
--- FAIL: TestSyncAPI (0.00s)
testing.go:730: race detected during execution of test
=== RUN   TestSyncAPI2
--- PASS: TestSyncAPI2 (0.00s)
=== RUN   TestSyncAPI
--- PASS: TestSyncAPI (0.00s)
=== RUN   TestSyncAPI2
--- PASS: TestSyncAPI2 (0.00s)
=== RUN   TestSyncAPI
--- PASS: TestSyncAPI (0.00s)
=== RUN   TestSyncAPI2
--- PASS: TestSyncAPI2 (0.00s)
=== RUN   TestSyncAPI
==
WARNING: DATA RACE
Read at 0x00c42010a380 by goroutine 36:
  _/Users/yanqing11/test/go/sync1.(*Pool).Get()
  /Users/yanqing11/test/go/sync1/pool.go:136 +0x5c
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func3()
  /Users/yanqing11/test/go/sync1/pool_test.go:23 +0x38

Previous write at 0x00c42010a380 by goroutine 34:
  _/Users/yanqing11/test/go/sync1.(*Pool).Put()
  /Users/yanqing11/test/go/sync1/pool.go:109 +0x2ad
  _/Users/yanqing11/test/go/sync1.TestSyncAPI.func1()
  /Users/yanqing11/test/go/sync1/pool_test.go:14 +0x5e

Goroutine 36 (running) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:22 +0x155
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d

Goroutine 34 (finished) created at:
  _/Users/yanqing11/test/go/sync1.TestSyncAPI()
  /Users/yanqing11/test/go/sync1/pool_test.go:12 +0xfd
  testing.tRunner()
  /Users/yanqing11/dev/go/src/testing/testing.go:777 +0x16d
==
--- FAIL: TestSyncAPI (0.00s)
testing.go:730: race detected during execution of test
=== RUN   TestSyncAPI2
--- PASS: TestSyncAPI2 (0.00s)
=== RUN   TestSyncAPI
--- PASS: TestSyncAPI (0.00s)
=== RUN   TestSyncAPI2
--- PASS: TestSyncAPI2 (0.00s)
FAIL
FAIL 

Re: [go-nuts] Does runtime.RaceDisable not work in non-std library ?

[nealhoo]

Hi lan,

The sync.Pool has memory accesses in per-p private storage. Why it not 
reports `DATA RACE`, but it reports when I copy those code outside stdlib?
I'm confused about this, or there is something wrong in 
https://gist.github.com/lrita/efa8c4ae555b4b7cceee29b4ed819652 
Thanks.

在 2018年8月24日星期五 UTC+8上午4:36:06，Ian Lance Taylor写道：
>
> On Thu, Aug 23, 2018 at 12:43 AM,  > 
> wrote: 
> > package main 
> > 
> > import "runtime" 
> > 
> > var a int 
> > 
> > func calc() { 
> > for i := 0; i < 10; i++ { 
> > go func() { 
> > for { 
> > runtime.RaceDisable() 
> > a++ 
> > runtime.RaceEnable() 
> > } 
> > }() 
> > 
> > } 
> > } 
> > 
> > func main() { 
> > calc() 
> > } 
> > 
> > go run -race a.go 
>
> Thanks for the example.  As the docs for runtime.RaceDisable say, it 
> only applies to synchronization events, like mutex locks.  It doesn't 
> apply to memory accesses. 
>
> Ian 
>
>
> > 在 2018年8月22日星期三 UTC+8下午10:34:35，Ian Lance Taylor写道： 
> >> 
> >> On Wed, Aug 22, 2018 at 3:25 AM,   wrote: 
> >> > 
> >> > When I copy the sync.Pool's source code in a repo, and using 
> >> > `//go:linkname` 
> >> > link those runtime functions manually. 
> >> > When I run `go test -race`, it reports `DATA RACE`. 
> >> > But the sync.Pool with the same test case does not report  `DATA 
> RACE`. 
> >> > 
> >> > Does runtime.RaceDisable not work in non-std library ? 
> >> 
> >> It should work.  I think you'll have to show us your code. 
> >> 
> >> Ian 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Does runtime.RaceDisable not work in non-std library ?

[nealhoo]

Thanks for your attention.
Here is the code 
: https://gist.github.com/lrita/efa8c4ae555b4b7cceee29b4ed819652
When I run `go test -v -race -count=10 .`  `TestSyncAPI` reports DATA RACE, 
`TestSyncAPI2` not reports. 

在 2018年8月22日星期三 UTC+8下午10:34:35，Ian Lance Taylor写道：
>
> On Wed, Aug 22, 2018 at 3:25 AM,  > wrote: 
> > 
> > When I copy the sync.Pool's source code in a repo, and using 
> `//go:linkname` 
> > link those runtime functions manually. 
> > When I run `go test -race`, it reports `DATA RACE`. 
> > But the sync.Pool with the same test case does not report  `DATA RACE`. 
> > 
> > Does runtime.RaceDisable not work in non-std library ? 
>
> It should work.  I think you'll have to show us your code. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Does runtime.RaceDisable not work in non-std library ?

[nealhoo]

Hi,

When I copy the sync.Pool's source code in a repo, and using `//go:linkname` 
link those runtime functions manually.
When I run `go test -race`, it reports `DATA RACE`.
But the sync.Pool with the same test case does not report  `DATA RACE`.

Does runtime.RaceDisable not work in non-std library ?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Does sync.(*Pool).Put slow to cause latency spikes sometimes?

[nealhoo]

Yes, there is only pointers in sync.Pool.

I think the sync.Pool only slow during GCing is a normal behavior. If the 
sync.Pool's method slow with out GCing is a runtime issue.

I donot agree:

sync.Pool just amortizes the cost of creating garbage


The sync.Pool is reusing object and reducing creating garbage during each 
gc.

在 2018年8月13日星期一 UTC+8下午1:59:02，Tamás Gulácsi写道：
>
> sync.Pool just amortizes the cost of creating garbage, does not eliminiate 
> it...
> You have to pay for your garbage somewhere!
>
> Are you sure you put only pointers into yout Pool?
>
> 2018. augusztus 13., hétfő 6:57:48 UTC+2 időpontban nea...@gmail.com a 
> következőt írta:
>>
>> Hi,
>>
>> My services show some latency spikes > 100ms sometimes. Then I use go 
>> trace to investigate this issue. And I found that sync.(*Pool).Put maybe 
>> cause the latency spikes.
>>
>> Is there someone encounter this issue also?
>> Is it a runtime issue?
>>
>> [image: sync-pool.png]
>>
>>
>>
>>
>> go version: go version go1.10 darwin/amd64 compile to linux amd64
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Does golang support SystemTap well?

[nealhoo]

Thanks your reply.

You say "Systemtap uses the literal Go symbol names as C identifiers".
But SystemTap 2.8 release say it support golang. It does do any adapting 
work for golang application? I cannot find more information for this.
Do you know any up to date information for this?

在 2017年9月19日星期二 UTC+8下午11:41:57，Aram Hăvărneanu写道：
>
> > I don't know why Go programs would not work with systemtap. 
>
> Because when tracing function call returns systemtap installs 
> trampolines in the return address slots. This confuses traceback, 
> which doesn't expect systemtap functions in the stack trace. It 
> also breaks stack frame copy for the same reason. 
>
> I don't know why the original SIGILL comes from exactly, but I have 
> experienced it also, among many other problems. Systemtap doesn't 
> work well with Go. In fact, a few years back it didn't work at all. 
> Systemtap uses the literal Go symbol names as C identifiers in the 
> generated C kernel driver (whtout mangling), which then didn't even 
> compile. 
>
> -- 
> Aram Hăvărneanu 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Does golang support SystemTap well?

[nealhoo]

Thanks your reply.

As I known, `pprof` only can tell your which function is busy on cpu, 
memory allocated/in-use and so on.
But It cannot tell you a function latency/elapsed, how long we wait on a 
mutex.
Is there any plan to add more feature in pprof ?

在 2017年9月19日星期二 UTC+8下午9:27:31，Ian Lance Taylor写道：
>
> On Tue, Sep 19, 2017 at 1:50 AM,   wrote: 
> > ENV : 
> >  SystemTap version : 3.1/0.160 
> >  Linux kernal : 3.10.0-229.el7.x86_64 
> >  Golang : 1.8.3 
> > 
> > Recently, I use SystemTap to do some tracing for a golang application. 
> But 
> > it occurred many fatal error, e.g: 
> > 
> > SIGILL: illegal instruction 
> > PC=0x7fffe015 m=41 sigcode=2 
> > 
> > ... 
> > 
> > SIGTRAP: trace trap 
> > PC=0x7fffe001 m=45 sigcode=0 
> > 
> >  
> > 
> > 
> > fatal error: unknown caller pc 
> >  
> > 
> > Is the PC  corrupted by SystemTap? 
> > Or SystemTap make some conflict with go runtime? 
> > The SystemTap 2.8 release say it support golang. Is it right? 
> > https://sourceware.org/ml/systemtap/2015-q2/msg00160.html 
> > 
> > I search some topic related to this here, but those topics is 
> obsoleted.Is 
> > there any newly update? 
>
> In Go 1.9 the Go compiler uses frame pointers by default.  That should 
> at least give you better tracebacks. 
>
> I don't know where those signals are coming from, though.  I don't 
> know why Go programs would not work with systemtap. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Does golang support SystemTap well?

[nealhoo]

ENV :
 SystemTap version : 3.1/0.160
 Linux kernal : 3.10.0-229.el7.x86_64
 Golang : 1.8.3

Recently, I use SystemTap to do some tracing for a golang application. But 
it occurred many fatal error, e.g:

SIGILL: illegal instruction
PC=0x7fffe015 m=41 sigcode=2

...

SIGTRAP: trace trap
PC=0x7fffe001 m=45 sigcode=0




fatal error: unknown caller pc


Is the PC  corrupted by SystemTap? 
Or SystemTap make some conflict with go runtime?
The SystemTap 2.8 release say it support golang. Is it 
right? https://sourceware.org/ml/systemtap/2015-q2/msg00160.html

I search some topic related to this here, but those topics is obsoleted.Is 
there any newly update?

Thanks.

 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.