Hi,

I’m writing a http poller that send concurrent HTTP requests to
retrieve the monitoring data, put the results into a queue then
consume them by several workers that send HTTP requests to our log
aggregators.

At the beginning everything works well, about 9 hours later the
program crashes and leaves the following messages:

The weird thing I don’t understand is that there is no panic or system
level messages.

Exception 0xc0000005 0x0 0xffffffffffffffff 0x7ffbea7bc8eb
PC=0x7ffbea7bc8ebx

syscall.Syscall6(0x7ffbea7b8a80, 0x4, 0x4, 0x1138ad0, 0xc04223d3d0,
0xc04223d3e0, 0x0, 0x0, 0xc0422f1a00, 0x300000002, ...)
        C:/Go/src/runtime/syscall_windows.go:174 +0x6b
syscall.CertVerifyCertificateChainPolicy(0x4, 0x1138ad0, 0xc04223d3d0,
0xc04223d3e0, 0x0, 0xc04249d3d8)
        C:/Go/src/syscall/zsyscall_windows.go:1208 +0xc1
crypto/x509.checkChainSSLServerPolicy(0xc04238d200, 0x1138ad0,
0xc04223d808, 0x38e8f10, 0xc04249d548)
        C:/Go/src/crypto/x509/root_windows.go:117 +0xfd
crypto/x509.(*Certificate).systemVerify(0xc04238d200, 0xc04249d808,
0x0, 0x0, 0x0, 0x0, 0x0)
        C:/Go/src/crypto/x509/root_windows.go:212 +0x484
crypto/x509.(*Certificate).Verify(0xc04238d200, 0xc042448940, 0x13,
0xc042251500, 0x0, 0xed0c3b03e, 0x25245214, 0xbb0b80, 0x0, 0x0, ...)
        C:/Go/src/crypto/x509/verify.go:279 +0x86c
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc04223de50,
0xc042203380, 0x51)
        C:/Go/src/crypto/tls/handshake_client.go:300 +0x4c0
crypto/tls.(*Conn).clientHandshake(0xc0422ef500, 0x969ae0, 0xc0422ef620)
        C:/Go/src/crypto/tls/handshake_client.go:228 +0xf97
crypto/tls.(*Conn).Handshake(0xc0422ef500, 0x0, 0x0)
        C:/Go/src/crypto/tls/conn.go:1307 +0x1aa
net/http.(*Transport).dialConn.func3(0x0, 0xc0422ef500, 0xc04200e9c0,
0xc0424201e0)
        C:/Go/src/net/http/transport.go:1082 +0x49
created by net/http.(*Transport).dialConn
        C:/Go/src/net/http/transport.go:1087 +0xff3
...

This is the main program, blocked by channel selecting, waiting for
signals. (SIGINT, SIGTERM and SIGQUIT)

goroutine 1 [chan receive, 732 minutes]:
cheyi/myprog/lib/poller.main(0xc042079b00, 0xc042146940, 0x0, 0x4)
        D:/Golang/src/cheyi/myprog/lib/poller/entry.go:52 +0x17c
cheyi/myprog/vendor/github.com/spf13/cobra.(*Command).execute(0xc042079b00,
0xc042146900, 0x4, 0x4, 0xc042079b00, 0xc042146900)
        D:/Golang/src/cheyi/myprog/vendor/github.com/spf13/cobra/command.go:648
+0x23b
cheyi/myprog/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xba9fc0,
0xc04201c0b8, 0x0, 0x1)
        D:/Golang/src/cheyi/myprog/vendor/github.com/spf13/cobra/command.go:734
+0x340
cheyi/myprog/vendor/github.com/spf13/cobra.(*Command).Execute(0xba9fc0,
0x850bc3, 0x946c4b)
        D:/Golang/src/cheyi/myprog/vendor/github.com/spf13/cobra/command.go:693
+0x32
cheyi/myprog/cmd.Execute()
        D:/Golang/src/cheyi/myprog/cmd/entry.go:28 +0x38
main.main()
        D:/Golang/src/cheyi/myprog/main.go:11 +0x27

goroutine 5 [syscall, 732 minutes]:
os/signal.signal_recv(0x0)
        C:/Go/src/runtime/sigqueue.go:116 +0x10b
os/signal.loop()
        C:/Go/src/os/signal/signal_unix.go:22 +0x29
created by os/signal.init.1
        C:/Go/src/os/signal/signal_unix.go:28 +0x48
...

After the backtrace of poller and output worker goroutines are the
backtrace of http related goroutines

goroutine 58972 [IO wait]:
net.runtime_pollWait(0x32b0780, 0x72, 0xb74500)
        C:/Go/src/runtime/netpoll.go:164 +0x60
net.(*pollDesc).wait(0xc042408a60, 0x72, 0xb74500, 0x0)
        C:/Go/src/net/fd_poll_runtime.go:75 +0x3f
net.(*ioSrv).ExecIO(0xc0421ea008, 0xc042408920, 0x949b9c, 0x7,
0x9692d0, 0x1, 0x0, 0xc04295d820)
        C:/Go/src/net/fd_windows.go:196 +0xfe
net.(*netFD).Read(0xc0424088c0, 0xc04249e000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
        C:/Go/src/net/fd_windows.go:452 +0x13d
net.(*conn).Read(0xc042232158, 0xc04249e000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
        C:/Go/src/net/net.go:181 +0x77
crypto/tls.(*block).readFromUntil(0xc04219c6c0, 0x32b0fc0,
0xc042232158, 0x5, 0xc042232158, 0x4200da00)
        C:/Go/src/crypto/tls/conn.go:488 +0x9f
crypto/tls.(*Conn).readRecord(0xc0423a4000, 0xc04295db14, 0x52dd0e,
0xc0424088c0)
        C:/Go/src/crypto/tls/conn.go:590 +0xcb
crypto/tls.(*clientHandshakeState).readFinished(0xc04295de50,
0xc0423a40f5, 0xc, 0xc, 0x0, 0x0)
        C:/Go/src/crypto/tls/handshake_client.go:584 +0x5a
crypto/tls.(*Conn).clientHandshake(0xc0423a4000, 0x969ae0, 0xc0423a4120)
        C:/Go/src/crypto/tls/handshake_client.go:244 +0x10a6
crypto/tls.(*Conn).Handshake(0xc0423a4000, 0x0, 0x0)
        C:/Go/src/crypto/tls/conn.go:1307 +0x1aa
net/http.(*Transport).dialConn.func3(0x0, 0xc0423a4000, 0xc042146e80,
0xc04218ee40)
        C:/Go/src/net/http/transport.go:1082 +0x49
created by net/http.(*Transport).dialConn
        C:/Go/src/net/http/transport.go:1087 +0xff3

goroutine 58945 [select]:
net/http.setRequestCancel.func3(0x0, 0xc0421b87b0, 0xc04200e8c0,
0xc0421b689c, 0xc0421b59e0)
        C:/Go/src/net/http/client.go:320 +0x183
created by net/http.setRequestCancel
        C:/Go/src/net/http/client.go:330 +0x28e
...

The crash messages end with register addresses/values:

rax     0x200f607ca72f723b
rbx     0x0
rcx     0x0
rdi     0x40
rsi     0x0
rbp     0x2e5fd59
rsp     0x2e5fcc0
r8      0x2e5fd40
r9      0x2e5fd18
r10     0x1
r11     0x246
r12     0x0
r13     0x0
r14     0x1
r15     0xc04249d408
rip     0x7ffbea7bc8eb
rflags  0x10256
cs      0x33
fs      0x53
gs      0x2b

The go version is 1.8.3 windows/amd64 and the program runs on a modern
Win10 laptop with i7 CPU and 16GB memory.

Thanks for your help!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to