Re: [go-nuts] Re: who does go compiler link executable to glibc without an apparent reason?
Thanks Ian. It came as a pleasant surprise that using statically compiled executable reduced(!) runtime memory usage by a noticeable portion. And gladly, till now the components are behaving well. Kind regards, Howard On Tuesday, 7 November 2017 16:23:17 UTC+1, Ian Lance Taylor wrote: > > On Tue, Nov 7, 2017 at 1:28 AM, Howard Guo <guoh...@gmail.com > > wrote: > > Just noticed this statement from src/net/net.go: > > > > -- > > On Unix systems, the resolver has two options for resolving names. > > It can use a pure Go resolver that sends DNS requests directly to the > > servers > > listed in /etc/resolv.conf, or it can use a cgo-based resolver that > calls C > > library routines such as getaddrinfo and getnameinfo. > > - > > > > However, using its recommended method to force usage of Go resolver > still > > results in a dynamic binary: > > > > export GODEBUG=netdns=go > > Setting the GODEBUG environment variable changes the behavior at run > time. It does not affect how the program is built. > > As you've seen setting CGO_ENABLED=0 is a quick way to ensure that you > get a static binary with no C code. But this does mean that the > os/user package, and less frequently the net package, will not behave > correctly in certain environments. If you are running in a Docker > container or otherwise constrained environment this does not matter. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: who does go compiler link executable to glibc without an apparent reason?
Thanks for the info! Kind regards, Howard On Tuesday, 7 November 2017 12:15:46 UTC+1, Dave Cheney wrote: > > Until recently some functions in the os/user package were only available > via glibc. > > The net package links to cgo by default to pick up the nss libraries, > which are important if your installation uses ldap. > > I argued that this should not be the default in the 1.5 release, that most > users deploying on Linux expected a static binary by default, however my > arguments were not sufficiently convincing. > > The tldr is rationale for the linking behaviour of a go binary is > implantation defined, and the go authors choose to make their > implementation use cgo where available. > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: who does go compiler link executable to glibc without an apparent reason?
Do you by any chance know what the rationale would be? Kind regards, Howard On Tuesday, 7 November 2017 11:55:22 UTC+1, Dave Cheney wrote: > > The net and os/user packages will force dynamic linking unless > CGO_ENABLED=0 or you are cross compiling. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: who does go compiler link executable to glibc without an apparent reason?
Thanks very much Karan, that's indeed a good approach for this challenge.
Here is the output from a dynamically linked executable:
abort
__errno_location
fprintf
fputc
free
freeaddrinfo
fwrite
gai_strerror
getaddrinfo
getnameinfo
malloc
mmap
munmap
nanosleep
pthread_attr_destroy
pthread_attr_getstacksize
pthread_attr_init
pthread_cond_broadcast
pthread_cond_wait
pthread_create
pthread_mutex_lock
pthread_mutex_unlock
pthread_sigmask
setenv
sigaction
sigaddset
sigemptyset
sigfillset
sigismember
stderr
strerror
unsetenv
vfprintf
Now this initiative of go compiler producing a dynamic executable looks
quite redundant.
I shall find some time to reduce this go project, file an issue, and see if
go developers can share more insights.
Kind regards,
Howard
On Tuesday, 7 November 2017 11:21:26 UTC+1, Karan Chaudhary wrote:
>
> With a quick look, it seems "net" is not the only pkg which depends on
> cgo, though by default it uses pure Go DNS resolver only.
>
> I tested on small program:
> package main
>
> import (
> "fmt"
> "net/http"
> )
>
> func main() {
>
> fmt.Println("Hello, playground")
> http.ListenAndServe(":", nil)
>
> }
>
> Which is dynamically linked if CGO_ENABLED=1 is set, and statically
> linked if CGO_ENABLED=0 is set, irrespective of GODEBUG=netdns=go .
> Diffing the cgo symbols from ELFs from the 2 cases shows that enabling cgo
> adds 61 more functions:
> _cgo_b0c710f30cfd_C2func_
> _cgo_b0c710f30cfd_C2func_
> _cgo_b0c710f30cfd_Cfunc__
> _cgo_b0c710f30cfd_Cfunc_f
> _cgo_b0c710f30cfd_Cfunc_f
> _cgo_b0c710f30cfd_Cfunc_g
> _cgo_b0c710f30cfd_Cfunc_g
> _cgo_b0c710f30cfd_Cfunc_g
> cgo_context_function
> _cgo_get_context_function
> _cgo_panic
> _cgo_panic
> _cgo_release_context
> _cgo_sys_thread_start
> _cgo_topofstack
> _cgo_topofstack
> _cgo_try_pthread_create
> _cgo_wait_runtime_init_do
> net._cgo_b0c710f30cfd_C2f
> net._cgo_b0c710f30cfd_Cfu
> net._cgo_b0c710f30cfd_Cfu
> net._cgo_b0c710f30cfd_Cfu
> net._cgo_b0c710f30cfd_Cfu
> net._cgo_cmalloc
> net.cgoIPLookup
> net.cgoLookupIP
> net.cgoLookupIPCNAME
> net.cgoLookupIPCNAME.func
> net.cgoLookupIPCNAME.func
> net.cgoLookupIPCNAME.func
> net.cgoLookupPort
> net.cgoLookupServicePort
> net.cgoLookupServicePort.
> net.cgoLookupServicePort.
> net.cgoLookupServicePort.
> net.cgoPortLookup
> runtime.cgoAlwaysFalse
> runtime/cgo(.bss)
> runtime.cgocallback
> runtime.cgoCheckArg
> runtime.cgoCheckPointer
> runtime.cgoCheckUnknownPo
> runtime._cgo_panic_intern
> runtime/cgo(.rodata)
> runtime/cgo(.rodata.str1.
> runtime/cgo(.rodata.str1.
> runtime/cgo(.text)
> runtime.cgoUse
> x_cgo_callers
> x_cgo_init
> x_cgo_inittls
> x_cgo_mmap
> x_cgo_munmap
> x_cgo_notify_runtime_init
> x_cgo_set_context_functio
> x_cgo_setenv
> x_cgo_sigaction
> x_cgo_sys_thread_create
> x_cgo_threadentry
> x_cgo_thread_start
> x_cgo_unsetenv
>
> Here we can see "net" is not the only pkg that uses CGO, but runtime uses
> it as well.
> These can be explored to understand more.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: who does go compiler link executable to glibc without an apparent reason?
Just noticed this statement from src/net/net.go: -- On Unix systems, the resolver has two options for resolving names. It can use a pure Go resolver that sends DNS requests directly to the servers listed in /etc/resolv.conf, or it can use a cgo-based resolver that calls C library routines such as getaddrinfo and getnameinfo. - However, using its recommended method to force usage of Go resolver still results in a dynamic binary: export GODEBUG=netdns=go Welp, have to dig deeper. On Sunday, 5 November 2017 17:12:52 UTC+1, Howard Guo wrote: > > Hello fellow gophers. > > I've a small number of go programs that only use standard library > functions without any 3rd party library dependency. A strange behaviour was > observed during compilation (go build) - most of them compile into static > executable, but few of them end up linking to glibc. Without an exception, > all of them compile into static executable when CGO_ENABLED=0 is set prior > to compilation, and their runtime behaviour doesn't change at all. > > What could be the obscure reasons for go compiler to link executable to > glibc? > > Thanks. > > Kind regards, > Howard > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] who does go compiler link executable to glibc without an apparent reason?
Hello fellow gophers. I've a small number of go programs that only use standard library functions without any 3rd party library dependency. A strange behaviour was observed during compilation (go build) - most of them compile into static executable, but few of them end up linking to glibc. Without an exception, all of them compile into static executable when CGO_ENABLED=0 is set prior to compilation, and their runtime behaviour doesn't change at all. What could be the obscure reasons for go compiler to link executable to glibc? Thanks. Kind regards, Howard -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Why does this program carry test flags once compiled?
Many thanks Chris, that's a wonderful idea and I've successfully got rid of
the test mode flags!
On Tuesday, 25 July 2017 05:47:30 UTC+2, Chris Broadfoot wrote:
>
> One way around the flag problem is to accept an interface, instead of
> using testing.T.
>
> Watch Mitchell Hashimoto's talk from GopherCon from about 39m30s:
> https://youtu.be/8hQG7QlcLBk?t=2368
>
> http://github.com/mitchellh/go-testing-interface
>
> On Mon, Jul 24, 2017 at 1:44 AM, roger peppe <rogp...@gmail.com
> > wrote:
>
>>
>>
>> On 24 July 2017 at 09:12, Howard Guo <guoh...@gmail.com >
>> wrote:
>>
>>> Thanks Nigel! I should definitely consider improving the test case.
>>>
>>> Go compiler appears to automatically places test flags into executable
>>> whenever "testing" package is included - no matter the function using
>>> "testing" package is dead code or otherwise. So here's a bug report for go:
>>> https://github.com/golang/go/issues/21141
>>>
>>
>> Although this could be considered a bug, it's not one that can be fixed,
>> because people rely on this behaviour of the testing package.
>>
>> Consider it a warning not to include testing dependencies in production
>> code.
>>
>>
>>>
>>> Kind regards,
>>> Howard
>>>
>>> On Sunday, 23 July 2017 06:54:28 UTC+2, Nigel Tao wrote:
>>>>
>>>> On Sat, Jul 22, 2017 at 7:08 PM, Howard Guo <guoh...@gmail.com> wrote:
>>>> > https://github.com/HouzuoGuo/laitos
>>>>
>>>> It's tangential to your question, but out of curiousity, I skimmed the
>>>> source code. In things like func TestUDPServer in
>>>> https://github.com/HouzuoGuo/laitos/blob/master/frontend/plain/udp.go,
>>>> you say:
>>>>
>>>> var stoppedNormally bool
>>>> go func() {
>>>> etc
>>>> stoppedNormally = true
>>>> }()
>>>> etc
>>>> server.Stop()
>>>> time.Sleep(1 * time.Second)
>>>> if !stoppedNormally { etc }
>>>>
>>>> This is an example of incorrect synchronization as per
>>>> https://golang.org/ref/mem#tmp_10
>>>>
>>>> There may be other such races in your code. I didn't do an exhaustive
>>>> review. It may be worthwhile to run your test suite under the race
>>>> detector.
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] Why does this program carry test flags once compiled?
Thanks Nigel! I should definitely consider improving the test case.
Go compiler appears to automatically places test flags into executable
whenever "testing" package is included - no matter the function using
"testing" package is dead code or otherwise. So here's a bug report for go:
https://github.com/golang/go/issues/21141
Kind regards,
Howard
On Sunday, 23 July 2017 06:54:28 UTC+2, Nigel Tao wrote:
>
> On Sat, Jul 22, 2017 at 7:08 PM, Howard Guo <guoh...@gmail.com
> > wrote:
> > https://github.com/HouzuoGuo/laitos
>
> It's tangential to your question, but out of curiousity, I skimmed the
> source code. In things like func TestUDPServer in
> https://github.com/HouzuoGuo/laitos/blob/master/frontend/plain/udp.go,
> you say:
>
> var stoppedNormally bool
> go func() {
> etc
> stoppedNormally = true
> }()
> etc
> server.Stop()
> time.Sleep(1 * time.Second)
> if !stoppedNormally { etc }
>
> This is an example of incorrect synchronization as per
> https://golang.org/ref/mem#tmp_10
>
> There may be other such races in your code. I didn't do an exhaustive
> review. It may be worthwhile to run your test suite under the race
> detector.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] Why does this program carry test flags once compiled?
Good day fellow gophers. Can anyone please help me to understand, why the command line help refers to test flags (e.g. test.bench) once this program is compiled? https://github.com/HouzuoGuo/laitos Thanks & kind regards, Howard -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Can ServerMux serve two different handler functions on / and a path beyond / ?
Hey fellow gophers. As it is currently implemented, if http.ServerMux serves a FileServer on a path beyond "/", for example "/mydir", and then serves a very different document on "/", the FileServer handler will not be triggered at all and the "/" handler gets all the requests. Is there a workaround so that FileServer(s) will continue to function, even though a very different handler is installed on path "/" ? Kind regards, Howard -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] An example web service implementation that runs on AWS Lambda and off-cloud
Hey fellow Gophers. This is a small but rather complete exercise to demonstrate writing a Go web service that runs on AWS Lambda while retaining the capability of running it off-cloud as a stand-alone server: https://github.com/HouzuoGuo/optionalcloud Features: - Authenticate a user via username/password combination read from SQL database, return a JWT if authorized. - Decrypt JWT from incoming HTTP request and respond with decrypted token structure. - Run stand-alone HTTP server if the executable is invoked with an arbitrary parameter. Kind regards, Howard -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
