Re: [aos-devel] optimizing go guru

[Luke Meyer]

If the answer is just "go guru is dog slow, use something else in that
case" then that seems like a useful thing to note in the README :) Along
with what people actually use in development. Seems like a number of tools
rely on guru but everyone complains about how slow it is on large projects.
So that's one more vote for VS...

On Tue, Dec 5, 2017 at 10:53 AM, Dan Mace  wrote:

>
>
> On Tue, Dec 5, 2017 at 10:43 AM, Luke Meyer  wrote:
>
>> In the context of the vim-go plugin. However behavior seems much the same
>> if I run the same command at the command line (I pulled it out of ps -ef).
>>
>> On Tue, Dec 5, 2017 at 10:40 AM, Sebastian Jug  wrote:
>>
>>> Are you using guru in some sort of editor/IDE or just standalone?
>>>
>>> On Dec 5, 2017 9:40 AM, "Luke Meyer"  wrote:
>>>


 On Tue, Dec 5, 2017 at 9:36 AM, Sebastian Jug  wrote:

> Sounds like you have got auto compile still on?
>
>
 What does this mean in the context of go guru? Is there an env var to
 set, an option to add, a config file to change to control this behavior?


>>>
>>
> ​The same query:
>
> guru -scope github.com/openshift/origin/cmd/oc whicherrs
> ./pkg/oc/admin/diagnostics/diagnostics.go:#7624
>
> was taking long enough for me (go1.8.3 darwin/amd64) that I killed it.
> It's hard to say without doing a deeper profile of that guru command. Even
> with your relatively narrow pointer analysis scope ​it seems really slow,
> but then again it's hard to gauge exactly how narrow that scope is without
> looking at a full import dependency graph...
>
> Guru has always been really slow for lots of useful pointer analysis
> queries, so I'm not entirely surprised. This is why vscode-go uses a
> variety of more optimized special purpose tools for most analysis[1].
>
> [1] https://github.com/Microsoft/vscode-go/blob/
> master/src/goInstallTools.ts#L21
>
> --
>
> Dan Mace
>
> Principal Software Engineer, OpenShift
>
> Red Hat
>
> dm...@redhat.com
>
>
>
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: [aos-devel] optimizing go guru

[Luke Meyer]

On Tue, Dec 5, 2017 at 10:51 AM, Clayton Coleman 
wrote:

> Openshift and Kubernetes are massive go projects - over 3 million lines of
> code (last I checked).  Initial compile can take a few minutes for these
> tools.  Things to check:
>
> 1. Go 1.9 uses less memory when compiling
> 2. Be sure you are reusing your go compiled artifacts dir between multiple
> tools (sometimes that is GOPATH/pkg, but openshift explicitly only compiles
> temp packages into _output/local/pkgdir for reasons)
>


So if I make clean all and then run my guru command, won't that be reusing
compiled artifacts? Is there some config that controls this? I don't think
I've customized anything.

It does seem to speed up a little bit after the first run but then it's
still pretty slow.



> 3. Get faster laptop :)
>
> On Dec 5, 2017, at 9:44 AM, Luke Meyer  wrote:
>
> In the context of the vim-go plugin. However behavior seems much the same
> if I run the same command at the command line (I pulled it out of ps -ef).
>
> On Tue, Dec 5, 2017 at 10:40 AM, Sebastian Jug  wrote:
>
>> Are you using guru in some sort of editor/IDE or just standalone?
>>
>> On Dec 5, 2017 9:40 AM, "Luke Meyer"  wrote:
>>
>>>
>>>
>>> On Tue, Dec 5, 2017 at 9:36 AM, Sebastian Jug  wrote:
>>>
 Sounds like you have got auto compile still on?


>>> What does this mean in the context of go guru? Is there an env var to
>>> set, an option to add, a config file to change to control this behavior?
>>>
>>>
>>
> ___
> dev mailing list
> dev@lists.openshift.redhat.com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>
>
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: [aos-devel] optimizing go guru

[Dan Mace]

On Tue, Dec 5, 2017 at 10:43 AM, Luke Meyer  wrote:

> In the context of the vim-go plugin. However behavior seems much the same
> if I run the same command at the command line (I pulled it out of ps -ef).
>
> On Tue, Dec 5, 2017 at 10:40 AM, Sebastian Jug  wrote:
>
>> Are you using guru in some sort of editor/IDE or just standalone?
>>
>> On Dec 5, 2017 9:40 AM, "Luke Meyer"  wrote:
>>
>>>
>>>
>>> On Tue, Dec 5, 2017 at 9:36 AM, Sebastian Jug  wrote:
>>>
 Sounds like you have got auto compile still on?


>>> What does this mean in the context of go guru? Is there an env var to
>>> set, an option to add, a config file to change to control this behavior?
>>>
>>>
>>
>
​The same query:

guru -scope github.com/openshift/origin/cmd/oc whicherrs
./pkg/oc/admin/diagnostics/diagnostics.go:#7624

was taking long enough for me (go1.8.3 darwin/amd64) that I killed it. It's
hard to say without doing a deeper profile of that guru command. Even with
your relatively narrow pointer analysis scope ​it seems really slow, but
then again it's hard to gauge exactly how narrow that scope is without
looking at a full import dependency graph...

Guru has always been really slow for lots of useful pointer analysis
queries, so I'm not entirely surprised. This is why vscode-go uses a
variety of more optimized special purpose tools for most analysis[1].

[1]
https://github.com/Microsoft/vscode-go/blob/master/src/goInstallTools.ts#L21

-- 

Dan Mace

Principal Software Engineer, OpenShift

Red Hat

dm...@redhat.com
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: [aos-devel] optimizing go guru

[Clayton Coleman]

Openshift and Kubernetes are massive go projects - over 3 million lines of
code (last I checked).  Initial compile can take a few minutes for these
tools.  Things to check:

1. Go 1.9 uses less memory when compiling
2. Be sure you are reusing your go compiled artifacts dir between multiple
tools (sometimes that is GOPATH/pkg, but openshift explicitly only compiles
temp packages into _output/local/pkgdir for reasons)
3. Get faster laptop :)

On Dec 5, 2017, at 9:44 AM, Luke Meyer  wrote:

In the context of the vim-go plugin. However behavior seems much the same
if I run the same command at the command line (I pulled it out of ps -ef).

On Tue, Dec 5, 2017 at 10:40 AM, Sebastian Jug  wrote:

> Are you using guru in some sort of editor/IDE or just standalone?
>
> On Dec 5, 2017 9:40 AM, "Luke Meyer"  wrote:
>
>>
>>
>> On Tue, Dec 5, 2017 at 9:36 AM, Sebastian Jug  wrote:
>>
>>> Sounds like you have got auto compile still on?
>>>
>>>
>> What does this mean in the context of go guru? Is there an env var to
>> set, an option to add, a config file to change to control this behavior?
>>
>>
>
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: [aos-devel] optimizing go guru

[Luke Meyer]

In the context of the vim-go plugin. However behavior seems much the same
if I run the same command at the command line (I pulled it out of ps -ef).

On Tue, Dec 5, 2017 at 10:40 AM, Sebastian Jug  wrote:

> Are you using guru in some sort of editor/IDE or just standalone?
>
> On Dec 5, 2017 9:40 AM, "Luke Meyer"  wrote:
>
>>
>>
>> On Tue, Dec 5, 2017 at 9:36 AM, Sebastian Jug  wrote:
>>
>>> Sounds like you have got auto compile still on?
>>>
>>>
>> What does this mean in the context of go guru? Is there an env var to
>> set, an option to add, a config file to change to control this behavior?
>>
>>
>
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: [aos-devel] optimizing go guru

[Luke Meyer]

In this case I'm also running into
https://github.com/openshift/origin/issues/17588 but perhaps it's all
related.

[origin] $ git describe
v3.9.0-alpha.0-11-ga5c80373e4
[origin] $ go version
go version go1.8.5 linux/amd64
[origin] $ echo $GOPATH
/home/lmeyer/go
[origin] $ time /home/lmeyer/go/bin/guru -scope
github.com/openshift/origin/cmd/oc whicherrs /home/lmeyer/go/src/
github.com/openshift/origin/pkg/oc/admin/diagnostics/diagnostics.go:#7624

/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/mtrmac/gpgme/data.go:4:11:
fatal error: gpgme.h: No such file or directory
 // #include 
   ^
compilation terminated.
cgo failed: [go tool cgo -objdir
/tmp/github.com_openshift_origin_vendor_github.com_mtrmac_gpgme_C519117980
-- -D_FILE_OFFSET_BITS=64 -I
/tmp/github.com_openshift_origin_vendor_github.com_mtrmac_gpgme_C519117980
data.go gpgme.go]: exit status 1
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:16:16:
Context not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:66:44:
Context not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:97:20:
NewDataBytes not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:101:14:
invalid operation: m.ctx (variable of type *invalid type) has no field or
method Import
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:106:20:
cannot range over res.Imports (invalid operand)
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:122:14:
invalid operation: m.ctx (variable of type *invalid type) has no field or
method GetKey
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:126:20:
NewDataBytes not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:131:18:
NewDataWriter not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:135:11:
invalid operation: m.ctx (variable of type *invalid type) has no field or
method Sign
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:135:25:
Key not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:135:61:
SigModeNormal not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:144:21:
NewDataWriter not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:148:34:
NewDataBytes not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:152:18:
invalid operation: m.ctx (variable of type *invalid type) has no field or
method Verify
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:161:42:
ValidityNever not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:67:14:
New not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:71:27:
ProtocolOpenPGP not declared by package gpgme
/home/lmeyer/go/src/
github.com/openshift/origin/vendor/github.com/containers/image/signature/mechanism_gpgme.go:75:28:
ProtocolOpenPGP not declared by package gpgme
guru: couldn't load packages due to errors:
github.com/openshift/origin/vendor/github.com/containers/image/signature,
github.com/openshift/origin/vendor/github.com/mtrmac/gpgme

real 0m10.041s
user 1m0.303s
sys 0m7.648s



On Tue, Dec 5, 2017 at 9:36 AM, Dan Mace  wrote:

>
>
> On Tue, Dec 5, 2017 at 9:31 AM, Luke Meyer  wrote:
>
>> I must be doing something wrong. Whenever go guru is fired off against
>> the origin codebase (for example, with godef or callstack or whicherrs) it
>> takes several seconds (or more) to do anything, sucking up GB of RAM and
>> all CPUs. I imagine it must be compiling the world, which is rather large.
>> Perhaps I am using the wrong scope. For instance, when working with oc, I
>> set scope to github.com/openshift/origin/cmd/oc. Should it be something
>> else? The guru docs are not as clear as they could be on exactly what
>> impact this has.
>>
>> How do you configure your development environment to do code