On Fri, Oct 15, 2021 at 2:13 PM Steven Hartland <ste...@multiplay.co.uk> wrote:
> I converted my code to x/tools/go/packages > <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> and while it > did solve the problem it's VERY slow in comparison. > > I have a set of 21 tests operating on a single package which has at most > two very basic types, no imports and using go/parser > <https://pkg.go.dev/go/parser> they take 0.011s but with go/packages > <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> that increases > to 3.548s a 300x slow down. > > I'm setting a basic mode: packages.NeedName | packages.NeedSyntax > > The package.Load call takes ~220ms whereas ast.NewPackage only takes 2.7µs. > Could you post a reproducer of your target package and analysis somewhere? 220ms for packages.Load sounds like a lot. It's true that packages does a lot more work than just the parser (*), but it's not supposed to be that slow. In my tests a simple Load with more functionality takes 60-70ms (*) The type checking takes a bit of time over just parsing to AST, but the biggest difference is loading multiple files from imports. For type checking you need to know, when you see: import foo x := foo.Foo() What the type of `x` is, so go/packages has to analyze the `foo` package as well. > > As the resulting ast.File's are pretty much the same, I'm wondering if for > my use case packages.Load is doing way more than I need? > > Another downside is for tests run in a temporary directory outside of the > package space package.Load fails with: > directory /tmp/tests76985775 outside available modules > > I fixed it by calling ioutil.TempDir with "." but that's not ideal. > > Thoughts? > > On Tue, 12 Oct 2021 at 13:42, Steven Hartland <ste...@multiplay.co.uk> > wrote: > >> Thanks David, much appreciated, I will have a look at both. >> >> When migrating from go/ast to go/types did you hit anything of note I >> should look out for? >> >> On Mon, 11 Oct 2021 at 17:06, David Finkel <david.fin...@gmail.com> >> wrote: >> >>> >>> >>> On Mon, Oct 11, 2021 at 5:48 AM Steven Hartland <ste...@multiplay.co.uk> >>> wrote: >>> >>>> If the ast.Files passed to ast.NewPackage includes built in types such >>>> as int it returns an error e.g. >>>> file1.go:5:6: undeclared name: int >>>> >>>> Is there a way to prevent that? >>>> >>> >>> Generally, I always add the `builtin` package to the list of packages >>> I'm parsing. >>> I wrote a little library for exactly this kind of package loading a few >>> years ago: >>> https://gitlab.com/dfinkel/goastpkg/-/blob/master/go_ast_parser.go >>> (https://pkg.go.dev/golang.spin-2.net/astpkg) >>> >>>> >>>> Playground example: https://play.golang.org/p/Yg30TTzoLHP >>>> >>>> My goal is to take multiple files, resolve inter file dependencies e.g. >>>> a type referencing another type in a different file and process the >>>> resulting ast.Files. So if there is a better way to achieve this I'm all >>>> ears. >>>> >>> >>> In general, I've stopped using the `go/ast` internal references as much >>> and have started using resolved `go/types` references as they're more >>> reliable and better-specified. >>> (golang.org/x/tools/go/packages >>> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> has a >>> LoadMode flag for generating `go/types.Info` (NeedTypesInfo >>> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages#NeedTypesInfo> >>> )) >>> >>>> >>>> Regards >>>> Steve >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/CAHEMsqbJoJxuo3c-mofMtzXXJhYCzV2skW2ZB3ZPY6WtA8%2BxHw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/golang-nuts/CAHEMsqbJoJxuo3c-mofMtzXXJhYCzV2skW2ZB3ZPY6WtA8%2BxHw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CAHEMsqYMSBUfuOUvptv6UrvBFTwFxjOhJZ5sMN-omOx5ESL5hw%40mail.gmail.com > <https://groups.google.com/d/msgid/golang-nuts/CAHEMsqYMSBUfuOUvptv6UrvBFTwFxjOhJZ5sMN-omOx5ESL5hw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAF-Rda9r8m7hGzpR2ZfK6xS3Sx76B%3D5Nks2BVE120xCKmj%3De5w%40mail.gmail.com.
