FWIW: I'm not surprised that it's slower because it's handling significantly more of the compilation. The parser is very fast. Adding in type-checking is going to be less fast.
IIRC, the packages package also has to make network calls and do filesystem I/O behind the scenes in some cases. The parser doesn't have to do anything more than parse the code and output the AST. It could certainly be the case that the packages library is more than you need. Without understanding more about your use-case it will be hard to say for sure. On Fri, Oct 15, 2021, 4:13 PM Steven Hartland <ste...@multiplay.co.uk> wrote: > I converted my code to x/tools/go/packages > <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> and while it > did solve the problem it's VERY slow in comparison. > > I have a set of 21 tests operating on a single package which has at most > two very basic types, no imports and using go/parser > <https://pkg.go.dev/go/parser> they take 0.011s but with go/packages > <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> that increases > to 3.548s a 300x slow down. > > I'm setting a basic mode: packages.NeedName | packages.NeedSyntax > > The package.Load call takes ~220ms whereas ast.NewPackage only takes 2.7µs. > > As the resulting ast.File's are pretty much the same, I'm wondering if for > my use case packages.Load is doing way more than I need? > > Another downside is for tests run in a temporary directory outside of the > package space package.Load fails with: > directory /tmp/tests76985775 outside available modules > > I fixed it by calling ioutil.TempDir with "." but that's not ideal. > > Thoughts? > > On Tue, 12 Oct 2021 at 13:42, Steven Hartland <ste...@multiplay.co.uk> > wrote: > >> Thanks David, much appreciated, I will have a look at both. >> >> When migrating from go/ast to go/types did you hit anything of note I >> should look out for? >> >> On Mon, 11 Oct 2021 at 17:06, David Finkel <david.fin...@gmail.com> >> wrote: >> >>> >>> >>> On Mon, Oct 11, 2021 at 5:48 AM Steven Hartland <ste...@multiplay.co.uk> >>> wrote: >>> >>>> If the ast.Files passed to ast.NewPackage includes built in types such >>>> as int it returns an error e.g. >>>> file1.go:5:6: undeclared name: int >>>> >>>> Is there a way to prevent that? >>>> >>> >>> Generally, I always add the `builtin` package to the list of packages >>> I'm parsing. >>> I wrote a little library for exactly this kind of package loading a few >>> years ago: >>> https://gitlab.com/dfinkel/goastpkg/-/blob/master/go_ast_parser.go >>> (https://pkg.go.dev/golang.spin-2.net/astpkg) >>> >>>> >>>> Playground example: https://play.golang.org/p/Yg30TTzoLHP >>>> >>>> My goal is to take multiple files, resolve inter file dependencies e.g. >>>> a type referencing another type in a different file and process the >>>> resulting ast.Files. So if there is a better way to achieve this I'm all >>>> ears. >>>> >>> >>> In general, I've stopped using the `go/ast` internal references as much >>> and have started using resolved `go/types` references as they're more >>> reliable and better-specified. >>> (golang.org/x/tools/go/packages >>> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> has a >>> LoadMode flag for generating `go/types.Info` (NeedTypesInfo >>> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages#NeedTypesInfo> >>> )) >>> >>>> >>>> Regards >>>> Steve >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/CAHEMsqbJoJxuo3c-mofMtzXXJhYCzV2skW2ZB3ZPY6WtA8%2BxHw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/golang-nuts/CAHEMsqbJoJxuo3c-mofMtzXXJhYCzV2skW2ZB3ZPY6WtA8%2BxHw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CAHEMsqYMSBUfuOUvptv6UrvBFTwFxjOhJZ5sMN-omOx5ESL5hw%40mail.gmail.com > <https://groups.google.com/d/msgid/golang-nuts/CAHEMsqYMSBUfuOUvptv6UrvBFTwFxjOhJZ5sMN-omOx5ESL5hw%40mail.gmail.com?utm_medium=email&utm_source=footer> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALJzkY-prCAwkqeWXxuGCZaEYQ4%2B7QukUWbqG0VzGpKF71F%2BFw%40mail.gmail.com.
