Re: [go-nuts] go/types#Check is not resolving external package source positions.
Glad you got things working. A few quick remarks:
1) You don't need to call ParseDir or CreateFromFiles. The loader will
figure out what files you need to load for each package; just call
config.Import(packagename).
2) The go/types tutorial says that export data doesn't contain position
information. That used to be true, but we have since extended the export
data format to include it. I'll update it.
3) That said, I did once notice a bug whereby the position of the package
declaration was reported instead of the correct location. If you are able
to reproduce that bug, please report it.
cheers
alan
> On Sun, Oct 29, 2017 at 5:00 PM, Keynan Pratt > wrote:
>
>> Can confirm the following works as expected:
>>
>>
>> package main
>>
>> import (
>> "fmt"
>> "go/parser"
>> "go/token"
>> "go/ast"
>> sourceLoader "golang.org/x/tools/go/loader"
>> )
>>
>> func main() {
>> pkgPath := "bitbucket.org/.../test_package"
>> dir := "../..//test_package"
>>
>> fset := token.NewFileSet()
>> packages, err := parser.ParseDir(fset, dir, nil, parser.AllErrors)
>> if err != nil {
>> fmt.Println(err)
>> return
>> }
>>
>> for _, p := range packages {
>> files := make([]*ast.File, 0, 8)
>> for _, file := range p.Files {
>> files = append(files, file)
>> }
>>
>> loader := sourceLoader.Config{Fset: fset}
>> loader.CreateFromFiles(pkgPath, files...)
>> prgm, err := loader.Load()
>> if err != nil {
>> fmt.Errorf(err.Error())
>> }
>>
>> info := prgm.Created[0].Info
>> fmt.Println("\tUsages from ", pkgPath)
>> for use := range info.Uses {
>> obj := info.Uses[use]
>>
>> pkgOfUse := obj.Pkg()
>> nameOfUse := obj.Name()
>> declPos := prgm.Fset.Position(obj.Pos())
>> if pkgOfUse == nil {
>>
>> fmt.Println("Package was nil for obj.id [ ", obj.Id(), " ]")
>> continue
>> }
>> fmt.Printf(
>> "\tUsage of %s # %s declared at %s:%d:%d\n",
>> pkgOfUse.Path(),
>> nameOfUse,
>> declPos.Filename, declPos.Line, declPos.Column)
>> }
>> }
>> }
>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Fwd: [go-nuts] go/types#Check is not resolving external package source positions.
-- Forwarded message --
From: Keynan Pratt
Date: Sun, Oct 29, 2017 at 5:01 PM
Subject: Re: [go-nuts] go/types#Check is not resolving external package
source positions.
To: roger peppe
Thanks for the help Roger.
On Sun, Oct 29, 2017 at 5:00 PM, Keynan Pratt wrote:
> Can confirm the following works as expected:
>
>
> package main
>
> import (
> "fmt"
> "go/parser"
> "go/token"
> "go/ast"
> sourceLoader "golang.org/x/tools/go/loader"
> )
>
> func main() {
> pkgPath := "bitbucket.org/.../test_package"
> dir := "../..//test_package"
>
> fset := token.NewFileSet()
> packages, err := parser.ParseDir(fset, dir, nil, parser.AllErrors)
> if err != nil {
> fmt.Println(err)
> return
> }
>
> for _, p := range packages {
> files := make([]*ast.File, 0, 8)
> for _, file := range p.Files {
> files = append(files, file)
> }
>
> loader := sourceLoader.Config{Fset: fset}
> loader.CreateFromFiles(pkgPath, files...)
> prgm, err := loader.Load()
> if err != nil {
> fmt.Errorf(err.Error())
> }
>
> info := prgm.Created[0].Info
> fmt.Println("\tUsages from ", pkgPath)
> for use := range info.Uses {
> obj := info.Uses[use]
>
> pkgOfUse := obj.Pkg()
> nameOfUse := obj.Name()
> declPos := prgm.Fset.Position(obj.Pos())
> if pkgOfUse == nil {
>
> fmt.Println("Package was nil for obj.id [ ", obj.Id(), " ]")
> continue
> }
> fmt.Printf(
> "\tUsage of %s # %s declared at %s:%d:%d\n",
> pkgOfUse.Path(),
> nameOfUse,
> declPos.Filename, declPos.Line, declPos.Column)
> }
> }
> }
>
>
> On Fri, Oct 27, 2017 at 6:28 PM, roger peppe wrote:
>
>> As I understand it, the default importer used by go/types looks in the
>> intermediate Go object files, which don't have position information,
>> instead of in the source code. This doesn't appear to be made very
>> clear in the stdlib doc comments, but is mentioned by the types
>> tutorial at https://golang.org/s/types-tutorial#imports That tutorial
>> document also says:
>>
>> "The golang.org/tools/x/go/loader package provides an alternative
>> Importer that addresses some of these problems. It loads a complete
>> program from source, performing cgo preprocessing if necessary,
>> followed by parsing and type-checking."
>>
>> As far as I can see, that's not quite right, as that package doesn't
>> seem to define an importer as such, but if you use it, it has an
>> importer internally that reads from source, so I'd suggest using it
>> rather than go/types directly.
>>
>> I've CC'd Alan Donovan, who can probably expand further on the design
>> choices made here.
>>
>> cheers,
>> rog.
>>
>> On 27 October 2017 at 01:51, wrote:
>> > I have a simple example file:
>> > ==
>> >
>> > package gobbledy_gook
>> >
>> > import (
>> > "log"
>> > turtle "github.com/rs/zerolog/log"
>> > "os"
>> > "github.com/rs/zerolog"
>> > )
>> >
>> > var LOG = turtle.Logger.Output(os.Stdout).Error()
>> >
>> > func g() []zerolog.Logger {
>> > return []zerolog.Logger{}
>> > }
>> >
>> > func DoThings() {
>> > log.Println("proof of concept")
>> > g()[3].Error()
>> > return
>> > }
>> >
>> > ==
>> >
>> > I want to use go/ast & go/types to correctly identify the source code
>> > position of the turtle.Logger declaration (.../zerolog/log.go:13:5).
>> >
>> > If I parse only the gobbledy_gook package then the position ends up
>> being
>> > :1:9
>> > If I parse gobbledy_gook & zerolog packages and put them in the fset /
>> list
>> > of files I get an unexpected package error
>> >
>> > Most of the below code is taken from the go/types#Info example code
>> >
>> > func main() {
>> > files := make([]*ast.File, 0, 8)
>> > fset := token.NewFileSet()
>> >
>&
Re: [go-nuts] go/types#Check is not resolving external package source positions.
As I understand it, the default importer used by go/types looks in the
intermediate Go object files, which don't have position information,
instead of in the source code. This doesn't appear to be made very
clear in the stdlib doc comments, but is mentioned by the types
tutorial at https://golang.org/s/types-tutorial#imports That tutorial
document also says:
"The golang.org/tools/x/go/loader package provides an alternative
Importer that addresses some of these problems. It loads a complete
program from source, performing cgo preprocessing if necessary,
followed by parsing and type-checking."
As far as I can see, that's not quite right, as that package doesn't
seem to define an importer as such, but if you use it, it has an
importer internally that reads from source, so I'd suggest using it
rather than go/types directly.
I've CC'd Alan Donovan, who can probably expand further on the design
choices made here.
cheers,
rog.
On 27 October 2017 at 01:51, wrote:
> I have a simple example file:
> ==
>
> package gobbledy_gook
>
> import (
> "log"
> turtle "github.com/rs/zerolog/log"
> "os"
> "github.com/rs/zerolog"
> )
>
> var LOG = turtle.Logger.Output(os.Stdout).Error()
>
> func g() []zerolog.Logger {
> return []zerolog.Logger{}
> }
>
> func DoThings() {
> log.Println("proof of concept")
> g()[3].Error()
> return
> }
>
> ==
>
> I want to use go/ast & go/types to correctly identify the source code
> position of the turtle.Logger declaration (.../zerolog/log.go:13:5).
>
> If I parse only the gobbledy_gook package then the position ends up being
> :1:9
> If I parse gobbledy_gook & zerolog packages and put them in the fset / list
> of files I get an unexpected package error
>
> Most of the below code is taken from the go/types#Info example code
>
> func main() {
> files := make([]*ast.File, 0, 8)
> fset := token.NewFileSet()
>
> for _, dir := range []string{
> "../../test_package",
> "go/src/github.com/rs/zerolog",
> "go/src/github.com/rs/zerolog/log",
> } {
> packages, err := parser.ParseDir(fset, dir, nil, parser.AllErrors)
> if err != nil {
> fmt.Println(err)
> return
> }
>
> for _, p := range packages {
> for _, file := range p.Files {
> files = append(files, file)
> }
> }
> }
> fmt.Println("Parse complete")
>
> info := types.Info{
> Types: make(map[ast.Expr]types.TypeAndValue),
> Defs: make(map[*ast.Ident]types.Object),
> Uses: make(map[*ast.Ident]types.Object),
> }
> var conf types.Config = types.Config{Importer: importer.Default()}
> pkg, err := conf.Check("fib", fset, files, &info)
> if err != nil {
> log.Println(err.Error())
> }
> fmt.Println("Check complete")
>
> // Print package-level variables in initialization order.
> fmt.Printf("InitOrder: %v\n\n", info.InitOrder)
>
> // For each named object, print the line and
> // column of its definition and each of its uses.
> fmt.Println("Defs and Uses of each named object:")
> usesByObj := make(map[types.Object][]string)
> for id, obj := range info.Uses {
> posn := fset.Position(id.Pos())
> lineCol := fmt.Sprintf("%d:%d", posn.Line, posn.Column)
> usesByObj[obj] = append(usesByObj[obj], lineCol)
> }
> var items []string
> for obj, uses := range usesByObj {
> sort.Strings(uses)
> item := fmt.Sprintf("%s:\n defined at %s\n used at %s",
> types.ObjectString(obj, types.RelativeTo(pkg)),
> fset.Position(obj.Pos()),
> strings.Join(uses, ", "))
> items = append(items, item)
> }
> sort.Strings(items) // sort by line:col, in effect
> fmt.Println(strings.Join(items, "\n"))
> fmt.Println()
>
> fmt.Println("Types and Values of each expression:")
> items = nil
> for expr, tv := range info.Types {
> var buf bytes.Buffer
> posn := fset.Position(expr.Pos())
> tvstr := tv.Type.String()
> if tv.Value != nil {
> tvstr += " = " + tv.Value.String()
> }
> // line:col | expr | mode : type = value
> fmt.Fprintf(&buf, "%s:%2d:%2d | %-19s | %s", posn.Filename,
> posn.Line, posn.Column, types.ExprString(expr), tvstr)
> items = append(items, buf.String())
> }
> sort.Strings(items)
> fmt.Println(strings.Join(items, "\n"))
> }
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe
[go-nuts] go/types#Check is not resolving external package source positions.
I have a simple example file:
==
package gobbledy_gook
import (
"log"
turtle "github.com/rs/zerolog/log"
"os"
"github.com/rs/zerolog"
)
var LOG = turtle.Logger.Output(os.Stdout).Error()
func g() []zerolog.Logger {
return []zerolog.Logger{}
}
func DoThings() {
log.Println("proof of concept")
g()[3].Error()
return
}
==
I want to use go/ast & go/types to correctly identify the source code
position of the turtle.Logger declaration (.../zerolog/log.go:13:5).
If I parse only the gobbledy_gook package then the position ends up being
:1:9
If I parse gobbledy_gook & zerolog packages and put them in the fset / list
of files I get an unexpected package error
Most of the below code is taken from the go/types#Info example code
func main() {
files := make([]*ast.File, 0, 8)
fset := token.NewFileSet()
for _, dir := range []string{
"../../test_package",
"go/src/github.com/rs/zerolog",
"go/src/github.com/rs/zerolog/log",
} {
packages, err := parser.ParseDir(fset, dir, nil, parser.AllErrors)
if err != nil {
fmt.Println(err)
return
}
for _, p := range packages {
for _, file := range p.Files {
files = append(files, file)
}
}
}
fmt.Println("Parse complete")
info := types.Info{
Types: make(map[ast.Expr]types.TypeAndValue),
Defs: make(map[*ast.Ident]types.Object),
Uses: make(map[*ast.Ident]types.Object),
}
var conf types.Config = types.Config{Importer: importer.Default()}
pkg, err := conf.Check("fib", fset, files, &info)
if err != nil {
log.Println(err.Error())
}
fmt.Println("Check complete")
// Print package-level variables in initialization order.
fmt.Printf("InitOrder: %v\n\n", info.InitOrder)
// For each named object, print the line and
// column of its definition and each of its uses.
fmt.Println("Defs and Uses of each named object:")
usesByObj := make(map[types.Object][]string)
for id, obj := range info.Uses {
posn := fset.Position(id.Pos())
lineCol := fmt.Sprintf("%d:%d", posn.Line, posn.Column)
usesByObj[obj] = append(usesByObj[obj], lineCol)
}
var items []string
for obj, uses := range usesByObj {
sort.Strings(uses)
item := fmt.Sprintf("%s:\n defined at %s\n used at %s",
types.ObjectString(obj, types.RelativeTo(pkg)),
fset.Position(obj.Pos()),
strings.Join(uses, ", "))
items = append(items, item)
}
sort.Strings(items) // sort by line:col, in effect
fmt.Println(strings.Join(items, "\n"))
fmt.Println()
fmt.Println("Types and Values of each expression:")
items = nil
for expr, tv := range info.Types {
var buf bytes.Buffer
posn := fset.Position(expr.Pos())
tvstr := tv.Type.String()
if tv.Value != nil {
tvstr += " = " + tv.Value.String()
}
// line:col | expr | mode : type = value
fmt.Fprintf(&buf, "%s:%2d:%2d | %-19s | %s", posn.Filename,
posn.Line, posn.Column, types.ExprString(expr), tvstr)
items = append(items, buf.String())
}
sort.Strings(items)
fmt.Println(strings.Join(items, "\n"))
}
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
