Hey Tim, thanks for writing this up. These kinds of reports are helpful. I'll try to respond to a few problems individually.
*1. Command installation with "go install"* There are basically two ways to do this, depending on whether you want to install the command within the context of your current module or "globally" ignoring go.mod, go.sum, and vendor. To install a command globally (you can replace @latest with a specific version like @v1.2.3): go install github.com/google/go-licenses@latest This form (with the @version suffix) is new in Go 1.16 <https://golang.org/doc/go1.16#modules>. Installing a command within the context of the current module is more complicated. I'd suggest this: # Add a dependency on the tool (only need to do this once) go get -d github.com/google/go-licenses # Import the command from tools.go # See below # Sync vendor directory (only if vendoring; only after changing go.mod with 'go get' above) go mod vendor # Install the command go install github.com/google/go-licenses # Alternatively, build and write to some other directory go build -o bin/go-licenses github.com/google/go-licenses The problem with this workflow is that 'go mod tidy' removes the requirement on the command's module because nothing imports it. 'go mod vendor' won't vendor the package for the same reason. There is a workaround documented on the Wiki <https://github.com/golang/go/issues/25922>: import the command package from a file (usually named tools.go) that is excluded by a build constraint (usually "tools"). // +build tools package tools import _ "github.com/google/go-licenses" This works, but personally, I don't find this to be an intuitive solution. At the moment, we don't have a concrete proposal to improve it though. There's some discussion at #25922 <https://github.com/golang/go/issues/25922>. *2. "writing go.mod cache" error messages* This error message should be a lot better. Sorry about that. Most module-aware commands need to download files into the module cache. The location of the cache can be set by setting the GOMODCACHE environment variable (or 'go env -w GOMODCACHE=/some/path'). That defaults to $GOPATH/pkg/mod, and GOPATH defaults to $HOME/go. So by default, the module cache is at $HOME/go/pkg/mod. It can't create that directory though. The error message should explain what went wrong and how to fix it, and it really shouldn't be repeated. I've opened #45113 <https://github.com/golang/go/issues/45113> for this. *3. 'go get' updating go.mod and go.sum* This is an unfortunate consequence of 'go get' being overloaded for downloading, installing, and managing package versions. Ideally, we want to separate these roles into different commands: 'go get' should be used to manage dependencies (changing requirements in go.mod and go.sum), and 'go install' should be used to install commands. We've taken a couple steps toward this in 1.16. The 'go install cmd@version' form is new. Also, 'go install', 'go build', and other commands no longer change go.mod or go.sum automatically when something is missing. We plan to deprecate 'go get' for installing commands. In Go 1.17, 'go get' will print a warning when the -d flag is not used and the arguments are main packages. In Go 1.18, we plan to remove that functionality entirely (the -d flag will always be on). #40276 <https://github.com/golang/go/issues/40276> is where that deprecation was proposed. On Thu, Mar 18, 2021 at 3:36 PM 'Tim Hockin' via golang-nuts < golang-nuts@googlegroups.com> wrote: > First: go version go1.16 linux/amd64 > > In one of my side-projects, I build a container image. It turns out > to be useful to people, but I got a request to put the license files > for all code used in the binary into the container. Lawyers, what can > you do? > > So these people sent me a PR to gather the licenses with `go install > github.com/google/go-licenses` <http://github.com/google/go-licenses> in > the `make container` rule. Here's > the friction. > > 1) Their PR includes dozens of new lines in go.sum, which makes me > grumpy. I set out to see if I can do better. > > 2) When I run that command on my workstation I get: > > ``` > $ go install github.com/google/go-licenses > cannot find package "." in: > /home/thockin/src/go/src/ > k8s.io/git-sync/vendor/github.com/google/go-licenses > ``` > > I don't know what to make of that message. It's not in vendor/ yet - > I want to put it there (I think?). > > So I am left wondering - what is the normal flow for "I want to use > this Go tool in my build" ? > > I tried `-mod=mod`: > > ``` > $ (unset GOPATH; go install -mod=mod github.com/google/go-licenses) > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: finding module for package github.com/google/go-licenses > go: downloading github.com/google/go-licenses > v0.0.0-20201026145851-73411c8fa237 > mkdir /home/thockin/go: not a directory > ``` > > I keep hearing that GOPATH is dead (and frankly good riddance) but > this tries to write to GOPATH or ~/go (which is also annoying). I > don't want this project's build spilling over into my homedir. > > I tried `go get` but it's the same. > > ``` > $ (unset GOPATH; go get github.com/google/go-licenses) > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: writing go.mod cache: mkdir /home/thockin/go: not a directory > go: downloading github.com/google/go-licenses > v0.0.0-20201026145851-73411c8fa237 > go get github.com/google/go-licenses: mkdir /home/thockin/go: not a > directory > ``` > > What do I *want*? I want to download a tool repo, build it, drop the > binary into ./bin/tools (or something) and then either vendor it or > have no artifacts. > > Since GOPATH seems to be a requirement, I thought maybe I can fake it: > > ``` > $ (export GOPATH="`pwd`/.gopath"; export GOBIN=`pwd`/bin/tools; mkdir > -p "$GOBIN"; go get github.com/google/go-licenses) > go: downloading github.com/google/go-licenses > v0.0.0-20201026145851-73411c8fa237 > go: downloading github.com/otiai10/copy v1.2.0 > go: downloading github.com/spf13/cobra v0.0.5 > go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b > go: downloading github.com/google/licenseclassifier > v0.0.0-20190926221455-842c0d70d702 > go: downloading golang.org/x/tools v0.0.0-20191118222007-07fc4c7f2b98 > go: downloading gopkg.in/src-d/go-git.v4 v4.13.1 > go: downloading github.com/spf13/pflag v1.0.5 > go: downloading github.com/inconshreveable/mousetrap v1.0.0 > go: downloading github.com/sergi/go-diff v1.0.0 > go: downloading gopkg.in/src-d/go-billy.v4 v4.3.2 > go: downloading golang.org/x/crypto v0.0.0-20191117063200-497ca9f6d64f > go: downloading golang.org/x/sys v0.0.0-20191119060738-e882bf8e40c2 > go: downloading github.com/emirpasic/gods v1.12.0 > go: downloading github.com/jbenet/go-context > v0.0.0-20150711004518-d14ea06fba99 > go: downloading github.com/src-d/gcfg v1.4.0 > go: downloading github.com/kevinburke/ssh_config > v0.0.0-20190725054713-01f96b0aa0cd > go: downloading github.com/mitchellh/go-homedir v1.1.0 > go: downloading github.com/xanzy/ssh-agent v0.2.1 > go: downloading golang.org/x/net v0.0.0-20191119073136-fc4aabc6c914 > go: downloading gopkg.in/warnings.v0 v0.1.2 > go get: added github.com/google/go-licenses > v0.0.0-20201026145851-73411c8fa237 > ``` > > This does actually build my binary (and leaves me one more thing to > `make clean`), but it also writes to go.mod and go.sum. I thought I'd > complete the task and vendor the code. `go mod vendor` adds a line to > vendor/modules.txt: > > ``` > # github.com/google/go-licenses v0.0.0-20201026145851-73411c8fa237 > ``` > > ...but doesn't actually vendor the code. OK. What if I just clean > up? `go mod tidy`. This leaves that vendor/modules.txt change but > removes almost all of the other changes to go.sum. Curiously, it > leaves some: > > ``` > $ git diff > diff --git a/go.mod b/go.mod > index ced314c..cfe7bca 100644 > --- a/go.mod > +++ b/go.mod > @@ -5,6 +5,7 @@ require ( > github.com/go-logr/logr v0.1.0 // indirect > github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // > indirect > github.com/prometheus/client_golang v0.9.2 > + golang.org/x/sync v0.0.0-20190423024810-112230192c58 // indirect > ) > > go 1.15 > diff --git a/go.sum b/go.sum > index b9f0527..4e159c8 100644 > --- a/go.sum > +++ b/go.sum > @@ -19,5 +19,6 @@ github.com/prometheus/common > v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7q > github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a > h1:9a8MnZMP0X2nLJdBg+pBmGgkJlSaKC2KaQmTCk1XDtE= > github.com/prometheus/procfs > v0.0.0-20181204211112-1dc9a6cbc91a/go.mod > h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= > golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod > h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= > -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f > h1:Bl/8QSvNqXvPGPGXa2z5xUTmV7VDcZyvRZ+QQXkXTZQ= > golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod > h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= > +golang.org/x/sync v0.0.0-20190423024810-112230192c58 > h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU= > +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod > h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= > ``` > > If I reset the whole repo and run `go mod tidy` it does not make this > change. > > Googling around I found a suggestion to use a different modfile. That > sounds like a hack, but I'll try it. > > I made a new, nearly-empty `go.tools.mod`: > > ``` > $ (export GOPATH="`pwd`/.gopath"; export GOBIN=`pwd`/bin/tools; mkdir > -p "$GOBIN"; go get -modfile=go.tools.mod > github.com/google/go-licenses) > ``` > > This works and wrote to go.tools.mod a created a new go.tools.sum. I > still can't figure how to vendor the tool. I guess I can just > .gitignore those, but this is starting to feel like I made a wrong > turn at Albuquerque. > > There has to be a "normal" path here, right? I feel like I shouldn't > have to fight so hard to make a more self-contained build. > > Can someone clue me in? > > Tim > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CAO_Rewb_5vBW4fuEa6i-XLLRiHqgEibxXTce3hHmOHfgAsRpfQ%40mail.gmail.com > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGCADba7USzJvW2T%3D3%3DHna5aHeUfp%2B%3DD_Y%3DPDj%2BqMpzJz_SOXg%40mail.gmail.com.
