Re: [go-nuts] Go += Package Versioning

[ivanov . maxim]

The proposed rules around semver and minimal version selection address 
> every pain point I've had so far, by aligning available authority with the 
> responsibility of the library authors: tell me what versions definitely 
> will *not* work, and I'll take it from there.
>
>
Maybe I miss something, but it is exactly what vgo doesn't let library 
authors to do. Sometimes "too new" is definitely not going to work for 
multiple reasons, but there is no way to convey that knowledge to library 
users.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Jon Calhoun]

Not ideal, but you *could* release a new major version where the "breaking"
change is the import path change. Eg if you are on v3 now, make a v4 folder
and the upgrade to v4 process would involve updating import paths. I
wouldn't do this right away since vgo is just a prototype and could change
to handle this situation better, but it is an option.


On Fri, Feb 23, 2018 at 10:00 AM, Nic Pottier  wrote:

> On Fri, Feb 23, 2018 at 7:59 AM, roger peppe  wrote:
> > As Russ pointed out to me, you can work around that by using a
> > 0.0.0-2101234543-4f34f456eeefdcba version in your go.mod require
> > section. If you've got that, it ignores the version tags.
>
> Right, that works for clients of the library (and I did manage to get
> my projects working with that and it now feels magical) but authors of
> modules above v1 are posed with a bit of a quandary as to how to give
> their users a nice vgo version without breaking current users on plain
> go.
>
> From Russ's latest post I think the answer is you put a v3 subdir in
> your repo with a copy of the code, add a mod.go and update both the
> root and v3 until vgo becomes adopted everywhere.
>
> -Nic
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/jFPz5yZCPcQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Nic Pottier]

Right, any self import needs to change for the vgo case. (but not for
the go case unless you force everybody currently using your library to
start using semantic versioning paths)

-Nic

On Fri, Feb 23, 2018 at 10:06 AM, Zellyn Hunter  wrote:
> On Fri, Feb 23, 2018 at 10:01 AM Nic Pottier  wrote:
>>
>> Right, that works for clients of the library (and I did manage to get
>> my projects working with that and it now feels magical) but authors of
>> modules above v1 are posed with a bit of a quandary as to how to give
>> their users a nice vgo version without breaking current users on plain
>> go.
>
>
>  Why not just commit a mod.go with v3 in it at the root, and tag it
> 3.{y+1}?, assuming you're already on v3.y? Adding mod.go doesn't break
> existing users. vgo users will use different import paths. If you like, you
> can branch off of your 1.x tag, add a mod.go with v1 in it there, and tag it
> 1.{x+1}. Same for v2.
>
> Oh wait, is it a problem because your self-import paths need to change (or
> not) depending on whether the client is using vgo? I think I'm unclear on
> that detail :-)
>
> Zellyn

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Zellyn Hunter]

On Fri, Feb 23, 2018 at 10:01 AM Nic Pottier  wrote:

> Right, that works for clients of the library (and I did manage to get
> my projects working with that and it now feels magical) but authors of
> modules above v1 are posed with a bit of a quandary as to how to give
> their users a nice vgo version without breaking current users on plain
> go.
>

 Why not just commit a mod.go with v3 in it at the root, and tag it
3.{y+1}?, assuming you're already on v3.y? Adding mod.go doesn't break
existing users. vgo users will use different import paths. If you like, you
can branch off of your 1.x tag, add a mod.go with v1 in it there, and tag
it 1.{x+1}. Same for v2.

Oh wait, is it a problem because your self-import paths need to change (or
not) depending on whether the client is using vgo? I think I'm unclear on
that detail :-)

Zellyn

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[roger peppe]

On 23 February 2018 at 15:00, Nic Pottier  wrote:
> On Fri, Feb 23, 2018 at 7:59 AM, roger peppe  wrote:
>> As Russ pointed out to me, you can work around that by using a
>> 0.0.0-2101234543-4f34f456eeefdcba version in your go.mod require
>> section. If you've got that, it ignores the version tags.
>
> Right, that works for clients of the library (and I did manage to get
> my projects working with that and it now feels magical) but authors of
> modules above v1 are posed with a bit of a quandary as to how to give
> their users a nice vgo version without breaking current users on plain
> go.
>
> From Russ's latest post I think the answer is you put a v3 subdir in
> your repo with a copy of the code, add a mod.go and update both the
> root and v3 until vgo becomes adopted everywhere.

Good point. Ugh.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Nic Pottier]

On Fri, Feb 23, 2018 at 7:59 AM, roger peppe  wrote:
> As Russ pointed out to me, you can work around that by using a
> 0.0.0-2101234543-4f34f456eeefdcba version in your go.mod require
> section. If you've got that, it ignores the version tags.

Right, that works for clients of the library (and I did manage to get
my projects working with that and it now feels magical) but authors of
modules above v1 are posed with a bit of a quandary as to how to give
their users a nice vgo version without breaking current users on plain
go.

>From Russ's latest post I think the answer is you put a v3 subdir in
your repo with a copy of the code, add a mod.go and update both the
root and v3 until vgo becomes adopted everywhere.

-Nic

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Zellyn]

Fantastic post. One small answer below:

On Friday, February 23, 2018 at 2:22:32 AM UTC-5, David Anderson wrote:
>
> The version to use in that replacement was hard to come by. Since vgo was 
> unable to compute the right dependency list without this replacement, it 
> bailed without giving me any version I might use. The date+hash format is 
> irritating to construct by hand, but could be trivially lifted into a tool 
> (perhaps even vgo itself).
>

git log -1 ae16178c028329cb0b979cd63bbd3921ddaf0945 
--date='format:%Y%m%d%H%M%S' --format='v.0.0.0-%cd-%h'
v.0.0.0-2018011942-ae16178c02

Zellyn
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[roger peppe]

On 23 February 2018 at 12:31, Nic Pottier  wrote:
> Thanks for the great write up Dave, you've inspired me to "try harder" to
> use it on a few of my projects after running into similar issues.
>
> As you noted, my remaining big question mark is how existing libraries that
> are above v1 transition to a vgo style while remaining backwards compatible
> during the transition.

As Russ pointed out to me, you can work around that by using a
0.0.0-2101234543-4f34f456eeefdcba version in your go.mod require
section. If you've got that, it ignores the version tags.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Nic Pottier]

Thanks for the great write up Dave, you've inspired me to "try harder" to 
use it on a few of my projects after running into similar issues.

As you noted, my remaining big question mark is how existing libraries that 
are above v1 transition to a vgo style while remaining backwards compatible 
during the transition. That seems like the biggest wrench to adoption. (if 
we can convince everyone to start using v# in import paths) I was preparing 
to open some PRs with dependencies of mine to introduce go.mod files, but 
as I understand it that won't be enough for those above v1.

On Friday, February 23, 2018 at 2:22:32 AM UTC-5, David Anderson wrote:
>
> I'm curious how we could upstream the changes I had to make to client-go. 
> I had to rename module-internal imports, which will break existing non-vgo 
> uses of the library, but vgo offers no alternative to this renaming that 
> I'm aware of. It looks to me like there's no graceful upgrade path for this 
> module. The repo structure works either for pre-vgo clients, or for vgo, 
> but not both at once.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Nicolas Grilly]

On Fri, Feb 23, 2018 at 8:20 AM, David Anderson  wrote:

> The date+hash format is irritating to construct by hand, but could be
> trivially lifted into a tool (perhaps even vgo itself).
>

Very true. I suffered from the same issue while trying vgo. This could be
improved to ease the adoption of vgo by projects with many dependencies
that are not already compatible with vgo semver.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Henrik Johansson]

A sidenote is that there seems to always be issues with using kubernetes
client.
Is it structured very un Go-ish?

Very nice writeup!

NB: I was also hit by the casing issue with the same viper dependency.

fre 23 feb. 2018 kl 08:28 skrev David Anderson :

> I should add: even though I spent several hours working through these
> issues, I feel very happy about the process and outcome. I have high
> confidence that I understand what vgo is doing with my module spec, and
> that I know how to tweak its thinking in future. I would contrast this with
> my experience with Glide. Last weekend, I tried to `glide up` this same
> project, and spent 4 hours trying to resolve the resulting mayhem of
> diamond dependencies and build errors. I failed, rolled back the change,
> and decided MetalLB could stay on old code for a while longer. I still have
> very low confidence that I understand what glide will do when I `glide up`,
> or that it will produce a working result.
>
> Additionally, some of the time spent is likely the learning curve. As
> https://github.com/golang/go/issues/24032 illustrates, I was initially
> confused and had to re-read some of rsc's writing to formulate my plan of
> action. Despite that, I still spent less time end to end than with glide,
> and I had a working build at the end of it.
>
> - Dave
>
> On Thu, Feb 22, 2018 at 11:20 PM, David Anderson  wrote:
>
>> This is an experience report onboarding vgo for a relatively complex
>> project (multiple binaries, vast import tree, tricky unidiomatic imports).
>> I leave it here in the hopes that it guides other people, and potentially
>> illustrates places where vgo falls short of great. TL;DR it went pretty
>> well, modulo a couple of UX speed bumps.
>>
>> The result is the `vgo-test` branch of https://github.com/google/metallb/
>> . Cloning that repository should allow you to `vgo test ./...`, `vgo build
>> ./controller`, `vgo build ./speaker`, and a few others. I don't make any
>> representation that the code does what it should, merely that it builds and
>> passes tests.
>>
>> The resultant go.mod,
>> https://github.com/google/metallb/blob/vgo-test/go.mod , is quite messy.
>> This is mostly due to the number of dependencies that have no semver at
>> all, forcing vgo to use commit hash "versions". The result is a lot of
>> visual noise in the file, but hopefully that will improve over time as both
>> vgo and dep nudge people towards semver releases.
>>
>> I encountered two major roadblocks on the road to `vgo test ./...`: the
>> Kubernetes client library, and mixed case packages. These are
>> https://github.com/golang/go/issues/24032 and
>> https://github.com/spf13/jWalterWeatherman/issues/22 respectively.
>>
>>
>> The Kubernetes client library is a worst case scenario for vgo. It
>> releases a new major version every 3 months under the same module name,
>> with real incompatibilities between versions; and it relies extensively on
>> a transitive version lock to force a specific package selection on its
>> dependents. Making this library usable from vgo required the following:
>>
>>- Fix up client-go in a fork
>>   - Fork github.com/kubernetes/client-go to
>>   github.com/danderson/client-go
>>   - Add a go.mod to the fork, containing only: module "
>>   k8s.io/client-go/v6"
>>   - Update all internal self-references within client-go: perl -pi
>>   -e 's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
>>   - Commit the result,
>>   
>> https://github.com/danderson/client-go/commit/7d8481153a9edbf8eacc176ce29f8d58949c3a77
>>   . Tag as v0.0.0-vgo-prototype-compat and push.
>>- Make my repository use my fork of client-go:
>>   - Update all uses of client-go to the new versioned package name: perl
>>   -pi -e 's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
>>   - Require the right version in mod.go: require "k8s.io/client-go/v6"
>>   v6.0.0
>>   - Replace upstream with my fork in mod.go: replace "
>>   k8s.io/client-go/v6" v6.0.0 => "github.com/danderson/client-go"
>>   v0.0.0-vgo-prototype-compat
>>
>> I'm curious how we could upstream the changes I had to make to client-go.
>> I had to rename module-internal imports, which will break existing non-vgo
>> uses of the library, but vgo offers no alternative to this renaming that
>> I'm aware of. It looks to me like there's no graceful upgrade path for this
>> module. The repo structure works either for pre-vgo clients, or for vgo,
>> but not both at once.
>>
>> The UX was lacking to explain the reason for failures, before I made any
>> of the changes. Given just my repository, vgo inferred that I wanted v1.5.2
>> of client-go (3+ years old), continued resolving dependencies, and
>> eventually barfed when it found an import for a client-go subpackage that
>> didn't exist in 1.5.2. The error was simply a bunch of "no such file or
>> directory", and required some head-scratching to 

Re: [go-nuts] Go += Package Versioning

[David Anderson]

I should add: even though I spent several hours working through these
issues, I feel very happy about the process and outcome. I have high
confidence that I understand what vgo is doing with my module spec, and
that I know how to tweak its thinking in future. I would contrast this with
my experience with Glide. Last weekend, I tried to `glide up` this same
project, and spent 4 hours trying to resolve the resulting mayhem of
diamond dependencies and build errors. I failed, rolled back the change,
and decided MetalLB could stay on old code for a while longer. I still have
very low confidence that I understand what glide will do when I `glide up`,
or that it will produce a working result.

Additionally, some of the time spent is likely the learning curve. As
https://github.com/golang/go/issues/24032 illustrates, I was initially
confused and had to re-read some of rsc's writing to formulate my plan of
action. Despite that, I still spent less time end to end than with glide,
and I had a working build at the end of it.

- Dave

On Thu, Feb 22, 2018 at 11:20 PM, David Anderson  wrote:

> This is an experience report onboarding vgo for a relatively complex
> project (multiple binaries, vast import tree, tricky unidiomatic imports).
> I leave it here in the hopes that it guides other people, and potentially
> illustrates places where vgo falls short of great. TL;DR it went pretty
> well, modulo a couple of UX speed bumps.
>
> The result is the `vgo-test` branch of https://github.com/google/metallb/
> . Cloning that repository should allow you to `vgo test ./...`, `vgo build
> ./controller`, `vgo build ./speaker`, and a few others. I don't make any
> representation that the code does what it should, merely that it builds and
> passes tests.
>
> The resultant go.mod, https://github.com/google/
> metallb/blob/vgo-test/go.mod , is quite messy. This is mostly due to the
> number of dependencies that have no semver at all, forcing vgo to use
> commit hash "versions". The result is a lot of visual noise in the file,
> but hopefully that will improve over time as both vgo and dep nudge people
> towards semver releases.
>
> I encountered two major roadblocks on the road to `vgo test ./...`: the
> Kubernetes client library, and mixed case packages. These are
> https://github.com/golang/go/issues/24032 and https://github.com/spf13/
> jWalterWeatherman/issues/22 respectively.
>
>
> The Kubernetes client library is a worst case scenario for vgo. It
> releases a new major version every 3 months under the same module name,
> with real incompatibilities between versions; and it relies extensively on
> a transitive version lock to force a specific package selection on its
> dependents. Making this library usable from vgo required the following:
>
>- Fix up client-go in a fork
>   - Fork github.com/kubernetes/client-go to
>   github.com/danderson/client-go
>   - Add a go.mod to the fork, containing only: module "
>   k8s.io/client-go/v6"
>   - Update all internal self-references within client-go: perl -pi -e
>   's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
>   - Commit the result, https://github.com/danderson/client-go/commit/
>   7d8481153a9edbf8eacc176ce29f8d58949c3a77
>   
> 
>   . Tag as v0.0.0-vgo-prototype-compat and push.
>- Make my repository use my fork of client-go:
>   - Update all uses of client-go to the new versioned package name: perl
>   -pi -e 's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
>   - Require the right version in mod.go: require "k8s.io/client-go/v6"
>   v6.0.0
>   - Replace upstream with my fork in mod.go: replace "
>   k8s.io/client-go/v6" v6.0.0 => "github.com/danderson/client-go"
>   v0.0.0-vgo-prototype-compat
>
> I'm curious how we could upstream the changes I had to make to client-go.
> I had to rename module-internal imports, which will break existing non-vgo
> uses of the library, but vgo offers no alternative to this renaming that
> I'm aware of. It looks to me like there's no graceful upgrade path for this
> module. The repo structure works either for pre-vgo clients, or for vgo,
> but not both at once.
>
> The UX was lacking to explain the reason for failures, before I made any
> of the changes. Given just my repository, vgo inferred that I wanted v1.5.2
> of client-go (3+ years old), continued resolving dependencies, and
> eventually barfed when it found an import for a client-go subpackage that
> didn't exist in 1.5.2. The error was simply a bunch of "no such file or
> directory", and required some head-scratching to understand what had
> happened. Once I understood why vgo was making that decision, and how to
> correct it, vgo provided the right tools (mostly replace-with-fork) to
> correct the issue without waiting on the library to fix itself.
>
>
> The second issue I hit is 

Re: [go-nuts] Go += Package Versioning

[David Anderson]

This is an experience report onboarding vgo for a relatively complex
project (multiple binaries, vast import tree, tricky unidiomatic imports).
I leave it here in the hopes that it guides other people, and potentially
illustrates places where vgo falls short of great. TL;DR it went pretty
well, modulo a couple of UX speed bumps.

The result is the `vgo-test` branch of https://github.com/google/metallb/ .
Cloning that repository should allow you to `vgo test ./...`, `vgo build
./controller`, `vgo build ./speaker`, and a few others. I don't make any
representation that the code does what it should, merely that it builds and
passes tests.

The resultant go.mod, https://github.com/google/metallb/blob/vgo-test/go.mod
, is quite messy. This is mostly due to the number of dependencies that
have no semver at all, forcing vgo to use commit hash "versions". The
result is a lot of visual noise in the file, but hopefully that will
improve over time as both vgo and dep nudge people towards semver releases.

I encountered two major roadblocks on the road to `vgo test ./...`: the
Kubernetes client library, and mixed case packages. These are
https://github.com/golang/go/issues/24032 and
https://github.com/spf13/jWalterWeatherman/issues/22 respectively.


The Kubernetes client library is a worst case scenario for vgo. It releases
a new major version every 3 months under the same module name, with real
incompatibilities between versions; and it relies extensively on a
transitive version lock to force a specific package selection on its
dependents. Making this library usable from vgo required the following:

   - Fix up client-go in a fork
  - Fork github.com/kubernetes/client-go to
  github.com/danderson/client-go
  - Add a go.mod to the fork, containing only: module "
  k8s.io/client-go/v6"
  - Update all internal self-references within client-go: perl -pi -e
  's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
  - Commit the result,
  
https://github.com/danderson/client-go/commit/7d8481153a9edbf8eacc176ce29f8d58949c3a77
  . Tag as v0.0.0-vgo-prototype-compat and push.
   - Make my repository use my fork of client-go:
  - Update all uses of client-go to the new versioned package name: perl
  -pi -e 's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
  - Require the right version in mod.go: require "k8s.io/client-go/v6"
  v6.0.0
  - Replace upstream with my fork in mod.go: replace "
  k8s.io/client-go/v6" v6.0.0 => "github.com/danderson/client-go"
  v0.0.0-vgo-prototype-compat

I'm curious how we could upstream the changes I had to make to client-go. I
had to rename module-internal imports, which will break existing non-vgo
uses of the library, but vgo offers no alternative to this renaming that
I'm aware of. It looks to me like there's no graceful upgrade path for this
module. The repo structure works either for pre-vgo clients, or for vgo,
but not both at once.

The UX was lacking to explain the reason for failures, before I made any of
the changes. Given just my repository, vgo inferred that I wanted v1.5.2 of
client-go (3+ years old), continued resolving dependencies, and eventually
barfed when it found an import for a client-go subpackage that didn't exist
in 1.5.2. The error was simply a bunch of "no such file or directory", and
required some head-scratching to understand what had happened. Once I
understood why vgo was making that decision, and how to correct it, vgo
provided the right tools (mostly replace-with-fork) to correct the issue
without waiting on the library to fix itself.


The second issue I hit is github.com/spf13/jwalterweatherman. The actual
repository name in github is "jWalterWeatherman", but everyone (including
the package's author :) ) imports it lowercase. vgo disallows this casing
mismatch.

To solve that, I just added a require + replacement to "fix" the
case: replace "github.com/spf13/jwalterweatherman"
v0.0.0-20180109140146-7c0cea34c8ec => "github.com/spf13/jWalterWeatherman"
v0.0.0-20180109140146-7c0cea34c8ec

The version to use in that replacement was hard to come by. Since vgo was
unable to compute the right dependency list without this replacement, it
bailed without giving me any version I might use. The date+hash format is
irritating to construct by hand, but could be trivially lifted into a tool
(perhaps even vgo itself). Instead, I opted to create a separate dummy
module, with a dummy main.go that imported and used the camelCased package,
and run vgo on that. This produced a tiny go.mod with a constructed commit
hash version, which I then used in MetalLB's go.mod. It would have been
nice for vgo to help me a bit more there.

- Dave



On Tue, Feb 20, 2018 at 9:20 AM, Russ Cox  wrote:

> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message 

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

On Tue, Feb 20, 2018 at 5:14 PM, Dave MacFarlane  wrote:

> I really like this, except for the claim that it the blog post that it
> will eliminate vendoring and deprecate GOPATH and the problems that
> will cause for backwards compatibility for things that are currently
> using them. If this is going to result in removing language features
> (ie. vendoring), shouldn't it be in Go2, not Go 1.11?
>

The proposal is only to remove vendoring when working with modules.
Non-module repos with vendor directories will keep building at least past
Go 1.11,
perhaps even forever.


> The other question I have is: where does the module name come from if
> you don't have a "// import" comment on the package, since it can't be
> filesystem based (because you're explicitly outside of GOPATH..)
>

It comes from go.mod. It only has to come from somewhere else if go.mod is
missing. That was a cute trick to ease transition more than anything else.
The places that vgo looks to try to guess a module path when filling in the
initial go.mod are:

- // import comments in the top-level directory or immediate
subdirectories
- Godeps/Godeps.json
- vendor/vendor.json
- .git/config

Perhaps this is not worth keeping in the real integration, but it was a
nice way to get off the ground.

Best,
Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

On Tue, Feb 20, 2018 at 4:07 PM, David Anderson  wrote:

> On Tue, Feb 20, 2018 at 11:37 AM, Russ Cox  wrote:
>
>> On Tue, Feb 20, 2018 at 1:55 PM, David Anderson  wrote:
>>
>>> I love this. I want it now.
>>>
>>
>> go get -u golang.org/x/vgo :-)
>>
>> I've struggled with `glide` and `dep` dependency hell a lot in my Go
>>> projects (especially those that use the Kubernetes client library, which
>>> abuses "I, the library author, can define byzantine constraints on my
>>> users" to the extreme). The way I've described it informally is that
>>> traditional "arbitrary version selection" algorithms create the wrong
>>> incentive structure for library authors, in the sense that they have all
>>> the power to lock versions arbitrarily, but bear none of the pain
>>> associated with their decisions - instead, the binary author at the apex of
>>> the dependency tree gets that pain. "Authority without responsibility" is
>>> the death of many incentive structures.
>>>
>>
>> Yes, I'll make that point again in tomorrow's posts elaborating minimal
>> version selection, but I think this is probably the most important
>> algorithmic policy detail. You get total control over your own build. You
>> only get limited control over other people's builds that happen to import
>> your code.
>>
>
> Big +1. I'm curious about your ideas on how to manage the transition from
> a glide/dep world to a vgo world. I look forward to reading more about
> that, assuming you have an article queued to discuss that facet :).
>

I don't have an article about this. There are two parts to the answer. The
first is that vgo reads all these tools' lock files and records the things
listed there are minimum requirements, as a starting point. I expect this
will be good enough for many projects. For example, on Tuesday when vgo was
first released, before anything had time to convert, you could git clone
upspin or dep, cd into the checkout (anywhere, not just in GOPATH), and
start working. Vgo seeded go.mod with the Gopkg.lock file. The second part
is that, as Sam posted , we are
trying to work out a way to contract him to build tools to help with more
complex conversions.

One thought that I hope you'll cover there: immutability. Reading through
> the vgo tour, it seems to assume that once v5.6.7 has been released, that
> .zip and its declared dependencies will never change.
>

You've probably seen this by now, but: research.swtch.com/vgo-repro.

Best,
Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[jimmy frasche]

Is there a story for a module that changes domains, like when google
code shutdown or moving to a custom domain for vanity import paths?

Having directives to say "find earlier versions here" or "find later
versions" here seems like it would help downgrades/upgrades. If one of
those old domains stops existing entirely the chain would be broken,
certainly, but it still seems like it would be useful.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

['Axel Wagner' via golang-nuts]

On Thu, Feb 22, 2018 at 4:21 AM David Anderson  wrote:

> Currently, per the vgo tour, the entire transitive closure gets updated,
> which, per the article's definition, results in a low-fidelity build... If
> you assume that the intent of `vgo get -u` was "the libraries I'm calling
> have new features/fixes, please use them," and not "I want to use the
> latest possible code the universe can offer me, throughout the stack."
>

I don't think this is the correct interpretation of what Russ calls
"low-fidelity". "low-fidelity build" means, that you install something I
built and gets a poor reproduction of what I intended you to get. This is
different from me upgrading the transitive closure of dependencies of my
stuff and you then installing that; this would still be a good reproduction
of my intent and thus high-fidelity.


> I believe both operations have their place, and I could be convinced
> either way on which should be the default for "update my stuff." I'm
> wondering if I'm alone in thinking that "upgrade only direct dependencies,
> minimal versions elsewhere" is desirable as a "suggested" commandline
> action.
>

I think it *may* have its place but it should be strongly discouraged (to
the degree that I'm not sure whether I want it to exist at all). I think
upgrading the transitive closure can be a valuable driver to prevent bitrot
in the Go community at large - but that requires that a critical mass of
projects actually use it.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[David Anderson]

On Wed, Feb 21, 2018 at 3:55 PM, Axel Wagner 
wrote:

> On Wed, Feb 21, 2018 at 11:55 PM David Anderson  wrote:
>
>> Reading the latest post, https://research.swtch.com/vgo-mvs , a
>> question...
>>
>> It feels to me like there's a missing 5th operation, in additions to the
>> one you proposed: "upgrade all my *direct* dependencies to their latest
>> version, but keep using minimal versions below that." I don't believe there
>> is an analog to this operation in the worlds of `go get` or `dep`, but I
>> think it might be an interesting middle-ground. I as the apex module
>> developer want to say "I'd like to use the latest versions of the things
>> I'm programming against, but I don't care about *their* dependencies –
>> please defer to the library authors's opinions for those."
>>
>> First of all, am I reading correctly that this is not currently one of
>> the options you described? If so, do you think this additional option has
>> merit, compared to "upgrade the entire transitive closure to the absolute
>> newest things" ?
>>
>
> I think this operation is algorithmically trivial: Just bump the versions
> in A's list to the newest ones available of each direct dependency. I
> understand Russ' Article to justify that these operations can be
> implemented efficient and that thus a trivial operation isn't interesting
> to mention.
>

Agreed, from the algorithmic perspective it simplifies down to the existing
operation add/remove operations, so the algorithms and properties are
unchanged.

What I'm arguing for (or at least, asking about), is whether `vgo get -u`
should only set direct dependencies to the latest version, or the entire
transitive closure. It's purely a UI/UX question, not an algorithms one.

Currently, per the vgo tour, the entire transitive closure gets updated,
which, per the article's definition, results in a low-fidelity build... If
you assume that the intent of `vgo get -u` was "the libraries I'm calling
have new features/fixes, please use them," and not "I want to use the
latest possible code the universe can offer me, throughout the stack."

I believe both operations have their place, and I could be convinced either
way on which should be the default for "update my stuff." I'm wondering if
I'm alone in thinking that "upgrade only direct dependencies, minimal
versions elsewhere" is desirable as a "suggested" commandline action.

- Dave


>
> If the "import compatibility rule" is true, there shouldn't be a downside
> to do a full, transitive closure upgrade either and if this is the
> standard/most common/only way to do upgrades, that would at least build
> *some* force into the system, to progress through upgrades regularly.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

['Axel Wagner' via golang-nuts]

On Wed, Feb 21, 2018 at 11:55 PM David Anderson  wrote:

> Reading the latest post, https://research.swtch.com/vgo-mvs , a
> question...
>
> It feels to me like there's a missing 5th operation, in additions to the
> one you proposed: "upgrade all my *direct* dependencies to their latest
> version, but keep using minimal versions below that." I don't believe there
> is an analog to this operation in the worlds of `go get` or `dep`, but I
> think it might be an interesting middle-ground. I as the apex module
> developer want to say "I'd like to use the latest versions of the things
> I'm programming against, but I don't care about *their* dependencies –
> please defer to the library authors's opinions for those."
>
> First of all, am I reading correctly that this is not currently one of the
> options you described? If so, do you think this additional option has
> merit, compared to "upgrade the entire transitive closure to the absolute
> newest things" ?
>

I think this operation is algorithmically trivial: Just bump the versions
in A's list to the newest ones available of each direct dependency. I
understand Russ' Article to justify that these operations can be
implemented efficient and that thus a trivial operation isn't interesting
to mention.

If the "import compatibility rule" is true, there shouldn't be a downside
to do a full, transitive closure upgrade either and if this is the
standard/most common/only way to do upgrades, that would at least build
*some* force into the system, to progress through upgrades regularly.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

Replying to this thread to drop golang-dev. Please try to keep the two
threads separate. I should not have used the same subject for each. Sorry.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[David Anderson]

Reading the latest post, https://research.swtch.com/vgo-mvs , a question...

It feels to me like there's a missing 5th operation, in additions to the
one you proposed: "upgrade all my *direct* dependencies to their latest
version, but keep using minimal versions below that." I don't believe there
is an analog to this operation in the worlds of `go get` or `dep`, but I
think it might be an interesting middle-ground. I as the apex module
developer want to say "I'd like to use the latest versions of the things
I'm programming against, but I don't care about *their* dependencies –
please defer to the library authors's opinions for those."

First of all, am I reading correctly that this is not currently one of the
options you described? If so, do you think this additional option has
merit, compared to "upgrade the entire transitive closure to the absolute
newest things" ?


I'm still completely in love with this overall approach, and so far the
only quibbles I might have would be over syntax or equivalently unimportant
details. I sincerely hope that this becomes Go's package management
solution of choice, I'm already finding it unpleasant to operate in a world
where this isn't the way to do things :).

- Dave

On Tue, Feb 20, 2018 at 9:20 AM, Russ Cox  wrote:

> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Henrik Johansson]

Why would that be a mistake? Working with tags and branches is what the VCS
is for. Extracting a given version and using that per the convention that
the import ends with the major version sounds very reasonable.

On Wed, Feb 21, 2018, 22:51 Zellyn  wrote:

> On Wednesday, February 21, 2018 at 4:45:25 PM UTC-5, Bakul Shah wrote:
>>
>> Something like
>> import "foo" v2
>> would be mildly preferable to me as
>> import "foo/v2"
>> can also refer to version 1 of a package named v2 (as opposed
>> to version 2 of foo) and the current naming rules continue to
>> work.
>>
>
> As I understood it, the fundamental philosophical point of today's rsc
> article was that you shouldn't use the the same "name" (identifier) for two
> different "things". If you look at it from that point of view, the "v2"
> should absolutely be part of the package URL. In fact, having the concept
> of version stored in some lookaside metadata outside of the name (eg. git
> tag) would be a mistake.
>
> Zellyn
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Zellyn]

On Wednesday, February 21, 2018 at 4:45:25 PM UTC-5, Bakul Shah wrote:
>
> Something like 
> import "foo" v2 
> would be mildly preferable to me as 
> import "foo/v2" 
> can also refer to version 1 of a package named v2 (as opposed 
> to version 2 of foo) and the current naming rules continue to 
> work. 
>

As I understood it, the fundamental philosophical point of today's rsc 
article was that you shouldn't use the the same "name" (identifier) for two 
different "things". If you look at it from that point of view, the "v2" 
should absolutely be part of the package URL. In fact, having the concept 
of version stored in some lookaside metadata outside of the name (eg. git 
tag) would be a mistake.

Zellyn

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Bakul Shah]

On Wed, 21 Feb 2018 22:24:38 +0100 Sebastien Binet  wrote:
> 
> On Feb 21, 2018 10:11 PM, "Bakul Shah"  wrote:
> > 
> > As v2 is only created due to some incompatible change, in my
> > view it is better to have v2 visible in the path.  Otherwise
> > it is easy for a reader to get confused when foo.F() in one
> > package (using v1) behaves differently from foo.F() in another
> > package (using v2) as they both will have the same import path
> > and only the "bindings" in their respective go.mod change.
> 
> Agreed but it seems a bit of a rather gratuitous constraint to require the
> version to be the last element:

Something like
import "foo" v2
would be mildly preferable to me as
import "foo/v2"
can also refer to version 1 of a package named v2 (as opposed
to version 2 of foo) and the current naming rules continue to
work.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

['Axel Wagner' via golang-nuts]

On Wed, Feb 21, 2018 at 10:17 PM Peter Bourgon  wrote:

> ·Any· change to the exported API of a package except adding new
> top-level identifiers can be considered an incompatible (breaking)
> change.
>
> https://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html


I should probably go on record that I acknowledge that post to be
potentially unreasonably nitpicky :)

FWIW, I *believe* that if you only ever release v1.x.x you'll end up at
essentially the same space as today, whether you actually end up adding
breaking changes or not. Unless vgo somehow tracks API changes and verifies
that your versioning is semantic, which I can't imagine working (which was
my point then, basically). Of course this makes semantic versions largely
useless, which was my other point then.

There is one significant difference, which is that people are currently
automatically held back, whereas previously they where automatically
updated, so it's somewhat more likely that people will sleep through
necessary API changes and then break the build at an unpredictable point in
the future.

Personally, that is (with the information currently published) what I
intend to do, most likely: Tag all my stuff at v1.x.x (or maybe not tag at
all, for a "pin on first use" kinda policy) and just be careful in if and
how I break stuff.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Sebastien Binet]

On Feb 21, 2018 10:11 PM, "Bakul Shah"  wrote:

On Wed, 21 Feb 2018 20:53:20 + Henrik Johansson 
wrote:
>
> That's not necessarily true.
> The tool can understand from the tag and assert or even rewrite the
imports
> to make the compiler happy.
>
> On Wed, Feb 21, 2018, 21:51 Sam Whited  wrote:
>
> > On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
> > > But wait. Wasn't there a mention of archive downloads instead of
relyin=
> g
> > on
> > > the different VCS's?
> > >
> > > In the GitHub case I guess it amount to downloading releases (or any
> > commit
> > > I guess) as a zip or tarball.
> >
> > My understanding was that the path is still relavant, so you'd need a
/v2
> > directory inside the zip file, which means you probably need it in your
> > source code. However, if that is incorrect it fixes the problem.

As v2 is only created due to some incompatible change, in my
view it is better to have v2 visible in the path.  Otherwise
it is easy for a reader to get confused when foo.F() in one
package (using v1) behaves differently from foo.F() in another
package (using v2) as they both will have the same import path
and only the "bindings" in their respective go.mod change.


Agreed but it seems a bit of a rather gratuitous constraint to require the
version to be the last element:

https://github.com/golang/go/issues/23960

I kind of like Gonum's way of versioning:
gonum.org/v1/gonum/stat
gonum.org/v1/gonum/mat
gonum.org/v1/plot/vg/vgpdf
(And, in the future:
gonum.org/v4/plot
gonum.org/v4/gonum/deeplearn
...)

-s


--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Peter Bourgon]

On Wed, Feb 21, 2018 at 1:10 PM, Bakul Shah  wrote:
> On Wed, 21 Feb 2018 20:53:20 + Henrik Johansson  
> wrote:
>>
>> That's not necessarily true.
>> The tool can understand from the tag and assert or even rewrite the imports
>> to make the compiler happy.
>>
>> On Wed, Feb 21, 2018, 21:51 Sam Whited  wrote:
>>
>> > On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
>> > > But wait. Wasn't there a mention of archive downloads instead of relyin=
>> g
>> > on
>> > > the different VCS's?
>> > >
>> > > In the GitHub case I guess it amount to downloading releases (or any
>> > commit
>> > > I guess) as a zip or tarball.
>> >
>> > My understanding was that the path is still relavant, so you'd need a /v2
>> > directory inside the zip file, which means you probably need it in your
>> > source code. However, if that is incorrect it fixes the problem.
>
> As v2 is only created due to some incompatible change, in my
> view it is better to have v2 visible in the path.

·Any· change to the exported API of a package except adding new
top-level identifiers can be considered an incompatible (breaking)
change.

https://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html


> Otherwise
> it is easy for a reader to get confused when foo.F() in one
> package (using v1) behaves differently from foo.F() in another
> package (using v2) as they both will have the same import path
> and only the "bindings" in their respective go.mod change.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Bakul Shah]

On Wed, 21 Feb 2018 20:53:20 + Henrik Johansson  
wrote:
> 
> That's not necessarily true.
> The tool can understand from the tag and assert or even rewrite the imports
> to make the compiler happy.
> 
> On Wed, Feb 21, 2018, 21:51 Sam Whited  wrote:
> 
> > On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
> > > But wait. Wasn't there a mention of archive downloads instead of relyin=
> g
> > on
> > > the different VCS's?
> > >
> > > In the GitHub case I guess it amount to downloading releases (or any
> > commit
> > > I guess) as a zip or tarball.
> >
> > My understanding was that the path is still relavant, so you'd need a /v2
> > directory inside the zip file, which means you probably need it in your
> > source code. However, if that is incorrect it fixes the problem.

As v2 is only created due to some incompatible change, in my
view it is better to have v2 visible in the path.  Otherwise
it is easy for a reader to get confused when foo.F() in one
package (using v1) behaves differently from foo.F() in another
package (using v2) as they both will have the same import path
and only the "bindings" in their respective go.mod change.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Sam Whited]

On Wed, Feb 21, 2018, at 14:57, Russ Cox wrote:
> My plan is that this is in one of tomorrows' post. Sorry for seeming like
> I'm withholding information. I'm posting as they are finished. :-)

Sounds good, I look forward to reading it and continuing to digest the 
proposal. I've had a hard time coming to grips with all the implications of 
this proposal (it being a radical departure from the current state of the art), 
but the detailed posts have been very helpful; thanks for writing about it!

—Sam

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

For people who haven't seen it yet: research.swtch.com/vgo-import is the
post Sam's referring to.

On Wed, Feb 21, 2018 at 2:22 PM, Sam Whited  wrote:

> Something that is still unclear to me even after reading todays post is
> how I would manage the various package import paths created by the import
> compatibility rule.
> The way this is currently done is through tools such as gopkg.in, will
> the import path be modified and rewritten by a package server in the future?
> If not,  do I maintain separate /v2 and /v3 trees in my directory and
> copy/paste changes between them when I want to release security fixes as a
> point release?
> Do I maintain separate branches that have a single /v2 or /v3 tree (if so,
> cherry-picking changes between them will be very difficult)?
> Do I maintain separate repos and cherry pick or copy/paste files between
> them?


> None of these sound especially appealing except perhapse having a server
> handle it for me, but this makes hosting my own packages harder.
>

My plan is that this is in one of tomorrows' post. Sorry for seeming like
I'm withholding information. I'm posting as they are finished. :-)

Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Henrik Johansson]

That's not necessarily true.
The tool can understand from the tag and assert or even rewrite the imports
to make the compiler happy.

On Wed, Feb 21, 2018, 21:51 Sam Whited  wrote:

> On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
> > But wait. Wasn't there a mention of archive downloads instead of relying
> on
> > the different VCS's?
> >
> > In the GitHub case I guess it amount to downloading releases (or any
> commit
> > I guess) as a zip or tarball.
>
> My understanding was that the path is still relavant, so you'd need a /v2
> directory inside the zip file, which means you probably need it in your
> source code. However, if that is incorrect it fixes the problem.
>
> —Sam
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Henrik Johansson]

But wait. Wasn't there a mention of archive downloads instead of relying on
the different VCS's?

In the GitHub case I guess it amount to downloading releases (or any commit
I guess) as a zip or tarball.

On Wed, Feb 21, 2018, 20:54 Sam Whited  wrote:

> On Wed, Feb 21, 2018, at 13:49, Peter Bourgon wrote:
> > > If not,  do I maintain separate /v2 and /v3 trees in my directory and
> copy/paste changes between them when I want to release security fixes as a
> point release?
> >
> > Yes, this is what vgo expects.
>
> This (and the other alternatives) seem like they break most common source
> control work flows to me. Any time we force users to change their workflow,
> I suspect we risk having to back track later. Even in cases where it may be
> better (eg. GOPATH which, like many people, I fought against when I first
> started using Go, but later came to love) if you force developers to
> drastically change their work flow, many of them will either push back, or,
> if it's only convention as this is, simply won't do it.
>
> —Sam
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Sam Whited]

On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
> But wait. Wasn't there a mention of archive downloads instead of relying on
> the different VCS's?
> 
> In the GitHub case I guess it amount to downloading releases (or any commit
> I guess) as a zip or tarball.

My understanding was that the path is still relavant, so you'd need a /v2 
directory inside the zip file, which means you probably need it in your source 
code. However, if that is incorrect it fixes the problem.

—Sam

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Sam Whited]

Something that is still unclear to me even after reading todays post is how I 
would manage the various package import paths created by the import 
compatibility rule.
The way this is currently done is through tools such as gopkg.in, will the 
import path be modified and rewritten by a package server in the future?
If not,  do I maintain separate /v2 and /v3 trees in my directory and 
copy/paste changes between them when I want to release security fixes as a 
point release?
Do I maintain separate branches that have a single /v2 or /v3 tree (if so, 
cherry-picking changes between them will be very difficult)?
Do I maintain separate repos and cherry pick or copy/paste files between them?

None of these sound especially appealing except perhapse having a server handle 
it for me, but this makes hosting my own packages harder.

—Sam

-- 
Sam Whited
s...@samwhited.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Peter Bourgon]

On Wed, Feb 21, 2018 at 11:22 AM, Sam Whited  wrote:
> Something that is still unclear to me even after reading todays post is how I 
> would manage the various package import paths created by the import 
> compatibility rule.
> The way this is currently done is through tools such as gopkg.in, will the 
> import path be modified and rewritten by a package server in the future?

No.

> If not,  do I maintain separate /v2 and /v3 trees in my directory and 
> copy/paste changes between them when I want to release security fixes as a 
> point release?

Yes, this is what vgo expects.

> Do I maintain separate branches that have a single /v2 or /v3 tree (if so, 
> cherry-picking changes between them will be very difficult)?
> Do I maintain separate repos and cherry pick or copy/paste files between them?

Both of these also seem possible, at first glance.

> None of these sound especially appealing except perhapse having a server 
> handle it for me, but this makes hosting my own packages harder.
>
> —Sam
>
> --
> Sam Whited
> s...@samwhited.com
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Zellyn]

On Wednesday, February 21, 2018 at 2:23:07 PM UTC-5, Sam Whited wrote:
>
> Something that is still unclear to me even after reading todays post is 
> how I would manage the various package import paths created by the import 
> compatibility rule. 
> The way this is currently done is through tools such as gopkg.in, will 
> the import path be modified and rewritten by a package server in the 
> future? 
> If not,  do I maintain separate /v2 and /v3 trees in my directory and 
> copy/paste changes between them when I want to release security fixes as a 
> point release? 
> Do I maintain separate branches that have a single /v2 or /v3 tree (if so, 
> cherry-picking changes between them will be very difficult)? 
> Do I maintain separate repos and cherry pick or copy/paste files between 
> them? 
>
> None of these sound especially appealing except perhapse having a server 
> handle it for me, but this makes hosting my own packages harder. 
>
> —Sam 
>

Maintaining fixes on two different branches doesn't seem much easier. I 
guess you could cherry-pick the same commit onto both branches, if it 
worked. With both versions visible in the same repository, you can send one 
PR that applies the same fix in two places, which is nice for review.

I guess every approach has pros and cons. vgo seems particularly goish :-)

Zellyn 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Sam Whited]

On Wed, Feb 21, 2018, at 13:49, Peter Bourgon wrote:
> > If not,  do I maintain separate /v2 and /v3 trees in my directory and 
> > copy/paste changes between them when I want to release security fixes as a 
> > point release?
> 
> Yes, this is what vgo expects.

This (and the other alternatives) seem like they break most common source 
control work flows to me. Any time we force users to change their workflow, I 
suspect we risk having to back track later. Even in cases where it may be 
better (eg. GOPATH which, like many people, I fought against when I first 
started using Go, but later came to love) if you force developers to 
drastically change their work flow, many of them will either push back, or, if 
it's only convention as this is, simply won't do it.

—Sam

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[sam boyer]

Unfortunately, the rules also create some new perverse incentives, with 
some, IMO, nastier failure modes and remediations. That's one of the things 
i'll be elaborating on in my doc (hopefully) next week.

That said, one of the points Russ and i agree on is that dep does allow 
more expressiveness than is wise.

On Tuesday, February 20, 2018 at 1:55:59 PM UTC-5, David Anderson wrote:
>
> I love this. I want it now.
>
> I've struggled with `glide` and `dep` dependency hell a lot in my Go 
> projects (especially those that use the Kubernetes client library, which 
> abuses "I, the library author, can define byzantine constraints on my 
> users" to the extreme). The way I've described it informally is that 
> traditional "arbitrary version selection" algorithms create the wrong 
> incentive structure for library authors, in the sense that they have all 
> the power to lock versions arbitrarily, but bear none of the pain 
> associated with their decisions - instead, the binary author at the apex of 
> the dependency tree gets that pain. "Authority without responsibility" is 
> the death of many incentive structures.
>
> The proposed rules around semver and minimal version selection address 
> every pain point I've had so far, by aligning available authority with the 
> responsibility of the library authors: tell me what versions definitely 
> will *not* work, and I'll take it from there.
>
> Questions that arose during reading:
>
>- In the "defining Go modules" section, you say: "Although semantic 
>versions are strongly preferred, referring to specific commits will be 
>supported as well." Then, you specify that such commits order before 
>v0.0.0. To me, this suggests that this feature will be useless, because 
> any 
>non-trivial example will have *some* semver constraint somewhere in 
>the dependency tree, such that in practice commit-hash versions will 
> always 
>be overridden by minimal version selection. I don't have a solution to 
> this 
>(other than removing support for commit-hash versions), but it seems like 
>it's something to think about
>   - One alternative I considered, but discarded, was to order all 
>   tags and commits by their date, and run minimal version selection on 
> that. 
>   However, this doesn't work because semver tag dates don't grow 
>   monotonically - you might have 0.1.2 released after 0.2.0, which leads 
> to 
>   confusing behavior to the user - why did 0.2.0 get selected when 0.1.2 
> was 
>   a "better" minimal version?
>- Modules as zip archives: how do I discover which versions are 
>available? Minimal version selection seems to rely on being able to list 
>the version continuum for a module, so that constraints can be applied. 
>What's the expected way to do that?
>- Writing module files sounds like a job for a machine, in cases where 
>I don't care which version gets used. Can `goimports` (or a new sibling 
>tool) be taught to update module definitions based on my code?
>- No GOPATH required: YES! Thank you. I share Zellyn's question on 
>what happens to `go install` in this world. Does it go away? Does it 
>install to some new $GOBINARYINSTALL or somesuch?
>
> Haven't read the vgo tour yet, it may answer some of these. I may be back 
> with more :)
>
> - Dave
>
>
> On Tue, Feb 20, 2018 at 9:20 AM, Russ Cox  
> wrote:
>
>> Hi everyone,
>>
>> I have a new blog post you might be interested in.
>> https://research.swtch.com/vgo.
>>
>> I'll try to watch this thread to answer any questions.
>>
>> Best,
>> Russ
>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Henrik Johansson]

Actually it seems viper imports it with lower case.
I don't remember if import paths are to be considered different if they
only differ in case.


ons 21 feb. 2018 kl 09:04 skrev Henrik Johansson :

> I am currently running `vgo build` on a decent sized project that has a
> few dependencies.
>
> It bailed on import case it seems:
>
> import "github.com/spf13/jwalterweatherman": module path of repo is
> github.com/spf13/jWalterWeatherman, not github.com/spf13/jwalterweatherman
> (wrong case)
>
> This is a transitive dep and it seems it indeed mixed upper and lower case.
> Does vgo assume lower case?
>
>
> ons 21 feb. 2018 kl 06:46 skrev :
>
>> Two more use cases of vendor that I would like to keep:
>> - A single audit trail for all go code going into production binaries in
>> a git log.
>> - A single place to enforce upgrades for multiple binaries in a monorepo.
>> Perhaps a top-level meta-module could do the trick?
>>
>> On Tuesday, February 20, 2018 at 6:42:50 PM UTC-8, Dominik Honnef wrote:
>>
>>> Hi,
>>>
>>> I'd like your input on two concern I have, one as a tool developer and
>>> one with regard to reproducible builds.
>>>
>>> With go and go get, it is expected that code will not compile until
>>> all dependencies have been explicitly downloaded. Hence, it is
>>> acceptable for tools such as staticcheck to also fail if dependencies
>>> are not present. Vgo seems to change this, by promising the user that
>>> dependencies will be automatically downloaded. Running `vgo build` at
>>> any point will "just work". Tools, however, do not have this luxury,
>>> for multiple reasons:
>>>
>>> - Multiple tools may run in parallel, either via helpers such as
>>> gometalinter, or simply because editors like Emacs launch multiple
>>> tools at once, for example when saving a file. This is likely prone to
>>> race conditions. File locking may be an option, but file locking is
>>> known to cause erratic problems on networked file systems, or be
>>> outright unsupported.
>>> - Fetching the dependencies, e.g. via 'vgo build', has the side-effect
>>> of modifying the go.mod file. This is probably unacceptable for tools
>>> that are run automatically, without the user's express wish to add new
>>> dependencies to the go.mod file.
>>> - If we do find a solution to these two problems, we'd probably still
>>> want a 'vgo get' (with no arguments), to just fetch the dependencies,
>>> without also incurring a costly build.
>>>
>>> In my opinion, tools should behave as closely as possible to (v)go
>>> build, so that users can operate on a single set of expectations.
>>>
>>> My second concern is regarding the promise of getting rid of the
>>> vendor directory. Reproducible builds not only depend on versioning,
>>> but also on ensuring that code doesn't go away, be it due to server
>>> downtime or more permanent reasons. Vendoring makes this rather easy:
>>> add your dependencies to vendor and commit them, and your project is
>>> now self-contained. Vgo seems to discourage this practice. Can you
>>> suggest an alternative that is as straightforward as vendoring? All
>>> other solutions - which amount to various forms of proxies and mirrors
>>> - are significantly more laboursome.
>>>
>>> I have noticed that currently, vgo downloads dependencies into GOPATH.
>>> Are there any plans to instead store them with the module itself,
>>> possibly in a way that facilitates committing them to version control?
>>>
>>> On Tue, Feb 20, 2018 at 6:20 PM, Russ Cox  wrote:
>>> > Hi everyone,
>>> >
>>> > I have a new blog post you might be interested in.
>>> > https://research.swtch.com/vgo.
>>> >
>>> > I'll try to watch this thread to answer any questions.
>>> >
>>> > Best,
>>> > Russ
>>> >
>>> >
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> Groups
>>> > "golang-nuts" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an
>>>
>> > email to golang-nuts...@googlegroups.com.
>>>
>> > For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Henrik Johansson]

I am currently running `vgo build` on a decent sized project that has a few
dependencies.

It bailed on import case it seems:

import "github.com/spf13/jwalterweatherman": module path of repo is
github.com/spf13/jWalterWeatherman, not github.com/spf13/jwalterweatherman
(wrong case)

This is a transitive dep and it seems it indeed mixed upper and lower case.
Does vgo assume lower case?


ons 21 feb. 2018 kl 06:46 skrev :

> Two more use cases of vendor that I would like to keep:
> - A single audit trail for all go code going into production binaries in a
> git log.
> - A single place to enforce upgrades for multiple binaries in a monorepo.
> Perhaps a top-level meta-module could do the trick?
>
> On Tuesday, February 20, 2018 at 6:42:50 PM UTC-8, Dominik Honnef wrote:
>
>> Hi,
>>
>> I'd like your input on two concern I have, one as a tool developer and
>> one with regard to reproducible builds.
>>
>> With go and go get, it is expected that code will not compile until
>> all dependencies have been explicitly downloaded. Hence, it is
>> acceptable for tools such as staticcheck to also fail if dependencies
>> are not present. Vgo seems to change this, by promising the user that
>> dependencies will be automatically downloaded. Running `vgo build` at
>> any point will "just work". Tools, however, do not have this luxury,
>> for multiple reasons:
>>
>> - Multiple tools may run in parallel, either via helpers such as
>> gometalinter, or simply because editors like Emacs launch multiple
>> tools at once, for example when saving a file. This is likely prone to
>> race conditions. File locking may be an option, but file locking is
>> known to cause erratic problems on networked file systems, or be
>> outright unsupported.
>> - Fetching the dependencies, e.g. via 'vgo build', has the side-effect
>> of modifying the go.mod file. This is probably unacceptable for tools
>> that are run automatically, without the user's express wish to add new
>> dependencies to the go.mod file.
>> - If we do find a solution to these two problems, we'd probably still
>> want a 'vgo get' (with no arguments), to just fetch the dependencies,
>> without also incurring a costly build.
>>
>> In my opinion, tools should behave as closely as possible to (v)go
>> build, so that users can operate on a single set of expectations.
>>
>> My second concern is regarding the promise of getting rid of the
>> vendor directory. Reproducible builds not only depend on versioning,
>> but also on ensuring that code doesn't go away, be it due to server
>> downtime or more permanent reasons. Vendoring makes this rather easy:
>> add your dependencies to vendor and commit them, and your project is
>> now self-contained. Vgo seems to discourage this practice. Can you
>> suggest an alternative that is as straightforward as vendoring? All
>> other solutions - which amount to various forms of proxies and mirrors
>> - are significantly more laboursome.
>>
>> I have noticed that currently, vgo downloads dependencies into GOPATH.
>> Are there any plans to instead store them with the module itself,
>> possibly in a way that facilitates committing them to version control?
>>
>> On Tue, Feb 20, 2018 at 6:20 PM, Russ Cox  wrote:
>> > Hi everyone,
>> >
>> > I have a new blog post you might be interested in.
>> > https://research.swtch.com/vgo.
>> >
>> > I'll try to watch this thread to answer any questions.
>> >
>> > Best,
>> > Russ
>> >
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>>
> > email to golang-nuts...@googlegroups.com.
>>
> > For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[jelle]

Two more use cases of vendor that I would like to keep:
- A single audit trail for all go code going into production binaries in a 
git log.
- A single place to enforce upgrades for multiple binaries in a monorepo. 
Perhaps a top-level meta-module could do the trick?

On Tuesday, February 20, 2018 at 6:42:50 PM UTC-8, Dominik Honnef wrote:
>
> Hi, 
>
> I'd like your input on two concern I have, one as a tool developer and 
> one with regard to reproducible builds. 
>
> With go and go get, it is expected that code will not compile until 
> all dependencies have been explicitly downloaded. Hence, it is 
> acceptable for tools such as staticcheck to also fail if dependencies 
> are not present. Vgo seems to change this, by promising the user that 
> dependencies will be automatically downloaded. Running `vgo build` at 
> any point will "just work". Tools, however, do not have this luxury, 
> for multiple reasons: 
>
> - Multiple tools may run in parallel, either via helpers such as 
> gometalinter, or simply because editors like Emacs launch multiple 
> tools at once, for example when saving a file. This is likely prone to 
> race conditions. File locking may be an option, but file locking is 
> known to cause erratic problems on networked file systems, or be 
> outright unsupported. 
> - Fetching the dependencies, e.g. via 'vgo build', has the side-effect 
> of modifying the go.mod file. This is probably unacceptable for tools 
> that are run automatically, without the user's express wish to add new 
> dependencies to the go.mod file. 
> - If we do find a solution to these two problems, we'd probably still 
> want a 'vgo get' (with no arguments), to just fetch the dependencies, 
> without also incurring a costly build. 
>
> In my opinion, tools should behave as closely as possible to (v)go 
> build, so that users can operate on a single set of expectations. 
>
> My second concern is regarding the promise of getting rid of the 
> vendor directory. Reproducible builds not only depend on versioning, 
> but also on ensuring that code doesn't go away, be it due to server 
> downtime or more permanent reasons. Vendoring makes this rather easy: 
> add your dependencies to vendor and commit them, and your project is 
> now self-contained. Vgo seems to discourage this practice. Can you 
> suggest an alternative that is as straightforward as vendoring? All 
> other solutions - which amount to various forms of proxies and mirrors 
> - are significantly more laboursome. 
>
> I have noticed that currently, vgo downloads dependencies into GOPATH. 
> Are there any plans to instead store them with the module itself, 
> possibly in a way that facilitates committing them to version control? 
>
> On Tue, Feb 20, 2018 at 6:20 PM, Russ Cox  
> wrote: 
> > Hi everyone, 
> > 
> > I have a new blog post you might be interested in. 
> > https://research.swtch.com/vgo. 
> > 
> > I'll try to watch this thread to answer any questions. 
> > 
> > Best, 
> > Russ 
> > 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Dominik Honnef]

Hi,

I'd like your input on two concern I have, one as a tool developer and
one with regard to reproducible builds.

With go and go get, it is expected that code will not compile until
all dependencies have been explicitly downloaded. Hence, it is
acceptable for tools such as staticcheck to also fail if dependencies
are not present. Vgo seems to change this, by promising the user that
dependencies will be automatically downloaded. Running `vgo build` at
any point will "just work". Tools, however, do not have this luxury,
for multiple reasons:

- Multiple tools may run in parallel, either via helpers such as
gometalinter, or simply because editors like Emacs launch multiple
tools at once, for example when saving a file. This is likely prone to
race conditions. File locking may be an option, but file locking is
known to cause erratic problems on networked file systems, or be
outright unsupported.
- Fetching the dependencies, e.g. via 'vgo build', has the side-effect
of modifying the go.mod file. This is probably unacceptable for tools
that are run automatically, without the user's express wish to add new
dependencies to the go.mod file.
- If we do find a solution to these two problems, we'd probably still
want a 'vgo get' (with no arguments), to just fetch the dependencies,
without also incurring a costly build.

In my opinion, tools should behave as closely as possible to (v)go
build, so that users can operate on a single set of expectations.

My second concern is regarding the promise of getting rid of the
vendor directory. Reproducible builds not only depend on versioning,
but also on ensuring that code doesn't go away, be it due to server
downtime or more permanent reasons. Vendoring makes this rather easy:
add your dependencies to vendor and commit them, and your project is
now self-contained. Vgo seems to discourage this practice. Can you
suggest an alternative that is as straightforward as vendoring? All
other solutions - which amount to various forms of proxies and mirrors
- are significantly more laboursome.

I have noticed that currently, vgo downloads dependencies into GOPATH.
Are there any plans to instead store them with the module itself,
possibly in a way that facilitates committing them to version control?

On Tue, Feb 20, 2018 at 6:20 PM, Russ Cox  wrote:
> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Wojciech S. Czarnecki]

On Tue, 20 Feb 2018 12:20:19 -0500
Russ Cox  wrote:

> https://research.swtch.com/vgo.
> Russ

Amazing! The 'minimal version selection' approach is briliant. 

Where it lacks for me is about keeping up with security patches.
If my module keeps libFoo at 1.2.3 it will not see a backported fix
that came with 1.2.98. At least not without me doing chores and
updating what I have to and leaving out what I don't.

So I'd like to see vgo to tackle also backported security patches.

Namely tags in the form of: 

x.y.z+spN where N is a security patch level.

If there is no +spN meta in the chosen tag, sp0 level is assumed.
If there is +spN, vgo SHOULD update dependency to the
highest N it sees - unless forbidden by the go.mod.

So:
require (
"rsc.io/quote" v1.5.2
)

Will use v1.5.2+sp7 if there is a such tag in the repo and 7 is the
highest sp level.

For testing (or govt certification) purposes one may
intentionally keep dependency with a hole; e.g:

"rsc.io/quote" v1.5.2 noSecurityPatchesPlease

or even

"rsc.io/quote" v1.5.2+sp3 noSecurityPatchesPlease

to have quote pinned to exact sp3 level.

In corporate environments it is not unusual to have decades old
software in production and maintenance, where 'maintenance'
is strictly regulated. Above scheme would help with that. IMO.

As vgo promises builds with tags immersed into binary any
security audit of go binaries may just list versions and compare
sp levels with ones at the source repo. If there is higher spN,
we will know something is amiss immediately - without grepping
changelogs for a CVE lines.

m0.2c

-- 
Wojciech S. Czarnecki
 << ^oo^ >> OHIR-RIPE

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Tom Mitchell]

On Tue, Feb 20, 2018 at 2:29 PM, Bakul Shah  wrote:

> On Tue, 20 Feb 2018 13:02:47 -0800 David Anderson 
> wrote:
> >
> > As a counterpoint to this: with vgo, I plan to make all my binary modules
> > specify that they want the latest versions of everything,
>
.

>
> The right choice here is not clear (at least to me).
>
.

>
> Usually more stable versions are tagged and "released"
>

My bias is to use the latest "released" tools libraries and source.
Released code has more interactions and probably a better chance
for the community discovery of an issue you care about.

Build and test often as part of the process.

It is not just the libraries but also the compiler and build system itself.

I spent a month+ trying to sort out a customers application bug that
resulted from promotion of an optimization to an intrinsic in a compiler
update.

Compilers and applications can be victims of a system lib change and more
including system test
resources.

This is a difficult problem and the answer involves knowing (best guess)
what it takes to debug the
entire pile of worms not just the application.  It is turtles all the way
down but one
possible solution is to put all the worms you can in a virtual machine that
can be canned and archived.
A 4.7 GB DVD or a 25 GB Blu-Ray archival quality disk for the self
contained
single purpose built VM.   A 25GB system is worthy for a lot of projects
but not all.


-- 
  T o mM i t c h e l l

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Bakul Shah]

On Tue, 20 Feb 2018 13:02:47 -0800 David Anderson  wrote:
> 
> As a counterpoint to this: with vgo, I plan to make all my binary modules
> specify that they want the latest versions of everything, and update that
> pinning every couple of weeks/months. If I encounter bugs, I'll report
> them, and move the pinning back a little, safe in the knowledge that my
> dependencies will tell me if I've moved it too far back.
> 
> I also believe the tooling around vgo should encourage/make default this
> behavior for binary modules (and maybe for library modules as well, though
> that's less clear to me). The default behavior when writing Go programs
> *should* be to use the latest of everything, unless you have a specific
> reason to hold back. Among other things, that ensures you are patched
> against CVEs and whatnot.

The right choice here is not clear (at least to me).  You do
want the latest security patches and you will want bug fixes
but you don't necessarily want the latest functional changes
as APIs may not have settled down or there may be more bugs
to be shaken out.

Usually more stable versions are tagged and "released" but you
may still choose to go with not the latest released version
but one or more earlier releases. Then there are issues such
as needing to run s/w on older versions of OS (not everyone
updates to the latest versions -- in large organizations the
delay can be very long) and often leading edge s/w will drop
support for older OS versions so ideally your s/w should work
on last 2-4 versions of an OS.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Dave MacFarlane]

I really like this, except for the claim that it the blog post that it
will eliminate vendoring and deprecate GOPATH and the problems that
will cause for backwards compatibility for things that are currently
using them. If this is going to result in removing language features
(ie. vendoring), shouldn't it be in Go2, not Go 1.11?

I've abused vendoring in the past for things like fixing bugs in
upstream libraries that I use where the maintainer hasn't been
responsive to pull requests and, while the "replace = .." of vgo is
definitely a better solution, removing vendoring would mean that I
would need to go through every repo I have to ensure they keep
building once this goes in.

The other question I have is: where does the module name come from if
you don't have a "// import" comment on the package, since it can't be
filesystem based (because you're explicitly outside of GOPATH..)

- Dave




On Tue, Feb 20, 2018 at 12:20 PM, Russ Cox  wrote:
> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
- Dave

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[David Anderson]

I think we're agreeing, with different words (but please correct me if I'm
wrong!).

Yes, active maintenance is required with vgo, as demonstrated in the tour.
But crucially, I do not get held back by library writers that don't update,
I'm only held back by my own laziness in failing to update at the apex of
the dependency tree. That's something I can automate away. It probably
means I'll be a beta tester for a new combination of versions more often
than not. But `vgo test` helps with that, and generally encourages the
writing of useful tests, so again I think the incentives all align. And
when they don't, there's the replacement mechanism to fork in place.

- Dave

On Tue, Feb 20, 2018 at 1:20 PM, Devon H. O'Dell 
wrote:

> 2018-02-20 13:02 GMT-08:00 David Anderson :
> > [snip]
> >
> > I also believe the tooling around vgo should encourage/make default this
> > behavior for binary modules (and maybe for library modules as well,
> though
> > that's less clear to me). The default behavior when writing Go programs
> > should be to use the latest of everything, unless you have a specific
> reason
> > to hold back. Among other things, that ensures you are patched against
> CVEs
> > and whatnot.
>
> AIUI, that is the opposite of what minimum version selection does;
> active maintenance is required. This is at least no worse than any
> other case where you need to make a determination on whether to
> consume some library or tool based on whether it is actively
> maintained. And when the tools are maintained, seems like it is
> probably better.
>
> --dho
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Devon H. O'Dell]

2018-02-20 13:02 GMT-08:00 David Anderson :
> [snip]
>
> I also believe the tooling around vgo should encourage/make default this
> behavior for binary modules (and maybe for library modules as well, though
> that's less clear to me). The default behavior when writing Go programs
> should be to use the latest of everything, unless you have a specific reason
> to hold back. Among other things, that ensures you are patched against CVEs
> and whatnot.

AIUI, that is the opposite of what minimum version selection does;
active maintenance is required. This is at least no worse than any
other case where you need to make a determination on whether to
consume some library or tool based on whether it is actively
maintained. And when the tools are maintained, seems like it is
probably better.

--dho

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Bakul Shah]

On Tue, 20 Feb 2018 15:51:39 -0500 Russ Cox  wrote:
> 
> On Tue, Feb 20, 2018 at 3:46 PM, Bakul Shah  wrote:
> 
> > May be one can write a tool around or similar to "vgo test" to
> > test and update not to the latest but some earlier version.
> > For example, if pkg A currently imports C v1.x and B imports C
> > v1.y, through iterative testing one may find C v1.z that works
> > for both and update go.mod files for both A & B.
> >
> > The underlying concern is that with better tooling to manage
> > versioning, there will also be more versions to manage as
> > there is less of an incentive to try to make things work with
> > the latest versions of imported packages.  I wondered if there
> > is a way to encourage/improve the quality/correctness aspect.
> > One vague idea is to consider specifying tests (in go.mod) to
> > be run prior to an update.  Sort of like asserts in code.
> 
> In the tour (research.swtch.com/vgo-tour) I show how "all" is redefined to
> be useful again. So it's completely reasonable to try an update, go test
> all, and if all the tests pass (and you trust your tests), then check in
> the go.mod file. Someone could build a bot to do this automatically, even.
> I don't think minimal version selection means you're always stuck with old
> versions. It just means you don't get new versions until you ask for them
> and are ready to evaluate how good they are, not just because you run 'go
> get' and a new version has appeared overnight.

Not automatically fetching the latest version is the right
choice.  Minimal version selection is also something I agree
with.  But an unintended consequence of this choice may be
binaries of large packages pulling in multiple versions.  So I
was wondering what can be done to encourage periodic updates.

I just wanted to provide this feedback but it may be too early
to worry about this.  As vgo is used more and more may be the
right tools will organically evolve!

Thanks,
Bakul

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[David Anderson]

On Tue, Feb 20, 2018 at 11:37 AM, Russ Cox  wrote:

> On Tue, Feb 20, 2018 at 1:55 PM, David Anderson  wrote:
>
>> I love this. I want it now.
>>
>
> go get -u golang.org/x/vgo :-)
>
> I've struggled with `glide` and `dep` dependency hell a lot in my Go
>> projects (especially those that use the Kubernetes client library, which
>> abuses "I, the library author, can define byzantine constraints on my
>> users" to the extreme). The way I've described it informally is that
>> traditional "arbitrary version selection" algorithms create the wrong
>> incentive structure for library authors, in the sense that they have all
>> the power to lock versions arbitrarily, but bear none of the pain
>> associated with their decisions - instead, the binary author at the apex of
>> the dependency tree gets that pain. "Authority without responsibility" is
>> the death of many incentive structures.
>>
>
> Yes, I'll make that point again in tomorrow's posts elaborating minimal
> version selection, but I think this is probably the most important
> algorithmic policy detail. You get total control over your own build. You
> only get limited control over other people's builds that happen to import
> your code.
>

Big +1. I'm curious about your ideas on how to manage the transition from a
glide/dep world to a vgo world. I look forward to reading more about that,
assuming you have an article queued to discuss that facet :).


>
> Questions that arose during reading:
>>
>>- In the "defining Go modules" section, you say: "Although semantic
>>versions are strongly preferred, referring to specific commits will be
>>supported as well." Then, you specify that such commits order before
>>v0.0.0. To me, this suggests that this feature will be useless, because 
>> any
>>non-trivial example will have *some* semver constraint somewhere in
>>the dependency tree, such that in practice commit-hash versions will 
>> always
>>be overridden by minimal version selection. I don't have a solution to 
>> this
>>(other than removing support for commit-hash versions), but it seems like
>>it's something to think about
>>
>> That's by design: if v1.2.3 is the latest tagged version and there's some
> experimental new stuff that you choose to depend on, and then vgo has to
> decide between those two requirements coming from two different
> dependencies in a larger build, I want it to pick the tagged one. That
> said, for builds of your own module, what you'd do is define that you want
> to replace the current latest (say, v1.2.3) with a different commit. In
> go.mod you'd say:
>
> replace "p.xyz/foo" v1.2.3 => "p.xyz/foo" commithash
>
> and then the version selection would transparently drop in commithash as
> if it were really v1.2.3, so that it would win over v1.2.2, but not v1.2.4.
> I suppose you could even make up a very large number like v1..0 and
> then depend on that. But again the replacement only applies to your build,
> not to other builds using your module.
>

That makes sense, thanks. What I was looking for was: it is impossible for
libraries to "work around" this system by using, say, `dep` to resolve
versions using more complex constraints, and then lock those versions in by
commit hash for all its consumers. But I now see you've explicitly thought
about that, and we can get the best of both worlds - apex modules can lock
to commits in order to work around bugs or achieve non-technical goals, but
library authors cannot impose arbitrary locks.


>
>>- Modules as zip archives: how do I discover which versions are
>>available? Minimal version selection seems to rely on being able to list
>>the version continuum for a module, so that constraints can be applied.
>>What's the expected way to do that?
>>
>> Yes, you have to have a way to list them. I'll write more about that
> Thursday.
>

One thought that I hope you'll cover there: immutability. Reading through
the vgo tour, it seems to assume that once v5.6.7 has been released, that
.zip and its declared dependencies will never change.

That is indeed what semver requires you to do, but I wonder how often
"oopsie" reuploads happen at the same version, and how well/poorly vgo
behaves under those conditions. IMO it's probably fine to say "wontfix,
don't do that", as long as it's loudly documented.


>
>>- Writing module files sounds like a job for a machine, in cases
>>where I don't care which version gets used. Can `goimports` (or a new
>>sibling tool) be taught to update module definitions based on my code?
>>
>> Goimports just needs to put the import in. "vgo build" will see the new
> import and take care of the rest. It rewrites go.mod for you (and formats
> it too, of course).
>

Excellent. I expected nothing less from Go tooling, but it's good to know
for sure :).


>
> Best,
> Russ
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To 

Re: [go-nuts] Go += Package Versioning

[David Anderson]

On Tue, Feb 20, 2018 at 12:46 PM, Bakul Shah  wrote:

> On Tue, 20 Feb 2018 11:55:00 -0800 "Devon H. O'Dell" <
> devon.od...@gmail.com> wrote:
> Devon H. O'Dell writes:
> > 2018-02-20 11:39 GMT-08:00 Russ Cox :
> > > On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell <
> devon.od...@gmail.com>
> > > wrote:
> > >>
> > >> With regards to minimum version selection, if I depend on some feature
> > >> present, this comes with two implicit dependencies: correctness and
> > >> safety. My knee-jerk reaction here is that the time folks spend
> > >> "telling the package manager, 'no, use at least Y,'" will largely be
> > >> in response to these sorts of issues. Maybe this is seen as "working
> > >> as intended;" my feeling is that this will become tiresome for
> > >> projects with large dependency chains. Might it be worthwhile (and
> > >> automatically possible) to pick the "maximally correct" minimum
> > >> version?
> > >
> > > I am not sure what you mean by "maximally correct".
> > > I think that's approximately what all the other package managers try
> to do.
> > > It makes things very complex (literally:
> research.swtch.com/version-sat).
> >
> > Thanks. I was thinking in terms of e.g. a "correctness counter" that
> > could be published. If I depend on e.g. foo.Bar, I automatically want
> > the least buggy version of foo.Bar. If foo.Bar appears in version 1,
> > the amount of work required to find the least buggy version is
> > directly proportional to the number of releases in which foo.Bar
> > appears, and is annotated as having some correctness fix. I don't
> > think this would necessarily be as complex as you point out in the
> > referenced post: this is a property of an individual package. Of
> > course, you want the same property for each package's dependencies,
> > but that can be decided without conflict using the same methodology
> > for each package. That is, I don't think there's a case for package X
> > with a dependency on package Y where the minimum version can't be
> > decided in linear time.
>
> I had a somewhat similar reaction as Devon.
>
> Reproducible builds are indeed very desirable but, as Devon
> hints at, it is also the case that differences between
> versions grow over time and implicit assumptions may break.
> For example, updating from v1.1 to v1.5 and then v1.10 may be
> easier than updating from v1.1 directly to v1.10. That is,
> updating version depdendencies from time to time may be
> considered part of regular maintenance.
>
> *Once* you release a package you want its dependencies
> explicitly stated for reproducible builds but when creating
> the next release of the same package, you may want to update
> to later versions of imported packages due to the above
> considerations.
>
> May be one can write a tool around or similar to "vgo test" to
> test and update not to the latest but some earlier version.
> For example, if pkg A currently imports C v1.x and B imports C
> v1.y, through iterative testing one may find C v1.z that works
> for both and update go.mod files for both A & B.
>
> The underlying concern is that with better tooling to manage
> versioning, there will also be more versions to manage as
> there is less of an incentive to try to make things work with
> the latest versions of imported packages.


As a counterpoint to this: with vgo, I plan to make all my binary modules
specify that they want the latest versions of everything, and update that
pinning every couple of weeks/months. If I encounter bugs, I'll report
them, and move the pinning back a little, safe in the knowledge that my
dependencies will tell me if I've moved it too far back.

I also believe the tooling around vgo should encourage/make default this
behavior for binary modules (and maybe for library modules as well, though
that's less clear to me). The default behavior when writing Go programs
*should* be to use the latest of everything, unless you have a specific
reason to hold back. Among other things, that ensures you are patched
against CVEs and whatnot.

In my mind's eye, the version declarations that modules make for
consumption by other modules is the tail end of the world's sliding window
of "what set of stuff can build together". It should not be the default
place that most Go binaries live.

- Dave


>   I wondered if there
> is a way to encourage/improve the quality/correctness aspect.
> One vague idea is to consider specifying tests (in go.mod) to
> be run prior to an update.  Sort of like asserts in code.
>
> Apart from this concern vgo seems like a huge improvement.
> Well done!
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

On Tue, Feb 20, 2018 at 3:46 PM, Bakul Shah  wrote:

> May be one can write a tool around or similar to "vgo test" to
> test and update not to the latest but some earlier version.
> For example, if pkg A currently imports C v1.x and B imports C
> v1.y, through iterative testing one may find C v1.z that works
> for both and update go.mod files for both A & B.
>
> The underlying concern is that with better tooling to manage
> versioning, there will also be more versions to manage as
> there is less of an incentive to try to make things work with
> the latest versions of imported packages.  I wondered if there
> is a way to encourage/improve the quality/correctness aspect.
> One vague idea is to consider specifying tests (in go.mod) to
> be run prior to an update.  Sort of like asserts in code.
>

In the tour (research.swtch.com/vgo-tour) I show how "all" is redefined to
be useful again. So it's completely reasonable to try an update, go test
all, and if all the tests pass (and you trust your tests), then check in
the go.mod file. Someone could build a bot to do this automatically, even.
I don't think minimal version selection means you're always stuck with old
versions. It just means you don't get new versions until you ask for them
and are ready to evaluate how good they are, not just because you run 'go
get' and a new version has appeared overnight.

Thanks.
Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Bakul Shah]

On Tue, 20 Feb 2018 11:55:00 -0800 "Devon H. O'Dell"  
wrote:
Devon H. O'Dell writes:
> 2018-02-20 11:39 GMT-08:00 Russ Cox :
> > On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell 
> > wrote:
> >>
> >> With regards to minimum version selection, if I depend on some feature
> >> present, this comes with two implicit dependencies: correctness and
> >> safety. My knee-jerk reaction here is that the time folks spend
> >> "telling the package manager, 'no, use at least Y,'" will largely be
> >> in response to these sorts of issues. Maybe this is seen as "working
> >> as intended;" my feeling is that this will become tiresome for
> >> projects with large dependency chains. Might it be worthwhile (and
> >> automatically possible) to pick the "maximally correct" minimum
> >> version?
> >
> > I am not sure what you mean by "maximally correct".
> > I think that's approximately what all the other package managers try to do.
> > It makes things very complex (literally: research.swtch.com/version-sat).
> 
> Thanks. I was thinking in terms of e.g. a "correctness counter" that
> could be published. If I depend on e.g. foo.Bar, I automatically want
> the least buggy version of foo.Bar. If foo.Bar appears in version 1,
> the amount of work required to find the least buggy version is
> directly proportional to the number of releases in which foo.Bar
> appears, and is annotated as having some correctness fix. I don't
> think this would necessarily be as complex as you point out in the
> referenced post: this is a property of an individual package. Of
> course, you want the same property for each package's dependencies,
> but that can be decided without conflict using the same methodology
> for each package. That is, I don't think there's a case for package X
> with a dependency on package Y where the minimum version can't be
> decided in linear time.

I had a somewhat similar reaction as Devon.

Reproducible builds are indeed very desirable but, as Devon
hints at, it is also the case that differences between
versions grow over time and implicit assumptions may break.
For example, updating from v1.1 to v1.5 and then v1.10 may be
easier than updating from v1.1 directly to v1.10. That is,
updating version depdendencies from time to time may be
considered part of regular maintenance.

*Once* you release a package you want its dependencies
explicitly stated for reproducible builds but when creating
the next release of the same package, you may want to update
to later versions of imported packages due to the above
considerations.

May be one can write a tool around or similar to "vgo test" to
test and update not to the latest but some earlier version.
For example, if pkg A currently imports C v1.x and B imports C
v1.y, through iterative testing one may find C v1.z that works
for both and update go.mod files for both A & B.

The underlying concern is that with better tooling to manage
versioning, there will also be more versions to manage as
there is less of an incentive to try to make things work with
the latest versions of imported packages.  I wondered if there
is a way to encourage/improve the quality/correctness aspect.
One vague idea is to consider specifying tests (in go.mod) to
be run prior to an update.  Sort of like asserts in code.

Apart from this concern vgo seems like a huge improvement.
Well done!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Devon H. O'Dell]

2018-02-20 11:39 GMT-08:00 Russ Cox :
> On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell 
> wrote:
>>
>> With regards to minimum version selection, if I depend on some feature
>> present, this comes with two implicit dependencies: correctness and
>> safety. My knee-jerk reaction here is that the time folks spend
>> "telling the package manager, 'no, use at least Y,'" will largely be
>> in response to these sorts of issues. Maybe this is seen as "working
>> as intended;" my feeling is that this will become tiresome for
>> projects with large dependency chains. Might it be worthwhile (and
>> automatically possible) to pick the "maximally correct" minimum
>> version?
>
> I am not sure what you mean by "maximally correct".
> I think that's approximately what all the other package managers try to do.
> It makes things very complex (literally: research.swtch.com/version-sat).

Thanks. I was thinking in terms of e.g. a "correctness counter" that
could be published. If I depend on e.g. foo.Bar, I automatically want
the least buggy version of foo.Bar. If foo.Bar appears in version 1,
the amount of work required to find the least buggy version is
directly proportional to the number of releases in which foo.Bar
appears, and is annotated as having some correctness fix. I don't
think this would necessarily be as complex as you point out in the
referenced post: this is a property of an individual package. Of
course, you want the same property for each package's dependencies,
but that can be decided without conflict using the same methodology
for each package. That is, I don't think there's a case for package X
with a dependency on package Y where the minimum version can't be
decided in linear time.

Upon further thought, I realize folks will occasionally rely on
particular forms of buggy behavior. I think that's probably fine if
you can always require a specific version. However, it'd probably
require some kinds of annotations on every portion of the API to work
well: if I rely only on foo.Bar, and there's no dependency on foo.Bat
which has a correctness change, that shouldn't leak into foo.Bar. That
is probably an unreasonable amount of work to require from developers,
so probably just ignore me :)

--dho

> Note that it's still super easy to say "update this package to latest"
> or "update everything to latest". It just only happens when you ask,
> not when the tool feels like it.
>
> Russ
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell 
wrote:

> With regards to minimum version selection, if I depend on some feature
> present, this comes with two implicit dependencies: correctness and
> safety. My knee-jerk reaction here is that the time folks spend
> "telling the package manager, 'no, use at least Y,'" will largely be
> in response to these sorts of issues. Maybe this is seen as "working
> as intended;" my feeling is that this will become tiresome for
> projects with large dependency chains. Might it be worthwhile (and
> automatically possible) to pick the "maximally correct" minimum
> version?
>

I am not sure what you mean by "maximally correct".
I think that's approximately what all the other package managers try to do.
It makes things very complex (literally: research.swtch.com/version-sat).

Note that it's still super easy to say "update this package to latest"
or "update everything to latest". It just only happens when you ask,
not when the tool feels like it.

Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Russ Cox]

On Tue, Feb 20, 2018 at 1:55 PM, David Anderson  wrote:

> I love this. I want it now.
>

go get -u golang.org/x/vgo :-)

I've struggled with `glide` and `dep` dependency hell a lot in my Go
> projects (especially those that use the Kubernetes client library, which
> abuses "I, the library author, can define byzantine constraints on my
> users" to the extreme). The way I've described it informally is that
> traditional "arbitrary version selection" algorithms create the wrong
> incentive structure for library authors, in the sense that they have all
> the power to lock versions arbitrarily, but bear none of the pain
> associated with their decisions - instead, the binary author at the apex of
> the dependency tree gets that pain. "Authority without responsibility" is
> the death of many incentive structures.
>

Yes, I'll make that point again in tomorrow's posts elaborating minimal
version selection, but I think this is probably the most important
algorithmic policy detail. You get total control over your own build. You
only get limited control over other people's builds that happen to import
your code.

Questions that arose during reading:
>
>- In the "defining Go modules" section, you say: "Although semantic
>versions are strongly preferred, referring to specific commits will be
>supported as well." Then, you specify that such commits order before
>v0.0.0. To me, this suggests that this feature will be useless, because any
>non-trivial example will have *some* semver constraint somewhere in
>the dependency tree, such that in practice commit-hash versions will always
>be overridden by minimal version selection. I don't have a solution to this
>(other than removing support for commit-hash versions), but it seems like
>it's something to think about
>
> That's by design: if v1.2.3 is the latest tagged version and there's some
experimental new stuff that you choose to depend on, and then vgo has to
decide between those two requirements coming from two different
dependencies in a larger build, I want it to pick the tagged one. That
said, for builds of your own module, what you'd do is define that you want
to replace the current latest (say, v1.2.3) with a different commit. In
go.mod you'd say:

replace "p.xyz/foo" v1.2.3 => "p.xyz/foo" commithash

and then the version selection would transparently drop in commithash as if
it were really v1.2.3, so that it would win over v1.2.2, but not v1.2.4. I
suppose you could even make up a very large number like v1..0 and then
depend on that. But again the replacement only applies to your build, not
to other builds using your module.

>
>- Modules as zip archives: how do I discover which versions are
>available? Minimal version selection seems to rely on being able to list
>the version continuum for a module, so that constraints can be applied.
>What's the expected way to do that?
>
> Yes, you have to have a way to list them. I'll write more about that
Thursday.

>
>- Writing module files sounds like a job for a machine, in cases where
>I don't care which version gets used. Can `goimports` (or a new sibling
>tool) be taught to update module definitions based on my code?
>
> Goimports just needs to put the import in. "vgo build" will see the new
import and take care of the rest. It rewrites go.mod for you (and formats
it too, of course).

Best,
Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Devon H. O'Dell]

With regards to minimum version selection, if I depend on some feature
present, this comes with two implicit dependencies: correctness and
safety. My knee-jerk reaction here is that the time folks spend
"telling the package manager, 'no, use at least Y,'" will largely be
in response to these sorts of issues. Maybe this is seen as "working
as intended;" my feeling is that this will become tiresome for
projects with large dependency chains. Might it be worthwhile (and
automatically possible) to pick the "maximally correct" minimum
version?

--dho

On Tue, Feb 20, 2018 at 9:20 AM Russ Cox  wrote:
>
> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[David Anderson]

I love this. I want it now.

I've struggled with `glide` and `dep` dependency hell a lot in my Go
projects (especially those that use the Kubernetes client library, which
abuses "I, the library author, can define byzantine constraints on my
users" to the extreme). The way I've described it informally is that
traditional "arbitrary version selection" algorithms create the wrong
incentive structure for library authors, in the sense that they have all
the power to lock versions arbitrarily, but bear none of the pain
associated with their decisions - instead, the binary author at the apex of
the dependency tree gets that pain. "Authority without responsibility" is
the death of many incentive structures.

The proposed rules around semver and minimal version selection address
every pain point I've had so far, by aligning available authority with the
responsibility of the library authors: tell me what versions definitely
will *not* work, and I'll take it from there.

Questions that arose during reading:

   - In the "defining Go modules" section, you say: "Although semantic
   versions are strongly preferred, referring to specific commits will be
   supported as well." Then, you specify that such commits order before
   v0.0.0. To me, this suggests that this feature will be useless, because any
   non-trivial example will have *some* semver constraint somewhere in the
   dependency tree, such that in practice commit-hash versions will always be
   overridden by minimal version selection. I don't have a solution to this
   (other than removing support for commit-hash versions), but it seems like
   it's something to think about
  - One alternative I considered, but discarded, was to order all tags
  and commits by their date, and run minimal version selection on that.
  However, this doesn't work because semver tag dates don't grow
  monotonically - you might have 0.1.2 released after 0.2.0, which leads to
  confusing behavior to the user - why did 0.2.0 get selected when
0.1.2 was
  a "better" minimal version?
   - Modules as zip archives: how do I discover which versions are
   available? Minimal version selection seems to rely on being able to list
   the version continuum for a module, so that constraints can be applied.
   What's the expected way to do that?
   - Writing module files sounds like a job for a machine, in cases where I
   don't care which version gets used. Can `goimports` (or a new sibling tool)
   be taught to update module definitions based on my code?
   - No GOPATH required: YES! Thank you. I share Zellyn's question on what
   happens to `go install` in this world. Does it go away? Does it install to
   some new $GOBINARYINSTALL or somesuch?

Haven't read the vgo tour yet, it may answer some of these. I may be back
with more :)

- Dave


On Tue, Feb 20, 2018 at 9:20 AM, Russ Cox  wrote:

> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Go += Package Versioning

[Russ Cox]

Hi everyone,

I have a new blog post you might be interested in.
https://research.swtch.com/vgo.

I'll try to watch this thread to answer any questions.

Best,
Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.