Re: [go-nuts] [generics] Data structures and the garbage collector

2020-07-11 Thread t hepudds
Hello Markus,

> "But apparently (see Ian's answer) this is not the case, and the GC 
always needs to trace all elements, keys and values."

I think Ian said something a bit different from what you said there.

As far as I understand, the GC does not scan the keys and values of a map 
if they are not pointers and do not contain pointers. See for example 
https://go-review.googlesource.com/c/go/+/3288

> "so the payload type is interface{} now."

For your case, it sounds like with generics you could substantially reduce 
the total count of pointers used by your data structure by storing the 
value directly as Ian said, rather than using interface{}.

Separately, a great overview of the GC system is in this golang.org blog 
post by Rick Hudson:

https://blog.golang.org/ismmkeynote

Also, the code itself is very readable, including there are some large 
comments that give overviews, such as:

https://github.com/golang/go/blob/master/src/runtime/mgc.go#L5
  
Those are worth a quick read if you are interested in more details of how 
the GC works.

Regards,
thepudds

On Friday, July 10, 2020 at 12:48:05 PM UTC-4, Markus Heukelom wrote:
>
>
> Many data structures are implemented using pointers. How does this perform 
>>> wrt garbage collecting? For example take any data structure that uses some 
>>> tree of nodes, or even simpler just a singly linked list: there is one 
>>> pointer from the left node to the next. Wouldn't this incur huge GC 
>>> (unavoidable) cycles for large linked lists that will look at all pointers? 
>>> I can imagine that a large AVL tree would then cause potentially large 
>>> hick-ups in the GC cycle? 
>>>
>>>
>> Is your worry latency in the form of pause times while doing GC, or 
>> bandwidth in the form of CPU cycles when doing GC? It isn't entirely clear.
>>
>>
> My "worry" is about CPU cycles that would not be (necessarily) needed in 
> let's say C++ or C. The fact that Go has a concurrent GC resulting in no 
> "stop the world"-hick-ups is really awesome, but was not my main point. 
>
> (btw I don't really "worry" about it for any real applications; it was 
> more a question out of interest)
>
> The reason I asked is because I use a custom tree data-structure that 
> manages parent-child, and leaf node prev-next, relationships using 
> pointers. I use the tree for multiple value types, so the payload type is 
> interface{} now. It would be a great place for me to use generics; this 
> would make my code more clear and safer. 
>
> In any case, you are right that these things require attention in a 
>> garbage collector, both of them. But Go has addressed this for many years 
>> now.
>>
>> As an example, imagine you have a binary AVL or Red/Black tree taking up 
>> 128 Gigabyte of memory space. You now delete the root node of said tree.
>>
>> Example 1: in C++ we have reference counted the tree. Deleting the root 
>> node sets its childrens refcounts to 0, so they get deleted, and so on 
>> recursively down the tree. It might take some time to delete 128 Gigabyte 
>> of data this way. So C++ isn't exempt from memory reclamation pauses either.
>>
>>
> In C/C++ you could better use an arena-allocator and not refcounts. 
> Clearing the tree would then use only (nodes / (nodes per region)) 
> deallocation calls. As the region is typically large, this would probably 
> be orders of magnitude quicker. I believe compilers use arena allocators 
> for this reason. (Deleting individual nodes would be cheaper maybe as 
> well.) 
>
> Refcounting I think is best reserved for situations where the lifetime of 
> an object is not determined by, or escapes, the package (class internals in 
> C++), or for reasons  of simplicity of implementation of course.
>
> I was under the (false) impression that one of the reasons of having a 
> built-in array, slice and map type in Go (besides it just being really 
> handy) is that for example a map[string]int would relieve the GC needing to 
> trace the keys and values (in contrast to for example map[int]*string). But 
> apparently (see Ian's answer) this is not the case, and the GC always needs 
> to trace all elements, keys and values. In that case there is (luckily) no 
> reason to choose build-in map over a custom map written with generics.
>  
>
>> Example 2: Imagine a language using Cheney's copying collection as the 
>> main garbage collection scheme. It will use 256 gigabyte of memory for its 
>> two copying spaces, but collection time will be 0, since the tree's data 
>> isn't live. You are paying with memory for faster collection in this case.
>>
>> Example 3: Concurrent collectors, like the one Go uses nowadays, will 
>> collect the tree while the program is running and doing productive work. 
>> While doing so incurs a bandwidth loss, latency pause times do not suffer. 
>> If you have spare CPU cores, they'll be used to handle the collection while 
>> the remaining cores run the program (more or less, this is a gross 
>> simplification).
>>
>> 

Re: [go-nuts] Generics feedback

2020-06-28 Thread t hepudds
Hello Calum,

One FYI that Tyler Compton pulled together a helpful list of dozen or so 
different alternative generics syntax suggestions along with their 
corresponding recent golang-nuts threads:

  https://groups.google.com/d/msg/golang-nuts/uQDrcHDwT_w/Y-Myzuw_AQAJ
  
That could be a helpful starting point, including to see what the prior 
response might have been from Ian Lance Taylor and others.

One link listed there that is especially worth a read is the second link in 
the list there, which is a longer explanation from Ian on his rationale for 
him personally wanting to take a "wait and see" approach on the syntax. 

For your particular example, I think it reads slightly cleaner in the Go2Go 
Playground:

  https://go2goplay.golang.org/p/S3Fgmgmblr0
  
The fixed-width font, the 2 extra spaces added by gofmt (compared to how 
you had formatted it in your post), and the 'Print' function that parallels 
your 'Combine' function all help at least slightly. That said, I don't 
personally know if the current syntax is the best syntax, but it also seems 
that the people who have spent the most time on the syntax are also saying 
they don't know if the current syntax is the best syntax and that time and 
more real examples are needed before really anyone will know.

In some sense, rather than you being "late to the party", you might 
actually be early to the party ;-)

Regards,
thepudds

On Sunday, June 28, 2020 at 8:58:56 AM UTC-4, Calum Shaw-Mackay wrote:
>
>
> " I believe over time, it will a) become clear that generic code will be 
> less common than people think (I hope) and b) that you get used to the 
> syntax either way. (also, yes, this has been discussed before, ad nauseam 
> in fact :) )”
>
> Yes I also hope that the need to generify everything is kept to an 
> absolute minimum.
>
> Apologies about discussing things already spoken about, I have arrived 
> fairly late to this party :)
>
>
> On 28 Jun 2020, at 12:23, 'Axel Wagner' via golang-nuts <
> golan...@googlegroups.com > wrote:
>
> Readability, at the end of the day, is subjective.
> So, personally: Yes, I absolutely find it more readable than any 
> alternative that has been suggested. Personally, when I see code using a 
> bunch of extra symbols that have special meaning (Perl and Haskell are 
> extreme examples), I tend to "zone out" and find it really hard to actually 
> make the effort of trying to parse it. I really like Go's general approach 
> of using fewer separators and keywords to convey meaning.
>
> But in any case: I believe over time, it will a) become clear that generic 
> code will be less common than people think (I hope) and b) that you get 
> used to the syntax either way. (also, yes, this has been discussed before, 
> ad nauseam in fact :) )
>
> On Sun, Jun 28, 2020 at 11:16 AM Calum Shaw-Mackay  > wrote:
>
>> Hi all -
>>
>> I know that there’s have been numerous threads regarding the syntax for 
>> declaring generic types etc, and at it’s core Go is a language that can do 
>> a lot without syntactic sugar just for the sake of it, but sometimes that 
>> syntactic sugar helps in a fundamental way - legibility.
>>
>> Observe this function declaration
>>
>> func Combine(type T)(s []T) (T,error){
>>  ….
>> }
>>
>> Do we think that 3 consecutive clauses wrapped in parentheses is legible, 
>> that the intent of both the function and its constraints are easily 
>> discernible?
>> To me, and it is only my opinion, unless (type T) has different 
>> ‘versions’ i.e. (struct T) or (func T), isn’t "(type " not only redundant 
>> but reduces readability?
>>
>> Apologies if this has been discussed before.
>>
>> Calum
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/6661399B-BF48-4D15-802C-E2B1C6C0F348%40gmail.com
>> .
>>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golan...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfHO0hvNKT8tVM2tXk1trDbRFqJaGH%3DWtParweJiJQNQbg%40mail.gmail.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1d7cbc1f-5351-4dda-8018-c019808e70f7o%40googlegroups.com.


Re: [go-nuts] [generics] Syntax feedback

2020-06-27 Thread t hepudds
Hi Tyler,

I just wanted to say a quick thank you for pulling those references together 
and your thoughtful response in this thread. 

Hopefully the time you spent doing that will help additional gophers as the 
generics discussion continues on this list.

Regards,
thepudds

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e2c1cb6a-06e7-48ad-9dce-145513ac0b61o%40googlegroups.com.


[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

2020-02-13 Thread t hepudds
Hello Dan,

Sorry, one quick clarification -- I had misremembered which image packages 
are already running in google/oss-fuzz. 

Of the three image packages you mentioned, I think it is just gif that is 
missing currently (and not both gif and jpeg missing):

https://github.com/google/oss-fuzz/blob/master/projects/golang/build.sh#L32

thepudds

On Thursday, February 13, 2020 at 1:40:58 PM UTC-5, t hepudds wrote:
>
> Hello Dan,
>
> I would definitely echo the fuzzing suggestion from Jake.
>
> In addition, in your quest to find broken images, I would suggest grabbing 
> the images from the dvyukov/go-fuzz corpus, which has a bunch of images 
> that are already broken in "creative" ways for you based on the 
> coverage-guided fuzzing that has happened these in the past. For example, 
> here are a bunch of images from there:
>
>   https://github.com/dvyukov/go-fuzz-corpus/tree/master/jpeg/corpus
>   https://github.com/dvyukov/go-fuzz-corpus/tree/master/png/corpus
>   https://github.com/dvyukov/go-fuzz-corpus/tree/master/gif/corpus
>
> You could just use those as inputs to your tests now, even if you don't 
> want to fuzz anything yet.  There are also other public fuzzing corpuses 
> for other languages or image libraries that you can find to run your tests 
> against.
>
> That go-fuzz corpus repository also contains sample fuzzing functions for 
> jpeg, png, and gif so that you don't have to start from scratch to fuzz 
> your changes.
>
> Also, some of those fuzzing functions from the go-fuzz corpus have also 
> somewhat recently been moved into the Go standard library. For example, 
> acln0 moved the png fuzz function from the go-fuzz corpus into the Go 
> standard library. 
>
> I mention that in part because as part of your effort to make sure you are 
> not introducing bugs, you could consider bringing over the jpeg and gif 
> fuzzing functions from the go-fuzz corpus into the Go standard library, 
> which would help exercise your code (including it would then be a ~2 line 
> PR to google/oss-fuzz to pick up continuous fuzzing funded by Google for 
> those new functions, which would give you on-going coverage of your 
> changes).
>
> Finally, one way to fuzz using the fuzzing functions in the standard 
> library (using image/png as the example here) is as follows:
>
>   $ go get -u github.com/dvyukov/go-fuzz/...
>   $ go get -u github.com/thepudds/fzgo
>   $ fzgo test -fuzz=. image/png
>
> fzgo is a wrapper on top of go-fuzz that provides some additional 
> conveniences, but perhaps more interesting is that fzgo is a simple 
> prototype of how cmd/go could behave under the proposal in #19109:
>
> #19109 proposal: make fuzzing a first class citizen, like tests or 
> benchmarks 
> https://github.com/golang/go/issues/19109
>
> Sorry for the long post, but I believe you'll get pretty quick payback on 
> the small amount of time it would take to start fuzzing your changes via 
> some of the starting points listed above.
>
> Regards,
> thepudds
>
> On Thursday, February 13, 2020 at 1:07:59 PM UTC-5, Dan Sugalski wrote:
>>
>> Also it's probably easiest if folks have these handy to just propose an 
>> addition to the test repository at 
>> https://github.com/drswork/image/tree/master/testdata since sending 
>> busted images via email is fraught with all sorts of exciting peril.
>> On Thursday, February 13, 2020 at 1:05:50 PM UTC-5 Dan Sugalski wrote:
>>
>>> Fuzz testing is absolutely reasonable and something I want to set up 
>>> once I've got the first pass of code done. 
>>>
>>> On Thursday, February 13, 2020 at 11:42:16 AM UTC-5 jake...@gmail.com 
>>> wrote:
>>>
>>>> This may be slightly tangential, but this seems like the kind of code 
>>>> that would benefit greatly from fuzz testing, like  go-fuzz 
>>>> <https://github.com/dvyukov/go-fuzz>. For this kind of code, go-fuzz 
>>>> <https://github.com/dvyukov/go-fuzz> can really help harden it against 
>>>> bad, malformed and malicious input. I have used it to really good effect. 
>>>> It requires thoughtful choices for the initial corpus, and a machine where 
>>>> you can let it run for a long time, preferable just leave it running for 
>>>> weeks. 
>>>>
>>>> On Wednesday, February 12, 2020 at 9:52:51 PM UTC-5, Dan Sugalski wrote:
>>>>>
>>>>> Specifically ones that are compatible with the Go license. The more 
>>>>> broken they are the better. Bonus points for ones that trigger degenerate 
>>>>> behaviour in the decoding libraries.
>>>>>
>>>>> The tldr h

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

2020-02-13 Thread t hepudds
Hello Dan,

I would definitely echo the fuzzing suggestion from Jake.

In addition, in your quest to find broken images, I would suggest grabbing 
the images from the dvyukov/go-fuzz corpus, which has a bunch of images 
that are already broken in "creative" ways for you based on the 
coverage-guided fuzzing that has happened these in the past. For example, 
here are a bunch of images from there:

  https://github.com/dvyukov/go-fuzz-corpus/tree/master/jpeg/corpus
  https://github.com/dvyukov/go-fuzz-corpus/tree/master/png/corpus
  https://github.com/dvyukov/go-fuzz-corpus/tree/master/gif/corpus

You could just use those as inputs to your tests now, even if you don't 
want to fuzz anything yet.  There are also other public fuzzing corpuses 
for other languages or image libraries that you can find to run your tests 
against.

That go-fuzz corpus repository also contains sample fuzzing functions for 
jpeg, png, and gif so that you don't have to start from scratch to fuzz 
your changes.

Also, some of those fuzzing functions from the go-fuzz corpus have also 
somewhat recently been moved into the Go standard library. For example, 
acln0 moved the png fuzz function from the go-fuzz corpus into the Go 
standard library. 

I mention that in part because as part of your effort to make sure you are 
not introducing bugs, you could consider bringing over the jpeg and gif 
fuzzing functions from the go-fuzz corpus into the Go standard library, 
which would help exercise your code (including it would then be a ~2 line 
PR to google/oss-fuzz to pick up continuous fuzzing funded by Google for 
those new functions, which would give you on-going coverage of your 
changes).

Finally, one way to fuzz using the fuzzing functions in the standard 
library (using image/png as the example here) is as follows:

  $ go get -u github.com/dvyukov/go-fuzz/...
  $ go get -u github.com/thepudds/fzgo
  $ fzgo test -fuzz=. image/png

fzgo is a wrapper on top of go-fuzz that provides some additional 
conveniences, but perhaps more interesting is that fzgo is a simple 
prototype of how cmd/go could behave under the proposal in #19109:

#19109 proposal: make fuzzing a first class citizen, like tests or 
benchmarks 
https://github.com/golang/go/issues/19109

Sorry for the long post, but I believe you'll get pretty quick payback on 
the small amount of time it would take to start fuzzing your changes via 
some of the starting points listed above.

Regards,
thepudds

On Thursday, February 13, 2020 at 1:07:59 PM UTC-5, Dan Sugalski wrote:
>
> Also it's probably easiest if folks have these handy to just propose an 
> addition to the test repository at 
> https://github.com/drswork/image/tree/master/testdata since sending 
> busted images via email is fraught with all sorts of exciting peril.
> On Thursday, February 13, 2020 at 1:05:50 PM UTC-5 Dan Sugalski wrote:
>
>> Fuzz testing is absolutely reasonable and something I want to set up once 
>> I've got the first pass of code done. 
>>
>> On Thursday, February 13, 2020 at 11:42:16 AM UTC-5 jake...@gmail.com 
>> wrote:
>>
>>> This may be slightly tangential, but this seems like the kind of code 
>>> that would benefit greatly from fuzz testing, like  go-fuzz 
>>> . For this kind of code, go-fuzz 
>>>  can really help harden it against 
>>> bad, malformed and malicious input. I have used it to really good effect. 
>>> It requires thoughtful choices for the initial corpus, and a machine where 
>>> you can let it run for a long time, preferable just leave it running for 
>>> weeks. 
>>>
>>> On Wednesday, February 12, 2020 at 9:52:51 PM UTC-5, Dan Sugalski wrote:

 Specifically ones that are compatible with the Go license. The more 
 broken they are the better. Bonus points for ones that trigger degenerate 
 behaviour in the decoding libraries.

 The tldr here is that I'm making a bunch of changes to the basic image 
 libraries to add in metadata support. Since I'm in there anyway I figure I 
 may as well add in some safety measures against files that consume an 
 unreasonable amount of time and/or space to decode. (Or encode if anyone's 
 got an image.Image that triggers bad behaviour) Having a good suite of 
 known-bad files to check against will be handy to make sure the code 
 actually works which is, y'know, kinda nice.

 (I am aware of some bad files kicking around the web, and I've grabbed 
 some for testing, but none I've run across so far have licenses on them 
 that'd allow me to toss them up on github as part of the tests for this 
 stuff. Hence the asking around)

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [go-nuts] Re: Why Discord is switching from Go to Rust

2020-02-10 Thread t hepudds
Sorry, sending again after correcting a couple of typos. (That's what I get 
for trying to walk and chew gum at the same time or typing while distracted 
;-)

They had said they were generating almost no garbage.

If that is true, I suspect they could have avoided the 2-minute forced GC 
by running with GOGC=off, or alternatively executed SetGCPercent(-1) 
(https://golang.org/pkg/runtime/debug/#SetGCPercent) to disable the GC 
programmatically after reaching steady-state.

Perhaps that would have worked for them. If needed, perhaps they could have 
also had a helper goroutine that ran, say, once an hour to run runtime.GC() 
manually, or once a day or whatever frequency might have seemed rational 
based on their workload.

I suspect that would have been sufficient as a simple workaround?

thepudds


On Monday, February 10, 2020 at 12:39:08 PM UTC-5, t hepudds wrote:
>
> They had said they were generating almost no garbage.
>
> If that is true, I suspect they could have avoided the 2-minute forced GC 
> by running with GOGC=off, or alternatively executed SetGCPercent(-1) (
> https://golang.org/pkg/runtime/debug/#SetGCPercent) to disable the GC 
> programmatically after reaching steady-state.
>
> Perhaps what would have worked for them. If needed, perhaps they could 
> have also had a helper goroutine that ran, say, once an hour to run 
> runtime.GC() manually, or once a day or whatever frequency might have 
> seemed rationale based on their workload.
>
> I suspect that would have been sufficient as a simple workaround?
>
> thepudds
>
> On Monday, February 10, 2020 at 12:02:45 PM UTC-5, Jake Montgomery wrote:
>>
>> runtime.GC() would not work for them. According to the docs - " 
>> <https://golang.org/pkg/runtime/#GC>GC runs a garbage collection and 
>> blocks the caller until the garbage collection is complete. It may also 
>> block the entire program." So that would be very counter productive in this 
>> case.
>>
>> On Monday, February 10, 2020 at 5:10:37 AM UTC-5, Kevin Chadwick wrote:
>>>
>>> On 2020-02-09 14:52, ffm...@web.de wrote: 
>>> > 
>>> > Aside from that it would be nice if that 2-minutes GC trigger, that is 
>>> mentioned 
>>> > in the text, could be removed or lessened. 
>>>
>>> runtime.GC()? 
>>>
>>> Though I recall that they found an alternative from the text? 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/afae7069-5fe2-4eaa-a3e0-3db416cb67a4%40googlegroups.com.


Re: [go-nuts] Re: Why Discord is switching from Go to Rust

2020-02-10 Thread t hepudds
They had said they were generating almost no garbage.

If that is true, I suspect they could have avoided the 2-minute forced GC 
by running with GOGC=off, or alternatively executed SetGCPercent(-1) 
(https://golang.org/pkg/runtime/debug/#SetGCPercent) to disable the GC 
programmatically after reaching steady-state.

Perhaps what would have worked for them. If needed, perhaps they could have 
also had a helper goroutine that ran, say, once an hour to run runtime.GC() 
manually, or once a day or whatever frequency might have seemed rationale 
based on their workload.

I suspect that would have been sufficient as a simple workaround?

thepudds

On Monday, February 10, 2020 at 12:02:45 PM UTC-5, Jake Montgomery wrote:
>
> runtime.GC() would not work for them. According to the docs - " 
> GC runs a garbage collection and 
> blocks the caller until the garbage collection is complete. It may also 
> block the entire program." So that would be very counter productive in this 
> case.
>
> On Monday, February 10, 2020 at 5:10:37 AM UTC-5, Kevin Chadwick wrote:
>>
>> On 2020-02-09 14:52, ffm...@web.de wrote: 
>> > 
>> > Aside from that it would be nice if that 2-minutes GC trigger, that is 
>> mentioned 
>> > in the text, could be removed or lessened. 
>>
>> runtime.GC()? 
>>
>> Though I recall that they found an alternative from the text? 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/66bcc92e-1627-4733-9b27-7d839fdae30c%40googlegroups.com.


[go-nuts] Re: go modules and internal packages

2020-01-27 Thread t hepudds
Hello Srini,

There is a decent answer covering some points on how to structure your module, 
options on where to place your go.mod file, and how to arrange your packages 
within a module here:

https://stackoverflow.com/a/57314494

One thing to note is that it says in that answer that you cannot use relative 
import paths like `import "../pkg2"` or `import "./subpkg"` when using modules. 

You seem to be using relative import paths in your example, which is not going 
to work.

You should instead import a package within your module using the full import 
path of that package, which starts with the module name, such as `import 
"github.com/me/mymod/pkg1"`

Finally, if you haven’t already, I highly recommend reading the official “How 
to Write Go Code” document, which has recently been updated to cover modules:

https://golang.org/doc/code.html

And of course, please don’t hesitate to post any follow-up questions here, or 
in a new thread.

Best,
thepudds

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3a7495cc-61f4-4f96-9476-33723d35289a%40googlegroups.com.


[go-nuts] module dependency & testing

2019-10-25 Thread t hepudds
To show the exact versions used in your build excluding test-only dependencies, 
I think this works:

go list -deps -f '{{with .Module}}{{.Path}} {{.Version}}{{end}}' ./... | sort -u

(Command originally from Bryan Mills). 

Regards,
thepudds 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/54fcc447-6a2a-4ffa-b9ca-71916b304f07%40googlegroups.com.


[go-nuts] Re: Empty go.mod file in root directory containing only version maps

2019-10-22 Thread t hepudds
Hello Gert,

It looks like you are experimenting with the "major subdirectory" layout 
for modules.

It looks like your v1 module is incorrectly specified, including with an 
incorrect trailing "/v1":

   https://github.com/gertcuykens/module/blob/v4.0.0/v1 


Usually you would not have a "/v1" subdirectory if you are following the 
"major subdirectory" layout. If you have the v1 module in the same branch 
as your v2, v3, and v4 modules, then usually the v1 module would be in the 
root directory of the repo, and not a /v1 subdirectory, and there would be 
no traililng "/v1" on the module line of the go.mod there.

This is described in more detail here:

   https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Note that section also describes an alternative layout called the "major 
branch" layout for modules, which does not use any "/vN" subdirectories, 
and which some people find simpler to manage at this point.

There is also a much longer discussion of those two alternative layouts 
here:

   https://research.swtch.com/vgo-module
  
In summary, I think the answer to your question is "no, that is not a 
standard practice".

Regards,
thepudds

On Tuesday, October 22, 2019 at 2:42:34 AM UTC-4, Gert wrote:
>
>
> Example https://github.com/gertcuykens/module/tree/v4.0.0 can I assume it 
> to be good practice to put a empty go.mod file in the root directory that 
> doesn't contain other go files, only the version folders, so the root 
> itself doesn't get recognised as a module itself?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/34d4ae99-f949-4850-b3c2-e2d1328dad32%40googlegroups.com.


[go-nuts] Re: GO Mod (modules) for dummies. Please help.

2019-10-22 Thread t hepudds
Hello Stuart,

I haven't digested everything here, but one comment is that all of the 
directives in go.mod like 'require' and 'replace' all deal with module 
paths, and not package import paths.

This 'replace' I think does nothing:

replace github.com/mygit/webserverbase/config v0.1.0-alpha => ../config

because it is using a package import path and not a module path. The module 
path was defined in the other go.mod on the 'module' line as:

module github.com/mygit/webserverbase

Backing up, I suspect part of what is tripping you up is the exact 
relationship between repositories vs. modules vs. packages vs. import paths.

Here is a summary of the relationships between those concepts which might 
help:

 * A _repository_ contains one or more Go _modules_ (usually exactly one 
module in the repository root).
 * Each module contains one or more Go _packages_.
 * Each package consists of one or more Go _source files_ that all reside 
in a _single directory_.
 * Go source code:
   * declares its own package with a `package foo` statement.
   * automatically has access to other Go source code in the same package.
   * imports code from another package via an _import path_ supplied in an 
import statement such as `import "github.com/some/repo/pkg1"`. The import 
path always starts with the module path of that package, regardless of 
whether that package is in the same module or a different module.

Also, I would like to repeat the advice that someone gave you earlier that 
it is best to avoid >1 module in a repo if you can, especially when 
starting out with modules, which sounds like the case here if I followed.

The most common and easiest approach is a single go.mod per repository, 
with the single go.mod file placed in the repository root, and using the 
repository name as the module path declared in the module line in the 
go.mod.

That avoids multiple subtleties...

One alternative you could consider is using a leading underscore for your 
examples directory, as shown for example here:

  https://github.com/loov/hrtime/tree/master/_example

>From the go documentation:

  > Directory and file names that begin with "." or "_" are ignored by the 
go tool, as are directories named "testdata".
  
Regards,
thepudds

On Tuesday, October 22, 2019 at 10:01:24 AM UTC-4, Stuart Davies wrote:
>
> Cannot make local changes. Dispite previous comments This is still a 
> frustrating issue!
>
> I have the following project structure:
>  ── webserverbase
> ├── config
> │   ├── config.go
> ├── example
> │   ├── go.mod  // Mod file 1
> │   ├── go.sum
> │   ├── webserver.go
> ├── exec
> │   ├── exec.go
> │   └── exec_test.go
> ├── go.mod // Mod file 2
> ├── go.sum
>
> Mod File 1:
> module webserver.go
> go 1.13
> require github.com/mygit/webserverbase v0.1.0-alpha // indirect
> replace github.com/mygit/webserverbase/config v0.1.0-alpha => 
> ../config
>
> Mod File 2:
> module github.com/mygit/webserverbase
> go 1.13
>
> At $GOPATH/pkg \mod\github.com\mygit\web!server!base@v0.1.0-alpha\ The 
> whole project source branch
>
> In the examples dir I can:
> go install webserver.go
> I get an exe in:
> C:\Users\user\go\bin
> 
> If I change webserverbase\config\config.go NOTHING CHANGES! Dispite the 
> replace in the go.mod file.
>
> I need to make local changes to config before pushing to git. What am I 
> dooing wrong!
>
> I even introduced a syntax error in to config.go. It is IGNORED!
>
> Please Please help. I cannot see where I am going wrong! It should not be 
> this hard!
>
> Stuart
>
> On Monday, 23 September 2019 16:25:30 UTC+1, Stuart Davies wrote:
>>
>> Hi. 
>>
>>
>> I have been using GO for about a year and I love the language and ideas 
>> behind the language. I am also a Java developer for many years, I switched 
>> from Delphi to Java 1, the new and exciting language from Sun (a bit like 
>> GO is now from Google).
>>
>>  
>>
>> In Java we have Maven and Gradle (sorry Ant) to make dependency hell more 
>> manageable so I understand the need for modules in Go. I have just 
>> installed GO 1.13 and thought I would convert an existing 'pet' project to 
>> use modules. It did NOT go well!
>>
>>  
>>
>> What I need is a dummies guide to the GO module so I can build good, 
>> reliable, standards compliant GO applications.
>>
>>  
>>
>> I needs to explain the new terminology in the context of a module, like 
>> 'vendor'.  Not just a one liner, I NEED to understand! 
>>
>>  
>>
>> I know how to use Google but the quality of the articles I have read on 
>> this subject is minimal and just brushes the surface.
>>
>>  
>>
>> If I have a reasonably large and complex (pet) project with local 
>> packages in it. I like to split my project in to small units with 
>> 'namespaces' to keep it manageable. These are NOT reusable components until 
>> I decide they qualify and publish on Github.
>>
>>- Why MUST I import them as 

[go-nuts] Re: go.mod changes on each build, -mod=readonly can never work

2019-09-19 Thread t hepudds
Hello Russ,

Usually, if you are not modifying your code, I think go.mod should be 
stable across repeated identical invocations of something like 'go build' 
or 'go install'.

I think there were a similar sounding set of bugs fixed for Go 1.13, but 
sounds like you are seeing this with Go 1.13.

A more recent bug was this one, which I think was reported against Go 1.13:

   https://github.com/golang/go/issues/34086

That particular bug has seemingly been fixed in the tip branch. It probably 
is worth trying to see if you can reproduce with that fix in place. A handy 
way to try with the latest from tip / master is:

$ go get golang.org/dl/gotip
$ gotip download 

However, that might not solve it for you. 
 
If you still see the same behavior using 'gotip', then I would suggest 
opening a new bug to make sure one of the people working on cmd/go takes a 
look. It looks like you already have nicely pulled together a public 
reproducer, which is great.

Finally, does 'go mod tidy' seem to be stable in terms of how it leaves 
go.mod?  If so, one approach in the short term might be to run 'go mod 
tidy' to put go.mod into a more stable state (e.g., after running 'go 
install'), but given you seem to be seeing a bug, I'm not sure if that will 
work for you.

Regards,
thepudds

On Wednesday, September 18, 2019 at 7:38:41 PM UTC-4, Russ Selph wrote:
>
> Hi,
>
> I've got a modules problem, and I'm not sure yet whether it's operator 
> error or something worthy of a bug report.  In short, I have a build where 
> go.mod oscillates between two states build by build.  The first build 
> changes it one way, the next build changes it back.  If I try the build 
> with -mod=readonly, it will always fail because every build wants to change 
> go.mod.  This is a real problem for source control.  :-)
>
> I've put together a toy project that demonstrates the problem.  It's based 
> on our real world project, which is pretty large.  This one has no code of 
> any consequence, but is an exact model of the external dependencies.
>
> https://github.com/rselph-tibco/go-unstable-mods
>
> There are two programs and two packages, all contained in two modules, 
> with a sort of twisted interdependency.  But it seems to be the external 
> dependencies that are messing things up.  On subsequent builds of sample1, 
> the go.mod file  does this:
>
> foo.com/me/sample2 v0.0.0-0001010100-
> github.com/coreos/bbolt v1.3.3
> github.com/coreos/etcd v3.3.15+incompatible
> -   github.com/coreos/go-semver v0.3.0 // indirect
> github.com/elazarl/go-bindata-assetfs v1.0.0
> github.com/gorilla/mux v1.7.3
> -   github.com/json-iterator/go v1.1.7 // indirect
> github.com/magiconair/properties v1.8.1
> -   github.com/modern-go/reflect2 v1.0.1 // indirect
> github.com/satori/go.uuid v1.2.0
> golang.org/x/net v0.0.0-20190918130420-a8b05e9114ab
>
> then this:
>
> foo.com/me/sample2 v0.0.0-0001010100-
> github.com/coreos/bbolt v1.3.3
> github.com/coreos/etcd v3.3.15+incompatible
> +   github.com/coreos/go-semver v0.3.0 // indirect
> github.com/elazarl/go-bindata-assetfs v1.0.0
> github.com/gorilla/mux v1.7.3
> +   github.com/json-iterator/go v1.1.7 // indirect
> github.com/magiconair/properties v1.8.1
> +   github.com/modern-go/reflect2 v1.0.1 // indirect
> github.com/satori/go.uuid v1.2.0
> golang.org/x/net v0.0.0-20190918130420-a8b05e9114ab
>
> What's the best way to debug what's going on here?  Does this look like 
> any known problem?  (Search didn't yield anything that looked similar to 
> me.)
>
> You can follow the exact steps and their consequences here: 
> https://github.com/rselph-tibco/go-unstable-mods/commits/master
> All of that was generated by the script at 
> https://github.com/rselph-tibco/go-unstable-mods/blob/master/run.sh
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/63ac661b-7d10-466f-9652-aaf0b2d8f90e%40googlegroups.com.


[go-nuts] Re: Is there a way to use the pakckages under vendor folder and from module cache?

2019-09-10 Thread t hepudds
Hello T L,

I think I might not fully understand the exact scenario, but one thing some 
people have done when they have something in their vendor directory that 
they can't otherwise find anywhere else (e.g., perhaps because it is 
modified, or maybe the only copy of a now-missing dependency is in your 
vendor directory) is to rename the vendor directory to something else 
("oldvendor", or whatever name makes sense), and then set 'replace' 
directives in the main project's go.mod file to point into the "oldvendor" 
directory for whatever pieces from there are needed.

For example, something like:

   require github.com/some/dependency v0.0.0

   replace github.com/some/dependency => 
./oldvendor/github.com/some/dependency

I think that would allow you to pick and choose what you want from your old 
vendor directory.

In terms of your other question, I think -mod=vendor is currently all or 
nothing. There is a proposal to perhaps one day support partial vendoring, 
but that is not in the current software.

Regards,
thepudds

On Tuesday, September 10, 2019 at 8:59:28 AM UTC-4, T L wrote:
>
> I maintain an private old Go project, which depends many many old packages.
> Before the module mode, I put all these dependency packages under the 
> vendor folder.
> In the developing process, from time to time, I modified some of 
> dependency packages.
>
> Meanwhile, I plan to migrate the project to modules mode and need to 
> import some new module packages.
> I found I encountered an embarrassing situation.
> If I delete the vendor folder and run "go mod vendor" in modules mode to 
> rebuild the vendor,
> the command will fail. One reason is some dependency packages disappeared.
> The other reason is many packages are updated and broke capabilities.
> And I don't want the new download to overwrite my local modifications for 
> some dependency packages.
>
> So, is there a way to let me continue use the these old dependency 
> packages and use some new modules based packages?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d9491904-9e36-4230-a3f4-6c3b19e68698%40googlegroups.com.


Re: [go-nuts] Re: go mod dependency hell is real

2019-09-10 Thread t hepudds
Hello Darko,

Rather than that 'replace' you found, a better solution is probably adding 
a 'require' for a more recent release of github.com/ugorji/go.

Adding this to my go.mod worked for me in a simple test just now to resolve 
a similar error to what you reported:

  require github.com/ugorji/go v1.1.7 

Using a 'require' for a recent version is better than the 'replace' you 
found for a couple reasons, including it is more forward looking, it ages 
out more gracefully, it helps anyone importing your project (avoiding the 
problem Sam W. just commented on), etc.

Or, you likely could use a later version than v1.1.7.

The v1.1.7 release of github.com/ugorji/go was specifically targeting 
resolving the error you reported, I believe. Some more details here if 
interested:
   https://github.com/ugorji/go/issues/299

Regards,
thepudds

On Tuesday, September 10, 2019 at 9:15:46 AM UTC-4, Sam Whited wrote:
>
> Note that replace directives are not transitive, so every single user of 
> your library will need to do this. You can put it into your go.mod file 
> to get your library building and get tests passing, but your users will 
> still have to do this work as well so you'll probably want to document 
> that they now have to jump through hoops to use your library. 
>
> —Sam 
>
> On Tue, Sep 10, 2019, at 13:10, Darko Luketic wrote: 
> > The answer is apparently 
> > https://github.com/gin-gonic/gin/issues/2039#issuecomment-527997733 
> > 
> > > Add this to your go.mod file: `replace github.com/ugorji/go v1.1.4 
> > > => github.com/ugorji/go v0.0.0-20190204201341-e444a5086c43` 
> > On Tuesday, September 10, 2019 at 2:48:25 PM UTC+2, Darko 
> > Luketic wrote: 
> > > What used to work pre go 1.13 now doesn't work anymore 
>
> -- 
> Sam Whited 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ef689ff2-fe8b-4e8b-b01f-c96e37260f6c%40googlegroups.com.


Re: [go-nuts] Re: v1.13: Altered go.mod file in project after updating vim-go binaries

2019-09-06 Thread t hepudds
FWIW,  I think this issue tracks a similar request:
 
  #30515 cmd/go: offer a consistent "global install" command
  https://github.com/golang/go/issues/30515
  
It was tagged as a release-blocker for Go 1.13, but I think time ran out.

One outcome considered there was a new argument to 'go get' such as 'go get 
-b', where '-b' might be for 'binary' or 'bare' and would just install a 
binary without updating the current go.mod.  A less likely outcome might be 
re-purposing 'go install' to do the same thing.

That issue could be a good place to comment if you have a specific idea, 
etc.

Regards,
thepudds

On Thursday, September 5, 2019 at 10:02:31 PM UTC-4, kortschak wrote:
>
> I also. 
>
> We have to add additional mess to our build scripts when we get testing 
> dependencies that are not part of our distribution to avoid 
> contaminating the go.{mod,sum} in the repo root. 
>
> This has repeatedly been a source of irritation and frustration. 
>
> Dan 
>
> On Thu, 2019-09-05 at 11:36 -0700, Michael Ellis wrote: 
> > I second Jan's request for light-shedding.  It's one thing to add 
> > real 
> > indirect dependencies. I can sort of understand that.  Can't think of 
> > a 
> > sensible reason for adding dependencies that are unrelated to the 
> > code in 
> > the module. 
> > 
> > On Thursday, September 5, 2019 at 2:23:11 PM UTC-4, Jan Mercl wrote: 
> > > 
> > > On Thu, Sep 5, 2019 at 8:18 PM > 
> > > wrote: 
> > > 
> > > > Running 'go get ...' inside a module can add indirect 
> > > > dependencies to 
> > > 
> > > go.mod. 
> > > 
> > > I'm surprised. There's probably some rationale behind `go get` 
> > > having 
> > > such side effects. Can anyone please shed some light on this? 
> > > 
> > 
> > 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b3f9ba20-563f-412f-b1a6-2e950a4033fa%40googlegroups.com.


[go-nuts] Re: Go module and local dependencies

2019-09-04 Thread t hepudds
Hello Guillaume,

I haven't had a chance to look at your example closely, but it seems you 
have two modules defined on your local filesystem, with two go.mod files 
total.

If that is what you are trying to do, one approach is to use a `replace` 
directive to let one module know about the on-disk location of the other 
module. Otherwise, they don't know how to find each other.

Also, you usually want to name your modules as if you will publish them 
someday, which means the `module` line in a go.mod would read something 
like `module github.com/some/repo`, rather I think you might have something 
like `module xxx/myLib`, which can be problematic.

In general, when importing code with an import statement, you should always 
use the full import path, which will start with the module path, such as 
`import "github.com/some/repo/some/pkg"`. You should always use the full 
import path when importing packages inside the same modules, as well as 
when importing packages from a different module.

You can read more about how to use `replace` to let modules find each other 
on your local filesystem in these two FAQs on the modules wiki:

FAQ: When should I use the `replace` directive?

https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive
  
FAQ: Can I work entirely outside of VCS on my local filesystem?

https://github.com/golang/go/wiki/Modules#can-i-work-entirely-outside-of-vcs-on-my-local-filesystem

Here is a link to a small runnable example that shows two modules 
side-by-side on the filesystem that use `replace` to make things work. I 
think that is at least somewhat similar to what you are trying to do, if I 
followed:

https://groups.google.com/d/msg/golang-nuts/1nYoAMFZVVM/eppaRW2rCAAJ

Hope that helps at least somewhat. Please don't hesitate to post more 
questions or comments.

Regards,
thepudds

On Sunday, September 1, 2019 at 6:02:21 AM UTC-4, Guillaume Lescure wrote:
>
> Hi,
>
> Thanks for the comments and the ideas :)
>
> 1. Try 'go build ./...'  from the root directory of the module to build 
>> all the packages in the module.  'go build'  without any arguments is the 
>> same as 'go build .'  which means just build the current directory/package. 
>
>
> I didn't know the command "go build ./..." and that directly answer my 1st 
> question, thanks a lot. Now my library is totally built.
> Thanks a lot for the tips and the explanation (maybe it sould be 
> documented ?).
>
> 2.  With only one go.mod, you should not need a 'replace'  to find local 
>> code in your own module. 
>
>
> I  have 2 go.mod, 1 in myLib folder and 1 other in myBin folder and I 
> don't want to merge them.
> The idea behind that is to simulate 2 repositories, 1 for the library and 
> 1 for a random application that use that library.
>
> 3.  Make sure you use the full import path in any import statements in 
>> your code when importing other packages, even when importing other packages 
>> in the same module.  The import path should always start with the module 
>> path. (The module path what you see on the 'module' line of your go.mod). 
>>
>  If that doesn’t help, you might need to share the exact error messages, 
>> along with the exact go.mod and exact import statements in question. 
>
>
> My code is attached.
>
> Regards,
>  
>
>>
>>
>> (I am on mobile, so sorry this is brief). 
>>
>> Regards, 
>> thepudds
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c27434cd-3ea9-4a29-b38d-6329051228da%40googlegroups.com.


[go-nuts] Re: How to update a module's direct dependencies only

2019-09-04 Thread t hepudds
Hello Mihai,

To upgrade your direct dependencies to their latest release (with your 
indirect dependencies using versions selected based on the requirements of 
your direct dependencies), this should work in most cases:

  go get $(go list -f '{{if not (or .Main .Indirect)}}{{.Path}}{{end}}' -m 
all)

Does that work for you?

There is an issue suggesting allowing this to be specified more easily:

#28424 cmd/go: add 'go get' options to update direct and indirect 
dependencies separately
https://github.com/golang/go/issues/28424  
  
If interested, you could add a comment there briefly explaining your use 
case, or whatever other comment makes sense.

Regards,
thepudds

On Wednesday, September 4, 2019 at 7:35:58 PM UTC-4, mihaib...@gmail.com 
wrote:
>
> Hello,
>
> I am looking for a way to regularly update my Go module's direct 
> dependencies. Without specifying indirect dependencies myself: let my 
> direct dependencies specify what they need in their own go.mod.
>
> After reading relevant documentation, I could only come up with this 
> partial workaround:
>
> go list -m -u -f '{{if and (and (not .Indirect) (not .Main)) 
> .Update}}{{.Path}}@{{.Update.Version}}{{end}}' all | xargs 
> --no-run-if-empty go get
>
> Suspecting this scenario is common and useful, I have prepared a minimal 
> concrete example below (each module is a repository under 
> github.com/MihaiB for now).
> "X → Y" means "module X depends directly on module Y".
> I have the following Go modules:
>
> goMain → goA → goB
> goMain's tests → goT
>
> All modules have published versions v1.1.0, v1.2.0 and v2.0.0 (except 
> goMain which has no version tags). All dependency arrows above (go.mod 
> entries) specify v1.1.0, so all can be updated.
>
> goMain's .go files are in a sub/dir/ to make sure the commands I type 
> inspect all packages inside the module, but this isn't mandatory: I can 
> also repeat a command in each package's directory if needed.
>
> 
> In goMain:
> $ cat go.mod 
> module github.com/MihaiB/goMain
>
> go 1.13
>
> require (
> github.com/MihaiB/goA v1.1.0
> github.com/MihaiB/goT v1.1.0
> )
> 
>
> 1) How to find out if goA or goT have a newer v1 version, and update to it?
> My 'go list … | xargs …' workaround above does this by listing the direct 
> dependencies and passing them to 'go get'.
> Would it make sense to support this functionality via a shorter command?
> I cannot use 'go get -u -t -d ./...' here because that brings in the 
> indirect dependency 'goB' into my go.mod.
>
> 2) How to find out if goA or goT have a newer major version (v2 or later)?
> I know that v2 and later have a different import path (ending in "/v2"), 
> so different major versions are different packages and they can be used 
> together in my module.
> But I would like a command which tells me if some of my direct 
> dependencies have a newer major version, otherwise I would have to check 
> each dependency by hand to find out.
>
> Mihai
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b6882ef2-7f48-4a92-af07-1b96ea190abf%40googlegroups.com.


[go-nuts] Re: Option to disable version validation in 1.13?

2019-09-03 Thread t hepudds
Hello Steve,

> https://golang.org/doc/go1.13#version-validation lists a number of 
> options if you're maintaining a module but nothing seems relevant for 
> CI/CD pipelines which are just trying to use tools for which they aren't 
> the maintainer. 

Regarding how to fix "invalid pseudo-version" errors, I think the release 
notes are also trying to give options for someone trying to use tools for 
which they aren't the maintainer (as well as options for maintainers).

The paragraph on the `replace` directive in that section of the release 
notes I think is something you can use, even if you are not the maintainer 
of a module with an invalid version. In general, the `replace` directive 
basically gives your module complete control over its own build in terms of 
what versions to use.

More specifically, using Go 1.13, this fails with an "invalid 
pseudo-version" error (as expected):

   export GOPATH=$(mktemp -d)   # using fresh module cache
   cd $(mktemp -d)   
   go mod init example.com/tempmod 
   go get github.com/golangci/golangci-lint/cmd/golangci-lint@v1.17.1 

which reports error:

  go: extracting github.com/golangci/golangci-lint v1.17.1
  go get: github.com/golangci/golangci-lint@v1.17.1 requires
github.com/go-critic/go-critic@v0.0.0-20181204210945-1df300866540: 
invalid pseudo-version: does not match version-control timestamp 
(2019-05-26T07:48:19Z)

It looks like there has been some discussion of solving this within 
golangci, including here and some related issues there:
  
   https://github.com/golangci/golangci-lint/pull/605
  
It looks like some of the core Go team has commented there with advice, but 
looks like it is still pending changes there.
   
However, we don't need to wait for that to be resolved within golangci.

If we follow the advice from the section of the Go 1.13 release notes on 
resolving version validation issues ( 
valhttps://golang.org/doc/go1.13#version-validation), we can make that same 
'go get' work:

   # re-do setup from scratch
   export GOPATH=$(mktemp -d)   # using fresh module cache
   cd $(mktemp -d)   
   go mod init example.com/tempmod

   # create 'replace' statements using *just* the commit hashes for each 
problematic module on the right-hand side
   echo 'replace github.com/go-critic/go-critic 
v0.0.0-20181204210945-1df300866540 => github.com/go-critic/go-critic 
1df300866540' >> go.mod
   echo 'replace github.com/golangci/errcheck 
v0.0.0-20181003203344-ef45e06d44b6 => github.com/golangci/errcheck 
ef45e06d44b6' >> go.mod
   echo 'replace github.com/golangci/go-tools 
v0.0.0-20180109140146-af6baa5dc196 => github.com/golangci/go-tools 
af6baa5dc196' >> go.mod
   echo 'replace github.com/golangci/gofmt 
v0.0.0-20181105071733-0b8337e80d98 => github.com/golangci/gofmt 
0b8337e80d98' >> go.mod
   echo 'replace github.com/golangci/gosec 
v0.0.0-20180901114220-66fb7fc33547 => github.com/golangci/gosec 
66fb7fc33547' >> go.mod
   echo 'replace github.com/golangci/ineffassign 
v0.0.0-20180808204949-42439a7714cc => github.com/golangci/ineffassign 
42439a7714cc' >> go.mod
   echo 'replace github.com/golangci/lint-1 
v0.0.0-20180610141402-ee948d087217 => github.com/golangci/lint-1 
ee948d087217' >> go.mod
   echo 'replace mvdan.cc/unparam v0.0.0-20190124213536-fbb59629db34 => 
mvdan.cc/unparam fbb59629db34' >> go.mod

   # this now works
   go get github.com/golangci/golangci-lint/cmd/golangci-lint@v1.17.1 

Regards,
thepudds

On Tuesday, September 3, 2019 at 8:17:47 PM UTC-4, Steven Hartland wrote:
>
> Trying to update to 1.13 but the new version validation breaks 
> golangci-lint installs with: 
> invalid pseudo-version: does not match version-control timestamp 
>
> https://golang.org/doc/go1.13#version-validation lists a number of 
> options if you're maintaining a module but nothing seems relevant for 
> CI/CD pipelines which are just trying to use tools for which they aren't 
> the maintainer. 
>
> On my local box I ended up downgrading to go 1.12.9 running the install, 
> which downloaded the dependencies into the cash then upgrading again and 
> reinstalling to get go 1.13 compiled versions of golangci-lint. 
>
> GO111MODULE=on go get 
> github.com/golangci/golangci-lint/cmd/golangci-lint@v1.17.1 
>
> The standard way to install golangci-lint from binaries fails due to 
> what seems like conflicts with the changes in go 1.13, erroring out in 
> various linters. 
>
> So the question is: 
> * Is there option to disable this behaviour so we don't have to get 
> every single tools dependencies fixed to pass this new validation just 
> to use them? 
>
>  Regards 
>  Steve 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

[go-nuts] Go module and local dependencies

2019-08-31 Thread t hepudds
 A few quick comments: 

1. Try 'go build ./...'  from the root directory of the module to build all the 
packages in the module.  'go build'  without any arguments is the same as 'go 
build .'  which means just build the current directory/package. 

2.  With only one go.mod, you should not need a 'replace'  to find local code 
in your own module. 

3.  Make sure you use the full import path in any import statements in your 
code when importing other packages, even when importing other packages in the 
same module.  The import path should always start with the module path. (The 
module path what you see on the 'module' line of your go.mod). 

 If that doesn’t help, you might need to share the exact error messages, along 
with the exact go.mod and exact import statements in question. 

(I am on mobile, so sorry this is brief). 

Regards,
thepudds

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2990d337-6f1e-46ea-9874-84b8e3089590%40googlegroups.com.


[go-nuts] Re: With Go modules, is that possible to reference a package which hasn't gotten a module support

2019-07-09 Thread t hepudds

Hi Roman,

> For a project considering switching to Go modules, is that possible to 
> reference a package without support of modules (doesn't have a 
go.mod/go.sum files)?

Yes, as far I understand, that should usually work.

Using your example of wanting to consume a non-module dependency 
anEnterpriseCompany.com/anotherProject, a module-based consumer should be 
able to do things like:

  $ go get anEnterpriseCompany.com/anotherProject@
  $ go get anEnterpriseCompany.com/anotherProject@
  $ go get anEnterpriseCompany.com/anotherProject@latest

There is a longer discussion in this FAQ on the Modules wiki:

  
 
https://github.com/golang/go/wiki/Modules#can-a-module-consume-a-package-that-has-not-opted-in-to-modules

Hope that helps,
thepudds


On Tuesday, July 9, 2019 at 9:21:48 AM UTC-4, Roman Gomoliako wrote:
>
> For a project considering switching to Go modules, is that possible to 
> reference a package without support of modules (doesn't have a 
> go.mod/go.sum files)?
>
> module anEnterpriseCompany.com/aProject
>
> require anEnterpriseCompany.com/anotherProject v0.0.0
>
> go 1.12
>
> "anEnterpriseCompany.com/anotherProject" can't be modified to support Go 
> modules for some internal reasons. Is there some ways to resolve this?
>
> Thanks,
>
> Roman
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/44ea25f2-522d-46e3-9cbd-8746e015908e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: simplified error handling

2019-07-09 Thread t hepudds
Hi Robert,

You might be interested in the discussion here:

   https://swtch.com/try.html#goto

which include 3-4 comments discussing a similar (but not identical) idea 
from @josharian and @griesemer.

Best,
thepudds

On Tuesday, July 9, 2019 at 1:25:36 PM UTC-4, Robert Engels wrote:
>
>
> There is probably a similar proposal out-there, but if there is it hasn't 
> surfaced, so I thought I would throw this out there. 
>
> It only requires 2 simple changes and is completely backwards compatible. 
>
> Currently, idiomatic Go error handling code looks like this: 
>
> v,err := somefunc() 
> if err!=nil { 
>  // do something, in many cases, just return err 
> } 
>
> but a substantial amount of Go code has handling code like this: 
>
> v,err := somefunc() 
> if err!=nil { 
>goto some_error_handler 
> } 
>
> especially when there is common wrapping of errors, or retries. 
>
> With the proposal https://github.com/golang/go/issues/27165 accepted (at 
> least Rob likes it), the error handling can be simplified  by 
>
> v,:some_error_handler := somefunc() 
>
> which is essentially an if with goto macro. 
>
> The only other addition is a compiler provided 'function scoped' variable 
> __error which is the last error returned by the last called function. This 
> allows for trivial writing of the error handler block, as an example: 
>
> some_error_handler: 
>  return errors.New("function failed",__error) 
>
> or even 
>
> some_error_handler: 
>  retry++ 
>  if !__error.Permanent() && retry < N { 
> goto try_again 
>  } 
>  return errors.New("function failed",__error) 
>
> And since goto/labels in Go are already scoped, it allows inner handlers, 
> as in: 
>
> for _,x := range blah { 
> v,:localerr := somefunc() 
> defer v.Close() 
> x,:localerr := anotherfunc(x) 
> continue 
>   localerr: 
>  return errors.New("unable to process item ...",__error) 
> } 
>
> I think this coupled with the xerrors enhancements would dramatically 
> improve Go error handling. It is still very explicit, no hidden flow, plays 
> nicely with defer, all returns are explicit, easy to read - no magic. 
>
> And if the function has minimal (or no) error handling required, it is 
> written just as before. 
>
> A simple enhancement might be a pseudo handler label :return that always 
> returns the 0 values for all results parameters, and __error as the err 
> parameter. In many ways this would be similar to the current 'try' 
> proposal. 
>
> The limitation of this solution is that nested calls are not possible as 
> with 'try', but they would instead be done on multiple lines, as in: 
>
> v,:return = somefunc() 
> x,:return = another(v) 
> y,:return = yetanother(x) 
>
> which I think is easier to reason about. 
>
> Anyway, just a thought. 
>
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d929c8ea-4048-4c08-aaf6-f2cf5316fe58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [go-nuts] goforward: What is it? How do I use it?

2019-06-28 Thread t hepudds
Hi AJ,

To reduce the suspense, I probably should have included the full help 
message:

 $ ./gofoward help
 Usage: goforward [-n] [-move | -filter=REGEXP] [-replace] [-filename=NAME] 
SOURCE DEST

  -filename string
destination file in which to write forwards (default "forward.go")
  -filter string
forward only the top-level identifiers matching this regexp 
(default ".*")
  -move
move the existing contents of SOURCE to DEST, leaving forwarding 
declarations at SOURCE
(otherwise, write declarations at DEST forwarding to SOURCE)
  -nprint the locations and contents of files instead of writing them
  -replace
replace conflicting declarations in the destination package
(otherwise, do not forward conflicting declarations)

HTH,
thepudds

On Friday, June 28, 2019 at 11:02:32 PM UTC-4, t hepudds wrote:
>
> Hi AJ,
>
> In terms of detailed write-ups on how gofoward works, to my knowledge the 
> best place to start is reading the commit message and then the help message 
> you get once you build goforward.
>
> The author (Bryan Mills) has stated there is still some work to do to 
> finish it up, including addressing some bugs, but people have used it 
> successfully.  A sample recent PR generated by someone from the broader 
> community using gofoward is here:
>https://github.com/mediocregopher/radix/pull/128
>
> In terms of how you get it, when you go to 
> https://go-review.googlesource.com/c/tools/+/137076/, you should see a 
> 'Download' button on the right side. If you click on that, it gives you 4-5 
> options ranging from 'git fetch' to downloading a zip.
>
> Here's a quick example (following one of the 'Download' options there):
>
>   $ cd $(mktemp -d)
>   $ git clone https://go.googlesource.com/tools .
>   $ git fetch "https://go.googlesource.com/tools; 
> refs/changes/76/137076/9 && git checkout FETCH_HEAD
>   $ cd cmd/goforward
>   $ go build
>   $ ./goforward help
>
> Usage: goforward [-n] [-move | -filter=REGEXP] [-replace] 
> [-filename=NAME] SOURCE DEST
> ...
>
> Hope that helps some,
> thepudds
>
> On Friday, June 28, 2019 at 8:53:52 PM UTC-4, Ian Lance Taylor wrote:
>>
>> On Fri, Jun 28, 2019 at 5:14 PM AJ ONeal  wrote: 
>> > 
>> > I was looking at the Module page on the Wiki ( 
>> https://github.com/golang/go/wiki/Modules ) for instruction on how to 
>> handle breaking API changes (v2, v3, etc) and I noticed mention of the 
>> mythical `goforward`: 
>> > 
>> >> A more sophisticated approach here could exploit type aliases 
>> (introduced in Go 1.9) and forwarding shims between major versions residing 
>> in different subdirectories. This can provide additional compatibility and 
>> allow one major version to be implemented in terms of another major 
>> version, but would entail more work for a module author. An in-progress 
>> tool to automate this is goforward. Please see here for more details and 
>> rationale, along with a functioning initial version of goforward. 
>> > 
>> > 
>> > I also found multiple mentions of it (by the author, I assume) on the 
>> Golang issues on Github, claiming that it may solve various problems. And I 
>> think "that's cool, but... how do I get it?" 
>> > 
>> > I went to the suggested link ( 
>> https://go-review.googlesource.com/c/tools/+/137076/ ), but there's no 
>> instruction on how to install, how to use it, and maybe even gives less 
>> digestible information than on the modules wiki. 
>> > 
>> > The googlesource.com interface is perhaps the strangest I've ever seen 
>> for git / code. I'm guessing it must be some sort of internal Google tool 
>> that you kinda have to be a googler to understand. 
>> > 
>> > Nevertheless, I see 
>> > 
>> >> Repo: tools 
>> >> 
>> >> Branch: master 
>> > 
>> > 
>> > and so I figure there must be a repo somewhere containing this code. 
>> > 
>> > I click on the tools link which takes me to 
>> https://go-review.googlesource.com/q/project:tools ... very confusing. 
>> > I found the repo settings link which lead me to 
>> https://go-review.googlesource.com/admin/repos/tools where I finally see 
>> something familiar: 
>> > 
>> >> git clone "https://go.googlesource.com/tools; 
>> > 
>> > 
>> >  Now I have the repo cloned... but there's no mention of "goforward" 
>> and hardly any mention of "forward" at all (which is not the forward I'm 
>> looking for). 
>> > 
>> > It appea

Re: [go-nuts] goforward: What is it? How do I use it?

2019-06-28 Thread t hepudds
Hi AJ,

In terms of detailed write-ups on how gofoward works, to my knowledge the 
best place to start is reading the commit message and then the help message 
you get once you build goforward.

The author (Bryan Mills) has stated there is still some work to do to 
finish it up, including addressing some bugs, but people have used it 
successfully.  A sample recent PR generated by someone from the broader 
community using gofoward is here:
   https://github.com/mediocregopher/radix/pull/128

In terms of how you get it, when you go to 
https://go-review.googlesource.com/c/tools/+/137076/, you should see a 
'Download' button on the right side. If you click on that, it gives you 4-5 
options ranging from 'git fetch' to downloading a zip.

Here's a quick example (following one of the 'Download' options there):

  $ cd $(mktemp -d)
  $ git clone https://go.googlesource.com/tools .
  $ git fetch "https://go.googlesource.com/tools; refs/changes/76/137076/9 
&& git checkout FETCH_HEAD
  $ cd cmd/goforward
  $ go build
  $ ./goforward help

Usage: goforward [-n] [-move | -filter=REGEXP] [-replace] 
[-filename=NAME] SOURCE DEST
...

Hope that helps some,
thepudds

On Friday, June 28, 2019 at 8:53:52 PM UTC-4, Ian Lance Taylor wrote:
>
> On Fri, Jun 28, 2019 at 5:14 PM AJ ONeal > 
> wrote: 
> > 
> > I was looking at the Module page on the Wiki ( 
> https://github.com/golang/go/wiki/Modules ) for instruction on how to 
> handle breaking API changes (v2, v3, etc) and I noticed mention of the 
> mythical `goforward`: 
> > 
> >> A more sophisticated approach here could exploit type aliases 
> (introduced in Go 1.9) and forwarding shims between major versions residing 
> in different subdirectories. This can provide additional compatibility and 
> allow one major version to be implemented in terms of another major 
> version, but would entail more work for a module author. An in-progress 
> tool to automate this is goforward. Please see here for more details and 
> rationale, along with a functioning initial version of goforward. 
> > 
> > 
> > I also found multiple mentions of it (by the author, I assume) on the 
> Golang issues on Github, claiming that it may solve various problems. And I 
> think "that's cool, but... how do I get it?" 
> > 
> > I went to the suggested link ( 
> https://go-review.googlesource.com/c/tools/+/137076/ ), but there's no 
> instruction on how to install, how to use it, and maybe even gives less 
> digestible information than on the modules wiki. 
> > 
> > The googlesource.com interface is perhaps the strangest I've ever seen 
> for git / code. I'm guessing it must be some sort of internal Google tool 
> that you kinda have to be a googler to understand. 
> > 
> > Nevertheless, I see 
> > 
> >> Repo: tools 
> >> 
> >> Branch: master 
> > 
> > 
> > and so I figure there must be a repo somewhere containing this code. 
> > 
> > I click on the tools link which takes me to 
> https://go-review.googlesource.com/q/project:tools ... very confusing. 
> > I found the repo settings link which lead me to 
> https://go-review.googlesource.com/admin/repos/tools where I finally see 
> something familiar: 
> > 
> >> git clone "https://go.googlesource.com/tools; 
> > 
> > 
> >  Now I have the repo cloned... but there's no mention of "goforward" and 
> hardly any mention of "forward" at all (which is not the forward I'm 
> looking for). 
> > 
> > It appears that the latest commit was yesterday (June 27th, 2019), but I 
> couldn't find the HEAD in the interface and I can't even checkout the 
> parent commit (which is shown in the interface): 
> > 
> >> git checkout a44989a 
> >> 
> >> 
> >> 
> >> error: pathspec 'a44989a' did not match any file(s) known to git 
> > 
> > 
> > Where is this thing and how can I try it out? 
>
>
> It's in https://golang.org/cl/137076.  Not sure what it needs before 
> being submitted, other than a review. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/90fd956a-b858-404b-a562-cabd55bb5100%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [go-nuts] how to prevent go1.13 go directive being written in an go directive-free go.mod?

2019-06-12 Thread t hepudds
Hi Ian,

Thank you for sending that CL 181840. 

One minor request for clarification on the discussion here. 

When you said earlier in this thread:

> “For almost all users the default
value will be correct.  At least, I hope that is the case.”

I *think* what you are saying is for almost everyone the default value set 
should be a valid choice when initially selected by the 'go’ command, even 
though someone might later need to manually change it in the future as new 
language features are released?

In other words, if hypothetically something like a new 'try' keyword lands in 
something like Go 1.15, then at that point someone might need to change the 
value in their go.mod from for example 'go 1.12'  to instead read 'go 1.15' if 
they want to use the new language feature?

Sorry if that is not correct.

Regards,
thepudds

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/39cd7ecc-501c-4eaa-8a00-cbd2a0fae328%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [go-nuts] how to prevent go1.13 go directive being written in an go directive-free go.mod?

2019-06-11 Thread t hepudds
 As far as I understand, it is not a minimum version of the language. 

 Also, a key point that is easy to miss is that the language version is 
distinct from tooling version, and newer tooling versions will know how to 
compile older language versions. At least, that is my understanding, though I 
am also aware there is some confusion on the topic, so I don’t fully trust my 
understanding. 

Some related text from the document Ian cited:

———
The Go compiler released with Go version 1.20 must be able to build packages 
using Go language 1.19. This can be done by adding an option to cmd/compile 
[...]. When cmd/compile sees the option, perhaps -lang=go1.19, it will compile 
the code using the Go 1.19 syntax.

Importantly, specifying the maximum version of the Go language should not be 
taken to imply the maximum version of the Go tools.

[...]

When cmd/compile sees the option, perhaps -lang=go1.19, it will compile the 
code using the Go 1.19 syntax.
This requires cmd/compile to support all previous versions, one way or another.

[...]

Naturally, even though the package is built with the language version 1.19 
syntax, it must in other respects be a 1.20 package: it must link with 1.20 
code, be able to call and be called by 1.20 code, and so forth.

The go tool will need to know the maximum language version so that it knows how 
to invoke cmd/compile. Assuming we continue with the modules experiment, the 
logical place for this information is the go.mod file. The go.mod file for a 
module M can specify the maximum language version for the packages that it 
defines
——

Regards,
thepudds

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/810a1cb4-c3fb-4b2f-bc86-02793902d422%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: tracking down broken module dependency golint

2019-06-03 Thread t hepudds
For this error:

  github.com/golang/lint@v0.0.0-20190409202823-959b441ac422: parsing 
go.mod: unexpected module path "golang.org/x/lint"

That is saying 'github.com/golang/lint' was loaded, but the 'go.mod' found 
at that location for that version had a different module name 
('golang.org/x/lint') declared in the go.mod there. The 'go' command does 
not allow that mismatch between how a module is referenced vs. how the 
module declares its identity in its go.mod.

You can probably make it work by adding a 'replace' statement for 
github.com/golang/lint using the version from just before the 'go.mod' was 
added there.

Here is an example.

First, verify we get a similar failure with 'go get -u' for this old 
version of zipkin-go:
$ cd $(mktemp -d)
$ git clone -b v0.1.5 --depth=1 https://github.com/openzipkin/zipkin-go 
.
$ go get -u 
  
...which fails with the same error you reported:
  github.com/golang/lint@v0.0.0-20190409202823-959b441ac422: parsing 
go.mod: unexpected module path "golang.org/x/lint"

We can add a replace for the version of github.com/golang/lint from just 
before the 'go.mod' was added there. This will be a temporary measure that 
allows this upgrade to succeed, and then 'go mod tidy' will drop 
github.com/golang/lint from the module graph as no longer needed in this 
case.

   $ echo 'replace github.com/golang/lint => github.com/golang/lint 
5614ed5bae' >> go.mod
   $ go get -u
   $ go mod tidy
   $ go mod graph | grep github.com/golang/lint

...which now succeeds, and the grep finds nothing for the older github.com 
name.

There is a good chance that technique will work for you. There is a longer 
explanation of why that error occurs and why that technique usually works 
in this troubleshooting FAQ on the "Modules" wiki:


https://github.com/golang/go/wiki/Modules#how-can-i-resolve-parsing-gomod-unexpected-module-path-and-error-loading-module-requirements-errors-caused-by-a-mismatch-between-import-paths-vs-declared-module-identity

An alternative solution can be to try with the tip / master version of Go, 
which has better upgrade logic that can often sidestep this problem, and 
also often has a better error message for this situation:

   $ go get golang.org/dl/gotip && gotip download
   $ gotip get -u all
   $ gotip mod tidy

Hope that helps, but reply back to the list with more details if it does 
not.

Regards,
thepudds

On Monday, June 3, 2019 at 9:52:52 AM UTC-4, Joseph Lorenzini wrote:
>
> Hi all,
>
> I've been happily using go modules for several months. A couple days ago, 
> I decided to update my dependencies with go get -u. This failed with:
>
> go: github.com/golang/lint@v0.0.0-20190409202823-959b441ac422: parsing 
> go.mod: unexpected module path "golang.org/x/lint"
>
> Which led me to: https://github.com/golang/go/issues/30831
>
> So then I added a replace directive but that failed too because of 
> https://github.com/golang/go/issues/26904
>
> Which means as long as I depend on the golint package,  then go get -u 
> will not work. And then it occurred to me: how and why is this package 
> being pulled in? If i can remove my module's transitive dependency on 
> golint, then go get -u will start working. So I first ran go mod tidy. 
> Then I used go mod why however that was no help because whenever i executed 
> go mod why on the module, this is the output returned:
>
> (main module does not need module github.com/golang/lint)
>
> I also verified that github.com/golang/lint is not in my go.mod after 
> running go mod tidy. My understanding of tidy is that it pulls in direct 
> and transitive module dependencies ergo if go get -u is trying to upgrade 
> my deps according to go.mod, then it should be in my go.mod. So I am 
> stumped why go get -u is trying to upgrade golint, when according to go mod 
> tidy, my go.mod file, and go mod why, I don't have a dependency on it.
>
> Any advice would be appreciated.
>
> Thanks,
> Joe
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3dc32542-d423-4920-a6ff-d4171b4667ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] modules: available GOPROXY instances or implementations?

2019-06-01 Thread t hepudds
Here is a partial list of Go modules proxy implementations or instances 
(sorted by url):

https://docs.gomods.io(Athens)
https://gocenter.io(JFrog)
https://goproxy.cn
https://goproxy.io
https://proxy.golang.org   (Go team)
https://thumbai.app  (THUMBAI)

Of those, as far I am aware GoCenter has declared itself production ready. 
The others in that list seem to be in beta or earlier stages to my 
knowledge, with the possible exception of goproxy.io, where I am not sure. 
Does anyone know if any additional ones on that list are production ready?

In addition, it looks like the core Go team is getting ready to publish the 
code for a reference implementation for a GOPROXY (currently in 
https://golang.org/cl/176541). This is part of an overall proposal to 
publish a series of 'golang.org/x/mod' packages focused on module mechanics 
and GOPROXY / GOSUMDB components. Some of those packages have been 
published at this point, with the planned list and status enumerated in 
https://github.com/golang/go/issues/31761.
 
>From some brief searching, it seems there are some additional personal 
projects that might be more experimental GOPROXY implementations, or 
otherwise fairly early stage.

Does anyone know of any additional GOPROXY instances or implementations 
that are either currently production ready or seem to be working towards a 
production ready status? 

Regards,
thepudds

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d97b4ffa-e864-428b-a76a-e0944b606f76%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: go modules and ambiguous import, how to fix it?

2019-05-17 Thread t hepudds
Regarding that specific playground error, bradfitz kindly chimed in via a 
more concise forum:

  
 

   https://twitter.com/bradfitz/status/1129508420925644802   
  
 

   It's because fsnotify doesn't support nacl/amd64p32.

   I get the same results locally:

   https://play.golang.org/p/IWR9KSQbUnV 
  
 


Regards,
thepudds
On Friday, May 17, 2019 at 5:50:38 PM UTC-4, t hepudds wrote:
>
> I only took a brief look at this, but this seems to be a tricky one.
>
> You could try: 
>   
>go get github.com/ugorji/go/codec@none
>
> That made your example then work for me locally.
>
> From the doc (https://golang.org/cmd/go/#hdr-Module_aware_go_get): 
>   
>   "The version suffix @none indicates that the dependency should be 
> removed entirely, downgrading or removing modules depending on it as 
> needed."
>   
> However, then using the resulting go.mod on the playground fails with a 
> different error:
>https://play.golang.org/p/EB5T7yuRQSC
>
> I did not look into that error, but I am not sure if that is a playground 
> specific problem with the playground's new module support, or maybe some 
> package has some NaCl build tag issue, or something else entirely. (It is 
> fun, though, to add a go.mod to the playground!)
>
> It seems viper might be depending on an older problematic version of 
> github.com/ugorji/go/codec. I would recommend you add your example to 
> this viper issue here:
>   https://github.com/spf13/viper/issues/658
>   
> The viper project might need to update their go.mod and issue a new viper 
> release, but not 100% sure.
>
> Some additional related discussion in:
>   https://github.com/gin-gonic/gin/issues/1897
>   https://github.com/golang/go/issues/29332#issuecomment-448669442
>   
> It probably also makes sense to open a clean new issue on the Go issue 
> tracker with your example, and ask for a clearer diagnostic message or a 
> simpler solution. (The Go issue mentioned above is closed).
>
> Sorry that is not a great answer,
> thepudds
>
> On Friday, May 17, 2019 at 4:52:12 PM UTC-4, Jérôme LAFORGE wrote:
>>
>> How can I import this both modules without ambiguous?
>>
>> https://play.golang.org/p/YgrEmcTallk
>>
>> Thx for your help.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/bd56612a-013c-48f5-9e02-167b0c3edfe6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[go-nuts] Re: go modules and ambiguous import, how to fix it?

2019-05-17 Thread t hepudds
I only took a brief look at this, but this seems to be a tricky one.

You could try: 
  
   go get github.com/ugorji/go/codec@none

That made your example then work for me locally.

>From the doc (https://golang.org/cmd/go/#hdr-Module_aware_go_get): 
  
  "The version suffix @none indicates that the dependency should be removed 
entirely, downgrading or removing modules depending on it as needed."
  
However, then using the resulting go.mod on the playground fails with a 
different error:
   https://play.golang.org/p/EB5T7yuRQSC

I did not look into that error, but I am not sure if that is a playground 
specific problem with the playground's new module support, or maybe some 
package has some NaCl build tag issue, or something else entirely. (It is 
fun, though, to add a go.mod to the playground!)

It seems viper might be depending on an older problematic version of 
github.com/ugorji/go/codec. I would recommend you add your example to this 
viper issue here:
  https://github.com/spf13/viper/issues/658
  
The viper project might need to update their go.mod and issue a new viper 
release, but not 100% sure.

Some additional related discussion in:
  https://github.com/gin-gonic/gin/issues/1897
  https://github.com/golang/go/issues/29332#issuecomment-448669442
  
It probably also makes sense to open a clean new issue on the Go issue 
tracker with your example, and ask for a clearer diagnostic message or a 
simpler solution. (The Go issue mentioned above is closed).

Sorry that is not a great answer,
thepudds

On Friday, May 17, 2019 at 4:52:12 PM UTC-4, Jérôme LAFORGE wrote:
>
> How can I import this both modules without ambiguous?
>
> https://play.golang.org/p/YgrEmcTallk
>
> Thx for your help.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e41cc46b-711d-4ac9-b16e-ae94b5f24290%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [go-nuts] go mod tidy pulls in too much

2018-09-29 Thread t hepudds
Hi Scott,

Regarding your comment, a related issue (which is still open) is this one:

   #26955: “cmd/go: provide straightforward way to see non-test dependencies”
  https://github.com/golang/go/issues/26955

--thepudds 

Terseness courtesy of my mobile device

> On Sep 29, 2018, at 9:53 AM, Scott Cotton  wrote:
> 
> Hi all,
> 
> I think this is related to https://github.com/golang/go/issues/26913 which is 
> closed but still has commentary that seems unresolved after it was closed .
> 
> Scott
> 
> 
>> On Friday, 28 September 2018 20:11:29 UTC+2, Harmen wrote:
>> On Fri, Sep 28, 2018 at 10:19:50AM -0700, thepud...@gmail.com wrote: 
>> > Hi Harmen, 
>> > 
>> > And my first sentence might not have been clear. When I said "even in your 
>> > current situation, 'go build' is still pulling in exactly what it needs", 
>> > I 
>> > was trying to reference the actual compilation process. 
>> > 
>> > In other words, I was just trying to make it clear that even if you have 
>> > "extra" dependencies appearing in your go.mod file, the resulting binary 
>> > produced by 'go build' does not have anything extra or any unused 
>> > dependencies. 
>> 
>> Hi thepudds, 
>> thanks for your anwers. 
>> 
>> It doesn't hurt per-se, but all those unused modules will be vendored in the 
>> repo (or the CI will have to download them every time). Maybe the consul 
>> repo 
>> is a particularly unlucky repo, since the /api package is small compared to 
>> the 
>> rest of the repo, but still. 
>> 
>> As for the argument that it's for test reproducibility, I do not follow 
>> that. 
>> Everything to test /api is there, and the stuff I don't import doesn't need 
>> to 
>> be tested in the first place. 
>> 
>> > In any event, I wanted to share at least my personal understanding, but of 
>> > course happy to learn more... 
>> 
>> I'm trying to do the same :) 
>> Thanks! 
>> 
>> > 
>> > --thepudds 
>> > 
>> > 
>> > On Friday, September 28, 2018 at 1:00:17 PM UTC-4, thepud...@gmail.com 
>> > wrote: 
>> > > 
>> > >  > "So if consul adds a go.mod file in the root, then `mod tidy` will 
>> > > suddenly 
>> > > behave as I would expect (i.e. not pull in unused dependencies)? " 
>> > > 
>> > > Hi Harmen, 
>> > > 
>> > > Just in case this isn't already clear-- note that even in your current 
>> > > situation, 'go build' is still pulling in exactly what it needs (and not 
>> > > pulling in unused dependencies). 
>> > > 
>> > > I think there is not a significant harm in your go.mod containing these 
>> > > indirect dependencies (aside from of course seeing that longer list, 
>> > > etc.). 
>> > > 
>> > > And there is some benefit -- this behavior is part of what provides for 
>> > > 100% reproducible builds and tests. The modules system records precise 
>> > > dependency version information, and in your case, that precise 
>> > > dependency 
>> > > version information for some of your indirect dependencies is being 
>> > > recorded in your go.mod (given that it is not yet recorded in the 
>> > > non-existent go.mod of some of your direct dependencies).   
>> > > 
>> > > As an example, this behavior helps make sure that `go test all` is 100% 
>> > > reproducible for you (where `go test all` runs tests for both your 
>> > > module, 
>> > > your direct dependencies, and your indirect dependencies, which is 
>> > > valuable 
>> > > as one way of validating that the currently selected packages versions 
>> > > are 
>> > > compatible -- the number of possible version combinations is exponential 
>> > > in 
>> > > the number of modules, so in general you cannot expect your dependencies 
>> > > to 
>> > > have tested against all possible combinations of *their* dependencies). 
>> > > 
>> > > --thepudds 
>> > > 
>> > > On Friday, September 28, 2018 at 12:48:20 PM UTC-4, Harmen wrote: 
>> > >> 
>> > >> On Fri, Sep 28, 2018 at 04:48:32PM +0100, Paul Jolly wrote: 
>> > >> > Hi Harmen 
>> > >> > 
>> > >> > I described the problem on https://github.com/golang/go/issues/27920, 
>> > >> which 
>> > >> > > got 
>> > >> > > closed within three minutes as being "documented", and "works as 
>> > >> > > expected" (which I assume also means "works as intended"). 
>> > >> > > Is this really the intented behaviour? It seems unexpected to me. 
>> > >> > > Or 
>> > >> > > should I 
>> > >> > > simply stay away from `go mod tidy`? 
>> > >> > > 
>> > >> > 
>> > >> > I replied to your issue earlier. But could arguably have been 
>> > >> > slightly 
>> > >> more 
>> > >> > detailed in my response beyond simply linking to 
>> > >> > 
>> > >> https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod
>> > >>  
>> > >> > 
>> > >> > As you note, github.com/hashicorp/consul/api is a package. It is a 
>> > >> package 
>> > >> > in the module github.com/hashicorp/consul. Despite there being no 
>> > >> go.mod in 
>> > >> > github.com/hashicorp/consul, the go tool simulates it as a module. 
>> >