[go-nuts] Re: Using underscore in return statements - a small convenience idea

[Nick Craig-Wood]

Ah ha! I thought there would be a related issue I couldn't find :-)

`zero` would remove most of the pain though IMHO `_` looks neater in the 
return statements!

Thanks for the pointer Brian.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9393cf35-0831-4c5d-a0c8-da639e7443a2n%40googlegroups.com.

[go-nuts] Using underscore in return statements - a small convenience idea

[Nick Craig-Wood]

// Sometimes you end up typing this having to return
// a value you made up but don't care about.
func f1() (int, error) {
return -1, errors.New("something went wrong")
}

// Or maybe like this with named return values.
func f2() (a int, err error) {
return a, errors.New("something went wrong")
}

// It would be nice if you could type this instead.
//
// The underscore being used here to return whatever is in
// the return slot when the return statement is executed.
func f() (a int, err error) {
return _, errors.New("something went wrong")
}

The last doesn't compile of course, it gives

cannot use _ as value or type

Being able to use _ seems like a nice convenience to me. What it actually 
returns is completely defined. And it would save us having to figure out 
types for stuff that you can't use `nil` for, eg `time.Time` is 
particularly annoying.

Thoughts?

Play: https://go.dev/play/p/TeNMV9zWemg

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ce73e522-fa63-4493-9376-daa16848b3c0n%40googlegroups.com.

Re: [go-nuts] Re: Making temporary changes to 3rd party modules without breaking go install

[Nick Craig-Wood]

This is super annoying - you'll see me in the bug report you linked!

The way to work around it is to fork the upstream repo into your own 
repository, apply the one line patch there, then change all the package 
lines and go.mod to point to your new repo.


After you've done all that then in the client code start using the your 
forked repo instead of the original.


I've had to do this lots of times with rclone. It works but is is a pain.

When the upstream merges your patch you can abandon your fork and revert 
the import changes.


Moral of the story - never commit a replace statement in go.mod as it 
breaks go install and go get.


On 24 February 2023 17:27:54 Adrian Hesketh 
 wrote:

I think I have two choices:

1 - stop recommending the use of `go install`, because it:
Doesn't support the replace directive.
Had an unexpected failure mode that I don't properly understand even after 
reading the error message and the big thread.
I think it means I'm not allowed to use replace directives at all, and the 
stuff about "interpreted differently" is irrelevant. I think "it" in the 
second sentence case means "The go.mod file" - I have no idea what "than if 
it were the main module" is about.
Doesn't set the version number in the binary like my build script does 
(using ldflags).
2 - work around the limitations of `go install` by forking the library 
_and_ changing the module name instead of using a redirect.


Not using `go install` is unappealing, because then I'd have to write my 
own cross-platform installation script, but it would solve people not 
having the build number (templ version would produce useful output). In 
addition, people might try to use `go install`, and then I'd have to try 
and explain why it doesn't work.


Working around the limitations is unappealing, because I have to change the 
go.mod names, and then package names in all .go filenames both in the fork 
I need to maintain, _and_ my own software. When the PR gets merged, if the 
go.mod replace directive is supported, it would only be one line change to 
the go.mod file to go back to the original package (not my fork), without 
it, it's a minimum 10 file change to update the package names again, and 
then to delete my fork, which also invalidates old builds.


On Thursday, 23 February 2023 at 18:02:47 UTC Adrian Hesketh wrote:
I've got a program I maintain that uses a 3rd party module. That module has 
a bug in it, and I've raised a pull request to fix it.


But... I need to release a new version of my program as soon as possible.

My understanding was that I can use a replace directive in the go.mod file 
to replace the implementation. So I forked the repo, made my single commit 
change and put a one-liner in the go.mod file:


replace go.lsp.dev/protocol => github.com/a-h/protocol 
v0.0.0-20230222141054-e5a15864b7f1


Locally, building the project with `go build` worked perfectly well, so I 
tagged a release, and called it a day.


However, users started reporting that they can't install the package with 
`go install` because it's broken with an error message of:


go install github.com/a-h/templ/cmd/templ@latest
go: github.com/a-h/templ/cmd/templ@latest (in github.com/a-h/te...@v0.2.198):
The go.mod file for the module providing named packages contains one or
more replace directives. It must not contain directives that would cause
it to be interpreted differently than if it were the main module.


It was a surprise, because it works perfectly well for me. Reading through 
various issues (linked below), I see lots of discussion, but no clear 
guidance on what to do as an author.

https://github.com/golang/go/issues/40276
https://groups.google.com/g/golang-nuts/c/10AJZaH9OFU/m/WobvkcB5BwAJ

I assume it's fairly common to need to patch a 3rd party dependency, so I 
must be missing something. What's the strategy to employ here?


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an 
email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/11d78bec-a188-4474-b05e-9d573b33d788n%40googlegroups.com.


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/18696f45768.2815.6b34a2a81515583dc95e5c0809db06bb%40craig-wood.com.

[go-nuts] Re: Considering dropping GO386=387

[Nick Craig-Wood]

I make a GO386=387 build for rclone, eg

https://github.com/rclone/rclone/issues/437

People love running rclone on ancient computers to rescue data off them I 
guess.

This would affect a very small percentage of users and there are always 
older versions of rclone they can use so I'm not too bothered if support is 
dropped.

I haven't tried compiling rclone with gccgo for a while.

It would be helpful if the build fails rather than silently ignoring the 
GO386 flag if this proposal does go forward.

On Tuesday, 14 July 2020 at 13:56:58 UTC+1 aus...@google.com wrote:

> Hi everyone. We’re exploring the possibility of dropping 387 
> floating-point support and requiring SSE2 support for GOARCH=386 in the 
> native gc compiler, potentially in Go 1.16. This would raise the minimum 
> GOARCH=386 requirement to the Intel Pentium 4 (released in 2000) or AMD 
> Opteron/Athlon 64 (released in 2003).
>
> There are several reasons we’re considering this:
>
>1. While 387 support isn’t a huge maintenance burden, it does take 
>time away from performance and feature work and represents a fair amount 
> of 
>latent complexity.
>2. 387 support has been a regular source of bugs (#36400 
>, #27516 
>, #22429 , 
>#17357 , #13923 
>, #12970 , 
>#4798 , just to name a few).
>3. 387 bugs often go undetected for a long time because we don’t have 
>builders that support only 387 (so unsupported instructions can slip in 
>unnoticed).
>4. Raising the minimum requirement to SSE2 would allow us to also 
>assume many other useful architectural features, such as proper memory 
>fences and 128 bit registers, which would simplify the compiler and 
> runtime 
>and allow for much more efficient implementations of core functions like 
>memmove on 386.
>5. We’re exploring switching to a register-based calling convention in 
>Go 1.16, which promises significant performance improvements, but 
> retaining 
>387 support will definitely complicate this and slow our progress.
>
>
> The gccgo toolchain will continue to support 387 floating-point, so this 
> remains an option for projects that absolutely must use 387 floating-point.
>
> We’d like to know if there are still significant uses of GO386=387, 
> particularly for which using gccgo would not be a viable option.
>
> Thanks!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/babd66fd-fe7f-4840-b1aa-8cb32b499b67n%40googlegroups.com.

[go-nuts] Generics and parentheses

[Nick Craig-Wood]

In my mind the use of [ ]  clashes horribly with the array declaration syntax. 
When I see [int] in particular my brain says, ah array declaration of size int, 
what, no wait...

This makes [identifier] mean two different things depending on whether 
indentifier is a const integer or whether identifier is a type. I think the 
suffix Vs prefix makes it unambiguous but I'm not sure.

I expect I'd get used to it, but I can see this being difficult for beginners.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8c37b668-5b4c-41fc-aac1-e8876901ff2bo%40googlegroups.com.

[go-nuts] Re: [ANN] pygor: yet another Python to Go regurgitator

[Nick Craig-Wood]

Very interesting :-)

It looks like a useful tool for producing a first rough cut of a python 
program.


-- Nick

On 20/10/18 05:16, Raffaele Sena wrote:
Inspired by ESR pytogo (and tired of writing python code), I went the 
complete opposite way and came up with pygor:


https://github.com/raff/pygorhttps://github.com/raff/pygor

pygor is written in Go, using the Python parser and AST from 
https://github.com/go-python/gpython (so right now it only targets 
Python 3.4). The origin of this was actually a python implementation 
that I started  a long time ago, using Python ast, but hat didn't go 
very far because it was hard to do Go things in python.


It doesn't generate runnable code, but aspire to generate at least 
code that passes the smell test of "go fmt" (with the problem that if 
"go fmt" fails to correctly format the code, the translation fails).


pygor also attempt to convert some of the python magic (list and dict 
comprehension, for example) to Go, again mainly in order to generate 
formatted code. And it also comes with a concept of "runtime" 
(automatically imported when needed) to implement a couple of things 
that are hard to do inline.


Enjoy,

Raffaele


--
You received this message because you are subscribed to the Google 
Groups "go-python" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to go-python+unsubscr...@googlegroups.com 
<mailto:go-python+unsubscr...@googlegroups.com>.
To post to this group, send email to go-pyt...@googlegroups.com 
<mailto:go-pyt...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-python/CANKfucYmWed786-ErNmO-Dyu-zVDSV43AfG17aDoMq1diq9Q9A%40mail.gmail.com 
<https://groups.google.com/d/msgid/go-python/CANKfucYmWed786-ErNmO-Dyu-zVDSV43AfG17aDoMq1diq9Q9A%40mail.gmail.com?utm_medium=email_source=footer>.

For more options, visit https://groups.google.com/d/optout.



--
Nick Craig-Wood  -- http://www.craig-wood.com/nick

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] What are the consequences of go install -ldflags "-s"

[Nick Craig-Wood]

On 04/11/16 14:55, Ian Lance Taylor wrote:
> On Fri, Nov 4, 2016 at 7:39 AM, Nick Craig-Wood <n...@craig-wood.com> wrote:
>> I know I can build go binaries passing the `-s` flag to the linker to
>> strip them and make them smaller.  In my tests (with rclone) it makes it
>> 60% of the size so a significant saving. I'd like to enable this for
>> binaries I distribute to end users.
>>
>>   go install -ldflags "-s"
>>
>> However what are the consequences?  The link docs say what it does but
>> not what the consequences are.
>>
>> https://golang.org/cmd/link/
>>
>> -s
>> Omit the symbol table and debug information.
>>
>> I can see that I still get source code lines in my backtraces which is
>> very important to me but what am I losing?  Why wouldn't I want to do this?
> 
> You won't be able to run gdb or various other non-Go-specific tools.

Thanks.  If that is the only consequence than I'm good with that.
Anyone who wants to run gdb can compile their own binary.

-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Architect a daemon program

[Nick Craig-Wood]

On 11/10/16 18:09, Tong Sun wrote:
> I've searched the mlist archive and have learned that I should
> avoid daemonize as much as possible. So instead of label my program as a
> daemon (or singleton or anything), let me describe what I need to do. 
> 
> I need the program (say, named as`myprog`) to fork into the background
> (if possible) with the "start" command-line option.

You could try this

https://github.com/sevlyar/go-daemon

This implements the daemonization by re-running the executable with an
environment variable.

-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Slow compile times

[Nick Craig-Wood]

On 04/10/16 09:27, rene.zbin...@gmail.com wrote:
> I use the following packages in a test file:
>
> import (
> "testing"
>
> "github.com/coreos/etcd/clientv3"
> "github.com/coreos/etcd/integration"
> "github.com/coreos/pkg/capnslog"
> )
>
>
> Now I have the problem, that I have really slow compile times (up to
> 10 seconds instead of 1-2 seconds).
>
> I know for normal packages, I have to do the following:
>
> go install github.com/coreos/etcd/clientv3
> go install github.com/coreos/etcd/integration
> go install github.com/coreos/pkg/capnslog
>
>
> But this does not help me here. Has anybody an idea to fix this 'issue'?

try `go test -i` to install the dependencies for the test.

-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] file or folder encryption tool

[Nick Craig-Wood]

On 19/09/16 10:49, DrGo wrote:
> Has any used Go to develop a cross-platform an easy-to-use utility to encrypt 
> individual files or folders? 
> 

You could use rclone to encrypt a directory of files.  It can also
transfer them to cloud storage systems too!

See rclone.org for docs and github.com/ncw/rclone for code.

What you would do is configure a crypt remote using a directory on your
disk.  You can then rclone copy /path/to/unencryptedfiles remote:

It perhaps isn't exactly the workflow you want but it is written in Go!

It uses the go port of secret box for encrypting stuff.

-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Copying http.DefaultTransport

[Nick Craig-Wood]

I'd like to be able to copy the http.DefaultTransport then tweak it for
my app.

http.DefaultTransport has a lot of good defaults in, so copying it then
tweaking it makes it forward compatible with changes (a good example of
a recent change would be the ExpectContinueTimeout).

You might think you could just do

  newTransport := new(http.Transport)
  *newTransport = *http.DefaultTransport
  new.Transport.MaxIdleConns = myTweakedValue

But if you do that then go vet moans about copying mutexes (for very
good reasons).

The best I could come up with was using a bit of reflection

https://play.golang.org/p/l2mkCTZSzL

Is there a better way?

Should making an http.Transport with the default settings in be
something the standard library should support?  Maybe a
NewDefaultTransport() function?

-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Speeding up Go Report Card

[Nick Craig-Wood]

On 02/09/16 14:41, Shawn Smith wrote:
> I'm one of the maintainers of Go Report Card http://goreportcard.com/
> and I am looking for advice on how to make it faster.
> 
> Basically we run a series of checks (gofmt, vet, etc.) on your Go
> repository and provide a grade based on how many errors each check
> returns. Each check runs in a separate goroutine
> (https://github.com/gojp/goreportcard/blob/master/handlers/checks.go#L124),
> and they are all shelling out using exec.Command
> (https://github.com/gojp/goreportcard/blob/master/check/utils.go#L170).

I'm pretty sure this means that command will use its own OS thread
rather than just a go routine since it will be blocked in a syscall
waiting for the command to end.

Threads are much more expensive than go-routines, eg see

http://stackoverflow.com/questions/344203/maximum-number-of-threads-per-process-in-linux

So I'd check to see if you are running out of threads, or memory because
of lots of threads.

> The server we run this on is a Digital Ocean droplet with 1 GB of RAM
> (would upgrading it help?)
> 
> We normally only see issues on repositories with a very large number of
> files.

What issues do you see?

-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Handling dynamic and unknown number of wait groups?

[Nick Craig-Wood]

Here is some code I wrote which demonstrates a solution.  This is a
recursive concurrent directory lister which I think is an identical
problem to yours - you start with one directory - you find new
directories in the course of listing them which you also want to list,
but you want to limit the concurrency and not return until they are all
done.

https://github.com/ncw/rclone/blob/master/dircache/list.go#L27

I used another WaitGroup and whenever I put anything into the job queue
I added 1 to it and whenever I removed something I ran .Done() it in.
This gives you something to wait on at the end.

Note I use an extra go routine to put the new directories into the queue
- this is to avoid a deadlock when all the workers are busy trying to do
the same.

On 29/06/16 17:52, Inspectre Gadget wrote:
> Hey everyone,
> 
> Here’s my issue, I will try to keep this short and concise:
> 
> I have written a program that will accept a URL, spider that URL’s
> domain and scheme (http/https), and return back all input fields found
> throughout to the console. The purpose is largely for web application
> security testing, as input fields are the most common vulnerability
> entry points (sinks), and this program automates that part of the
> reconnaissance phase.
> 
> Here is the problematic
> code: 
> https://github.com/insp3ctre/input-field-finder/blob/ce7983bd336ad59b2e2b613868e49dfb44110d09/main.go
> 
> 
> The issue lies in the last for loop in the main() function. If you were
> to run this program, it would check the queue and workers so frequently
> that it is bound to find a point where there are both no workers
> working, and no URLs in the queue (as proved by the console output
> statements before it exits). Nevermind that the problem is exacerbated
> by network latency. The number of URLs actually checked varies on every
> run, which causes some serious inconsistencies, and prevents the program
> from being at all reliable.
> 
> The issue was fixed
> here: 
> https://github.com/insp3ctre/input-field-finder/blob/f0032bb550ced0b323e63be9c4f40d644257abcd/main.go
> 
> 
> I fixed it by removing all concurrency from network requests, leaving it
> only in the internal HTML processing functions.
> 
> So, the question is- how does one run efficient concurrent code when the
> number of wait groups is dynamic, and unknown at program initialization?
> 
> I have tried:
> 
>   * Using “worker pools”, which consist of channels of workers. The for
> loop checks the length of the URL queue and the number of workers
> available. If the URL queue is empty and all the workers are
> available, then it exits the loop.
>   * Dynamically adding wait groups (wg.Add(1)) every time I pull a URL
> from the URL queue. *I can’t set the wait group numbers before the
> loop, because I can never know how many URLs are going to be checked.*
> 
> 
> So I have tried using both channels and wait groups to check alongside
> the URL queue length to determine whether more concurrent network
> requests are needed. In both cases, the for loop checks the values so
> fast that it eventually stumbles upon a non-satisfied loop condition,
> and exits. This usually results in either the program hanging as it
> waits for wait groups to exit that never do, or it simply exits
> prematurely, as more URLs are added to the queue after the fact.
> 
> I would really like to know if there is a way to actually do this well
> in Go.
> 
> Cheers,
> 
> Inspectre 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com
> <mailto:golang-nuts+unsubscr...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


-- 
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.