Re: [go-nuts] Re: Why not tuples?

[burak serdar]

On Sat, Dec 3, 2022 at 8:47 PM Diogo Baeder  wrote:

> Hi there, sorry for weighting in so late in the game, but I just started
> again to learn Go and was thinking why the language still doesn't have a
> tuple type.
>
> Now, imagine this scenario: I have a web application which has to access a
> webservice that responds with JSON payloads; These payloads are a list of
> values, where each value is a smaller list like '[20220101, 1.234, "New
> York"]'. And these smaller lists follow the same type sequence: int64,
> float64, string. Suppose that I want to filter those values and send a
> response to the client, with the data structure unchanged (same format and
> types). Today, it doesn't seem to be possible to do that in Go, unless I do
> some dirty hack like decoding to '[]any' and then cast to the other types,
> and then hack again to put these values in the response to the client.
>

What you described above is a struct.


>
> I totally understand the reasoning for preferring the usage of structs for
> heterogeneous data (and I myself do prefer them, they're much more powerful
> in general), but there's real world data that's available like in the
> example above, and we just can't go on changing them at their sources. I
> might be mistaken (please let me know if it's the case), but it seems like
> Go is missing an opportunity to interoperate with what's a fundamental data
> structure in many other languages (Python, Rust etc). I'm having a lot of
> fun learning to use the language, and would be happy to see this feature
> being implemented at the core.
>

In general, you can implement most tuple functionality using []any.
However, when dealing with unpredictable data structures like an unknown
JSON document, third-party libraries might be better suited than a
map[string]any. When you have a tuple like that, you have to "discover" the
type of each variable and parse it to do any nontrivial processing. You
can't really unmarshal a string from a JSON document and expect to get a
date in the tuple. My current work is on the interoperability of
heterogeneous data, so for XML documents we use DOM libraries, for JSON
documents we wrote https://github.com/bserdar/jsonom, etc.


>
> (Maybe what I said above is total BS, I acknowledge that since I'm an
> almost complete ignorant in the language)
>
> Cheers!
>
> On Thursday, April 19, 2018 at 1:03:55 PM UTC-3 Louki Sumirniy wrote:
>
>> Multiple return values. They do kinda exist in a declarative form of
>> sorts, in the type signature, this sets the number and sequence and types
>> of return values. You could even make functions accept them as also input
>> values, I think, but I don't think it works exactly like this. I'm not a
>> fan of these things because of how you have to nominate variables or _ and
>> type inference will make these new variables, if you  := into whatever the
>> return was.
>>
>> I'm not sure what the correct word is for them. Untyped in the same way
>> that literals can be multiple types (especially integers) but singular in
>> their literal form.
>>
>>
>> On Thursday, 19 April 2018 16:06:42 UTC+3, Jan Mercl wrote:
>>>
>>> On Thu, Apr 19, 2018 at 2:51 PM Louki Sumirniy 
>>> wrote:
>>>
>>> > Sorry for the self-promotion but it was relevant in that I was working
>>> on how to tidy up the readability of my code and needed multiple returns
>>> and simple untyped tuples were really not nearly as convenient as using a
>>> type struct.
>>>
>>> I have no idea what you mean by 'untyped tuples' because Go does not
>>> have tuples, or at least not as a well defined thing. I can only guess if
>>> you're trying to implement tuples in Go with an array, slice or a struct,
>>> ...? To add to my confusion, Go functions can have as many return values as
>>> one wishes just fine, ie. I obviously do not even understand what problem
>>> you're trying to solve. Sorry.
>>>
>>>
>>> --
>>>
>>> -j
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/8e728e0f-341d-4340-a868-aac028dfc443n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqq%2BDmRDuqcxW-DF%2ByDqTrAu%3DNv51wLWZsx2ERiYEv%3DZ1w%40mail.gmail.com.

Re: [go-nuts] Re: Why not tuples?

[Diogo Baeder]

Hi there, sorry for weighting in so late in the game, but I just started 
again to learn Go and was thinking why the language still doesn't have a 
tuple type.

Now, imagine this scenario: I have a web application which has to access a 
webservice that responds with JSON payloads; These payloads are a list of 
values, where each value is a smaller list like '[20220101, 1.234, "New 
York"]'. And these smaller lists follow the same type sequence: int64, 
float64, string. Suppose that I want to filter those values and send a 
response to the client, with the data structure unchanged (same format and 
types). Today, it doesn't seem to be possible to do that in Go, unless I do 
some dirty hack like decoding to '[]any' and then cast to the other types, 
and then hack again to put these values in the response to the client.

I totally understand the reasoning for preferring the usage of structs for 
heterogeneous data (and I myself do prefer them, they're much more powerful 
in general), but there's real world data that's available like in the 
example above, and we just can't go on changing them at their sources. I 
might be mistaken (please let me know if it's the case), but it seems like 
Go is missing an opportunity to interoperate with what's a fundamental data 
structure in many other languages (Python, Rust etc). I'm having a lot of 
fun learning to use the language, and would be happy to see this feature 
being implemented at the core.

(Maybe what I said above is total BS, I acknowledge that since I'm an 
almost complete ignorant in the language)

Cheers!

On Thursday, April 19, 2018 at 1:03:55 PM UTC-3 Louki Sumirniy wrote:

> Multiple return values. They do kinda exist in a declarative form of 
> sorts, in the type signature, this sets the number and sequence and types 
> of return values. You could even make functions accept them as also input 
> values, I think, but I don't think it works exactly like this. I'm not a 
> fan of these things because of how you have to nominate variables or _ and 
> type inference will make these new variables, if you  := into whatever the 
> return was.
>
> I'm not sure what the correct word is for them. Untyped in the same way 
> that literals can be multiple types (especially integers) but singular in 
> their literal form.
>
>
> On Thursday, 19 April 2018 16:06:42 UTC+3, Jan Mercl wrote:
>>
>> On Thu, Apr 19, 2018 at 2:51 PM Louki Sumirniy  
>> wrote:
>>
>> > Sorry for the self-promotion but it was relevant in that I was working 
>> on how to tidy up the readability of my code and needed multiple returns 
>> and simple untyped tuples were really not nearly as convenient as using a 
>> type struct.
>>
>> I have no idea what you mean by 'untyped tuples' because Go does not have 
>> tuples, or at least not as a well defined thing. I can only guess if you're 
>> trying to implement tuples in Go with an array, slice or a struct, ...? To 
>> add to my confusion, Go functions can have as many return values as one 
>> wishes just fine, ie. I obviously do not even understand what problem 
>> you're trying to solve. Sorry.
>>
>>
>> -- 
>>
>> -j
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8e728e0f-341d-4340-a868-aac028dfc443n%40googlegroups.com.

Re: [go-nuts] cgo C.uint64_t compatiblity on multiple platforms?

[David Stainton]

Thanks, yes switching those type casts to C.size_t worked.

On Saturday, December 3, 2022 at 2:39:24 PM UTC-5 kortschak wrote:

> On Sat, 2022-12-03 at 10:51 -0800, David Stainton wrote:
> > Greetings,
> >
> > I think my question is something like: "how to make my usage of
> > C.uint64_t work on all platforms?" or "Is there something obviously
> > wrong with my cgo bindings that is causing this not to build on all
> > platforms?"
> >
> > I wrote this cgo binding for the reference implementation of Sphincs+
> > (it's a hash based post quantum signature scheme):
> >
> > 
> https://github.com/katzenpost/katzenpost/blob/main/sphincsplus/ref/binding.go
> >
> > Recently we noticed it fails to build on MacOS and Windows:
> >
> > 
> https://github.com/katzenpost/katzen/actions/runs/3609590801/jobs/6082872618
> >
> > So I "fixed" it by changing the typecast to use C.uint64_t instead of
> > C.ulong:
> >
> > https://github.com/katzenpost/katzenpost/pull/110/files
> >
> > However that causes the build to fail using Go1.19.3 on MacOS,
> > although it works on Windows:
> >
> > 
> https://github.com/katzenpost/katzen/actions/runs/3609771045/jobs/6083165054
> >
> >
> > Sincerely,
> > David Stainton
>
> The header file definition for crypto_sign_signature uses size_t for
> the parameters that are passed as ulong in the calls that are causing
> the issue. These types are not always the same size, so perhaps in e.g.
>
> https://github.com/katzenpost/katzenpost/blob/a165cb2a13e40f3a15dd1fa296a7763d8b638ae0/sphincsplus/ref/binding.go#L120-L125
> making the ulong params be size_t would help.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ff839488-59ff-45d8-bd01-ebc47a9a473an%40googlegroups.com.

Re: [go-nuts] vendors and modules

[Andrew Athan]

Thanks Ian ... so a couple of comments & clarifications

(1) I see various other questions popping up in the list regarding go mod, 
go work, vendoring etc, and yes I certainly did review  
https://go.dev/blog/using-go-modules before posting my original question 
here. Point being, I think the documentation as currently presented is way 
too dense, leaves many degrees of freedom open, and thus it remains 
generally hard to quickly get to a basic understanding of the best practice 
of common usecases. I think the docs would greatly benefit from a list of 
those use cases that present a very brief outline of the commands that 
would be used to implement each, perhaps with some abridged contents of the 
go.mod's etc that result.

(2) With respect to my original question the answer for me was (as you 
mentioned)

go get github.com/dustinxie/lockfree
... add the import to appropriate places in the source
go mod tidy
go mod vendor

My original mistake was to run go mod tidy too early, such that it 
(silently) removed the lockfree package from go.mod because it did not 
appear anywhere in the source code.
Yes, I acknowledge that go mod tidy is documented to clean up go.mod in 
that manner, but for whatever reason, I missed that, and it is silent in 
its cleanup.
I was motivate to try go mod tidy because I noticed the the original go get 
dropped the module into go.mod as an indirect reference, and I wasn't sure 
that was correct
I might suggest that the -v option be made the default, and that a -q 
"silent" option be provided instead? Where can I make that suggestion to 
the golang maintainers?

(3) Now, imagine I wan't to make modifications to lockfree in such a way 
that those modifications can be submitted to my company's private repo, so 
that other developers can build the overall project successfully, and the 
modifications to lockfree can eventually be PR'ed to the original lockfree 
project. If I only fork lockfree to a new github repo, and I try to work 
with the go get/etc commands I will run into problems because in that new 
github repo, the lockfree/go.mod will contain the "wrong" reference for the 
module (it will point to the original github repo instead of the forked 
one).

To overcome this issue, in the repo that wants to use the modified lockfree 
I create a directory to contain all my git submodules and in there I do:

cd ~/my-go-project
mkdir submodules
cd submodules
git submodule add --depth=1 g...@github.com:my-company/lockfree.git

within that new clone I can then do the necessary feature additions to 
lockfree. Now, to resolve the issues with go's module system I then either:

(3a) add a  `replace github.com/dustinxie/lockfree => 
./submodules/lockfree` in the project's go.mod

OR

(3b) yes, I verified this works ok even in the project dir that contains 
its go.mod:

cd ~/my-go/project
go work init
got work add .
go work add ./submodules/lockfree

(4) Note: My understanding of the intent for `go mod vendor` is to snapshot 
the "external" sources used to build a project.

Reasonably: I might expect the contents of the vendor'ed lockfree project 
to reflect what was actually being used to build the project in the context 
that go mod vendor executed in.
Empirically:   `go mod vendor` instead deposits a copy of lockfree from the 
original project source, not the source referenced in my go.work file.

Should this difference be reported for consideration as a bug? If so, where 
should I do that?

Thanks in advance for your responses.
A.


On Thursday, December 1, 2022 at 3:53:13 PM UTC-8 Ian Lance Taylor wrote:

> On Thu, Dec 1, 2022 at 3:23 PM Andrew Athan  wrote:
> >
> > I am new to go devops (but not to the language). First, I must say, the 
> go module documentation is thorough but absolutely inscrutable for new 
> users. Googling for things like "golang add github module vendor" brings up 
> what appear to be either outdated or incorrect cookbooks. I've already 
> wasted 1 hour trying to figure out "the right way" to a module provided by 
> a github repo to an existing vendor'ed golang project.
> >
> > In short: HLP!
> >
> > All I want to do is use `github.com/dustinxie/lockfree.git` 
>  in the project, which *does* 
> commit the vendor directory to its git repo. I cannot seem to find any 
> semver defined in the lockfree project so I have no idea what to specify by 
> hand to go.mod, and there is a bewildering set possible commands to use ... 
> from `go get` to `go install` to `govendor` to `godep` (the latter appear 
> to be old news?)
> >
> > I would be eternally grateful for a few simple examples added to the 
> golang reference and/or tutorials for how to do this.
>
> First make sure that you have a go.mod file, typically by running "go
> mod init my-package-path". Then run "go get
> github.com/dustinxie/lockfree". The "go get" command will add the
> package to your go.mod file, and you will be able to import it in your
> Go files.
>
> 

Re: [go-nuts] Performance for concurrent requests

[Diogo Baeder]

Hey guys!

Turns out that one of the biggest bottlenecks was the type guessing when 
parsing the JSON content to a "map[string]any"; As soon as I implemented 
more appropriate structs to unmarshall the bytes I immediately got faster 
responses. Another big improvement was when changing from the standard JSON 
library to go-json ( https://github.com/goccy/go-json ).

Right now, I get response times close enough to Rust (just a tad slower, 
not much), which is good enough for me! :-)

The experiment project is up-to-date with my findings now. Thank you all 
for the help!

Cheers!

On Saturday, December 3, 2022 at 9:21:10 AM UTC-3 Diogo Baeder wrote:

> Hi all!
>
> Thanks for the inputs, I'll do some profiling here and update about my 
> findings.
>
> I don't want to change anything on Nginx because the comparison I'm doing 
> between different stacks is on the same basis. If I try to tune Nginx or 
> anything like that I'll be comparing apples to oranges, so that's not the 
> point of my experiment.
>
> Cheers!
>
> On Saturday, December 3, 2022 at 6:07:42 AM UTC-3 Brian Candler wrote:
>
>> Have you tried a tcpdump of the packets between the Go program and 
>> nginx?  Is it using HTTP/1.1 or HTTP/2?  If it's HTTP/1.1, does tcpdump 
>> show that it actually starts all 50 HTTP client requests simultaneously?
>>
>> You are making all these concurrent requests to the same host. The 
>> default of MaxConnsPerHost I believe is 0 (unlimited), but 
>> DefaultMaxIdleConnsPerHost is 2.  It could be worth cranking that up. I 
>> wonder if it's being forced to close down 48 of those connections 
>> immediately because it can't return them to the pool.
>>
>> I also wonder whether there's tuning required at the Nginx side, e.g. for 
>> the backlog queue.
>>
>> On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote:
>>
>>> I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In 
>>> general, though, using pprof is the best way to start to break down a 
>>> question like this. Would the actual workload involve more structured JSON, 
>>> or more computation with decoded values?
>>>
>>> On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org 
>>> wrote:
>>>
 On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder  wrote:

> Hi guys,
>
> I've been working on some experiments with different web application 
> stacks to check their performances under a specific scenario: one in 
> which 
> I have to make several concurrent requests and then gather the results 
> together (in order) and throw them out as JSON in the response body. 
> (This 
> project is only an experiment, but it's informing me for decisions that 
> have to be made for a real-world project where we have a similar 
> scenario.)
>
> However, probably due to my ignorance in Go, I cannot make it perform 
> as well as I expected - actually the best I'm getting are results that 
> are 
> even slower than Python, which was a surprise to me. Here they are: 
> https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin 
>
> So, looking at the code here: 
> https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go
>  
> - does anybody see any problem in the implementation that could be 
> hurting 
> performance? I tried using a WaitGroup, tried sharing memory (nasty, I 
> know, but just for the sake of experimentation), tried multiple JSON 
> codecs, different web frameworks, and nothing worked so far. I have a 
> feeling that I'm doing something fundamentally wrong and stupid, and that 
> somehow I can make a small change to make the experiment much faster.
>

 Have you measured how much time is spent on the http.Get calls? It is 
 likely that the 50 concurrent http.Get calls is the bottleneck. 

 Also note that you don't need a channel there. You can simply use a 
 waitgroup and set the results from inside the goroutine, because each 
 goroutine knows the index. But that is unlikely to change anything 
 measurable when compared to the Get calls.

  

>
> Thanks in advance, I'm sure this will help me learning more about the 
> language! :-)
>
> Cheers!
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com
>  
> 
> .
>


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To 

Re: [go-nuts] Go Memory Model question

[robert engels]

happens-before in this case would only guarantee 0, not 1. e.g. change the 
initializer to x = 3, then 3 must be guaranteed to be seen rather than the 
default value of 0

> On Dec 2, 2022, at 8:12 AM, burak serdar  wrote:
> 
> The way I read the memory model, this program can print 01, 00, and 11, but 
> not 10. This is because goroutine creation creates a synchronized before 
> relationship between x=1 and x=0, so even though there is a race, x=0 happens 
> before x=1.
> 
> On Fri, Dec 2, 2022 at 6:56 AM のびしー  > wrote:
> > I believe your example is basically equivalent to the ones in 
> > https://go.dev/ref/mem#badsync  which also 
> > contains an explanation of how the memory model implies this
> 
> @Wagner <>
> Thanks for your opinion, I think so too. I was not confident that my example 
> is equivalent to https://go.dev/ref/mem#badsync 
> , since my example reads from the same 
> variable.
> 
> 
> > Programs that modify data being simultaneously accessed by multiple 
> > goroutines must serialize such access. 
> @peterGo <>
> 
> I know that my example has data races. But the Go Memory Model guarantees a 
> limited number of outcomes for programs with races(unless runtime reports the 
> race and terminates). My question is about the limited outcomes.
>  <>
> 2022年12月2日(金) 22:02 Axel Wagner  >:
> I believe your example is basically equivalent to the ones in 
> https://go.dev/ref/mem#badsync  which also 
> contains an explanation of how the memory model implies this (or rather, how 
> it does not imply the opposite).
> 
> On Fri, Dec 2, 2022, 13:11 のびしー  > wrote:
> Hello, I have another question regarding the Go Memory Model. 
> 
> (1) Can this program print "10", according to the Memory Model?
> (2) If that is forbidden, which part of the Go Memory Model excludes such 
> behavior?
> 
> https://go.dev/play/p/Fn5I0fjSiKj 
> 
> ```go
> package main
> 
> var x int = 0
> 
> func main() {
> go func() {
> x = 1
> }()
> print(x) // first read: 1
> print(x) // second read: 0
> }
> ```
> 
> I draw a picture of the happens-before relation of this program here:
> 
> https://gist.github.com/nobishino/8150346c30101e2ca409ed83c6c25add?permalink_comment_id=4388680#gistcomment-4388680
>  
> 
> 
> I think the answer to (1) is yes. 
> Both reads are concurrent with x = 1, so each read can observe both x = 0 and 
> x = 1.
> And there is no constraint between the results of the first read and the 
> second read.
> So the second read can observe x = 0 even if the first read observes x = 0.
> 
> But I'm not very sure. Is this understanding correct?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAGoET5HyPxEhBETGWNPuYpDgXbm0JM3jeaKFdoQdtc5eGUR0Uw%40mail.gmail.com
>  
> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAGoET5Fk6Wwd7JOJOY%3DJVix9NymsP3yJ_P%2BaDPgtPTorL-X_Wg%40mail.gmail.com
>  
> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqphhx8S0sZDv%3Dj1rskhPvCBstzLo%2BAscz5X7pMnNFzWNQ%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this 

Re: [go-nuts] cgo C.uint64_t compatiblity on multiple platforms?

['Dan Kortschak' via golang-nuts]

On Sat, 2022-12-03 at 10:51 -0800, David Stainton wrote:
> Greetings,
>
> I think my question is something like: "how to make my usage of
> C.uint64_t work on all platforms?" or "Is there something obviously
> wrong with my cgo bindings that is causing this not to build on all
> platforms?"
>
> I wrote this cgo binding for the reference implementation of Sphincs+
> (it's a hash based post quantum signature scheme):
>
> https://github.com/katzenpost/katzenpost/blob/main/sphincsplus/ref/binding.go
>
> Recently we noticed it fails to build on MacOS and Windows:
>
> https://github.com/katzenpost/katzen/actions/runs/3609590801/jobs/6082872618
>
> So I "fixed" it by changing the typecast to use C.uint64_t instead of
> C.ulong:
>
> https://github.com/katzenpost/katzenpost/pull/110/files
>
> However that causes the build to fail using Go1.19.3 on MacOS,
> although it works on Windows:
>
> https://github.com/katzenpost/katzen/actions/runs/3609771045/jobs/6083165054
>
>
> Sincerely,
> David Stainton

The header file definition for crypto_sign_signature uses size_t for
the parameters that are passed as ulong in the calls that are causing
the issue. These types are not always the same size, so perhaps in e.g.
https://github.com/katzenpost/katzenpost/blob/a165cb2a13e40f3a15dd1fa296a7763d8b638ae0/sphincsplus/ref/binding.go#L120-L125
making the ulong params be size_t would help.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0dff332833277c47b106d5417a159981d6e4d6bd.camel%40kortschak.io.

Re: [go-nuts] HTTPS proxy issue using CONNECT method

['Sean Liao' via golang-nuts]

How are you setting the proxy?

- sean

On Sat, Dec 3, 2022, 16:46 Mauro Monteiro  wrote:

> Hello all,
>
> I have been facing an issue when I try to create a HTTP client which needs
> to connect through a HTTPS proxy using the HTTP CONNEC method. I know that
> it can be achieved setting my own http.Transport object. However the issue
> seems to be in the current implementation of /net/http/transport.go code.
>
> In my environment, I am developing a HTTP client which ALWAYS use a HTTPS
> proxy using HTTP CONNECT method. This client is allowed to reach HTTP or
> HTTPS targets. Therefore, I noticed that when I try to reach a HTTPS
> target, the the transport layer works as expected and it uses the HTTP
> CONNECT method. However, when I try to reach a HTTP target, the transport
> does not use the CONNECT  method.
>
> Looking at the transport.go code, I realized that the check to use the
> CONNECT method is based on the protocol of the target instead of being on
> the protocol of the proxy URL. Below is a link showing that:
>
> 1. HTTP check
>
>
> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1092
>
> 2. HTTPS check
>
>
> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1099
>
> As can be seen on the links above, the condition is based on cm
> 
> .targetScheme
> 
>  instead
> of cm
> 
> .proxyURL
> 
> .Scheme
> .
> Is it a bug?
>
> *Go version: go version go1.19.3 linux/amd64*
>
> Mauro
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d06cd78b-c012-425f-8b5e-52d4bd7a3cbcn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAGabyPrBxj2m9kgNZrL5k3maGBu2Ljb%2B%3DONe2m-kr7Z8K3ArHA%40mail.gmail.com.

Re: [go-nuts] go install of forked repo

['Sean Liao' via golang-nuts]

`go install pkg@version` doesn't support installing from a fork.

The argument is best understood as a module identity rather than source
code location.

- sean

On Sat, Dec 3, 2022, 11:55 Duncan Harris  wrote:

> This seems a noddy question but can't easily find an answer with Google
> apparently. I may have lost the plot :-)
>
> There is a repo which contains source for a go executable that I want to
> use.
> Normally I install this with: go install original.domain/path@latest
> I want to fork that repo and make some experimental changes.
> So then I want to: go install my.forked.domain/path@commit
> But if I don't also change the go.mod file in my fork I get something like:
>
> go: downloading my.forked.domain/path v0.0.0-20221203102529-commit
> go: my.forked.domain/path@commit
> : my.forked.domain/path@v0.0.0-20221203102529-commit: parsing go.mod:
> module declares its path as: original.domain/path
> but was required as: my.forked.domain/path
>
> Obviously I could change the go.mod in my fork, but that seems wrong if I
> later want to submit a PR on the original repo.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/9a9542bd-1718-4ea7-a0ed-42b343bb9699n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAGabyPoRcekTD0R7JB51%3DOSSVsyDLUsTEuGYjNvYUvOOGZuw7Q%40mail.gmail.com.

[go-nuts] cgo C.uint64_t compatiblity on multiple platforms?

[David Stainton]

Greetings,

I think my question is something like: "how to make my usage of C.uint64_t 
work on all platforms?" or "Is there something obviously wrong with my cgo 
bindings that is causing this not to build on all platforms?"

I wrote this cgo binding for the reference implementation of Sphincs+ (it's 
a hash based post quantum signature scheme):

https://github.com/katzenpost/katzenpost/blob/main/sphincsplus/ref/binding.go

Recently we noticed it fails to build on MacOS and Windows:

https://github.com/katzenpost/katzen/actions/runs/3609590801/jobs/6082872618

So I "fixed" it by changing the typecast to use C.uint64_t instead of 
C.ulong:

https://github.com/katzenpost/katzenpost/pull/110/files

However that causes the build to fail using Go1.19.3 on MacOS, although it 
works on Windows:

https://github.com/katzenpost/katzen/actions/runs/3609771045/jobs/6083165054


Sincerely,
David Stainton

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/86bdb5ce-d4ce-474e-bc98-01f7a8dce9fcn%40googlegroups.com.

[go-nuts] How to Install Goyacc

[Peter Bočan]

Hello Gophers!

Quick question: what is the best way to install goyacc at this time? Is it 
even recommended to use it these days?

Kindly,
Peter.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b0e77640-6b65-419b-8a57-b67b6f1d997an%40googlegroups.com.

[go-nuts] HTTPS proxy issue using CONNECT method

[Mauro Monteiro]

Hello all,

I have been facing an issue when I try to create a HTTP client which needs 
to connect through a HTTPS proxy using the HTTP CONNEC method. I know that 
it can be achieved setting my own http.Transport object. However the issue 
seems to be in the current implementation of /net/http/transport.go code.

In my environment, I am developing a HTTP client which ALWAYS use a HTTPS 
proxy using HTTP CONNECT method. This client is allowed to reach HTTP or 
HTTPS targets. Therefore, I noticed that when I try to reach a HTTPS 
target, the the transport layer works as expected and it uses the HTTP 
CONNECT method. However, when I try to reach a HTTP target, the transport 
does not use the CONNECT  method.

Looking at the transport.go code, I realized that the check to use the 
CONNECT method is based on the protocol of the target instead of being on 
the protocol of the proxy URL. Below is a link showing that:

1. HTTP check

https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1092

2. HTTPS check

https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1099

As can be seen on the links above, the condition is based on cm 

.targetScheme 

 instead 
of cm 

.proxyURL 

.Scheme 
.
 
Is it a bug?

*Go version: go version go1.19.3 linux/amd64*

Mauro

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d06cd78b-c012-425f-8b5e-52d4bd7a3cbcn%40googlegroups.com.

Re: [go-nuts] Go Memory Model question

[Quân Anh Mai]

A. From the theoretical point of view given this program:

Initial:
x = 0

Thread 1:
x = 1

Thread 2:
x1 = x
x2 = x

If x1 = 1, we have the following happen-before relationship:
- x = 0 happens before x = 1
- x = 0 happens before x1 = x
- x1 = x happens before x2 = x
Note that there is no x = 1 happening before x1 = x since these are not 
synchronising operations.
As a result, there is no happen-before relationship between x = 1 and x2 = 
x, which means these 2 are racy, and the read can observe either value of 0 
or 1.

B. From the practical point of view, given the 2 reads:

x1 = x
x2 = x

The compiler, CPU, etc can freely reorder these 2 operations, which leads 
to the second load executing before the first one. Worse, even if the first 
load seems to come before the second one from the perspective of a thread, 
there is no guarantee that another thread may observe the opposite 
behaviour, that is the second load may result in some value that apparently 
impossible at that point. This may lead to paradoxes when trying to reason 
the behaviour of a program using some sequentially consistent order of a 
permutation of a program order.

The ability to prevent the reordering of 2 loads with respect to every 
thread is called a load-load fence. Suppose Go will support memory barriers 
in the future, your program can be rewritten as:

```go
package main

import "memory"

var x int = 0

func main() {
go func() {
x = 1
}()

print(x) // first read: 1
memory.LoadLoadFence()
print(x) // second read: 0
}
```

and we can hope that the program will never print out 1, 0.

Thanks.

On Friday, 2 December 2022 at 22:12:48 UTC+8 bse...@computer.org wrote:

> The way I read the memory model, this program can print 01, 00, and 11, 
> but not 10. This is because goroutine creation creates a synchronized 
> before relationship between x=1 and x=0, so even though there is a race, 
> x=0 happens before x=1.
>
> On Fri, Dec 2, 2022 at 6:56 AM のびしー  wrote:
>
>> > I believe your example is basically equivalent to the ones in 
>> https://go.dev/ref/mem#badsync which also contains an explanation of how 
>> the memory model implies this
>>
>> @Wagner
>> Thanks for your opinion, I think so too. I was not confident that my 
>> example is equivalent to https://go.dev/ref/mem#badsync, since my 
>> example reads from the same variable.
>>
>>
>> > Programs that modify data being simultaneously accessed by multiple 
>> goroutines must serialize such access. 
>> @peterGo
>>
>> I know that my example has data races. But the Go Memory Model guarantees 
>> a limited number of outcomes for programs with races(unless runtime reports 
>> the race and terminates). My question is about the limited outcomes.
>>
>> 2022年12月2日(金) 22:02 Axel Wagner :
>>
>>> I believe your example is basically equivalent to the ones in 
>>> https://go.dev/ref/mem#badsync which also contains an explanation of 
>>> how the memory model implies this (or rather, how it does not imply the 
>>> opposite).
>>>
>>> On Fri, Dec 2, 2022, 13:11 のびしー  wrote:
>>>
 Hello, I have another question regarding the Go Memory Model. 

 (1) Can this program print "10", according to the Memory Model?
 (2) If that is forbidden, which part of the Go Memory Model excludes 
 such behavior?

 https://go.dev/play/p/Fn5I0fjSiKj

 ```go
 package main

 var x int = 0

 func main() {
 go func() {
 x = 1
 }()
 print(x) // first read: 1
 print(x) // second read: 0
 }
 ```

 I draw a picture of the happens-before relation of this program here:


 https://gist.github.com/nobishino/8150346c30101e2ca409ed83c6c25add?permalink_comment_id=4388680#gistcomment-4388680

 I think the answer to (1) is yes. 
 Both reads are concurrent with x = 1, so each read can observe both x = 
 0 and x = 1.
 And there is no constraint between the results of the first read and 
 the second read.
 So the second read can observe x = 0 even if the first read observes x 
 = 0.

 But I'm not very sure. Is this understanding correct?

 -- 
 You received this message because you are subscribed to the Google 
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to golang-nuts...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/golang-nuts/CAGoET5HyPxEhBETGWNPuYpDgXbm0JM3jeaKFdoQdtc5eGUR0Uw%40mail.gmail.com
  
 
 .

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>>
> To view this discussion on 

Re: [go-nuts] Performance for concurrent requests

[Diogo Baeder]

Hi all!

Thanks for the inputs, I'll do some profiling here and update about my 
findings.

I don't want to change anything on Nginx because the comparison I'm doing 
between different stacks is on the same basis. If I try to tune Nginx or 
anything like that I'll be comparing apples to oranges, so that's not the 
point of my experiment.

Cheers!

On Saturday, December 3, 2022 at 6:07:42 AM UTC-3 Brian Candler wrote:

> Have you tried a tcpdump of the packets between the Go program and nginx?  
> Is it using HTTP/1.1 or HTTP/2?  If it's HTTP/1.1, does tcpdump show that 
> it actually starts all 50 HTTP client requests simultaneously?
>
> You are making all these concurrent requests to the same host. The default 
> of MaxConnsPerHost I believe is 0 (unlimited), but 
> DefaultMaxIdleConnsPerHost is 2.  It could be worth cranking that up. I 
> wonder if it's being forced to close down 48 of those connections 
> immediately because it can't return them to the pool.
>
> I also wonder whether there's tuning required at the Nginx side, e.g. for 
> the backlog queue.
>
> On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote:
>
>> I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In 
>> general, though, using pprof is the best way to start to break down a 
>> question like this. Would the actual workload involve more structured JSON, 
>> or more computation with decoded values?
>>
>> On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org 
>> wrote:
>>
>>> On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder  wrote:
>>>
 Hi guys,

 I've been working on some experiments with different web application 
 stacks to check their performances under a specific scenario: one in which 
 I have to make several concurrent requests and then gather the results 
 together (in order) and throw them out as JSON in the response body. (This 
 project is only an experiment, but it's informing me for decisions that 
 have to be made for a real-world project where we have a similar scenario.)

 However, probably due to my ignorance in Go, I cannot make it perform 
 as well as I expected - actually the best I'm getting are results that are 
 even slower than Python, which was a surprise to me. Here they are: 
 https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin 

 So, looking at the code here: 
 https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go
  
 - does anybody see any problem in the implementation that could be hurting 
 performance? I tried using a WaitGroup, tried sharing memory (nasty, I 
 know, but just for the sake of experimentation), tried multiple JSON 
 codecs, different web frameworks, and nothing worked so far. I have a 
 feeling that I'm doing something fundamentally wrong and stupid, and that 
 somehow I can make a small change to make the experiment much faster.

>>>
>>> Have you measured how much time is spent on the http.Get calls? It is 
>>> likely that the 50 concurrent http.Get calls is the bottleneck. 
>>>
>>> Also note that you don't need a channel there. You can simply use a 
>>> waitgroup and set the results from inside the goroutine, because each 
>>> goroutine knows the index. But that is unlikely to change anything 
>>> measurable when compared to the Get calls.
>>>
>>>  
>>>

 Thanks in advance, I'm sure this will help me learning more about the 
 language! :-)

 Cheers!

 -- 
 You received this message because you are subscribed to the Google 
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to golang-nuts...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com
  
 
 .

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/66d94fac-a7f2-4fd3-a809-b1be1f668181n%40googlegroups.com.

Re: [go-nuts] go install of forked repo

[Steven Hartland]

Check out go work I achieve it’s exactly what you are looking for, allowing
for projects to use temporary local versions of packages

On Sat, 3 Dec 2022 at 11:55, Duncan Harris  wrote:

> This seems a noddy question but can't easily find an answer with Google
> apparently. I may have lost the plot :-)
>
> There is a repo which contains source for a go executable that I want to
> use.
> Normally I install this with: go install original.domain/path@latest
> I want to fork that repo and make some experimental changes.
> So then I want to: go install my.forked.domain/path@commit
> But if I don't also change the go.mod file in my fork I get something like:
>
> go: downloading my.forked.domain/path v0.0.0-20221203102529-commit
> go: my.forked.domain/path@commit
> : my.forked.domain/path@v0.0.0-20221203102529-commit: parsing go.mod:
> module declares its path as: original.domain/path
> but was required as: my.forked.domain/path
>
> Obviously I could change the go.mod in my fork, but that seems wrong if I
> later want to submit a PR on the original repo.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/9a9542bd-1718-4ea7-a0ed-42b343bb9699n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAHEMsqa7HahGdG0-Fz5_WZaVSa90TvFCX0dZ4O4516TP6drB3A%40mail.gmail.com.

[go-nuts] go install of forked repo

[Duncan Harris]

This seems a noddy question but can't easily find an answer with Google 
apparently. I may have lost the plot :-)

There is a repo which contains source for a go executable that I want to 
use.
Normally I install this with: go install original.domain/path@latest
I want to fork that repo and make some experimental changes.
So then I want to: go install my.forked.domain/path@commit
But if I don't also change the go.mod file in my fork I get something like:

go: downloading my.forked.domain/path v0.0.0-20221203102529-commit
go: my.forked.domain/path@commit: 
my.forked.domain/path@v0.0.0-20221203102529-commit: 
parsing go.mod:
module declares its path as: original.domain/path
but was required as: my.forked.domain/path

Obviously I could change the go.mod in my fork, but that seems wrong if I 
later want to submit a PR on the original repo.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9a9542bd-1718-4ea7-a0ed-42b343bb9699n%40googlegroups.com.

Re: [go-nuts] What's go's convention for specifying the equivolent of configure --prefix?

[Nick White]

On Thu, Dec 01, 2022 at 08:42:46PM -0800, hey...@gmail.com wrote:
> Using a Makefile makes sense.
> 
> And thanks for bringing embed to my attention. I never knew it exists. Sounds
> like it can solve my problem.

Yeah, embed is brilliant, I'm glad it can help you.

One thing I forgot to mention is build tags. If you only have a few 
options of different things to embed / change, they're a great way 
to specify them. Just have a few small different files containing a 
line like '//go:build mytag', defining things as you need them, and 
set the tags appropriately when you build / install with '-tags 
mytag'. It's easier and nicer than replacing things with a makefile.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/Y4s3ujz0dpD%2BPt6s%40mussel.lan.

Re: [go-nuts] Performance for concurrent requests

[Brian Candler]

Have you tried a tcpdump of the packets between the Go program and nginx?  
Is it using HTTP/1.1 or HTTP/2?  If it's HTTP/1.1, does tcpdump show that 
it actually starts all 50 HTTP client requests simultaneously?

You are making all these concurrent requests to the same host. The default 
of MaxConnsPerHost I believe is 0 (unlimited), but 
DefaultMaxIdleConnsPerHost is 2.  It could be worth cranking that up. I 
wonder if it's being forced to close down 48 of those connections 
immediately because it can't return them to the pool.

I also wonder whether there's tuning required at the Nginx side, e.g. for 
the backlog queue.

On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote:

> I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In 
> general, though, using pprof is the best way to start to break down a 
> question like this. Would the actual workload involve more structured JSON, 
> or more computation with decoded values?
>
> On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org wrote:
>
>> On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder  wrote:
>>
>>> Hi guys,
>>>
>>> I've been working on some experiments with different web application 
>>> stacks to check their performances under a specific scenario: one in which 
>>> I have to make several concurrent requests and then gather the results 
>>> together (in order) and throw them out as JSON in the response body. (This 
>>> project is only an experiment, but it's informing me for decisions that 
>>> have to be made for a real-world project where we have a similar scenario.)
>>>
>>> However, probably due to my ignorance in Go, I cannot make it perform as 
>>> well as I expected - actually the best I'm getting are results that are 
>>> even slower than Python, which was a surprise to me. Here they are: 
>>> https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin 
>>>
>>> So, looking at the code here: 
>>> https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go
>>>  
>>> - does anybody see any problem in the implementation that could be hurting 
>>> performance? I tried using a WaitGroup, tried sharing memory (nasty, I 
>>> know, but just for the sake of experimentation), tried multiple JSON 
>>> codecs, different web frameworks, and nothing worked so far. I have a 
>>> feeling that I'm doing something fundamentally wrong and stupid, and that 
>>> somehow I can make a small change to make the experiment much faster.
>>>
>>
>> Have you measured how much time is spent on the http.Get calls? It is 
>> likely that the 50 concurrent http.Get calls is the bottleneck. 
>>
>> Also note that you don't need a channel there. You can simply use a 
>> waitgroup and set the results from inside the goroutine, because each 
>> goroutine knows the index. But that is unlikely to change anything 
>> measurable when compared to the Get calls.
>>
>>  
>>
>>>
>>> Thanks in advance, I'm sure this will help me learning more about the 
>>> language! :-)
>>>
>>> Cheers!
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0894c03c-ff5b-403f-97c8-74193821c10en%40googlegroups.com.