[go-nuts] Re: HTTPS proxy issue using CONNECT method

[gbarr]

Hi Mauro,

In your curl examples the CONNECT method was used in both cases because you 
forced it to with the --proxytunnel option. If you run those same commands 
without that option then you will find that curl will operate the same as 
Go and will only use the CONNECT method when the target is HTTPS

So yes the transport could operate in a way that you describe, as 
demonstrated by your curl commands. But doing so would add overhead that is 
unnecessary for a plain HTTP request. By using CONNECT you impose an extra 
round trip between the client and the proxy because first it has to 
establish the connection to the target site with the CONNECT then it sends 
the request.

Graham.

On Monday, December 5, 2022 at 11:32:40 PM UTC maumon...@gmail.com wrote:

> Hello Graham,
>
> Would not make more sense to deal with both cases HTTPS and HTTP in the 
> same way? The transport could send the CONNECT method to establish the TCP 
> tunnel and then send the request after that. Of course, if the target 
> request is based on HTTPS, the TLS handshake would start otherwise the 
> request would be sent in plain text. For instance, I tested both scenarios 
> using curl command and they  behave exactly the same way, sending a CONNECT 
> request.
>
> *curl -v --proxytunnel --proxy-insecure -x https://localhost:7443 
>  http://localhost:8001  ->* 
> The proxy is HTTPS and the target is HTTP, curl sends a CONNECT and then 
> the GET method
>
> *curl -v --proxytunnel --proxy-insecure -x https://localhost:7443 
>  https://www.google.com  *
> *->* The proxy is HTTPS and the target is HTTPS, curl sends a CONNECT and 
> then the GET method
>
> Based on what i understood from you comment, please sorry whether I 
> misunderstood, the proxy would need to have a logic to check whether the 
> target is HTTPS or HTTP and the act differently depending on the case. For 
> HTTPS it would create the TCP tunnel while for HTTP it would forward the 
> request received. Is it true?
>
> Mauro
> On Monday, December 5, 2022 at 8:23:18 AM UTC gbarr wrote:
>
>> On Saturday, December 3, 2022 at 4:46:54 PM UTC maumon...@gmail.com 
>> wrote:
>>
>>> Hello all,
>>>
>>> I have been facing an issue when I try to create a HTTP client which 
>>> needs to connect through a HTTPS proxy using the HTTP CONNEC method. I know 
>>> that it can be achieved setting my own http.Transport object. However the 
>>> issue seems to be in the current implementation of /net/http/transport.go 
>>> code.
>>>
>>> In my environment, I am developing a HTTP client which ALWAYS use a 
>>> HTTPS proxy using HTTP CONNECT method. This client is allowed to reach HTTP 
>>> or HTTPS targets. Therefore, I noticed that when I try to reach a HTTPS 
>>> target, the the transport layer works as expected and it uses the HTTP 
>>> CONNECT method. However, when I try to reach a HTTP target, the transport 
>>> does not use the CONNECT  method.
>>>
>>
>> This is normal. The CONNECT method allows a client to create a TCP tunnel 
>> through your gateway. This allows your client to perform all TLS 
>> negotiation.
>>
>> However for HTTP requests this extra layering is not required. In the 
>> standard library you can see the setting of pconn.isProxy=true at 
>> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1093
>>   
>> this is later used when writing the request at 
>> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/request.go;l=521
>>
>> Essentially it changes the form of the http method from GET 
>> /path/to/resource to GET http://hostname/path/to/resource so your 
>> gateway would then know that this is a proxy request and perform the 
>> external request
>>
>> Graham.
>>
>>
>>> Looking at the transport.go code, I realized that the check to use the 
>>> CONNECT method is based on the protocol of the target instead of being on 
>>> the protocol of the proxy URL. Below is a link showing that:
>>>
>>> 1. HTTP check
>>>
>>>
>>> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1092
>>>
>>> 2. HTTPS check
>>>
>>>
>>> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1099
>>>
>>> As can be seen on the links above, the condition is based on cm 
>>> 
>>> .targetScheme 
>>> 
>>>  instead 
>>> of cm 
>>> 
>>> .proxyURL 
>>> 
>>> .Scheme 
>>> 

Re: [go-nuts] Re: Golang Github Package References

[Peter Galbavy]

Also, if you are concerned about the availability of the packages in the 
future, use vendoring to pull in a copy of the sources to the 
repo: https://go.dev/ref/mod#go-mod-vendor

Peter

On Monday, 5 December 2022 at 13:35:25 UTC Brian Candler wrote:

> On Monday, 5 December 2022 at 13:12:26 UTC loji...@gmail.com wrote:
>
>> The problem as I see it, is that when the security of the code relies on 
>> a package is outside the main program/executable, it can open potential 
>> problems of code injection; code changes or forks or if the url has moved, 
>> how these issues could cause the collapse of the integrity of the software. 
>>
>
> Go is a compiled language, and there are a few things you need to 
> understand about the compilation process.
>
> Firstly, there is no dynamic linking.  The *source code* for those 
> third-party libraries is fetched at compile time, and then compiled into 
> your binary, and that's it.  The binary is flat, containing your code and 
> the third-party code, all compiled together into a single blob.  The binary 
> does not change, and at runtime no new dependencies are pulled in or 
> updated; there is no route for code injection.
>
> Secondly, using go modules, in your source code you point to the *exact* 
> version of the third-party module that you want to import - either using 
> its version number, or a git commit reference. This is stored in 
> go.mod/go.sum.  The build of your application is reproducible.  If the 
> third-party module author releases a new or different version of their 
> module, then you have to explicitly pick it up - e.g. with "go mod tidy".
>
> Therefore, if someone else builds your code from source, version X, then 
> they will get exactly the same dependencies as you had when you released 
> version X.
>
> Thirdly, the imports are really module names, not urls.  Some of these 
> *happen* to look a bit like urls, and indeed there are mechanisms to fetch 
> the module source code automatically at compile time, if the module name 
> has the correct format.  But apart from that these are not used as urls, 
> and certainly not as urls embedded in the binary..
>
> Finally, if you are concerned about security problems, then you are right 
> to be aware of the versions of third-party libraries that you are using.  
> Of course, using newer versions of those libraries is likely to result in 
> fewer security holes, rather than more.  Therefore you need to track them, 
> and when they are updated, you'll want to recompile and re-release your own 
> software to take advantage of the fixes.
>
> HTH,
>
> Brian.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9c9b6f0d-a59d-445d-93e9-7d7dc1547c60n%40googlegroups.com.

Re: [go-nuts] Re: Why not tuples?

[Robert Engels]

I worked on a system one time - in Java - that used a Sextuplet. I kid you not. 
Please. Please. Please use typed structures and save the futures maintainers 
they pain. It’s a little extra work up front that pays big dividends. 

> On Dec 5, 2022, at 9:39 PM, Diogo Baeder  wrote:
> 
> 
> Hi folks,
> 
> Thanks for all the inputs, I really appreciate the effort and help :-)
> 
> Regardless of the whole discussion of whether or not tuples are a good or bad 
> data structure, they exist out there in the wild, and we have to deal with 
> them in many situations - of course, in our own code we can opt by continuing 
> to use them or not, depending on the language we're using. It would be nice 
> if Go could support them as first-class citizens - I know it's not the only 
> language that doesn't, though, Java for example doesn't support them either 
> (BTW there's a nice Java library, javatuples, which gives nice names like 
> Pair, Triplet, Quartet etc, to n-sized "tuples" - I find that enjoyable).
> 
> Anyways, thanks for the discussion, that helps me get a better grasp of the 
> language!
> 
> Cheers!
> 
>> On Sunday, December 4, 2022 at 11:39:41 PM UTC-3 Kevin Chowski wrote:
>> If you really need anonymous tuple types that support decoding that sort of 
>> JSON, it isn't too hard to write one: https://go.dev/play/p/Fn_wUXh2drs
>> 
>> Go's generics don't support varargs types (...yet? who knows) so there'd be 
>> a little copypasta if you needed many different tuple lengths, but Java has 
>> been doing that for years ;)
>> 
>> (IMO, using these anonymous tuple types across a whole codebase is not 
>> great: data should be labeled with useful names as it is passed around a 
>> program. But if you really are just using this to make json parsing code 
>> easier, that seems reasonable to me.)
>> 
>>> On Saturday, December 3, 2022 at 8:47:03 PM UTC-7 diogo...@gmail.com wrote:
>>> Hi there, sorry for weighting in so late in the game, but I just started 
>>> again to learn Go and was thinking why the language still doesn't have a 
>>> tuple type.
>>> 
>>> Now, imagine this scenario: I have a web application which has to access a 
>>> webservice that responds with JSON payloads; These payloads are a list of 
>>> values, where each value is a smaller list like '[20220101, 1.234, "New 
>>> York"]'. And these smaller lists follow the same type sequence: int64, 
>>> float64, string. Suppose that I want to filter those values and send a 
>>> response to the client, with the data structure unchanged (same format and 
>>> types). Today, it doesn't seem to be possible to do that in Go, unless I do 
>>> some dirty hack like decoding to '[]any' and then cast to the other types, 
>>> and then hack again to put these values in the response to the client.
>>> 
>>> I totally understand the reasoning for preferring the usage of structs for 
>>> heterogeneous data (and I myself do prefer them, they're much more powerful 
>>> in general), but there's real world data that's available like in the 
>>> example above, and we just can't go on changing them at their sources. I 
>>> might be mistaken (please let me know if it's the case), but it seems like 
>>> Go is missing an opportunity to interoperate with what's a fundamental data 
>>> structure in many other languages (Python, Rust etc). I'm having a lot of 
>>> fun learning to use the language, and would be happy to see this feature 
>>> being implemented at the core.
>>> 
>>> (Maybe what I said above is total BS, I acknowledge that since I'm an 
>>> almost complete ignorant in the language)
>>> 
>>> Cheers!
>>> 
 On Thursday, April 19, 2018 at 1:03:55 PM UTC-3 Louki Sumirniy wrote:
 Multiple return values. They do kinda exist in a declarative form of 
 sorts, in the type signature, this sets the number and sequence and types 
 of return values. You could even make functions accept them as also input 
 values, I think, but I don't think it works exactly like this. I'm not a 
 fan of these things because of how you have to nominate variables or _ and 
 type inference will make these new variables, if you  := into whatever the 
 return was.
 
 I'm not sure what the correct word is for them. Untyped in the same way 
 that literals can be multiple types (especially integers) but singular in 
 their literal form.
 
 
> On Thursday, 19 April 2018 16:06:42 UTC+3, Jan Mercl wrote:
> On Thu, Apr 19, 2018 at 2:51 PM Louki Sumirniy  
> wrote:
> 
> > Sorry for the self-promotion but it was relevant in that I was working 
> > on how to tidy up the readability of my code and needed multiple 
> > returns and simple untyped tuples were really not nearly as convenient 
> > as using a type struct.
> 
> I have no idea what you mean by 'untyped tuples' because Go does not have 
> tuples, or at least not as a well defined thing. I can only guess if 
> you're trying to implement tuples in Go with an 

Re: [go-nuts] Re: Why not tuples?

[Diogo Baeder]

Hi folks,

Thanks for all the inputs, I really appreciate the effort and help :-)

Regardless of the whole discussion of whether or not tuples are a good or 
bad data structure, they exist out there in the wild, and we have to deal 
with them in many situations - of course, in our own code we can opt by 
continuing to use them or not, depending on the language we're using. It 
would be nice if Go could support them as first-class citizens - I know 
it's not the only language that doesn't, though, Java for example doesn't 
support them either (BTW there's a nice Java library, javatuples, which 
gives nice names like Pair, Triplet, Quartet etc, to n-sized "tuples" - I 
find that enjoyable).

Anyways, thanks for the discussion, that helps me get a better grasp of the 
language!

Cheers!

On Sunday, December 4, 2022 at 11:39:41 PM UTC-3 Kevin Chowski wrote:

> If you really need anonymous tuple types that support decoding that sort 
> of JSON, it isn't too hard to write one: https://go.dev/play/p/Fn_wUXh2drs
>
> Go's generics don't support varargs types (...yet? who knows) so there'd 
> be a little copypasta if you needed many different tuple lengths, but Java 
> has been doing that for years ;)
>
> (IMO, using these anonymous tuple types across a whole codebase is not 
> great: data should be labeled with useful names as it is passed around a 
> program. But if you really are just using this to make json parsing code 
> easier, that seems reasonable to me.)
>
> On Saturday, December 3, 2022 at 8:47:03 PM UTC-7 diogo...@gmail.com 
> wrote:
>
>> Hi there, sorry for weighting in so late in the game, but I just started 
>> again to learn Go and was thinking why the language still doesn't have a 
>> tuple type.
>>
>> Now, imagine this scenario: I have a web application which has to access 
>> a webservice that responds with JSON payloads; These payloads are a list of 
>> values, where each value is a smaller list like '[20220101, 1.234, "New 
>> York"]'. And these smaller lists follow the same type sequence: int64, 
>> float64, string. Suppose that I want to filter those values and send a 
>> response to the client, with the data structure unchanged (same format and 
>> types). Today, it doesn't seem to be possible to do that in Go, unless I do 
>> some dirty hack like decoding to '[]any' and then cast to the other types, 
>> and then hack again to put these values in the response to the client.
>>
>> I totally understand the reasoning for preferring the usage of structs 
>> for heterogeneous data (and I myself do prefer them, they're much more 
>> powerful in general), but there's real world data that's available like in 
>> the example above, and we just can't go on changing them at their sources. 
>> I might be mistaken (please let me know if it's the case), but it seems 
>> like Go is missing an opportunity to interoperate with what's a fundamental 
>> data structure in many other languages (Python, Rust etc). I'm having a lot 
>> of fun learning to use the language, and would be happy to see this feature 
>> being implemented at the core.
>>
>> (Maybe what I said above is total BS, I acknowledge that since I'm an 
>> almost complete ignorant in the language)
>>
>> Cheers!
>>
>> On Thursday, April 19, 2018 at 1:03:55 PM UTC-3 Louki Sumirniy wrote:
>>
>>> Multiple return values. They do kinda exist in a declarative form of 
>>> sorts, in the type signature, this sets the number and sequence and types 
>>> of return values. You could even make functions accept them as also input 
>>> values, I think, but I don't think it works exactly like this. I'm not a 
>>> fan of these things because of how you have to nominate variables or _ and 
>>> type inference will make these new variables, if you  := into whatever the 
>>> return was.
>>>
>>> I'm not sure what the correct word is for them. Untyped in the same way 
>>> that literals can be multiple types (especially integers) but singular in 
>>> their literal form.
>>>
>>>
>>> On Thursday, 19 April 2018 16:06:42 UTC+3, Jan Mercl wrote:

 On Thu, Apr 19, 2018 at 2:51 PM Louki Sumirniy <
 louki.sumir...@gmail.com> wrote:

 > Sorry for the self-promotion but it was relevant in that I was 
 working on how to tidy up the readability of my code and needed multiple 
 returns and simple untyped tuples were really not nearly as convenient as 
 using a type struct.

 I have no idea what you mean by 'untyped tuples' because Go does not 
 have tuples, or at least not as a well defined thing. I can only guess if 
 you're trying to implement tuples in Go with an array, slice or a struct, 
 ...? To add to my confusion, Go functions can have as many return values 
 as 
 one wishes just fine, ie. I obviously do not even understand what problem 
 you're trying to solve. Sorry.


 -- 

 -j

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To 

Re: [go-nuts] Golang 1.19+ preemption model on a preemptive linux kernel?

[robert engels]

I don’t think the analysis by alphadose in the issue you cite is correct. Go 
added async pre-emption (via signals) and I believe it can preempt a Go thread 
anywhere but in an internal critical section. Ian would know for sure. It is 
certainly not only at a function call boundary because that would cause the 
“infinite spin loop blocking GC” issue. Once a routine is preempted any 
available routine could run on the same processor/OS thread.

> On Dec 5, 2022, at 5:20 PM, Andrew Athan  wrote:
> 
> In brief then, in the presence of os threads running on a kernel that can 
> preempt threads, it is incorrect to assume that any two statements within a 
> goroutine will run atomically (both run, or not at all), irrespective of the 
> simplicity of those statements, or whether any function calls involved are 
> inlined, etc...
> 
> On Monday, December 5, 2022 at 3:18:09 PM UTC-8 Andrew Athan wrote:
> The statement I made is actually a bit stronger than the one you confirmed: 
> Even if the goroutines are using atomic load/store, the preemtive nature of 
> the os threads could easily interrupt one goroutine between an atomic load 
> and its use of that value, giving another goroutine the opportunity to change 
> the stored value at the atomic load location, potentially leading to a data 
> race -- particularly when the loader is storing to a related but different 
> location and/or without a CAS on the original loaded value.
> 
> To answer your question, no ... within the go runtime, I DO NOT claim to have 
> found instances of that kind of load/check/store as separate operations 
> within a function such that those operations are assumed to run atomically.
> 
> However, I've run into a couple of such instances in public repos. Thus, I 
> wanted to confirm my thinking. For example, I initiated a discussion about 
> same here:
> 
> https://github.com/alphadose/haxmap/issues/26 
> 
> 
> (I ran across haxmap while reviewing issues in the golang repo relative to 
> sync.Map)
> 
> On Monday, December 5, 2022 at 10:45:10 AM UTC-8 Ian Lance Taylor wrote:
> On Mon, Dec 5, 2022 at 10:03 AM Andrew Athan > wrote: 
> > 
> > I'm having trouble finding definitive information about golang's preemption 
> > model when running with GOMAXPROCS>1 on a multi-core Intel with preemtive 
> > linux kernel. 
> > 
> > As I understand it, in such a scenario, two goroutines may be scheduled by 
> > the go runtime onto two separate system threads on two separate CPUs. 
> > 
> > Isn't it true then, that irrespective of whether the two goroutines are 
> > scheduled onto separate CPUs, the OS may preemptively interrupt either of 
> > those goroutines to let the other run? Also, that if they are indeed 
> > scheduled onto separate CPUs, that the memory accesses made by those 
> > goroutines are interleaved even if no individual goroutine is "preempted"? 
> > 
> > I'm asking because in reviewing the code of some "lock free" concurrent 
> > data structures written in go, it appears the authors have made certain 
> > assumptions about race conditions (e.g., the code of on goroutine assumes 
> > that another goroutine won't perform a load/store between two function 
> > calls the first goroutine makes to check a boolean and store a value). 
> > 
> > Given that goroutines are mappes onto system threads, and that in a 
> > preemptive kernel those threads may be preempted and/or irrespective of the 
> > preemptive kernel issue may run on separate cores, no assumptions should be 
> > made about the "atomicity" of multiple statements in a goroutine relative 
> > to other goroutines. Right? 
> > 
> > Where can I get the most current and accurate information about this topic? 
> 
> I'm not aware of any detailed documentation of the issues you are 
> looking at. So the most current and accurate information is the 
> source code. Or this mailing list. 
> 
> You are correct that non-atomic memory operations made by different 
> goroutines can be freely interleaved. If there is any code in the 
> runtime that assumes otherwise, that code is badly broken. If you can 
> point to the specific code that you are concerned about, perhaps we 
> can allay your concerns, or perhaps we can fix the bug. Thanks. 
> 
> Ian 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0b44c5d7-7c45-4392-9046-eb56f8830a52n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this 

[go-nuts] Re: HTTPS proxy issue using CONNECT method

[Mauro Monteiro]

Hello Graham,

Would not make more sense to deal with both cases HTTPS and HTTP in the 
same way? The transport could send the CONNECT method to establish the TCP 
tunnel and then send the request after that. Of course, if the target 
request is based on HTTPS, the TLS handshake would start otherwise the 
request would be sent in plain text. For instance, I tested both scenarios 
using curl command and they  behave exactly the same way, sending a CONNECT 
request.

*curl -v --proxytunnel --proxy-insecure -x https://localhost:7443 
http://localhost:8001 ->* The proxy is HTTPS and the target is HTTP, curl 
sends a CONNECT and then the GET method

*curl -v --proxytunnel --proxy-insecure -x https://localhost:7443 
https://www.google.com **->* The proxy is HTTPS and the target is HTTPS, 
curl sends a CONNECT and then the GET method

Based on what i understood from you comment, please sorry whether I 
misunderstood, the proxy would need to have a logic to check whether the 
target is HTTPS or HTTP and the act differently depending on the case. For 
HTTPS it would create the TCP tunnel while for HTTP it would forward the 
request received. Is it true?

Mauro
On Monday, December 5, 2022 at 8:23:18 AM UTC gbarr wrote:

> On Saturday, December 3, 2022 at 4:46:54 PM UTC maumon...@gmail.com wrote:
>
>> Hello all,
>>
>> I have been facing an issue when I try to create a HTTP client which 
>> needs to connect through a HTTPS proxy using the HTTP CONNEC method. I know 
>> that it can be achieved setting my own http.Transport object. However the 
>> issue seems to be in the current implementation of /net/http/transport.go 
>> code.
>>
>> In my environment, I am developing a HTTP client which ALWAYS use a HTTPS 
>> proxy using HTTP CONNECT method. This client is allowed to reach HTTP or 
>> HTTPS targets. Therefore, I noticed that when I try to reach a HTTPS 
>> target, the the transport layer works as expected and it uses the HTTP 
>> CONNECT method. However, when I try to reach a HTTP target, the transport 
>> does not use the CONNECT  method.
>>
>
> This is normal. The CONNECT method allows a client to create a TCP tunnel 
> through your gateway. This allows your client to perform all TLS 
> negotiation.
>
> However for HTTP requests this extra layering is not required. In the 
> standard library you can see the setting of pconn.isProxy=true at 
> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1093
>   
> this is later used when writing the request at 
> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/request.go;l=521
>
> Essentially it changes the form of the http method from GET 
> /path/to/resource to GET http://hostname/path/to/resource so your gateway 
> would then know that this is a proxy request and perform the external 
> request
>
> Graham.
>
>
>> Looking at the transport.go code, I realized that the check to use the 
>> CONNECT method is based on the protocol of the target instead of being on 
>> the protocol of the proxy URL. Below is a link showing that:
>>
>> 1. HTTP check
>>
>>
>> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1092
>>
>> 2. HTTPS check
>>
>>
>> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1099
>>
>> As can be seen on the links above, the condition is based on cm 
>> 
>> .targetScheme 
>> 
>>  instead 
>> of cm 
>> 
>> .proxyURL 
>> 
>> .Scheme 
>> .
>>  
>> Is it a bug?
>>
>> *Go version: go version go1.19.3 linux/amd64*
>>
>> Mauro
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6a05b385-cea5-45d0-bda9-ae8caba73e51n%40googlegroups.com.

Re: [go-nuts] Golang 1.19+ preemption model on a preemptive linux kernel?

[Andrew Athan]

In brief then, in the presence of os threads running on a kernel that can 
preempt threads, it is incorrect to assume that any two statements within a 
goroutine will run atomically (both run, or not at all), irrespective of 
the simplicity of those statements, or whether any function calls involved 
are inlined, etc...

On Monday, December 5, 2022 at 3:18:09 PM UTC-8 Andrew Athan wrote:

> The statement I made is actually a bit stronger than the one you 
> confirmed: Even if the goroutines are using atomic load/store, the 
> preemtive nature of the os threads could easily interrupt one goroutine 
> between an atomic load and its use of that value, giving another goroutine 
> the opportunity to change the stored value at the atomic load location, 
> potentially leading to a data race -- particularly when the loader is 
> storing to a related but different location and/or without a CAS on the 
> original loaded value.
>
> To answer your question, no ... within the go runtime, I DO NOT claim to 
> have found instances of that kind of load/check/store as separate 
> operations within a function such that those operations are assumed to run 
> atomically.
>
> However, I've run into a couple of such instances in public repos. Thus, I 
> wanted to confirm my thinking. For example, I initiated a discussion about 
> same here:
>
> https://github.com/alphadose/haxmap/issues/26
>
> (I ran across haxmap while reviewing issues in the golang repo relative to 
> sync.Map)
>
> On Monday, December 5, 2022 at 10:45:10 AM UTC-8 Ian Lance Taylor wrote:
>
>> On Mon, Dec 5, 2022 at 10:03 AM Andrew Athan  wrote: 
>> > 
>> > I'm having trouble finding definitive information about golang's 
>> preemption model when running with GOMAXPROCS>1 on a multi-core Intel with 
>> preemtive linux kernel. 
>> > 
>> > As I understand it, in such a scenario, two goroutines may be scheduled 
>> by the go runtime onto two separate system threads on two separate CPUs. 
>> > 
>> > Isn't it true then, that irrespective of whether the two goroutines are 
>> scheduled onto separate CPUs, the OS may preemptively interrupt either of 
>> those goroutines to let the other run? Also, that if they are indeed 
>> scheduled onto separate CPUs, that the memory accesses made by those 
>> goroutines are interleaved even if no individual goroutine is "preempted"? 
>> > 
>> > I'm asking because in reviewing the code of some "lock free" concurrent 
>> data structures written in go, it appears the authors have made certain 
>> assumptions about race conditions (e.g., the code of on goroutine assumes 
>> that another goroutine won't perform a load/store between two function 
>> calls the first goroutine makes to check a boolean and store a value). 
>> > 
>> > Given that goroutines are mappes onto system threads, and that in a 
>> preemptive kernel those threads may be preempted and/or irrespective of the 
>> preemptive kernel issue may run on separate cores, no assumptions should be 
>> made about the "atomicity" of multiple statements in a goroutine relative 
>> to other goroutines. Right? 
>> > 
>> > Where can I get the most current and accurate information about this 
>> topic? 
>>
>> I'm not aware of any detailed documentation of the issues you are 
>> looking at. So the most current and accurate information is the 
>> source code. Or this mailing list. 
>>
>> You are correct that non-atomic memory operations made by different 
>> goroutines can be freely interleaved. If there is any code in the 
>> runtime that assumes otherwise, that code is badly broken. If you can 
>> point to the specific code that you are concerned about, perhaps we 
>> can allay your concerns, or perhaps we can fix the bug. Thanks. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0b44c5d7-7c45-4392-9046-eb56f8830a52n%40googlegroups.com.

Re: [go-nuts] Golang 1.19+ preemption model on a preemptive linux kernel?

[Andrew Athan]

The statement I made is actually a bit stronger than the one you confirmed: 
Even if the goroutines are using atomic load/store, the preemtive nature of 
the os threads could easily interrupt one goroutine between an atomic load 
and its use of that value, giving another goroutine the opportunity to 
change the stored value at the atomic load location, potentially leading to 
a data race -- particularly when the loader is storing to a related but 
different location and/or without a CAS on the original loaded value.

To answer your question, no ... within the go runtime, I DO NOT claim to 
have found instances of that kind of load/check/store as separate 
operations within a function such that those operations are assumed to run 
atomically.

However, I've run into a couple of such instances in public repos. Thus, I 
wanted to confirm my thinking. For example, I initiated a discussion about 
same here:

https://github.com/alphadose/haxmap/issues/26

(I ran across haxmap while reviewing issues in the golang repo relative to 
sync.Map)

On Monday, December 5, 2022 at 10:45:10 AM UTC-8 Ian Lance Taylor wrote:

> On Mon, Dec 5, 2022 at 10:03 AM Andrew Athan  wrote:
> >
> > I'm having trouble finding definitive information about golang's 
> preemption model when running with GOMAXPROCS>1 on a multi-core Intel with 
> preemtive linux kernel.
> >
> > As I understand it, in such a scenario, two goroutines may be scheduled 
> by the go runtime onto two separate system threads on two separate CPUs.
> >
> > Isn't it true then, that irrespective of whether the two goroutines are 
> scheduled onto separate CPUs, the OS may preemptively interrupt either of 
> those goroutines to let the other run? Also, that if they are indeed 
> scheduled onto separate CPUs, that the memory accesses made by those 
> goroutines are interleaved even if no individual goroutine is "preempted"?
> >
> > I'm asking because in reviewing the code of some "lock free" concurrent 
> data structures written in go, it appears the authors have made certain 
> assumptions about race conditions (e.g., the code of on goroutine assumes 
> that another goroutine won't perform a load/store between two function 
> calls the first goroutine makes to check a boolean and store a value).
> >
> > Given that goroutines are mappes onto system threads, and that in a 
> preemptive kernel those threads may be preempted and/or irrespective of the 
> preemptive kernel issue may run on separate cores, no assumptions should be 
> made about the "atomicity" of multiple statements in a goroutine relative 
> to other goroutines. Right?
> >
> > Where can I get the most current and accurate information about this 
> topic?
>
> I'm not aware of any detailed documentation of the issues you are
> looking at. So the most current and accurate information is the
> source code. Or this mailing list.
>
> You are correct that non-atomic memory operations made by different
> goroutines can be freely interleaved. If there is any code in the
> runtime that assumes otherwise, that code is badly broken. If you can
> point to the specific code that you are concerned about, perhaps we
> can allay your concerns, or perhaps we can fix the bug. Thanks.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/998087d0-642a-4bad-ba9a-87c6a369496dn%40googlegroups.com.

Re: [go-nuts] Re: Looking Backend Dev Learning Resources

[Christoph Berger]

> that would mean the progress made did not invalidate any of effective Go 
(which seems not possible given the addition of generics)

Adding new features makes Effective Go *incomplete* but not *invalid*. 
There is separate documentation available for Generics and other new 
features. Go documentation can only be invalidated if breaking changes are 
introduced to the language. But the Go 1 compatibility promise 
 prevents this from happening except for very 
rare cases like, for example, security fixes that cannot be implemented 
without breaking existing code. 

On Monday, December 5, 2022 at 9:55:18 PM UTC+1 ren...@ix.netcom.com wrote:

> Can you elaborate on that reference? At first review, it means you are 
> putting in lots of effort making lots of progress (anti red queen) but that 
> would mean the progress made did not invalidate any of effective Go (which 
> seems not possible given the addition of generics). I’m doubting you 
> implied little effort was put in over 10 years so little progress has been 
> made :)
>
> > On Dec 5, 2022, at 2:19 PM, 'Dan Kortschak' via golang-nuts <
> golan...@googlegroups.com> wrote:
> > 
> > On Mon, 2022-12-05 at 09:27 -0800, Tsvetomir Lazarov wrote:
> >> How relevant is Effective Go actually, given the January 2022 update
> >> that this document has not been updated significantly since 2009?
> > 
> > Still relevant. This is one of the virtues of having a language that is
> > not built on the Red Queen model of progress[1].
> > 
> > [1]https://en.wikipedia.org/wiki/Red_Queen%27s_race
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0af2089eb4f34cd5f8e67f1c9b1cad62c1b76e57.camel%40kortschak.io
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1005a951-495b-4ad4-9303-f8f63109d160n%40googlegroups.com.

Re: [go-nuts] Re: Looking Backend Dev Learning Resources

['Dan Kortschak' via golang-nuts]

On Mon, 2022-12-05 at 14:54 -0600, Robert Engels wrote:
> Can you elaborate on that reference? At first review, it means you
> are putting in lots of effort making lots of progress (anti red
> queen) but that would mean the progress made did not invalidate any
> of effective Go (which seems not possible given the addition of
> generics).  I’m doubting you implied little effort was put in over 10
> years so little progress has been made :)

The Red Queen was the definition of churn. She never made any progress.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/dc76e280f6529cc12703bc77323547167b64e22f.camel%40kortschak.io.

Re: [go-nuts] Re: Looking Backend Dev Learning Resources

[Robert Engels]

Can you elaborate on that reference? At first review, it means you are putting 
in lots of effort making lots of progress (anti red queen) but that would mean 
the progress made did not invalidate any of effective Go (which seems not 
possible given the addition of generics).  I’m doubting you implied little 
effort was put in over 10 years so little progress has been made :)

> On Dec 5, 2022, at 2:19 PM, 'Dan Kortschak' via golang-nuts 
>  wrote:
> 
> On Mon, 2022-12-05 at 09:27 -0800, Tsvetomir Lazarov wrote:
>> How relevant is Effective Go actually, given the January 2022 update
>> that this document has not been updated significantly since 2009?
> 
> Still relevant. This is one of the virtues of having a language that is
> not built on the Red Queen model of progress[1].
> 
> [1]https://en.wikipedia.org/wiki/Red_Queen%27s_race
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0af2089eb4f34cd5f8e67f1c9b1cad62c1b76e57.camel%40kortschak.io.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/C9BDCD17-4499-453C-A74A-A165EFEBD8A2%40ix.netcom.com.

Re: [go-nuts] Re: Looking Backend Dev Learning Resources

['Dan Kortschak' via golang-nuts]

On Mon, 2022-12-05 at 09:27 -0800, Tsvetomir Lazarov wrote:
> How relevant is Effective Go actually, given the January 2022 update
> that this document has not been updated significantly since 2009?

Still relevant. This is one of the virtues of having a language that is
not built on the Red Queen model of progress[1].

[1]https://en.wikipedia.org/wiki/Red_Queen%27s_race

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0af2089eb4f34cd5f8e67f1c9b1cad62c1b76e57.camel%40kortschak.io.

Re: [go-nuts] Golang 1.19+ preemption model on a preemptive linux kernel?

[Robert Engels]

I have some examples of lock-free structures at 
github.com/robaho/go-concurrency-test

Most lock-free structures should not depend on OS scheduling. Some that use 
“cpu id “ to partition would. 

> On Dec 5, 2022, at 12:02 PM, Andrew Athan  wrote:
> 
> I'm having trouble finding definitive information about golang's preemption 
> model when running with GOMAXPROCS>1 on a multi-core Intel with preemtive 
> linux kernel.
> 
> As I understand it, in such a scenario, two goroutines may be scheduled by 
> the go runtime onto two separate system threads on two separate CPUs.
> 
> Isn't it true then, that irrespective of whether the two goroutines are 
> scheduled onto separate CPUs, the OS may preemptively interrupt either of 
> those goroutines to let the other run? Also, that if they are indeed 
> scheduled onto separate CPUs, that the memory accesses made by those 
> goroutines are interleaved even if no individual goroutine is "preempted"?
> 
> I'm asking because in reviewing the code of some "lock free" concurrent data 
> structures written in go, it appears the authors have made certain 
> assumptions about race conditions (e.g.,  the code of on goroutine assumes 
> that another goroutine won't perform a load/store between two function calls 
> the first goroutine makes to check a boolean and store a value).
> 
> Given that goroutines are mappes onto system threads, and that in a 
> preemptive kernel those threads may be preempted and/or irrespective of the 
> preemptive kernel issue may run on separate cores, no assumptions should be 
> made about the "atomicity" of multiple statements in a goroutine relative to 
> other goroutines. Right?
> 
> Where can I get the most current and accurate information about this topic?
> 
> A.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0912663b-d4cb-41ee-a335-466c74b00d8cn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8F0BDFB7-231D-4E55-AED8-86BE1DD286C5%40ix.netcom.com.

Re: [go-nuts] Golang 1.19+ preemption model on a preemptive linux kernel?

[Ian Lance Taylor]

On Mon, Dec 5, 2022 at 10:03 AM Andrew Athan  wrote:
>
> I'm having trouble finding definitive information about golang's preemption 
> model when running with GOMAXPROCS>1 on a multi-core Intel with preemtive 
> linux kernel.
>
> As I understand it, in such a scenario, two goroutines may be scheduled by 
> the go runtime onto two separate system threads on two separate CPUs.
>
> Isn't it true then, that irrespective of whether the two goroutines are 
> scheduled onto separate CPUs, the OS may preemptively interrupt either of 
> those goroutines to let the other run? Also, that if they are indeed 
> scheduled onto separate CPUs, that the memory accesses made by those 
> goroutines are interleaved even if no individual goroutine is "preempted"?
>
> I'm asking because in reviewing the code of some "lock free" concurrent data 
> structures written in go, it appears the authors have made certain 
> assumptions about race conditions (e.g.,  the code of on goroutine assumes 
> that another goroutine won't perform a load/store between two function calls 
> the first goroutine makes to check a boolean and store a value).
>
> Given that goroutines are mappes onto system threads, and that in a 
> preemptive kernel those threads may be preempted and/or irrespective of the 
> preemptive kernel issue may run on separate cores, no assumptions should be 
> made about the "atomicity" of multiple statements in a goroutine relative to 
> other goroutines. Right?
>
> Where can I get the most current and accurate information about this topic?

I'm not aware of any detailed documentation of the issues you are
looking at.  So the most current and accurate information is the
source code.  Or this mailing list.

You are correct that non-atomic memory operations made by different
goroutines can be freely interleaved.  If there is any code in the
runtime that assumes otherwise, that code is badly broken.  If you can
point to the specific code that you are concerned about, perhaps we
can allay your concerns, or perhaps we can fix the bug.  Thanks.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUYxpZbNmf4iG6r23RcL%2B%2BDJBEUfUxJff1Y59y0dXu7fQ%40mail.gmail.com.

[go-nuts] Re: Looking Backend Dev Learning Resources

[Tsvetomir Lazarov]

How relevant is Effective Go actually, given the January 2022 update that 
this document has not been updated significantly since 2009?

On Thursday, December 1, 2022 at 6:33:01 PM UTC+2 marc...@gmail.com wrote:

> https://mehdihadeli.github.io/awesome-go-education/
>
> Op woensdag 30 november 2022 om 14:12:46 UTC+1 schreef 
> christoph...@gmail.com:
>
>> If you know the basics of Go already, Effective Go 
>>  and the Go FAQ are 
>> good starting points for learning what idiomatic Go is like and why certain 
>> decisions in the language design were made.
>>
>> On Saturday, November 26, 2022 at 11:37:29 AM UTC+1 sksin...@gmail.com 
>> wrote:
>>
>>> Dear fellow Gophers, 
>>>
>>> I have been looking for resources to learn the backend development in 
>>> Go. I want some good resources which talk about how to write Idiomatic Go 
>>> Code, the Best Practices to follow, Anti-Patterns to avoid and writing an 
>>> overall secure server. Any good resources are highly appreciated. 
>>>
>>> Thanks and Regards
>>> SK Singh
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9a340711-4af7-4519-af09-fd34da45ea15n%40googlegroups.com.

Re: [go-nuts] proposal: Qo

[gonads]

Less specific.

Qo Language (Quantum Computing Language). Qo Cloud (Computing). Qo 
Serverless (Computing). Qo Commercials (Go Team Members). Set the Status 
Quo, with Qo.


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f09282c98302e47e6a8d5982860e131f%40switchupcb.com.

[go-nuts] Valgo is a type-safe, expressive, and extensible validator library for Golang.

[Carlos Forero]

https://github.com/cohesivestack/valgo

Valgo is in its early stages. We would like community feedback before 
releasing version 1.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0bbe4b3f-6fc3-4279-8cab-d7e9c715c84bn%40googlegroups.com.

[go-nuts] Golang 1.19+ preemption model on a preemptive linux kernel?

[Andrew Athan]

I'm having trouble finding definitive information about golang's preemption 
model when running with GOMAXPROCS>1 on a multi-core Intel with preemtive 
linux kernel.

As I understand it, in such a scenario, two goroutines may be scheduled by 
the go runtime onto two separate system threads on two separate CPUs.

Isn't it true then, that irrespective of whether the two goroutines are 
scheduled onto separate CPUs, the OS may preemptively interrupt either of 
those goroutines to let the other run? Also, that if they are indeed 
scheduled onto separate CPUs, that the memory accesses made by those 
goroutines are interleaved even if no individual goroutine is "preempted"?

I'm asking because in reviewing the code of some "lock free" concurrent 
data structures written in go, it appears the authors have made certain 
assumptions about race conditions (e.g.,  the code of on goroutine assumes 
that another goroutine won't perform a load/store between two function 
calls the first goroutine makes to check a boolean and store a value).

Given that goroutines are mappes onto system threads, and that in a 
preemptive kernel those threads may be preempted and/or irrespective of the 
preemptive kernel issue may run on separate cores, no assumptions should be 
made about the "atomicity" of multiple statements in a goroutine relative 
to other goroutines. Right?

Where can I get the most current and accurate information about this topic?

A.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0912663b-d4cb-41ee-a335-466c74b00d8cn%40googlegroups.com.

Re: [go-nuts] Get network interface values

['Axel Wagner' via golang-nuts]

This isn't really something that can be solved, in Go or otherwise. The
operating system only knows that a given interface is connected to a
network, it doesn't know whether that network is routes to the internet.

Any piece of software that actually checks of you are "connected to the
internet" does some variant of what you described as "connecting to
google.com". If you want to check that you are connected to the internet,
you'll have to do the same.

On Mon, Dec 5, 2022, 13:49 Nikhilesh Susarla 
wrote:

> Hi,
>
> I want to check if my system is connected to internet or not.
> 1. Say I may be connected to wifi but there is no internet.
> 2. Connected in lan but there is no internet
>
> So, I want a low level package to get the status of the network card or
> related in Go. Rather than just querying google.com
>
>
> Is there any open source package or C supported package where Go has a
> wrapper.
>
> Thank you
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3f0df694-37c8-4e1f-9a0e-ba39969927a4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGGpAS-vaTbqPhhaJou6hWpUv65ohVizefN%3DhRBu2wVXA%40mail.gmail.com.

Re: [go-nuts] Re: Golang Github Package References

[Brian Candler]

On Monday, 5 December 2022 at 13:12:26 UTC loji...@gmail.com wrote:

> The problem as I see it, is that when the security of the code relies on a 
> package is outside the main program/executable, it can open potential 
> problems of code injection; code changes or forks or if the url has moved, 
> how these issues could cause the collapse of the integrity of the software. 
>

Go is a compiled language, and there are a few things you need to 
understand about the compilation process.

Firstly, there is no dynamic linking.  The *source code* for those 
third-party libraries is fetched at compile time, and then compiled into 
your binary, and that's it.  The binary is flat, containing your code and 
the third-party code, all compiled together into a single blob.  The binary 
does not change, and at runtime no new dependencies are pulled in or 
updated; there is no route for code injection.

Secondly, using go modules, in your source code you point to the *exact* 
version of the third-party module that you want to import - either using 
its version number, or a git commit reference. This is stored in 
go.mod/go.sum.  The build of your application is reproducible.  If the 
third-party module author releases a new or different version of their 
module, then you have to explicitly pick it up - e.g. with "go mod tidy".

Therefore, if someone else builds your code from source, version X, then 
they will get exactly the same dependencies as you had when you released 
version X.

Thirdly, the imports are really module names, not urls.  Some of these 
*happen* to look a bit like urls, and indeed there are mechanisms to fetch 
the module source code automatically at compile time, if the module name 
has the correct format.  But apart from that these are not used as urls, 
and certainly not as urls embedded in the binary..

Finally, if you are concerned about security problems, then you are right 
to be aware of the versions of third-party libraries that you are using.  
Of course, using newer versions of those libraries is likely to result in 
fewer security holes, rather than more.  Therefore you need to track them, 
and when they are updated, you'll want to recompile and re-release your own 
software to take advantage of the fixes.

HTH,

Brian.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5a6d52e5-c99d-424a-b3de-4147546991f8n%40googlegroups.com.

Re: [go-nuts] Re: Golang Github Package References

[Mucas Schlack]

Ok,
The package code in question looks like this: -

import (
"context"
"encoding/json"
"flag"
"fmt"
"io"
"log"
"net/http"
"net/http/httputil"
"net/url"
"os"
"time"
"github.com/go-oauth2/oauth2/v4/generates"
"github.com/go-oauth2/oauth2/v4/errors"
"github.com/go-oauth2/oauth2/v4/manage"
"github.com/go-oauth2/oauth2/v4/models"
"github.com/go-oauth2/oauth2/v4/server"
"github.com/go-oauth2/oauth2/v4/store"
"github.com/go-session/session"
) The problem as I see it, is that when the security of the code relies on
a package is outside the main program/executable, it can open potential
problems of code injection; code changes or forks or if the url has moved,
how these issues could cause the collapse of the integrity of the software.
Snot-bag

Am Mo., 5. Dez. 2022 um 13:05 Uhr schrieb Robert Engels <
reng...@ix.netcom.com>:

> The op is trying to say, they wrote the code using github.com/robaho/fixed
> but then they decide they want to use a fork of fixed (maybe it is being
> maintained better, performance enhancements, etc) but they don’t want to
> change all of their code.
>
> Can they use a replace directive to point at the fork?
>
> I’m uncertain - because the package references in the fork would all need
> to change ? (unless it used relative references which are discouraged).
>
> I think that is the problem.
>
> On Dec 5, 2022, at 5:36 AM, Brian Candler  wrote:
>
> This question makes no sense to me.  If your code actually *uses* "
> github.com/installed-package" then it will fail to build or run without
> it.  On the other hand, if it doesn't use it, then just remove the import
> (the compiler will complain about unused imports anyway)
>
> I also don't know what you mean by "keep referencing" (you only need to
> import it once in a given source file), nor the "security reasons" you
> mention.
>
> Can you be more specific about what exactly you're trying to do?  Can you
> point to the real code in question?
>
> On Monday, 5 December 2022 at 10:46:17 UTC loji...@gmail.com wrote:
>
>> Hello Everyone,
>> I would like to install a Golang Github package, but I do not want to
>> keep referencing the installed package in the "imports" section of the code
>> - for security reasons, and because I want my code to be able to run
>> without those references. Does anyone know if there is a way to do this?
>>
>> For example:
>>
>> import (
>> "log"
>> "net/http"
>> "github.com/installed-package?"
>> )
>>
>> So, how can I remove the "github.com/installed-package"  reference above
>> and still be able to run the package?
>>
>> Regards,
>>
>> Snot-Bag
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/1a747ce2-8cbc-496d-a3bb-65211315e024n%40googlegroups.com
> 
> .
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/JqHFwcFQDdI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3F50C814-566E-4C72-B836-D669130FFE8D%40ix.netcom.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA8AC4UYo4hZ3-4KTGtoKkGsxu9VtZUBxaFjAwEYH9U8pdU7zg%40mail.gmail.com.

[go-nuts] Get network interface values

[Nikhilesh Susarla]

Hi, 

I want to check if my system is connected to internet or not.
1. Say I may be connected to wifi but there is no internet. 
2. Connected in lan but there is no internet

So, I want a low level package to get the status of the network card or 
related in Go. Rather than just querying google.com


Is there any open source package or C supported package where Go has a 
wrapper. 

Thank you

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3f0df694-37c8-4e1f-9a0e-ba39969927a4n%40googlegroups.com.

[go-nuts] Re: Issue with regular expression processing

[Brian Candler]

For the documentation of the RE2 regular expression dialect, see
https://github.com/google/re2/wiki/Syntax

^ matches only at start of string, unless you set multi-line mode:
https://go.dev/play/p/GCglWdjqEHq

On Monday, 5 December 2022 at 12:10:20 UTC sbezv...@cisco.com wrote:

> Hello,
>
>  
>
> I came across a strange behavior in regular expression processing, please 
> see the following simple code which is supposed to match the second line of 
> data variable.
>
>  
>
> https://go.dev/play/p/QPrNRuhCv8r
>
>  
>
> The matching data starts at the beginning of the 3rd line so I expect the 
> pattern `^` + l + `\s+([0-9]+)\s+` would result in a match, but it does 
> not. If I remove `^` from the regular expression, then the match is found. 
> Running `grep -E` with regular expression which includes `^` also finds the 
> expected match.
>
>  
>
> Appreciate some feedback. Is it a bug or I do something incorrectly?
>
>  
>
> Thank you
>
> Serguei 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/809137c4-66c6-42b2-a30f-fa62e1883dadn%40googlegroups.com.

[go-nuts] Issue with regular expression processing

[sbezverk]

Hello,

 

I came across a strange behavior in regular expression processing, please see 
the following simple code which is supposed to match the second line of data 
variable.

 

https://go.dev/play/p/QPrNRuhCv8r

 

The matching data starts at the beginning of the 3rd line so I expect the 
pattern `^` + l + `\s+([0-9]+)\s+` would result in a match, but it does not. If 
I remove `^` from the regular expression, then the match is found. Running 
`grep -E` with regular expression which includes `^` also finds the expected 
match.

 

Appreciate some feedback. Is it a bug or I do something incorrectly?

 

Thank you

Serguei 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1DF73419-C0C8-48CD-ACE5-8745927C70F9%40gmail.com.

Re: [go-nuts] Re: go install of forked repo

[Robert Engels]

As I mentioned in the other thread - isn’t that made more complicated because 
the module author (the module that is imported) doesnt use relative packages 
references? If they did wouldn’t this be simpler?

> On Dec 5, 2022, at 5:32 AM, Brian Candler  wrote:
> 
> Or there is vendoring.
> 
>> On Monday, 5 December 2022 at 09:16:24 UTC stephane@gmail.com wrote:
>> Hi, 
>> 
>> If you don't want to commit go.mod, you can use go.work 
>> https://go.dev/ref/mod#go-work-file
>> 
>>> On Monday, December 5, 2022 at 9:52:17 AM UTC+1 kortschak wrote:
>>> On Sun, 2022-12-04 at 23:47 -0800, 'Christian Stewart' via golang-nuts 
>>> wrote: 
>>> > I definitely fork things and use "replace" quite frequently. 
>>> 
>>> It *can* be used, but it is not the solution in the general case as 
>>> Volker said. A replace in a library's go.mod file has no effect on 
>>> consumers of that library. 
>>> 
>>> https://go.dev/ref/mod#go-mod-file-replace: 
>>> 
>>> > replace directives only apply in the main module’s go.mod file and 
>>> are ignored in other modules. See Minimal version selection for 
>>> details. 
>>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/bb56e11c-e6c5-425a-8243-df7929021014n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/135F3BF8-8D03-4279-A201-F641CCD6120C%40ix.netcom.com.

Re: [go-nuts] Re: Golang Github Package References

[Robert Engels]

The op is trying to say, they wrote the code using github.com/robaho/fixed but 
then they decide they want to use a fork of fixed (maybe it is being maintained 
better, performance enhancements, etc) but they don’t want to change all of 
their code. 

Can they use a replace directive to point at the fork?

I’m uncertain - because the package references in the fork would all need to 
change ? (unless it used relative references which are discouraged). 

I think that is the problem. 

> On Dec 5, 2022, at 5:36 AM, Brian Candler  wrote:
> 
> This question makes no sense to me.  If your code actually *uses* 
> "github.com/installed-package" then it will fail to build or run without it.  
> On the other hand, if it doesn't use it, then just remove the import (the 
> compiler will complain about unused imports anyway)
> 
> I also don't know what you mean by "keep referencing" (you only need to 
> import it once in a given source file), nor the "security reasons" you 
> mention.
> 
> Can you be more specific about what exactly you're trying to do?  Can you 
> point to the real code in question?
> 
>> On Monday, 5 December 2022 at 10:46:17 UTC loji...@gmail.com wrote:
>> Hello Everyone,
>> I would like to install a Golang Github package, but I do not want to keep 
>> referencing the installed package in the "imports" section of the code - for 
>> security reasons, and because I want my code to be able to run without those 
>> references. Does anyone know if there is a way to do this?
>> 
>> For example: 
>> 
>> import (
>> "log"
>> "net/http"
>> "github.com/installed-package?"
>> )
>> 
>> So, how can I remove the "github.com/installed-package"  reference above and 
>> still be able to run the package?
>> 
>> Regards,
>> 
>> Snot-Bag
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/1a747ce2-8cbc-496d-a3bb-65211315e024n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3F50C814-566E-4C72-B836-D669130FFE8D%40ix.netcom.com.

[go-nuts] Re: Golang Github Package References

[Brian Candler]

This question makes no sense to me.  If your code actually 
*uses* "github.com/installed-package" then it will fail to build or run 
without it.  On the other hand, if it doesn't use it, then just remove the 
import (the compiler will complain about unused imports anyway)

I also don't know what you mean by "keep referencing" (you only need to 
import it once in a given source file), nor the "security reasons" you 
mention.

Can you be more specific about what exactly you're trying to do?  Can you 
point to the real code in question?

On Monday, 5 December 2022 at 10:46:17 UTC loji...@gmail.com wrote:

> Hello Everyone,
> I would like to install a Golang Github package, but I do not want to keep 
> referencing the installed package in the "imports" section of the code - 
> for security reasons, and because I want my code to be able to run without 
> those references. Does anyone know if there is a way to do this?
>
> For example: 
>
> import (
> "log"
> "net/http"
> "github.com/installed-package?"
> )
>
> So, how can I remove the "github.com/installed-package"  reference above 
> and still be able to run the package?
>
> Regards,
>
> Snot-Bag
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1a747ce2-8cbc-496d-a3bb-65211315e024n%40googlegroups.com.

Re: [go-nuts] Re: go install of forked repo

[Brian Candler]

Or there is vendoring .

On Monday, 5 December 2022 at 09:16:24 UTC stephane@gmail.com wrote:

> Hi, 
>
> If you don't want to commit *go.mod*, you can use *go.work 
>  *https://go.dev/ref/mod#go-work-file
>
> On Monday, December 5, 2022 at 9:52:17 AM UTC+1 kortschak wrote:
>
>> On Sun, 2022-12-04 at 23:47 -0800, 'Christian Stewart' via golang-nuts 
>> wrote: 
>> > I definitely fork things and use "replace" quite frequently. 
>>
>> It *can* be used, but it is not the solution in the general case as 
>> Volker said. A replace in a library's go.mod file has no effect on 
>> consumers of that library. 
>>
>> https://go.dev/ref/mod#go-mod-file-replace: 
>>
>> > replace directives only apply in the main module’s go.mod file and 
>> are ignored in other modules. See Minimal version selection for 
>> details. 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/bb56e11c-e6c5-425a-8243-df7929021014n%40googlegroups.com.

[go-nuts] Golang Github Package References

[Mucas Schlack]

Hello Everyone,
I would like to install a Golang Github package, but I do not want to keep 
referencing the installed package in the "imports" section of the code - 
for security reasons, and because I want my code to be able to run without 
those references. Does anyone know if there is a way to do this?

For example: 

import (
"log"
"net/http"
"github.com/installed-package?"
)

So, how can I remove the "github.com/installed-package"  reference above 
and still be able to run the package?

Regards,

Snot-Bag

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1ec2ea3f-efbd-431d-9654-0879079770edn%40googlegroups.com.

Re: [go-nuts] Re: go install of forked repo

[Stéphane Jeandeaux]

Hi, 

If you don't want to commit *go.mod*, you can use *go.work *
https://go.dev/ref/mod#go-work-file

On Monday, December 5, 2022 at 9:52:17 AM UTC+1 kortschak wrote:

> On Sun, 2022-12-04 at 23:47 -0800, 'Christian Stewart' via golang-nuts
> wrote:
> > I definitely fork things and use "replace" quite frequently.
>
> It *can* be used, but it is not the solution in the general case as
> Volker said. A replace in a library's go.mod file has no effect on
> consumers of that library.
>
> https://go.dev/ref/mod#go-mod-file-replace:
>
> > replace directives only apply in the main module’s go.mod file and
> are ignored in other modules. See Minimal version selection for
> details.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1ec27070-6521-4b4c-acd7-ad9956e6ea67n%40googlegroups.com.

Re: [go-nuts] Re: go install of forked repo

['Dan Kortschak' via golang-nuts]

On Sun, 2022-12-04 at 23:47 -0800, 'Christian Stewart' via golang-nuts
wrote:
> I definitely fork things and use "replace" quite frequently.

It *can* be used, but it is not the solution in the general case as
Volker said. A replace in a library's go.mod file has no effect on
consumers of that library.

https://go.dev/ref/mod#go-mod-file-replace:

> replace directives only apply in the main module’s go.mod file and
are ignored in other modules. See Minimal version selection for
details.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/57399c8014247d4c72a8e830b326e8674ad18247.camel%40kortschak.io.

[go-nuts] Re: HTTPS proxy issue using CONNECT method

[gbarr]

On Saturday, December 3, 2022 at 4:46:54 PM UTC maumon...@gmail.com wrote:

> Hello all,
>
> I have been facing an issue when I try to create a HTTP client which needs 
> to connect through a HTTPS proxy using the HTTP CONNEC method. I know that 
> it can be achieved setting my own http.Transport object. However the issue 
> seems to be in the current implementation of /net/http/transport.go code.
>
> In my environment, I am developing a HTTP client which ALWAYS use a HTTPS 
> proxy using HTTP CONNECT method. This client is allowed to reach HTTP or 
> HTTPS targets. Therefore, I noticed that when I try to reach a HTTPS 
> target, the the transport layer works as expected and it uses the HTTP 
> CONNECT method. However, when I try to reach a HTTP target, the transport 
> does not use the CONNECT  method.
>

This is normal. The CONNECT method allows a client to create a TCP tunnel 
through your gateway. This allows your client to perform all TLS 
negotiation.

However for HTTP requests this extra layering is not required. In the 
standard library you can see the setting of pconn.isProxy=true 
at 
https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1093
  
this is later used when writing the request 
at 
https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/request.go;l=521

Essentially it changes the form of the http method from GET 
/path/to/resource to GET http://hostname/path/to/resource so your gateway 
would then know that this is a proxy request and perform the external 
request

Graham.


> Looking at the transport.go code, I realized that the check to use the 
> CONNECT method is based on the protocol of the target instead of being on 
> the protocol of the proxy URL. Below is a link showing that:
>
> 1. HTTP check
>
>
> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1092
>
> 2. HTTPS check
>
>
> https://cs.opensource.google/go/go/+/refs/tags/go1.9.5:src/net/http/transport.go;l=1099
>
> As can be seen on the links above, the condition is based on cm 
> 
> .targetScheme 
> 
>  instead 
> of cm 
> 
> .proxyURL 
> 
> .Scheme 
> .
>  
> Is it a bug?
>
> *Go version: go version go1.19.3 linux/amd64*
>
> Mauro
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/885c9184-6696-4dc2-8c75-bfc48b510214n%40googlegroups.com.