Re: [go-nuts] How to use atomic_int in cgo?

[Devon H. O'Dell]

Forgot to reply to the list. Oops. Sorry for the second delivery, Changkun.

On Mon, Feb 15, 2021 at 12:39 changkun  wrote:

> Hi Ian,
>
> Thanks for the hint, but I have some follow-up questions:
>
>
>> Even if there were a way to do this, an atomic variable is not a good
>> synchronization mechanism, because the other side has to poll the
>> variable.
>
> Indeed, it the other side will be a spin loop to poll the atomic variable,
> which ...
>
>
>
>> You can do this as a last resort, but it doesn't sound like
>> you are at a last resort here. I suggest that you have your C
>> function call a Go function to write a value on a channel.
>
> seems easy to write than this (?).
>
> Say a Go function foo is called from the C side, to be able to send to the
> corresponding channel, C must pass that channel to the Go function, which
> brings to the initial question:  pass a channel from Go to C, is it
> supported at the moment (?)
>
> How could a Go function be able to send content to differently allocated
> channels in correspondingly invoked C functions?
>


I think this is only a problem if you need a separate channel per
invocation for some reason, which seems unlikely. But if you did, you could
have a  map of int to chan T and pass the integer key through C.

Kind regards,

--dho


>
>
> Sincerely,
> Changkun
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7adba3a6-ee2f-4923-905d-9afc2b9f796an%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAFgOgC_zWPuPtNAc74CM7aimNh7k-ZbNT_TsFbCntWDT6mX_VQ%40mail.gmail.com.

Re: [go-nuts] tcp/ip stack for bare metal

[Devon H. O'Dell]

Hi Steve,

You might take a look at the stack implemented in
https://github.com/google/gvisor.

—dho

On Wed, Jan 1, 2020 at 13:16 Steve Simon  wrote:

> hi,
>
> please forgive a newbie question but is there a production ready tcp/ip
> stack written in go?
>
> i would like to run go on bare metal on a fairly grunty cpu - not running
> hosted simplifies the dependency tree for product release. i need only a
> trivial filesystem and (i think) no other OS services.
>
> i simple C based tcp/ip stack would be a reasonable alternative but i fear
> lwip misses some features i would need.
>
> thoughts gratefully received.
>
> -Steve
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/42DB02DC-0CBC-44F1-BBE6-5256FDEBE1A7%40quintile.net
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAFgOgC9k%3DHmkQMaJ5djTgbQSxv3AX96SB2BOtD2-DjnN%3DPRebA%40mail.gmail.com.

Re: [go-nuts] Standard library ETag/Last-Modified conditional request logic and best practices?

[Devon H. O'Dell]

Op di 6 aug. 2019 om 09:10 schreef :
>
> Thanks Devon!

You're welcome!

> So just to clarify our request flow is:
>
> Client > CDN > Go Reverse Proxy > Origin
>
> Our Go Reverse Proxy has historically been responsible for adding caching 
> headers (e.g. Cache-Control and Surrogate-Control) when the origins have 
> failed to do so (as a way to ensure things are cached appropriately).
>
>> It's unclear to me why you should be setting an etag header if you're a 
>> proxy.
>
> That's why when it came to looking at setting serve stale defaults for our 
> origins (e.g. stale-while-revalidate and stale-if-error) I realized that 
> somewhere along the chain an appropriate ETag/Last-Modified should be set and 
> that's why I started wondering if our proxy should be responsible for setting 
> them.
>
> Even then I felt like setting Last-Modified was way outside the 
> responsibility of our proxy, but that maybe setting of ETag would have 
> sufficed.

Ah, I see. So you're still the content owner; you're just further
offloading work from between your origin and the CDN. Assuming you're
not still multi-tenant behind your proxy (i.e. your proxy only serves
_your_ assets), then I think it's probably reasonable for you to make
that determination at your proxy. And from that perspective, I agree
that you'd be more interested in ETag/INM than LM/IMS on your proxy.

>> Unless you're serving from the filesystem handler (which does
>> implement IMS/INM), you'll need to implement these yourself.
>
> I think your other related answers might explain to me why the go reverse 
> proxy doesn't support conditional requests, in that it's NOT a 'caching 
> proxy' and so being able to handle that revalidation logic wouldn't make 
> sense.

Right -- it boils down to whether a proxy is transparent or not. A
transparent proxy observes traffic and makes no changes to the
protocol or the discussion over it. The only impact it can really have
is if it stops servicing requests. A transparent proxy assumes that
both sides of the connection are speaking the same protocol, and so it
doesn't really have to know about protocol semantics.

A caching proxy isn't transparent. It looks like it because it ends up
having very good knowledge of the protocol it's proxying, but every
request isn't passed through unmodified, so it's by definition opaque.

>> Note that you _could_ simply proxy this to the origin and let it
>> handle the validation. This is often overkill for what people actually
>> need, but it is guaranteed to work.
>
> OK, so as we are indeed just proxying the request pretty much 'as is' to the 
> origin, i.e. the CDN is making the revalidation conditional request when our 
> stale-while-revalidate TTL expires, I'm guessing (I appreciate this is the 
> 'basics' of how a proxy works, but I want to talk it through in case I'm 
> mistaken in any way!) the go proxy will transparently keep that information 
> for the origin to respond with the appropriate ETag/Last-Modified, and the go 
> proxy again will transparently pass back their response through to the CDN to 
> then update its cache if it indeed got a `200 OK` from origin or to continue 
> serving stale if the origin returned a `304 Not Modified` (and in either case 
> I expect the origin should send ETag/Last-Modified headers regardless of 
> 200/304 status').

It's been a while since I was working at a CDN (Fastly) so I may be a
bit fuzzy here; what you've written sounds like a correct
understanding. Again, as the proxy is transparent, its knowledge of
the protocol is really not meaningful; as long as your CDN and origin
both implement the protocol correctly, your transparent proxy will
also be by definition correct. (Though I'd note that one could argue
that X-Forwarded-For makes most HTTP proxies not strictly transparent;
it's also not super meaningful anyway except for logging to make sure
you understand your topology when things go wrong.)

As you've already got a CDN in the picture, it seems to me (especially
if you're using origin shielding) that it won't be super helpful for
you to implement LM/IMS or ETag/INM in your proxy. Lack of explicit
support for this in Go is therefore hopefully not an issue for you
because a CDN supporting stale-while-revalidate and stale-if-error
will already be shielding your origins from heaps of revalidation
traffic.

However, it sounds like your origin isn't setting ETag or other cache
control headers everywhere it could. Adding strong ETags at your proxy
should be reasonably cheap since the CDN is shielding you from
revalidation storms, and it can also save you on your CDN's bandwidth
bill as it will allow your CDN to respond with 304s rather than 200s
for more objects.

>> A hash function over the body of the response would constitute strong
>> validation. I'm not sure why you'd need to mix in the path; there's
>> nothing wrong with serving the exact same content between two
>> endpoints, and the ETag is tied to a response object.
>
> Ah ok, so 

Re: [go-nuts] Standard library ETag/Last-Modified conditional request logic and best practices?

[Devon H. O'Dell]

Hi Mark,

Whether or not your proxy is caching, you may find RFC7234[1] relevant
in addressing some of your questions (as well as many you may later
encounter). I think you may find section 5.2 to be of particular
interest, though any proxy author should be familiar with the full
text.

Op di 6 aug. 2019 om 05:14 schreef :
>
> Hello,
>
> I'm using Go's standard library reverse proxy and I'm trying to figure out if 
> the standard library HTTP web server (e.g. http.ListenAndServe) implements 
> the relevant conditional request handling logic for ETag/Last-Modified 
> headers.
>
> I did some Googling and noticed the HTTP file system request handler 
> (https://golang.org/src/net/http/fs.go) does implement that logic, but I 
> couldn't find the same for the HTTP web server.
>
> I also couldn't find any examples of setting ETags/Last-Modified (other than 
> this basic implementation for setting ETags: 
> https://github.com/go-http-utils/etag/blob/master/etag.go).
>
> What's confusing me there is the concept of "strong" and "weak" validation 
> and how certain scenarios might influence whether an ETag is marked as either 
> strong or weak (depending on the implementation -- see 
> https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests#Validators).
>
> So to recap, my questions are (and I appreciate some of these are outside the 
> scope of just Go -- so apologies if that's not allowed in this forum):

I think this is a fine question for this list, which isn't necessarily
constrained to questions about Go, but also for how to achieve things
while using Go. Lines get blurred since many technologies touch each
other. I don't think any apologies are necessary :).

> 1. Should I set ETag/Last-Modified in a proxy? Last-Modified feels like it's 
> not the responsibility of the proxy but the origin, where as an ETag is 
> something I feel is "ok" to do in the proxy as a 'fallback' (as we already 
> set 'serve stale' caching headers on behalf of our origins if they neglect to 
> include them).

ETag and Last-Modified should be sent by the origin to any proxy to
let the proxy know when the content is stale (assuming the proxy is
caching). The only case in which a proxy might set these things is if
there are configurations provided by the content owner that allow the
proxy to determine what the lifetime of the response object is outside
of response headers. This is most useful in cases where the content is
synthetically generated by the proxy as a result of the content
owner's configuration. If you don't have such a system in place, your
proxy should never be generating these response headers, and you
should be working with your customers / users to help them understand
when to set cache control headers.

> 2. Do I need to implement `If-None-Match` and `If-Modified-Since` behaviours 
> myself (i.e. is it not provided by the Go standard library's HTTP web server)?

Unless you're serving from the filesystem handler (which does
implement IMS/INM), you'll need to implement these yourself.

Note that you _could_ simply proxy this to the origin and let it
handle the validation. This is often overkill for what people actually
need, but it is guaranteed to work.

One trick that many CDN providers leverage is to offer their customers
the option to serve the stale object while revalidating it. If that
option is set, an asynchronous revalidation request is spawned -- new
requests are blocked on the completion of that request -- and the
potentially stale content is served to the original requester without
blocking that request on revalidation.

> 3. I was planning on setting an ETag header on the response from within 
> httputil.ReverseProxy#ModifyResponse but wasn't sure if that would be the 
> correct place to set it.

It's unclear to me why you should be setting an etag header if you're a proxy.

> 4. What constitutes a strong/weak validator (e.g. would a simple hash 
> function generating a digest of the URL path + response body suffice)?

A hash function over the body of the response would constitute strong
validation. I'm not sure why you'd need to mix in the path; there's
nothing wrong with serving the exact same content between two
endpoints, and the ETag is tied to a response object.

Weak validation is signified by an additional "W/" in the etag
identifier. In practice, this means that you mustn't use weak
identifiers for serving byte-range requests. Weak identifiers may be
more useful for dynamically generated content where you might for
example have a date added in, or an ad server link that is rotated
each time the page is served, or a counter, or something like this. An
example of weak validation would be something that is version and
encoding based -- each time the content changes materially, you'd
increment the version, and some identifier for the content-encoding
would also be mixed in.

> Thanks for any help/insights/opinions y'all can share with me.
>
> Kind regards,
> Mark
>

Hope that helps!

Re: [go-nuts] prevent alteration of binaries once distributed in the wild?

[Devon H. O'Dell]

The signature would probably be computed only over data segments (or
equivalent) in the executable file format, and stored outside of those
sections. This approach doesn't work when the person with the binary
can write to the binary (which is usually always). They can just
change the signature to match whatever changes they've made to the
binary. Further tamper-proofing is a game of cat-and-mouse. A
determined user with a debugger can circumvent most of what you can
bake into your binary.

Other anti-theft and anti-tampering approaches exist. The only way to
ensure what code is being run is to not have the client run it; you
can hide functionality behind a network. This places an additional
burden on you to provide reliable resources. If you believe your
software has been tampered with, you revoke access to the license (and
therefore to some functionality of the program). This ends up turning
into a legal hassle, so you'll want to consult a lawyer about how and
when you can actually do this, and how to craft such a contract that
enables you to do this.

I don't think I would be worried about this sort of thing unless I was
in some very specific circumstances. And the circumstances I can think
of where I'd be concerned about this, I might use other approaches (OS
security features, audited remote access requirements, etc.) to ensure
the software was tamper-proof.

Without specifics on what you are writing (or have written), it's hard
to offer more specific thoughts.

Kind regards,

--dho

Op di 23 jul. 2019 om 11:51 schreef clement auger :

>
> Hi,
>
> I m looking for a technique to prevent binary alteration once distributed in 
> the wild.
>
> I have no clue what i m asking for.
>
> I was imagining a solution where a signature is prepended to the binary and 
> checked during the startup sequence.
>
> However i do understand (well ... i imagine it) the chicken and egg problem 
> behind this question (no way to sign a binary that will gets its signature 
> inserted right after its signature was computed)
>
> Is there anything possible ?
>
> Is it something i should be worried of, to start with ? (independently of the 
> interest that altering such binary might raises, i d prefer a strictly 
> technical analysis)
>
> thanks for feedbacks.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/194e2e4f-e41b-4c23-b241-f8fe1f5da154%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAFgOgC8hDDUD2NDOKw5BF3i3P%2BdxMZLTqVR3b7ZN9Es6ke5eFw%40mail.gmail.com.

Re: [go-nuts] Re: Interesting public commentary on Go...

[Devon H. O'Dell]

Maybe this story about suggesting the murder of a colleague is supposed to
be a tongue-in-cheek joke, but I want to push back heavily against it. And
I’m sorry that this is devolving significantly from the original topic, but
I don’t think this should slide by.

Though my professional experience is limited to “just” a bit under 20
years, this story embodies a culture I want to see change in both
professional and open source environments. This is a culture that makes
neophyte programmers — indeed even some with experience — live in a
constant state of fear. I think back to a job I had about a decade ago
where I was so bent on outperforming all my colleagues, I missed an
opportunity to help a junior developer improve. This person was doing
terribly by all measurable metrics (at least compared to the other
colleagues), but nobody intervened to help them improve. I remember that
person saying to several people — and this was said in their exit interview
— that a big reason they quit was that they lived in a constant state of
fear about being fired. That’s neither healthy nor ok for the work culture
to have supported.

Eventually, I decided to try to help this person after they left. This was
through  comments left on their blog. In retrospect these comments read
more as attacks or attempts to boost my own image over theirs. This person
blocked me on Twitter many years ago and disabled comments on their blog.

I regret this in its entirety. I can’t imagine looking back at a time I
suggested shooting someone, thinking it was funny, and sharing that
globally on an open source list with a CoC to be inviting and welcoming to
neophytes.

Having since successfully mentored individuals into systems programming
teams, I can’t imagine working in an environment that would tolerate such a
comment. If I heard such a thing today, I would make it clear that such
commentary is unacceptable, if not file a complaint with HR.

Having also worked with people who seem immune to learning, I understand
that helping folks can be a drain, especially when it’s not successful. But
suggesting shooting a person is just not ok, and to be frank, it doesn’t
make the story funny and you owe that person an apology. There are plenty
of other more constructive ways to handle such a situation.

Kind regards,

—dho


On Thu, May 30, 2019 at 16:33 David Skinner  wrote:

> I only rarely use generics in Go. When I do so, it is implemented using
> the +generate. The repos with my generics stuff is not public. If they
> were, they might be incomprehensible. While I rather like Fo, the thought
> of C++ style generics makes me cringe. Code must compile but it needs to be
> readable.
>
> I am very old school, I started programming with 8008 machine code. If
> something does not meet my needs, I may complain, but I may just write what
> I need. Go does not have generics but it is very easy for any user to
> implement generics in a variety of ways on an as needed basis. The thing
> is, I am not committed to Go, I am willing to use whatever works best for
> me, and right now that is Go, and I believe that that is the result of the
> experience of the Go team residing at Google in working as a team.
>
> I remember doing a code review at Sierra Online, it was a metrics project
> to evaluate employee performance, one programmer was so bad, I asked the
> head of the programming department to have him shot. He said, you want him
> fired? No, I want him shot, if you fire him, he will go and write bad code
> somewhere else. For some reason I do not understand, the company had a
> policy against shooting programmers that violated the style guidelines.
>
> When this is your life and your livelihood, it is easy to get emotional.
> Right now, I am still saying Thank you Google, and Thank you to the Go Dev
> Team. Well Done! Hope you do better next year. :)
>
>
>
> On Thursday, May 23, 2019 at 8:18:25 AM UTC-5, lgo...@gmail.com wrote:
>>
>> https://utcc.utoronto.ca/~cks/space/blog/programming/GoIsGooglesLanguage
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3e72df2e-e27d-4dbe-9545-8527bdce1e35%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [go-nuts] Is this a valid way of implementing a mutex?

[Devon H. O'Dell]

Op zo 17 mrt. 2019 om 15:28 schreef Louki Sumirniy
:
> As my simple iterative example showed, given the same sequence of events, 
> channels are deterministic, so this is an approach that is orthogonal but in 
> the same purpose - to prevent multiple concurrent agents from desynchronising 
> shared state, without blocking everything before access. It's not a journal 
> but the idea is to have each goroutine acting on the final state of the value 
> at the time it is invoked to operate on it. So you let everyone at it but 
> everyone stops at this barrier and checks if anyone else changed it and then 
> they try again to have a conflict free access.

Channels are deterministic in the sense that if items A and B are
enqueued in that order, they will be delivered in that order. They are
not deterministic in two ways: if items A and B are enqueued
concurrently, the order could be A B or B A; if two consumers of the
channel read from it concurrently, it is not deterministic which will
read.

This doesn't seem to me to be related in any way to locking / mutual
exclusion. The purpose of mutual exclusion is to serialize some
execution history. In other words, the idea is to make an interleaved
/ concurrent execution history of some set of instructions impossible.
That's not what a channel does. While a channel will act like a FIFO,
when multiple concurrent producers and / or consumers are present, one
cannot say which concurrent producer or consumer "wins" reading or
writing. So this pattern is really only meaningful for SPSC workloads.

What you term "access contention" seems to me somewhat analogous to
"data race". Assuming "somestate" is a pointer to some shared mutable
state, you might be able to determine whether that state changed after
the channel operation, but:

a) You're still susceptible to ABA problems when the state does not
change deterministically.
b) Even with a deterministically mutated state (such as a counter
increasing by a constant factor), you still have TOCTOU (time-of-check
versus time-of-use) issues. If your test of the state after the
channel operation doesn't show that the state has changed, what
guarantees that the state hasn't changed after your check? It's
turtles all the way down from here.
c) Both the assignment of "somestate" and a read from a channel create
a copy of the thing that was yielded from the function or was sent
over the channel, respectively. Unless that thing is a pointer to
shared mutable state, that state can't be observed to change via
external influence, anyway.

The channel use you've demonstrated is much more analogous to a binary
semaphore than any sort of lock in the sense that readers block on a
binary condition (whether or not the channel contains a value). You
can implement a mutex on a binary semaphore, but you need some notion
of "lock" and "unlock". To achieve locking semantics with an
unbuffered channel, the channel must begin in a non-empty state. When
the channel is read from, the state transitions to "locked". When the
lock is meant to be released, the releasing process writes to the
channel. This is perilous:

1) if you ever accidentally write twice, you no longer have the
guarantee of mutual exclusion, and
2) "unlocking" blocks until there's a a new process wishing to acquire the lock.

You can solve the second point with a buffered channel with a capacity
of 1, but you seem to be explicitly talking about unbuffered channels.
The first point can be solved, but I don't think it's super relevant.

If all you want to do is determine that some state was valid at the
time it was accessed, atomics or mutexes are the correct approach. If
you want to have multiple concurrent processes "agree" that they're
both working with the same state for the entire duration of some
execution history, you still need some sort of consensus protocol.

--dho

> On Sunday, 17 March 2019 20:52:12 UTC+1, Robert Engels wrote:
>>
>> https://g.co/kgs/2Q3a5n
>>
>> On Mar 17, 2019, at 2:36 PM, Louki Sumirniy  wrote:
>>
>> So I am incorrect that only one goroutine can access a channel at once? I 
>> don't understand, only one select or receive or send can happen at one 
>> moment per channel, so that means that if one has started others can't start.
>>
>> I was sure this was the case and this seems to confirm it:
>>
>> https://stackoverflow.com/a/19818448
>>
>> https://play.golang.org/p/NQGO5-jCVz
>>
>> In this it is using 5 competing receivers but every time the last one in 
>> line gets it, so there is scheduling and priority between two possible 
>> receivers when a channel is filled.
>>
>> This is with the range statement, of course, but I think the principle I am 
>> seeing here is that in all cases it's either one to one between send and 
>> receive, or one to many, or many from one, one side only receives the other 
>> only sends. If you consider each language element in the construction, and 
>> the 'go' to be something like a unix fork(), this means that the first 
>> 

Re: [go-nuts] Is this a valid way of implementing a mutex?

[Devon H. O'Dell]

I like to think of a channel as a concurrent messaging queue. You can
do all sorts of things with such constructs, including implementing
mutual exclusion constructs, but that doesn't mean that one is the
other.

Your playground example is a bit weird and very prone to various kinds
of race conditions that indicate that it may not be doing what you
expect. At a high level, your program loops 10 times. Each loop
iteration spawns a new concurrent process that attempts to read a
value off of a channel three times. Each iteration of the main loop
writes exactly one value into the channel.

As a concurrent queue, writes to the channel can be thought of as
appending an element, reads can be thought of as removing an element
from the front.

Which goroutine will read any individual value is not deterministic.
Since you're only sending 11 values over the channel, but spawn 10
goroutines that each want to read 3 values, you have at best 6
goroutines still waiting for data to be sent (and at worse, all 10) at
the time the program exits.

I would also point out that this is not evidence of mutual exclusion.
Consider a case where the work performed after the channel read
exceeds the time it takes for the outer loop to write a new value to
the channel. In that case, another goroutine waiting on the channel
would begin executing. This is not mutual exclusion. In this regard,
the example you've posted is more like a condition variable or monitor
than it is like a mutex.

Also note that in your second playground post, you're spawning 12
goroutines, so I'm not sure what "goroutine1" and "goroutine2" are
supposed to mean.

Kind regards,

--dho

Op zo 17 mrt. 2019 om 13:07 schreef Louki Sumirniy
:
>
> https://play.golang.org/p/13GNgAyEcYv
>
> I think this demonstrates how it works quite well, it appears that threads 
> stick to channels, routine 0 always sends first and 1 always receives, and 
> this makes sense as this is the order of their invocation. I could make more 
> parallel threads but clearly this works as a mutex and only one thread gets 
> access to the channel per send/receive (one per side).
>
> On Sunday, 17 March 2019 14:55:58 UTC+1, Jan Mercl wrote:
>>
>> On Sun, Mar 17, 2019 at 1:04 PM Louki Sumirniy  
>> wrote:
>>
>> > My understanding of channels is they basically create exclusion by control 
>> > of the path of execution, instead of using callbacks, or they bottleneck 
>> > via the cpu thread which is the reader and writer of this shared data 
>> > anyway.
>>
>> The language specification never mentions CPU threads. Reasoning about the 
>> language semantics in terms of CPU threads is not applicable.
>>
>> Threads are mentioned twice in the Memory Model document. In both cases I 
>> think it's a mistake and we should s/threads/goroutines/ without loss of 
>> correctness.
>>
>> Channel communication establish happen-before relations (see Memory Model). 
>> I see nothing equivalent directly to a critical section in that behavior, at 
>> least as far as when observed from outside. It was mentioned before that 
>> it's possible to _construct a mutex_ using a channel. I dont think that 
>> implies channel _is a mutex_ from the perspective of a program performing 
>> channel communication. The particular channel usage pattern just has the 
>> same semantics as a mutex.
>>
>> --
>>
>> -j
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Go could really use a while statement

[Devon H. O'Dell]

Interesting. I've never had that experience in the past 15ish years
doing POSIXish systems stuff. I've always found it the more natural
expression for handling EINTR from syscalls like read/write. I've also
never seen anyone seriously discourage its use in ##c on freenode over
the same-ish timeframe. I think the only place I may have seen it come
up may have been if someone was doing something egregious with a
macro. You've of course worked on C with far more people for far
longer than I have, but I'm really surprised to hear that this is a
widespread thing, and would be curious to hear more about where that
came from. Do C++ folks hate it and try to force that on C, perhaps?

--dho

2018-05-10 13:32 GMT-07:00 Rob Pike :
> There is a widespread dislike of do-while in C. Pretty much every time I've
> wanted it, my code reviewer made me take it out. I agree it has its place
> but it's one of those style things that social pressures seem to force. I
> would not be the one to try to argue for its return; I couldn't handle the
> backlash.
>
> -rbo
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: The &(*x) idiom for copying.

[Devon H. O'Dell]

2018-03-28 9:39 GMT-07:00 Devon H. O'Dell <devon.od...@gmail.com>:
> CopyExplicitDeref gets a pointer to the struct in its receiver. If you
> have a pointer to T, then taking a pointer to the dereferenced T is a
> no-op: you get the pointer of the thing you just dereferenced. Any
> statement &*whatever will always yield the value of whatever. Copy
> happens on assignment, and no assignment occurs in this statement.

I just realized that one additional thing that might be confusing is
that you're expecting CopyExplicitDeref to get a copy of _something_
since everything in Go is done by-value. Indeed, CopyExplicitDeref
does get a copy of something: the pointer to T. So if you change
CopyExplicitDeref to:

func (t *T) CopyExplicitDeref() **T {
return 
}

and in main:

a := {0}
b := a.CopyExplicitDeref()
fmt.Println( == b)

you will see that it's actually the pointer that is copied.

--dho

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: The &(*x) idiom for copying.

[Devon H. O'Dell]

CopyExplicitDeref gets a pointer to the struct in its receiver. If you
have a pointer to T, then taking a pointer to the dereferenced T is a
no-op: you get the pointer of the thing you just dereferenced. Any
statement &*whatever will always yield the value of whatever. Copy
happens on assignment, and no assignment occurs in this statement.

CopyImplicitDeref does not get a pointer to the struct in its
receiver; it gets a copy. The pointer returned will not be equal to
the original pointer as you've returned a pointer to the copy the
receiver got.

Your latest example dereferences t and copies it into a new variable.
Because an assignment happens between the dereference and address-of
operators, you've created a copy and thus a new object. You then
return a pointer to this copy.

All these behaviors are desired and expected.

--dho


2018-03-28 9:29 GMT-07:00 thwd :
> Even more surprising, make this small change to the previous playground link
> code:
>
> func (t *T) CopyExplicitDeref() *T {
> x := *t
> return 
> }
>
> Merely introducing a local variable changes the behavior of the method.
>
> On Wednesday, March 28, 2018 at 6:21:49 PM UTC+2, thwd wrote:
>>
>> https://play.golang.org/p/pjyoPX99Zr1
>>
>> Taking the address of an explicit dereference has different behavior than
>> implicitly dereferencing and taking address.
>>
>> Is this the desired behavior? It surprised me.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Would this race condition be considered a bug?

[Devon H. O'Dell]

2018-03-18 10:27 GMT-07:00 Jan Mercl <0xj...@gmail.com>:
> On Sun, Mar 18, 2018 at 6:21 PM Devon H. O'Dell <devon.od...@gmail.com>
> wrote:
>
>> * There is no way to force at least 16 byte alignment of data per the
> language spec, so there is no way to implement DCAS (notably missing
> from sync/atomic) on amd64 (cmpxchg16b requires 16 byte alignment of
> its target operand).
>
> https://golang.org/ref/spec#Size_and_alignment_guarantees

I don't think implementing things in terms of complex128 and using
unsafe.Pointer to convert e.g.

type stackForAlignment struct {
header complex128
}

to

type Stack struct {
head *node
gen   uint64
}

is anybody's idea of a good idea. But well received, I guess it is possible.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Would this race condition be considered a bug?

[Devon H. O'Dell]

2018-03-18 9:34 GMT-07:00 Michael Jones <michael.jo...@gmail.com>:
> Subtle observation--it is not fundamentally the memory model theorized in Go
> or C++ specifications that causes this--rather it is the least strong
> guarantee of the hardware on which these languages promise that standard
> programs will work as specified, and following that, the decision of
> language designers how efficiently they want to use the fastest of that
> hardware.

If I might reflect this differently: it's a combination of what
language designers want to target based on the hardware platforms they
intend to support. Is that understanding your point correctly?

> If one is designing a language that is more demanding than the execution
> hardware promises, that's fine. It just means that the generated/evaluated
> code must virtualize what appears to be simple things. For example, if we
> want multiple threads to be able to do "i++" against the same variable, we
> can generate an implicit mutex around every access to every "i" that could
> ever be visible to a concurrent thread. There is nothing wrong with this,
> but the resulting code will not generally be as fast as the code of other
> languages where the offer is less general, say, requiring developers to put
> locks around critical sections or use atomic increments when they know
> memory could be accessed concurrently.

Some (maybe less academic) examples of where Go does this:

 * There is no way to force at least 16 byte alignment of data per the
language spec, so there is no way to implement DCAS (notably missing
from sync/atomic) on amd64 (cmpxchg16b requires 16 byte alignment of
its target operand). DCAS is useful in lock-free protocols to avoid
ABA problems. These issues can be avoided in Go by copying, but this
usually inefficient (and maybe ends up holding locks anyway if you
need to allocate to copy)
.
 * The sync.atomic interface implements full memory barriers around
all operations. This is designed to obviate the need for understanding
memory barriers, but this is inefficient if all you need is (for
example) a load fence to guarantee loads observed in some order.

(I get that the applications for such things are "clever" and folks
are encouraged not to be "clever" in this way. But if one has a
relatively good understanding of this stuff, it's rather unfortunate.)

> As it happens, the "auto-magic" mode can be just as fast in some cases. On
> IBM POWER9+ there are special memory ranges where the synchronization is
> hardware facilitated. On the CPUs most people have, though, the code would
> be much slower and this would likely be seen as a flaw.

This is an excellent point. I wish that people would more often talk
about applications for these things (like lock-freedom) in terms of
system predictability (and therefore scalability) than performance
(and therefore speed). Scalability doesn't automatically imply
performance, but it sometimes results in improved performance, and is
sometimes more preferable (predictability is hugely important for
correct capacity modeling) even when it does not improve performance.

[1] provides another example of this kind of thing. Table 4 shows
uncontended latency for push/pop operations on a stack on x86-64 vs
POWER7. Spinlocks are easier to acquire on x86-64 than on POWER7, so
if writing for that platform, you might protect mostly-uncontended
access with spinlocks instead of using a lock-free stack. You'd come
to the opposite conclusion on POWER7. And so targeting either
platform, one might be likely to see the less performant approach as a
flaw. But at the end of the day, Figure 6 (just below) does a great
job at illustrating what you actually get from the progress guarantees
of lock-freedom, once you introduce contention.

(I'd really like the ability to do this in Go...)

--dho

[1] https://queue.acm.org/detail.cfm?id=2492433 "Nonblocking
Algorithms and Scalable Multicore Programming", Samy Al Bahra, 2013

> On Sun, Mar 18, 2018 at 2:30 AM, Devon H. O'Dell <devon.od...@gmail.com>
> wrote:
>>
>> 2018-03-18 1:42 GMT-07:00 David Anderson <d...@natulte.net>:
>> > There's a difference though. The program that uses sync/atomic will
>> > behave
>> > non-deterministically, but within a small set of possible outcomes. The
>> > one
>>
>> While a fair point, I'd also note the set of possible outcomes is not
>> small and my point is more that the term "race" doesn't just mean
>> "access that violates a particular language's memory model."
>>
>> > without could do anything. In addition to producing an incorrect numeric
>> > result, it could deadlock, segfault, jump execution to some random place
>> > in
>> > the program code, ...
>>
>> "Incorre

Re: [go-nuts] Re: Would this race condition be considered a bug?

[Devon H. O'Dell]

2018-03-18 1:42 GMT-07:00 David Anderson <d...@natulte.net>:
> There's a difference though. The program that uses sync/atomic will behave
> non-deterministically, but within a small set of possible outcomes. The one

While a fair point, I'd also note the set of possible outcomes is not
small and my point is more that the term "race" doesn't just mean
"access that violates a particular language's memory model."

> without could do anything. In addition to producing an incorrect numeric
> result, it could deadlock, segfault, jump execution to some random place in
> the program code, ...

"Incorrect" is subjective WRT the counters themselves; there is no
concurrent access to those. I agree that, per the Go specification,
the program is not guaranteed to halt without the use of sync/atomic;
I should've worded that better. But the non-determinism in the
behavior of the program is precisely because of racing. (The term
itself comes from EE where effectively the length of wire can cause
races in signals.)

> I highly recommend
> https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
> . It's an excellent article (by the author of the go race detector) that
> explains why safe, benign data races are never safe or benign. The examples
> are in C++, but the same applies to Go - the compiler makes assumptions
> about code execution in the absence of synchronization points, and even
> simple optimizations can have hilarious consequences on code that violates
> those assumptions.

It's a good article; I've read it several times before and should've
qualified things better. Some points related to the article:

 * Prior to C11, C had no concurrency model and as such, any
multithreaded program exhibits undefined behavior when compared to the
specification.
 * POSIX avoids a formal specification of a memory model entirely and
instead encourages what can effectively be summed up as "best
practices" for concurrent access, so it's no help either.
 * Both C and C++ have adopted memory models that are provably
inconsistent (though I seem to have lost links to the papers).

So while it's easy for this sort of thing to be "mis"-compiled, it's
still very much true that we're already relying on compiler behavior
(as opposed to some formal specification) in many languages.

While the languages themselves may leave this behavior undefined, it's
easy enough to link against things written in other languages in most
of them. So we can take my invalid program, link it with some
racy_stop.s that does (for example) MOVB $1, (DI), relying on whatever
ordering and visibility semantics the machine provides, and implement
a similar interface for reading that memory. At that point which point
language specification is moot, and my crappy version is guaranteed to
halt again.

Anyway, I had hoped to not get into the weeds here. Ignoring the
version that is not guaranteed to halt when specified in pure Go,
non-determinism in both other examples occurs due to racing on data.
"Any race is a bug" is not true, and that was really the only point I
was trying to make.

> - Dave

--dho

> On Sun, Mar 18, 2018 at 1:19 AM, Devon H. O'Dell <devon.od...@gmail.com>
> wrote:
>>
>> Perhaps better illustrated:
>>
>> package main
>>
>> import (
>> "fmt"
>> "time"
>> )
>>
>> type info struct {
>> count int
>> idint
>> }
>>
>> var stop int32
>>
>> func runner(id int, c chan info) {
>> info := info{id: id}
>> for stop == 0 {
>> info.count++
>> time.Sleep(1 * time.Microsecond)
>> }
>> c <- info
>> }
>>
>> func main() {
>> c := make(chan info)
>> for i := 0; i < 10; i++ {
>> go runner(i, c)
>> }
>>
>> time.Sleep(10 * time.Millisecond)
>> stop = 1
>>
>> for i := 0; i < 10; i++ {
>> fmt.Printf("Returned %+v\n", <-c)
>> }
>> }
>>
>> This program is guaranteed to halt, regardless of any GOMAXPROCS
>> setting. If you use the race detector on this program, it will
>> complain. You can "fix" "the race" by changing the loop condition to
>> "atomic.LoadInt32() == 0" and use atomic.StoreInt32(, 1) as
>> your signal. Is the program still racy? Yep. Is either version of this
>> program buggy? That just depends. It's not too hard to think of cases
>> where this behavior might be desirable; even lossy counters are a
>> thing.
>>
>> --dho
>>
>> 2018-03-18 0:

Re: [go-nuts] Re: Would this race condition be considered a bug?

[Devon H. O'Dell]

Perhaps better illustrated:

package main

import (
"fmt"
"time"
)

type info struct {
count int
idint
}

var stop int32

func runner(id int, c chan info) {
info := info{id: id}
for stop == 0 {
info.count++
time.Sleep(1 * time.Microsecond)
}
c <- info
}

func main() {
c := make(chan info)
for i := 0; i < 10; i++ {
go runner(i, c)
}

time.Sleep(10 * time.Millisecond)
stop = 1

for i := 0; i < 10; i++ {
fmt.Printf("Returned %+v\n", <-c)
}
}

This program is guaranteed to halt, regardless of any GOMAXPROCS
setting. If you use the race detector on this program, it will
complain. You can "fix" "the race" by changing the loop condition to
"atomic.LoadInt32() == 0" and use atomic.StoreInt32(, 1) as
your signal. Is the program still racy? Yep. Is either version of this
program buggy? That just depends. It's not too hard to think of cases
where this behavior might be desirable; even lossy counters are a
thing.

--dho

2018-03-18 0:43 GMT-07:00 Devon H. O'Dell <devon.od...@gmail.com>:
> 2018-03-16 2:16 GMT-07:00 Jérôme Champion <champ...@gmail.com>:
>> Any race is a bug. When there is a race, the compiler is free to do whatever
>> it wants.
>
> This is a false statement; whether or not a race constitutes a bug
> depends on the encapsulating protocol. Consider the following:
>
> package main
>
> import (
> "fmt"
> "sync"
> "sync/atomic"
> )
>
> var wg sync.WaitGroup
>
> type Spinlock struct {
> state uint32
> }
>
> func (s *Spinlock) Lock() {
> for atomic.SwapUint32(, 1) == 1 {
> }
> }
>
> func (s *Spinlock) Unlock() {
> atomic.StoreUint32(, 0)
> }
>
> func locker(s *Spinlock, i int) {
> defer wg.Done()
> s.Lock()
> fmt.Printf("Locker %d acquired lock\n", i)
> s.Unlock()
> }
>
> func main() {
> s := {}
> for i := 0; i < 10; i++ {
> wg.Add(1)
> go locker(s, i)
> }
> wg.Wait()
> }
>
> This program does not have deterministic output because each goroutine
> races to acquire the lock, and which gets it first relies on
> scheduling, which is effectively stoachastic. (Playground caches
> output, so no use illustrating it there.) Despite this, the program
> compiles just fine and behaves correctly for any reasonable definition
> of correct.
>
> I recognize this is not the issue mentioned in OP, but hopefully it's
> clear that this is not just a semantics argument (race doesn't just
> mean "only things the race detector can find") and that the statement
> "any race is a bug" is false. This race only constitutes a bug when
> the requirements of this program require deterministic output.
>
> --dho
>
>> What do you want to do? Do you want to println before or after
>> cancelContext? In your example, println could even be executed during
>> cancelContext!
>> Your example is not enough, such program :
>> https://play.golang.org/p/KJsYwu-ivjb would fix your problem, but I don't
>> think that's what you asked for :)
>>
>>
>> Le vendredi 16 mars 2018 03:11:49 UTC+1, Anmol Sethi a écrit :
>>>
>>> https://play.golang.org/p/82Um1jSntBo
>>>
>>> Please run with the race detector to see the race.
>>>
>>> This race condition arises because private variables are read via
>>> reflection by the fmt package. Is there any elegant way to avoid such a
>>> race?
>>>
>>> --
>>> Best,
>>> Anmol
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Experience report on coming to Go from a C perspective

[Devon H. O'Dell]

I like this, thanks for pointing me to it. I think that's a good way
to go for about half of it; if make / new can be considered as well,
that'd be great. I suppose adg's point still stands in terms of gofmt
output and a detailed proposal? I agree such a change should only be
considered for Go 2.

--dho

2018-02-22 15:37 GMT-08:00 'Bryan Mills' via golang-nuts
<golang-nuts@googlegroups.com>:
> You might be interested in https://golang.org/issue/12854 for maps, slices,
> and structs.
>
> (It wouldn't help with channels, because there is currently no such thing as
> a channel literal.)
>
> On Thursday, February 22, 2018 at 5:32:40 PM UTC-5, Devon H. O'Dell wrote:
>>
>> 2018-02-22 14:19 GMT-08:00 Caleb Spare <ces...@gmail.com>:
>> > I occasionally run into this, but in my experience it's exclusively with
>> > maps:
>>
>> I think all of the cases I ran into this were for maps and channels.
>>
>> > - Initializing empty slices is ~never necessary; (in my opinion the
>> > "members: []int{}" from the blog post is a code smell).
>>
>> Probably, but as mentioned in the post, this example was to illustrate
>> the point. It was also to keep consistency with the C example. This
>> would have been convoluted to do for maps or channels.
>>
>> > - Channels need initialization but that's where the important buffered
>> > vs.
>> > unbuffered choice is made, so I'm happy that's explicit. (I suppose you
>> > can
>> > say the same about initializing maps with a size, but that's fairly rare
>> > in
>> > my experience.)
>>
>> That's fair, but why require expressing the type in the call to make?
>> You already know the type of the channel; you should only need to
>> specify the buffer length (if any). Type inference is so pervasive
>> through the rest of the language; the compiler should absolutely be
>> able to help here.
>>
>> For example:
>>
>> return {c: make(chan T)}
>> return {c: make(chan T, 100)}
>> return {m: make()}
>> return {m: make(100)}
>>
>> and
>>
>> return {c: make()}
>> return {c: make(100)}
>> return {m: make()}
>> return {m: make(100)}
>>
>> I think the latter set of examples is much nicer. Yes, you don't see
>> _what_ make is allocating, but you knew that if you looked at the
>> specification for the type (which you probably already did). Other
>> idiomatic expressions already hide this information. For example, c :=
>> foo.c also doesn't tell you what the type of the assignment is. And
>> bar := foo is even more indirect.
>>
>> --dho
>>
>> > - If the struct has a nested pointer to another struct type, then having
>> > it
>> > be nil is definitely what you want since the nested type may or may not
>> > have
>> > a valid zero value.
>> >
>> > On Thu, Feb 22, 2018 at 12:02 PM, Devon H. O'Dell <devon...@gmail.com>
>> > wrote:
>> >>
>> >> Hi all,
>> >>
>> >> It's been some time since I really contributed much of anything to the
>> >> project (sorry!), but after 8 years, I'm finally writing Go outside of
>> >> the project itself (and outside of porting efforts). I was lamenting
>> >> to some coworkers about the lack of a comparable feature to C's
>> >> "malloc idiom" and they suggested I write an experience report on it.
>> >> I wrote the bulk of the article a month ago, but finally put in some
>> >> finishing touches and published.
>> >>
>> >> For whatever it's worth (probably not much):
>> >> https://9vx.org/post/a-malloc-idiom-in-go/
>> >>
>> >> --dho
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "golang-nuts" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> >> an
>> >> email to golang-nuts...@googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Experience report on coming to Go from a C perspective

[Devon H. O'Dell]

2018-02-22 14:19 GMT-08:00 Caleb Spare <cesp...@gmail.com>:
> I occasionally run into this, but in my experience it's exclusively with
> maps:

I think all of the cases I ran into this were for maps and channels.

> - Initializing empty slices is ~never necessary; (in my opinion the
> "members: []int{}" from the blog post is a code smell).

Probably, but as mentioned in the post, this example was to illustrate
the point. It was also to keep consistency with the C example. This
would have been convoluted to do for maps or channels.

> - Channels need initialization but that's where the important buffered vs.
> unbuffered choice is made, so I'm happy that's explicit. (I suppose you can
> say the same about initializing maps with a size, but that's fairly rare in
> my experience.)

That's fair, but why require expressing the type in the call to make?
You already know the type of the channel; you should only need to
specify the buffer length (if any). Type inference is so pervasive
through the rest of the language; the compiler should absolutely be
able to help here.

For example:

return {c: make(chan T)}
return {c: make(chan T, 100)}
return {m: make()}
return {m: make(100)}

and

return {c: make()}
return {c: make(100)}
return {m: make()}
return {m: make(100)}

I think the latter set of examples is much nicer. Yes, you don't see
_what_ make is allocating, but you knew that if you looked at the
specification for the type (which you probably already did). Other
idiomatic expressions already hide this information. For example, c :=
foo.c also doesn't tell you what the type of the assignment is. And
bar := foo is even more indirect.

--dho

> - If the struct has a nested pointer to another struct type, then having it
> be nil is definitely what you want since the nested type may or may not have
> a valid zero value.
>
> On Thu, Feb 22, 2018 at 12:02 PM, Devon H. O'Dell <devon.od...@gmail.com>
> wrote:
>>
>> Hi all,
>>
>> It's been some time since I really contributed much of anything to the
>> project (sorry!), but after 8 years, I'm finally writing Go outside of
>> the project itself (and outside of porting efforts). I was lamenting
>> to some coworkers about the lack of a comparable feature to C's
>> "malloc idiom" and they suggested I write an experience report on it.
>> I wrote the bulk of the article a month ago, but finally put in some
>> finishing touches and published.
>>
>> For whatever it's worth (probably not much):
>> https://9vx.org/post/a-malloc-idiom-in-go/
>>
>> --dho
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Experience report on coming to Go from a C perspective

[Devon H. O'Dell]

(Re-adding list in here with Bakul's okay.)

2018-02-22 12:27 GMT-08:00 Bakul Shah <ba...@bitblocks.com>:
> On Thu, 22 Feb 2018 12:02:35 -0800 "Devon H. O'Dell" <devon.od...@gmail.com> 
> wrote:
>> Hi all,
>>
>> It's been some time since I really contributed much of anything to the
>> project (sorry!), but after 8 years, I'm finally writing Go outside of
>> the project itself (and outside of porting efforts). I was lamenting
>> to some coworkers about the lack of a comparable feature to C's
>> "malloc idiom" and they suggested I write an experience report on it.
>> I wrote the bulk of the article a month ago, but finally put in some
>> finishing touches and published.
>
> I still run FreeBSD at home and test any go programs I write
> on it as well as MacOS (and sometime on plan9)! But FreeBSD
> support seems to be falling further and further behind.

I'm aware of some strange crash, but I don't have any FreeBSD systems
anymore to investigate. Are there other areas that FreeBSD is lagging
in? (Maybe tangent should go to separate thread :))

>> For whatever it's worth (probably not much):
>> https://9vx.org/post/a-malloc-idiom-in-go/
>
> I think you are suggesting inferring types of subobjects given
> that the parent type specified in a literal. So that you can
> type
>
> func NewSet(sz int)*Set {
> return {cap: sz, members:{}}
> }
>
> Or even
>
> func NewSet(sz int)*Set {
> return &{cap: sz, members:{}}
> }
>
> Or even
>
> func NewSet(sz int)*Set {
> return &{sz, {}}
> }
>
> I agree. Though some people won't like the extreme brevity of
> the last form!

Yes, this is exactly the idea. I like this empty-compound-literal
form, but I agree that the last form is probably asking for trouble.
For example, if the last two members of type Set are e.g. *Foo and
map[T]T', you probably meant to allocate the map. I think compound
literals for structs should retain field names. I definitely like the
second idea where you infer the type based on the return value (after
all, it _is_ known!).

--dho

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Experience report on coming to Go from a C perspective

[Devon H. O'Dell]

Thanks for pointing this out; I wasn't aware of this new syntax. I
think this gets to about halfway of what I'd hope for. That said, I'd
really like to avoid needing to jump through the hoop of indirection
for this. I occasionally find it hard to recognize what the type of a
thing is due to heavy use of type inference, so I'd hope for something
that doesn't require me to do additional backtracking to figure out
what the real underlying type is.

2018-02-22 12:31 GMT-08:00  <jake6...@gmail.com>:
> Have you considered using the new type alias feature introduced in 1.9? Line
> so:
>
> https://play.golang.org/p/6HK8qcuh9UU
>
> This would allow you to change the type of the map in a singe place (the
> type alias) and all the {} or make would adapt appropriately.
> Disclaimer: I am not espousing this style, just pointing out that it is
> possible.
>
>
>
> On Thursday, February 22, 2018 at 3:03:13 PM UTC-5, Devon H. O'Dell wrote:
>>
>> Hi all,
>>
>> It's been some time since I really contributed much of anything to the
>> project (sorry!), but after 8 years, I'm finally writing Go outside of
>> the project itself (and outside of porting efforts). I was lamenting
>> to some coworkers about the lack of a comparable feature to C's
>> "malloc idiom" and they suggested I write an experience report on it.
>> I wrote the bulk of the article a month ago, but finally put in some
>> finishing touches and published.
>>
>> For whatever it's worth (probably not much):
>> https://9vx.org/post/a-malloc-idiom-in-go/
>>
>> --dho
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Experience report on coming to Go from a C perspective

[Devon H. O'Dell]

Hi all,

It's been some time since I really contributed much of anything to the
project (sorry!), but after 8 years, I'm finally writing Go outside of
the project itself (and outside of porting efforts). I was lamenting
to some coworkers about the lack of a comparable feature to C's
"malloc idiom" and they suggested I write an experience report on it.
I wrote the bulk of the article a month ago, but finally put in some
finishing touches and published.

For whatever it's worth (probably not much):
https://9vx.org/post/a-malloc-idiom-in-go/

--dho

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Devon H. O'Dell]

2018-02-20 13:02 GMT-08:00 David Anderson :
> [snip]
>
> I also believe the tooling around vgo should encourage/make default this
> behavior for binary modules (and maybe for library modules as well, though
> that's less clear to me). The default behavior when writing Go programs
> should be to use the latest of everything, unless you have a specific reason
> to hold back. Among other things, that ensures you are patched against CVEs
> and whatnot.

AIUI, that is the opposite of what minimum version selection does;
active maintenance is required. This is at least no worse than any
other case where you need to make a determination on whether to
consume some library or tool based on whether it is actively
maintained. And when the tools are maintained, seems like it is
probably better.

--dho

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Devon H. O'Dell]

2018-02-20 11:39 GMT-08:00 Russ Cox <r...@golang.org>:
> On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell <devon.od...@gmail.com>
> wrote:
>>
>> With regards to minimum version selection, if I depend on some feature
>> present, this comes with two implicit dependencies: correctness and
>> safety. My knee-jerk reaction here is that the time folks spend
>> "telling the package manager, 'no, use at least Y,'" will largely be
>> in response to these sorts of issues. Maybe this is seen as "working
>> as intended;" my feeling is that this will become tiresome for
>> projects with large dependency chains. Might it be worthwhile (and
>> automatically possible) to pick the "maximally correct" minimum
>> version?
>
> I am not sure what you mean by "maximally correct".
> I think that's approximately what all the other package managers try to do.
> It makes things very complex (literally: research.swtch.com/version-sat).

Thanks. I was thinking in terms of e.g. a "correctness counter" that
could be published. If I depend on e.g. foo.Bar, I automatically want
the least buggy version of foo.Bar. If foo.Bar appears in version 1,
the amount of work required to find the least buggy version is
directly proportional to the number of releases in which foo.Bar
appears, and is annotated as having some correctness fix. I don't
think this would necessarily be as complex as you point out in the
referenced post: this is a property of an individual package. Of
course, you want the same property for each package's dependencies,
but that can be decided without conflict using the same methodology
for each package. That is, I don't think there's a case for package X
with a dependency on package Y where the minimum version can't be
decided in linear time.

Upon further thought, I realize folks will occasionally rely on
particular forms of buggy behavior. I think that's probably fine if
you can always require a specific version. However, it'd probably
require some kinds of annotations on every portion of the API to work
well: if I rely only on foo.Bar, and there's no dependency on foo.Bat
which has a correctness change, that shouldn't leak into foo.Bar. That
is probably an unreasonable amount of work to require from developers,
so probably just ignore me :)

--dho

> Note that it's still super easy to say "update this package to latest"
> or "update everything to latest". It just only happens when you ask,
> not when the tool feels like it.
>
> Russ
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go += Package Versioning

[Devon H. O'Dell]

With regards to minimum version selection, if I depend on some feature
present, this comes with two implicit dependencies: correctness and
safety. My knee-jerk reaction here is that the time folks spend
"telling the package manager, 'no, use at least Y,'" will largely be
in response to these sorts of issues. Maybe this is seen as "working
as intended;" my feeling is that this will become tiresome for
projects with large dependency chains. Might it be worthwhile (and
automatically possible) to pick the "maximally correct" minimum
version?

--dho

On Tue, Feb 20, 2018 at 9:20 AM Russ Cox  wrote:
>
> Hi everyone,
>
> I have a new blog post you might be interested in.
> https://research.swtch.com/vgo.
>
> I'll try to watch this thread to answer any questions.
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: [ANN] reverse proxy with ssl in less than 100 lines of code

[Devon H. O'Dell]

The best way (currently) to do http->https redirects is to use HSTS:

https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security

You may want to investigate HPKP as well.

—dho
On Fri, Dec 29, 2017 at 5:10 AM anmol via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> Here is a better way to redirect from HTTPS to HTTP:
> https://github.com/nhooyr/redirecthttp/blob/d87881e4fbfddb1614c9d0934ea97c4163903301/main.go#L10-L19
>
> Also, you don't need a secure variable in the handler, just check for
> r.TLS.
>
>
> On Thursday, December 28, 2017 at 12:10:16 PM UTC-5, jzs wrote:
>>
>> For those interested in hosting multiple golang apps on the same host
>> without using nginx/apache/[insert reverse proxy here].
>>
>> Shameless plug. I'm the author. But without
>> golang.org/x/crypto/acme/autocert it wouldn't have been possible.
>>
>>
>> https://journal.sketchground.dk/entry/2017-11-25-minimal-reverse-proxy-in-go
>>
>> Any feedback is more than welcome,
>>
>> Thanks,
>> Jens.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] 99c

[Devon H. O'Dell]

(I’d love to contribute, mostly “99c is a c99 compiler” made me wonder if
c99 was a requirement. Sorry for inbox spam.)
On Sat, Oct 7, 2017 at 7:45 PM Devon H. O'Dell <devon.od...@gmail.com>
wrote:

> More curiosity than anything, was just wondering whether patches to make
> it c11 (or c2x whenever that happens) would be accepted :)
> On Sat, Oct 7, 2017 at 6:42 PM Jan Mercl <0xj...@gmail.com> wrote:
>
>> Do you have any particular feature(s) of C11 on your mind? Because maybe
>> some of them are possibly easy to add.  Also, IIRC, one or two are already
>> there, just disabled by default, so adding support of the -std flag can
>> make them available.
>>
>> On Sun, Oct 8, 2017, 01:37 Devon H. O'Dell <devon.od...@gmail.com> wrote:
>>
>>> Is there any desire to support c11?
>>> On Sat, Oct 7, 2017 at 1:01 PM Jan Mercl <0xj...@gmail.com> wrote:
>>>
>>>> Command 99c is a c99 compiler targeting a virtual machine:
>>>> https://github.com/cznic/99c
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> -j
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>>
>> -j
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] 99c

[Devon H. O'Dell]

More curiosity than anything, was just wondering whether patches to make it
c11 (or c2x whenever that happens) would be accepted :)
On Sat, Oct 7, 2017 at 6:42 PM Jan Mercl <0xj...@gmail.com> wrote:

> Do you have any particular feature(s) of C11 on your mind? Because maybe
> some of them are possibly easy to add.  Also, IIRC, one or two are already
> there, just disabled by default, so adding support of the -std flag can
> make them available.
>
> On Sun, Oct 8, 2017, 01:37 Devon H. O'Dell <devon.od...@gmail.com> wrote:
>
>> Is there any desire to support c11?
>> On Sat, Oct 7, 2017 at 1:01 PM Jan Mercl <0xj...@gmail.com> wrote:
>>
>>> Command 99c is a c99 compiler targeting a virtual machine:
>>> https://github.com/cznic/99c
>>>
>>>
>>>
>>> --
>>>
>>> -j
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>
> -j
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] [ANN] 99c

[Devon H. O'Dell]

Is there any desire to support c11?
On Sat, Oct 7, 2017 at 1:01 PM Jan Mercl <0xj...@gmail.com> wrote:

> Command 99c is a c99 compiler targeting a virtual machine:
> https://github.com/cznic/99c
>
>
>
> --
>
> -j
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.