Re: [go-nuts] Re: Give the zero time value a location, and it won't survive a roundtrip to JSON?

[Michael Jones]

Matt’s formatting example suggests interval arithmetic for time. That is a
nice idea.

On Sat, Nov 28, 2020 at 2:22 AM 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> It would also be possible to implement `json.Marshaler` to use a different
> time format. In particular, it might be reasonable to encode the zero
> value of time.Time as `null`, instead of a string (though mixed types in
> json messages are… icky).
>
> Personally, I'm always very cautious about encoding and decoding times.
> There isn't really a standard way to transmit timezone information,
> especially as timezone definitions can even change over time, AIUI. So,
> even if you specify which timezone a point in time should be in, you can't
> rely on the receiver of the message having the same understanding of that
> timezone.
>
> Using UTC for storage and transmission is probably a better option. You
> essentially treat the stored data purely as "a point in time", while
> timezone info is specific to the presentation. A time.Time isn't just a
> point in time, though, so if you really marshal "a time.Time", you have to
> somehow include timezone info - so I kinda understand why the default
> MarshalJSON method doesn't convert it to UTC first.
>
> On Sat, Nov 28, 2020 at 10:35 AM Matt Harden 
> wrote:
>
>>
>>
>> On Fri, Nov 27, 2020 at 4:14 PM 'Robert Ma' via golang-nuts <
>> golang-nuts@googlegroups.com> wrote:
>>
>>> Is this because the 2-second offset of LMT gets lost in RFC 3339
>>> representation?
>>>
>>
>> Yes, that's exactly it.
>>
>>
>>> On Fri, Nov 27, 2020 at 6:33 PM Robert Ma  wrote:
>>>
 Hi all,

 Time is complicated.

>>>
>> Wibbly-wobbly, even.
>>
>> Today I found myself in a rabbit hole. For some unrelated reason, I got a
 time value in a non-UTC location that's otherwise zero (IsZero=true). This
 value doesn't seem to survive a roundtrip to JSON. See this playground for
 a minimal reproduction: https://play.golang.org/p/QdglfKYkstS

 Is this expected, a bug, or an undefined behaviour? I checked RFC 3339,
 which time uses as the JSON serialization format, and it seems to support
 AD to AD, but I have to admit I know little about time.

>>>
>> RFC 3339 doesn't support sub-minute timezone offsets, so it's not
>> possible to format the LMT zone precisely.
>>
>> I am concerned that the time printed is incorrect by 2 seconds in this
>> case, and (I imagine) could be anywhere from 0 to 59 seconds off depending
>> on the particular sub-minute timezone offset used. That seems like a real
>> bug, and I don't know what a proper fix would look like. Perhaps when the
>> timezone is formatted in numeric form, the printed time should be adjusted
>> to account for the loss of precision in the zone info. In your case this
>> would print "-12-31T19:04:00-04:56" instead
>> of "-12-31T19:03:58-04:56". That solution has its own issues though.
>>
>> You probably should only use IsZero() to detect uninitialized time
>> values; never on a value that's been parsed from any source, even JSON.
>>
>> To get arbitrary time values to survive a JSON round trip, I think using
>> UTC exclusively would be the best option.
>>
>>
 Cheers,
 Robert

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/CAOPAaN%2BWvZ73Z-oMVaGmDt-Gr5DEk7ZtQeU43x5fKrCFW42%3DqQ%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CALRNMm0VN8030%2BQLY0sLr%2BuHfz4WoG7NfWe1RUHj-d2Zqh393Q%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfFx13J%3DiSx7y_hPKNFd8WPO9gVSUz9H_PKsNKjfgnNMPA%40mail.gmail.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 

Re: [go-nuts] Evaluation order of return

[Michael Jones]

Another way to understand the general topic is by comparison to Lindenmayer
systems. The compiler is an iterated rewrite framework and the exact code
that you get can be a surprise the same way L-System graphics vary.

On Thu, Sep 24, 2020 at 10:36 PM cs.ali...@gmail.com <
cs.alikoyu...@gmail.com> wrote:

> I understood perfectly now, thanks for the explanations and the link! I
> really appreciate you guys!
>
> On Thursday, September 24, 2020 at 3:28:10 AM UTC+3 Ian Lance Taylor wrote:
>
>> On Wed, Sep 23, 2020 at 1:10 AM cs.ali...@gmail.com
>>
>>
>>  wrote:
>>
>>
>> >
>>
>>
>> > I am not actually questioning the current design, as you both said it
>> is not a good practice to call a return statement as I wrote above, I am
>> trying to understand the relation between memory, interface and order of
>> evaluation. It is clear that the compiler takes account of whether a return
>> statement is an interface or a struct and the memory size of the returned
>> value, If I return a struct rather than an interface, it changes the order,
>> If I add fields to the structs it changes the order. Is there a paper that
>> I can find why the compiler considers them for ordering, why it is
>> important for performance or anything else?
>>
>>
>>
>>
>>
>> I'm not aware of any paper specific to the Go compiler.
>>
>>
>>
>>
>>
>> I find it most useful to consider a compiler as creating a set of
>>
>>
>> constraints derived from the input based on the language semantics.
>>
>>
>> These are constraints like in "a = b; b = c" the read of b in the
>>
>>
>> first assignment must be completed before the store to b in the second
>>
>>
>> assignment. Once the set of constraints is created, the compiler must
>>
>>
>> solve those constraints given an instruction architecture including a
>>
>>
>> set of registers. The goal is to optimize execution time while
>>
>>
>> minimizing compilation time without violating any constraints.
>>
>>
>> Because compilation time matters, compilers do not fully analyze all
>>
>>
>> possible solutions; instead, when it comes to things like memory
>>
>>
>> load/store order, instruction selection, and register allocation, they
>>
>>
>> are full of heuristics that tend to give good results in practice.
>>
>>
>>
>>
>>
>> When you view a compiler in that way, a question like "why does adding
>>
>>
>> fields to a struct change the order of memory loads and stores"
>>
>>
>> becomes uninteresting. The reason has to do with the details of all
>>
>>
>> the constraints that applied while compiling that particular package.
>>
>>
>> There is no rule that says "if the struct has more fields, do this."
>>
>>
>> It's just that the set of heuristics happened to produce a particular
>>
>>
>> result. Changing some other piece of code in some other part of the
>>
>>
>> package might produce a different result. Or a different version of
>>
>>
>> the compiler might apply different heuristics and get different
>>
>>
>> results.
>>
>>
>>
>>
>>
>> Ian
>>
>>
>>
>
>
>
>
>
>
>
> --
>
>
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
>
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
>
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/b2aa2ac3-a7da-4415-9534-7076551da5e7n%40googlegroups.com
> 
> .
>
>
> --

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxuSfsGaiExNCdvHZcYkZn9qKZxD3i_yhzct4PFoz9U-Q%40mail.gmail.com.

Re: [go-nuts] flag: bug with shared values?

[Michael Jones]

> 2. if other gophers has ideas of workaround?


Use distinct values, but never access them directly.

Write setter/getter functions that do the value sharing as you prefer and
return the desired result.

On Fri, Sep 18, 2020 at 11:30 AM Ian Lance Taylor  wrote:

> On Fri, Sep 18, 2020 at 12:50 AM Manfred Touron  wrote:
> >
> > 1. if I need to open an issue or not on the go repo, if the fix is not
> about code, it can be about doc maybe?
>
> Documentation fixes are always welcome.  You can open an issue or just
> send a change.
>
> > 2. if other gophers has ideas of workaround?
>
> I don't understand why it is important for these two different flags
> to share the same variable.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU9cMZnf4PaozoBEa4kL7DJgK1PDmnkibDmQ%2BLhbgm%3DJw%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwNNNfMuNQxKHbD3Lo%3Db85Sn8HrNd3YK%3DqMiZGNq8wuMw%40mail.gmail.com.

Re: [go-nuts] How to find repeated string?

[Michael Jones]

Maybe this way?
https://play.golang.org/p/_OM4RvO83Cf

On Wed, Aug 19, 2020 at 4:04 AM Ian Davis  wrote:

> On Wed, 19 Aug 2020, at 12:02 PM, Ian Davis wrote:
>
>
>
> On Wed, 19 Aug 2020, at 11:54 AM, Brian Candler wrote:
>
> Or another thought - perhaps you are looking for *adjacent* repeats only?
>
> In that case strings.Count doesn't help because it counts all occurrences,
> not just adjacent ones.
>
>
> I suspect the OP is looking for the longest repeating substring.
>
>
> Hmm, that's not correct. Sorry for the noise.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/bbc89836-e955-45ed-aaa5-f06669128281%40www.fastmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwHU3trJrnLUFCqo7%2Bq0q8_K1EDsqv3oJ6-nfOP151kFQ%40mail.gmail.com.

Re: [go-nuts] [ generics ] Added constraint type inference to the design draft

[Michael Jones]

Yes, thank you. In intellectual shame I must say that I somehow understood
that the generics needed to be "leaf" functions. Thank you for
demonstrating my oversight. Happier now.

Michael

On Thu, Aug 13, 2020 at 3:47 PM Bakul Shah  wrote:

> On Aug 13, 2020, at 3:29 PM, Michael Jones 
> wrote:
> >
> > The all-or-none aspect would be sidestepped if it were allowed to define
> "partial generics":
> >
> > func A (T1, T2) (...){}
> >
> > func B(T)(...){ A(T,int)(...){...} }
> >
> > Allowing B to exist just means running the whole generic thing
> iteratively until no resolution is changed. If the fixed point still has
> generic types then the compilation is a failure, otherwise, a success.
>
> Do you mean something like this?
> https://go2goplay.golang.org/p/4I4y-dLC-Yp
>
> >
> > Seems useful to me. A generic balanced tree infrastructure could be
> "meta-instantiated" as a LLRB instance that still allows generic leaf types.
>
>

-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxTgQdhS2x4MU%3D_FcwBAJ09D68XnNYOQxy5YV7%2B6QOi0w%40mail.gmail.com.

Re: [go-nuts] [ generics ] Added constraint type inference to the design draft

[Michael Jones]

The all-or-none aspect would be sidestepped if it were allowed to define
"partial generics":

func A (T1, T2) (...){}

func B(T)(...){ A(T,int)(...){...} }

Allowing B to exist just means running the whole generic thing iteratively
until no resolution is changed. If the fixed point still has generic types
then the compilation is a failure, otherwise, a success.

Seems useful to me. A generic balanced tree infrastructure could be
"meta-instantiated" as a LLRB instance that still allows generic leaf types.


On Thu, Aug 13, 2020 at 3:09 PM roger peppe  wrote:

> That's interesting; thanks for the heads up.
>
> My initial reaction is that this perhaps feels a bit "clever", but perhaps
> this feeling will go away in time.
>
> Constraint type inference is useful when a function wants to have a type
>> name for an element of some other type parameter, or when a function wants
>> to apply a constraint to a type that is based on some other type parameter.
>
>
> I found this initial sentence describing the motivation behind the feature
> a bit abstract and hence hard to understand:
>
> Perhaps a tiny example might help at that point to orient the reader in
> the right direction?
>
>  Constraint type inference can only infer types if some type parameter has
>> a constraint that has a type list with exactly one type in it.
>
>
> When I first read the above, my first thought was: but it's not very
> useful then because when would you have a type list with only one type in?
> Of course, it becomes clear later that you'd define this kind of
> constraint precisely to use this feature, but perhaps the phrasing could be
> changed a little to make that more obvious?
>
> Overall, I like the fact that this feature uses the existing semantics of
> type lists and only extends the type-inference algorithm.
>
> I do feel that the "all or nothing" nature of type parameter lists (if you
> specify one type parameter, you must explicitly specify all of the others
> too, even if they could otherwise be inferred) is likely to get in the way
> here. I'd like to see a way to specify some type parameters and not others,
> so that constraint type inference can work even when, for example, there's
> a generic return type parameter that can't be inferred from the arguments.
> We could potentially use an underscore for that.
>
> So, for example:
>
>var f = DoubleDefined[MySlice, _]
>
> would be valid. f would be a function value of type func(MySlice) MySlice.
>
>   cheers,
> rog.
>
>
>
>
> On Thu, 13 Aug 2020 at 03:31, Ian Lance Taylor  wrote:
>
>> I just added a new section to the generics design draft describing
>> constraint type inference.  This is a generalization and
>> simplification of the earlier pointer method approach, which has now
>> been removed.  I would be happy to hear any comments.  Thanks.
>>
>>
>> https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md#constraint-type-inference
>>
>> Ian
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcX6AcGUt4e6JTMrxXkAtMRq%2Bo6zSVnjqchEroheYBP%2BBw%40mail.gmail.com
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAJhgaciL%3D0p3rwMOmXW3oxNFhKVWFa6bT6-SsBi-E5iaeFxYqQ%40mail.gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxJM0%2Bi9ri9FMV0zQ2L6R9zvH5eHHbXEQYci5XW6dzwFg%40mail.gmail.com.

Re: [go-nuts] map without default values

[Michael Jones]

Joe, your question is perfectly answered by Axel.

I'll just share a few (personal) Go "style" comments:

Go likes you to test explicitly for failure where that is possible: did the
open fail, did the pipe break, etc.Multiple return values make this clear
by avoiding the need for a "reserved" error value of the normal-case return.

Go likes untested actions to work. In your case adding an existing key to a
map replaces the old entry, accessing a missing entry returns the default
value. ["no surprises"]

Go likes you to combine these approaches to make your own more elaborate
behaviors using the "comma OK" approach that Axel shared. In your case,
adding, and deleting could complain if there is already a matching entry,
or not a matching entry, or -- and this is the real reason -- by looking at
the payload of a new or matching entry to use application logic to decide
if that's ok or not. Only you can know so Go won't second guess you, and
the test-rather-than-panic style is because testing right there is deemed
the right way.


On Thu, Aug 13, 2020 at 11:42 AM 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> No, there isn't, but you can check if the value exists:
>
> x, ok := m[k]
> if !ok {
> panic("does not exist")
> }
>
> You can also wrap that with methods, if you want to avoid the extra check.
>
> I disagree with you that panicking on a non-existent key is better -
> either in general, or in the majority of cases. Personally, I don't think I
> encountered many use-cases where the zero-value wasn't the perfect thing to
> use. I'm not saying your use-case doesn't exist or even that it's rare,
> just that I don't think you can generalize from your experience here.
>
> On Thu, Aug 13, 2020 at 8:36 PM Joe Marty 
> wrote:
>
>> I'm very new to Go - apologies in advance if I'm missing something:
>>
>> I find it frustrating that there's no way to create a map that does *not*
>> automatically return a zero value for undefined key access by default.
>>
>> I love the fact that Go doesn't return "nil" for this use case (I love
>> Ruby, but I don't think they got that quite right), but returning a default
>> value is worse!  It would make so much more sense if it would just panic
>> when accessing an undefined key.  I'm not a Python guy, but I think this is
>> something Python got right.
>>
>> Creating a map that returns some default value when you access an
>> undefined key should be a special kind of map, or a special argument to
>> initializing the map (which Ruby does allow).  In Go, is there even any way
>> to create a map that panics when you access an undefined key?
>>
>> -Joe
>>
>> *Connect with us here:* LinkedIn
>> , Facebook
>> , Instagram
>> , Twitter
>> , BuiltInAustin
>> 
>>
>> [image: https://www.smartersorting.com/]
>> 
>> 
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/f53377a0-ddc1-4842-ac7a-a796d85b7077n%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEc5MO-FS1esLDJGoKRUq0tneJ5v6GR5T8JQMDFgjx9pQ%40mail.gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwSDQ78K%3DVD6jjH7izCj6%3DTtfCWObFRioxsxz22Zx0ZtQ%40mail.gmail.com.

Re: [go-nuts] Newbie question about type-casts

[Michael Jones]

var cash, change, bet int64
:
bet = cash/4

// change is 0,1,2,3 meaning 0/4, 1/4, 2/4, 3/4 dineros or zlottys or
whatever
change = cash - 4*bet // or, change = cash%4

// choose one of these
if change>0{
  bet++ // a: round up means 1/4 and 2/4 and 3/4 go to 1
}
// ...or...
if change>=2{
  bet++ // b: round up means 2/4 and 3/4 go to 1
}

cash = cash - bet

once you understand the above compare to these versions:

bet = (cash+3)/4 // this is Kurtis's advice, mode (a)
cash = cash - bet
...or...
bet = (cash+2)/4 // this is mode (b)
cash = cash - bet

On Tue, Aug 4, 2020 at 3:37 AM Martin Møller Skarbiniks Pedersen <
traxpla...@gmail.com> wrote:

> On Sun, Aug 2, 2020 at 2:09 PM Martin Møller Skarbiniks Pedersen <
>> traxp...@gmail.com> wrote:
>>
>>> I have written my first little piece of Go-code.
>>> However it took some time and code for me to divide a int64 by 4 and
>>> round down.
>>>
>>> [...]
>>> var bet int64
>>> bet = int64(math.Ceil(float64(cash)/float64(4)))
>>>
>>>
> On Sunday, 2 August 2020 23:25:23 UTC+2, Kurtis Rader wrote:
>>
>> In addition to Jake's question regarding whether you want to round up or
>> down I'll point out there isn't any to cast to float64 and back to int64.
>> If you want to "round down" just divide by four. If you want to "round up"
>> add one less than the divisor before dividing; e.g., bet := (cash + 3) /
>> 4. Notice the ":=" which avoids the need for the "var bet int64" statement.
>>
>
>
> OK. Good idea. But what is I need to integer divide with a variable and
> not the number 4?
>
> Cheers
> Martin
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/025665ff-21eb-4139-93c6-57f1adfc5cc9o%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQy2mSaMnyrAjJvH3QQXONjSgOnRLKodN8moFord36U5iw%40mail.gmail.com.

Re: [go-nuts] Generic symbol metaproposal

[Michael Jones]

I'm never sure how much detail to share. The colorForth idea is a perfect
example of the thought about "another dimension" of specification. (Albeit
difficult to implement in practice). The upper/lower case signal is a good
one, and easy; it is limiting, but not terribly, and offers easy
frictionless specification.

The earlier comment about ";" being everywhere and nowhere in Go is
important. It leads to the question "what can we intuit?" That is a fine
starting place. For example, if the generic specification requires "()" or
"[]" or whatever around type-concretization lists, but the parser will add
them for you, then you'll never see either except in "pedantic" machine
generated code or the rareish case of a generic type list that needs
another generic type list as an element (the inner ones need delimiters).

I vote for this whatever symbol is chosen.

On Fri, Jul 24, 2020 at 3:16 PM David Riley  wrote:

> On Jul 24, 2020, at 5:50 PM, Ian Lance Taylor  wrote:
> >
> > On Fri, Jul 24, 2020 at 2:22 PM  wrote:
> > >
> > > On 7/23/20, Michael Jones  wrote:
> > > ...
> > > > Another delight is the uppercase signal for external.
> > > >
> > > > Maybe the “how to signal it” aspect of type instantiation could look
> to
> > > > these approaches as well—make the automatic understanding so magical
> that
> > > > the complete specification is unnecessary in most all cases, or the
> > > > signaling so intrinsic to the variables that no identification
> Symbol or
> > > > BracketedPair is necessary.
> > >
> > > Maybe braille?
> > > ⠓⠑⠇⠇⠕⠺⠕⠗⠇⠙
> > > https://www.royalblind.org/sites/www.royalblind.org/files/alphavet.PNG
> > >
> > > ⠊⠞ ⠊⠎ ⠑⠁⠎⠽
> >
> > I've always felt that color is underused in modern programming languages.
> >
> > func Max(T)(s []T) T
>
> Perhaps there could be some cross-pollination with ColorForth, wherein
> color actually is semantically meaningful.
>
> https://colorforth.github.io
>
>
> - Dave



-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQz%3D5WA0K8OV5q_ONpxESzP7wEtkBZtF32AMT08bvFR0DQ%40mail.gmail.com.

[go-nuts] Generic symbol metaproposal

[Michael Jones]

One interesting fact of go is that semicolons are required at the end of
statements. A fact forgotten perhaps because of the automatic ‘we’ll insert
one for you’ process. This duality, required but auto-supplied in nearly
every case is a delightful outcome.

Another delight is the uppercase signal for external.

Maybe the “how to signal it” aspect of type instantiation could look to
these approaches as well—make the automatic understanding so magical that
the complete specification is unnecessary in most all cases, or the
signaling so intrinsic to the variables that no identification Symbol or
BracketedPair is necessary.

Do I have a perfect example? No. Imperfect, sure.

Choice a: leading upper case means an exported symbol, all upper case means
generic type. (Not Go 1 promise consistent) no special syntax/keyword
needed!

Choice b: struct name a=int, b=MyType {...}
No parse issue about ‘struct name’ followed by an identifier, right?

Choice c: leading and trailing underscore on generic placeholder
identifiers. Func MySqrt(x _t_)

These Examples are not deep thoughts. But the notion of no keyword or funky
symbol In the 99.99% of cases is a deep thought.

Michael
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwX5S70Zf5dwRCjaSO3Pppm2WpbD%3Dsrp9VLhD5UTO7srQ%40mail.gmail.com.

Re: [go-nuts] Generics and parentheses

[Michael Jones]

So it seems! nothing substantive on the hard part.

On Wed, Jul 22, 2020 at 5:02 PM Russ Cox  wrote:

> So it sounds like everyone is in favor of the entire generics proposal and
> all the semantics, and all we have left to hammer out is the bracket
> characters? Do I have that right?
>
> Best,
> Russ
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAA8EjDRrEBeOgs2Yax%2BgtHp-oJqCApjGJJEsop-S1OzOJbU3xA%40mail.gmail.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyUQPGYKGLfVvdHLrvyPq4X2Vc5RLZcWSv6Kqm8hghZ6Q%40mail.gmail.com.

Re: [go-nuts] Pointers to allocated memory objects?

[Michael Jones]

interior.

https://blog.golang.org/ismmkeynote

On Fri, Jul 17, 2020 at 7:57 PM joe mcguckin 
wrote:

> Do pointers have to point to the beginning of a heap object or can they
> point to the interior of the object?
>
> Thanks,
>
> Joe
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/67380fb0-4677-471a-8fce-fb7073ff0a07o%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxy3haWhRVovVyEYeGqMRUk6bvex89sz-jBk543VQ8p%3DA%40mail.gmail.com.

Re: [go-nuts] the go archive size must be smaller？

[Michael Jones]

What are you asking for?

1. GZip to compress more completely?
2. Use of different compression tools? (Zstd?)
3. Go to output binaries that are more compression friendly?
4. The Go binaries to just be smaller?
:

If you are specific, you will get specific answers.

On Fri, Jul 17, 2020 at 2:33 PM yangw...@gmail.com 
wrote:

> This issue is only the binary file, I said is go archive size .
> Such as go1.14.6.darwin-amd64.tar.gz
>  or
> go1.14.5.darwin-amd64.tar.gz
> 's size etc.
>
> 在2020年7月17日星期五 UTC+8 下午5:38:04 写道：
>
>> Note that progress for reducing binary size is tracked under this issue:
>> https://github.com/golang/go/issues/6853
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/dd1edcfe-698b-4005-be0e-56083c35878fn%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQz9dn3PCDXNBwGMgr3XZfd91C5i%2BEU1B8LzWUB_GXr%3D6g%40mail.gmail.com.

Re: [go-nuts] Re: Generics and parentheses

[Michael Jones]

Jon, this is a special case where a "tr '«»' '[]'' is enough.

On Fri, Jul 17, 2020 at 8:56 AM Jon Conradt  wrote:

> In the spirit of “show me, don’t tell me” and experience reports. How hard
> would it be to release a beta which supports both [] and guillamets? We try
> them. We see where we have compatibility problems. We give editor writers
> and plugin writers a chance to simplify they keystrokes?
>
> Jon
>
> On Tuesday, July 14, 2020 at 11:37:21 PM UTC-4 Ian Lance Taylor wrote:
>
>> On Tue, Jul 14, 2020 at 7:45 PM Watson Ladd  wrote:
>> >
>> > Guillamets are worth consideration. They are common on European
>> keyboards and avoid all the syntax ambiguities.
>>
>>
>> https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md#why-not-use
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/af5929bc-2e49-48dc-a1bc-3a9a7ef63051n%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxooXkN0REjKy92h-4mhbirwfD4KKkZKg0Fas8NS5wWsw%40mail.gmail.com.

Re: [go-nuts] Generics and parentheses

[Michael Jones]

nice. "gen" here is akin to the existing forward declaration of recursive
inner functions. it says, "you are about to see something special and you
need to know *this* about it."

On Tue, Jul 14, 2020 at 11:06 PM Bakul Shah  wrote:

> I don't much like square brackets or angle brackets or guillemets. But
> here is a different way
> of *reducing* the need for parentheses at least for the common case:
>
> A proposal for fewer irritating parentheses!
>
> One thing to note is that generic functions & types are *different* from
> existing things like const, func, type, var. As such they should have their
> own declaration marker. For example
>
> gen T   type pair struct { a, b T } // contrast with type pair(type T) ...
> gen T,U type W struct { a T; b U } // contrast with type W(type T, U) ...
> gen T   func Print(s []T) {...} // print a slice of T
>
> These function/type/method generics are used by *prepending* the type:
>
> var x int pair // a pair of ints
> var y (int, int pair) W // here we have to use parentheses
> int Print([]int{1,2,3}) // print a slice of ints
> qq := int pair pair{{1,2},{3,4}} // a pair of a pair of ints
> ww := (int, int) W pair{{1,2},{3,4}}
>
> This use may seem weird if you are used to C/C++. I find it more readable
> than having to deal with extra parentheses. "int pair" clearly says a
> pair of ints and so on. What is more, if in future types are allowed to be
> *inferred* for generic function calls, you can simply drop the type prefix.
>
> If there is still a parsing ambiguity, I'd suggest adding a - as in
> int-pair.
>
> Additional type syntax rule:
>
> type: ... | type generic-type| (type-list) generic-type
>
> or
>
> type: ... | type "-" generic-type | (type-list) "-" generic-type
>
> FWIW I thought of this four weeks ago (June 16).
>
> On Jul 14, 2020, at 3:29 PM, 'gri' via golang-nuts <
> golang-nuts@googlegroups.com> wrote:
>
> Correction: The last paragraph in the post below should have said: "In Go,
> type information is not available at *parse* time". (Of course, type
> information is available at compile time.)
>
> On Tuesday, July 14, 2020 at 2:56:01 PM UTC-7 gri wrote:
>
>> We have received a variety of feedback on the generics draft design
>> 
>> (blog ). Thanks to everyone
>> who took the time to read it, play with generics code in the playground
>> , file issues, and send us their thoughts.
>>
>> Not unexpectedly, many people raised concerns about the syntax,
>> specifically the choice of parentheses for type parameter declarations and
>> generic type and function instantiations.
>>
>> A typical computer keyboard provides four easily accessible pairs of
>> single-character symmetrical "brackets": parentheses ( and ), square
>> brackets [ and ], curly braces { and }, and angle brackets < and >. Go uses
>> curly braces to delineate code blocks, composite literals, and some
>> composite types, making it virtually impossible to use them for generics
>> without severe syntactic problems. Angle brackets require unbounded parser
>> look-ahead or type information in certain situations (see the end of this
>> e-mail for an example). This leaves us with parentheses and square
>> brackets. Unadorned square brackets cause ambiguities in type declarations
>> of arrays and slices, and to a lesser extent when parsing index
>> expressions. Thus, early on in the design, we settled on parentheses as
>> they seemed to provide a Go-like feel and appeared to have the fewest
>> problems.
>>
>> As it turned out, to make parentheses work well and for
>> backward-compatibility, we had to introduce the type keyword in type
>> parameter lists. Eventually, we found additional parsing ambiguities in
>> parameter lists, composite literals, and embedded types which required more
>> parentheses to resolve them. Still, we decided to proceed with parentheses
>> in order to focus on the bigger design issues.
>>
>> The time has come to revisit this early decision. If square brackets
>> alone are used to declare type parameters, the array declaration
>>
>> type A [N]E
>>
>> cannot be distinguished from the generic type declaration
>>
>> type A[N] E
>>
>> But if we are comfortable with the extra type keyword, the ambiguity
>> disappears:
>>
>> type A[type N] E
>>
>> (When we originally dismissed square brackets, the type keyword was not
>> yet on the table.)
>>
>> Furthermore, the ambiguities that arise with parentheses appear not to
>> arise with square brackets. Here are some examples where extra parentheses
>> are not needed with square brackets:
>>
>> using () using []
>> func f((T(int))  func f(T[int])
>> struct{ (T(int)) }   struct{ T[int] }
>> interface{ (T(int)) }interface{ T[int] }
>> [](T(int)){} []T[int]{}
>>
>> To test this better understanding, and to get a feel for this alternative
>> 

Re: [go-nuts] pure function on golang

[Michael Jones]

The standard math library is a natural, easy gateway; a simple preprocessor
could do it. maybe i'll prototype  it.

On Tue, Jul 7, 2020 at 4:43 AM Brian Candler  wrote:

> On Tuesday, 7 July 2020 01:46:48 UTC+1, Kurniagusta Dwinto wrote:
>>
>> > i would value "pure" if it were a contract for early evaluation
>> does this "early evaluation" concern about IO? like loading blob data
>> with ioutil.ReadFile into global variable at compile time?
>>
>
> I think by definition, any function which accesses the filesystem is not
> pure, since it depends on external state (the state of the filesystem).
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/94ca32bc-bc07-4ca5-93ef-57a4d7327387o%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxpwpbJE1SZ4hV_7EKTGBEaH%3DFHLs4xKQmVft4ddxuBdA%40mail.gmail.com.

Re: [go-nuts] pure function on golang

[Michael Jones]

i would value "pure" if it were a contract for early evaluation. in my post
on this from 2018 (linked above), the reasoning was so that "x :=
math.Sin(0.23)" would be a compile-time event.

On Mon, Jul 6, 2020 at 5:11 PM Kurniagusta Dwinto 
wrote:

> > It's not obvious to me that "pure" is a characteristic that is important
> enough
> > to be singled out and added to the language
>
> the name "pure" may be debatable, but the characteristic is the same with
> "constexpr" in C++, although I also don't have a strong reason why this is
> important beside separation IO and non-IO
>
> On Tue, Jul 7, 2020 at 7:05 AM Kurniagusta Dwinto 
> wrote:
>
>> Additionally, this feature complement new generic feature,
>> this feature will help anyone that trying to use functional programming
>> pattern (like monad pattern) in their code
>>
>> On Tuesday, July 7, 2020 at 6:52:31 AM UTC+7 Kurniagusta Dwinto wrote:
>>
>>> Adding pure marker will give information to the programmer that the
>>> function will not do any side effect, the compiler just gives compile error
>>> when the programmer disagrees about the contract, like doing IO operation
>>> on pure function.
>>> So in the end, this feature focuses on helping the programmer, not the
>>> compiler, to make sure the function does not do any io operation inside it.
>>> I like how Haskell separate IO and non-IO function, they create a clear
>>> separation between those worlds,
>>>
>>> On the other side, the compiler can evaluate some function in
>>> compile-time, although this feature maybe not really needed yet, this will
>>> help the programmer to create pre-computed value instead of copying some
>>> magic blob data,
>>>
>>>
>>> > I agree that adding the keyword would let the compiler enforce it, but
>>> > that doesn't seem all that big a benefit to me. It also seems like
>>> > something that could be done by an analysis tool rather than requiring
>>> > a change to the language.
>>>
>>> That wouldn't work with interfaces, like
>>>
>>> purefunc Hai(x interface{}) int {
>>>   val := 42
>>>   if x, ok := x.(interface { pure Value() int }); ok {
>>> val += x.Value()
>>>   }
>>>   return val
>>> }
>>>
>>> here, without knowing the implementation, the caller of Hai know that
>>> Hai will not do any IO operation at all.
>>>
>>> I've tried to create an analysis tool to do that before. I need to mark
>>> the pure function with "Pure" suffix,
>>> the code above will be
>>>
>>> func HaiPure(x interface{}) int {
>>>   val := 42
>>>   if x, ok := x.(interface { ValuePure() int }); ok {
>>> val += x.ValuePure()
>>>   }
>>>   return val
>>> }
>>>
>>> But when it comes to passing a function as a parameter, it will become
>>> more subtle
>>>
>>> purefunc Hai(x purefunc() int) int {
>>>   return 42 + x()
>>> }
>>>
>>> // this should generate a compile error
>>> a := 20
>>> fmt.Println(Hai(purefunc() int {
>>>   a += 1 // side effect
>>>   fmt.Println("something") // side effect
>>>   return a
>>> }))
>>> On Tuesday, July 7, 2020 at 5:56:23 AM UTC+7 Ian Lance Taylor wrote:
>>>
 On Mon, Jul 6, 2020 at 3:11 PM bugpowder  wrote:
 >
 > I'd guess the compiler could then enforce it (see if any non-pure
 marked function is called from a pure one), it could exploit it (e.g. play
 with evaluation order, cache, etc), and other such things?

 The compiler can already tell whether a function is pure, so I don't
 think that adding a keyword would lead to any better optimization.

 I agree that adding the keyword would let the compiler enforce it, but
 that doesn't seem all that big a benefit to me. It also seems like
 something that could be done by an analysis tool rather than requiring
 a change to the language.

 Ian


 > On Tue, Jul 7, 2020 at 1:00 AM Ian Lance Taylor 
 wrote:
 >>
 >> On Mon, Jul 6, 2020 at 9:47 AM  wrote:
 >> >
 >> > Hi, I don't know if this kind of idea is already discussed before.
 >> >
 >> > I have an idea of adding pure function marker/type on golang, it
 is just like "constexpr" on C++ or "const fn" on Rust, whether this
 function is evaluated at compile time if the input is known at compile time
 is another discussion,
 >> > I don't think this idea is hard to implement
 >> >
 >> > to my understanding, a pure function is a function that doesn't
 have a side effect, so we can limit pure function to:
 >> > - unable to call non-pure function
 >> > - unable to modify a variable that is not declared on current
 function (like a global variable)
 >> >
 >> > for this purpose, we can think receiver as input to the function
 >>
 >> ...
 >>
 >> > what do you guys think about this idea?
 >>
 >> You didn't really explain what we would gain by adding this to the
 >> language. It's clearly already possible to write pure functions. How
 >> does it help to add the ability to explicitly mark a function as
 

Re: [go-nuts] pure function on golang

[Michael Jones]

prior discussion:
https://groups.google.com/forum/#!searchin/Golang-Nuts/pure$20functions/golang-nuts/ZVeMxBBVpa4/slidZL9KBAAJ

On Mon, Jul 6, 2020 at 3:11 PM bugpowder  wrote:

> I'd guess the compiler could then enforce it (see if any non-pure marked
> function is called from a pure one), it could exploit it (e.g. play with
> evaluation order, cache, etc), and other such things?
>
> On Tue, Jul 7, 2020 at 1:00 AM Ian Lance Taylor  wrote:
>
>> On Mon, Jul 6, 2020 at 9:47 AM  wrote:
>> >
>> > Hi, I don't know if this kind of idea is already discussed before.
>> >
>> > I have an idea of adding pure function marker/type on golang, it is
>> just like "constexpr" on C++ or "const fn" on Rust, whether this function
>> is evaluated at compile time if the input is known at compile time is
>> another discussion,
>> > I don't think this idea is hard to implement
>> >
>> > to my understanding, a pure function is a function that doesn't have a
>> side effect, so we can limit pure function to:
>> > - unable to call non-pure function
>> > - unable to modify a variable that is not declared on current function
>> (like a global variable)
>> >
>> > for this purpose, we can think receiver as input to the function
>>
>> ...
>>
>> > what do you guys think about this idea?
>>
>> You didn't really explain what we would gain by adding this to the
>> language.  It's clearly already possible to write pure functions.  How
>> does it help to add the ability to explicitly mark a function as pure?
>>
>> Ian
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXOdCc8Zz8mXAmghLm%2B6%3Dvi8S8zG_3Phrv2Hy-w%3Dm70kQ%40mail.gmail.com
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAACdnTAKTKQxU_K5xRqHGDKKBEhyTAq6%3D6q1HK%2BgDUewgJW1aw%40mail.gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxmwWTgOSXdzpS%3DpF3KH65xRULhfTDPePc%2BYnzc-KNmgQ%40mail.gmail.com.

Re: [go-nuts] Worker Pool vs. New Goroutine For Each Task

[Michael Jones]

If you change then will anything limit the peak degree of parallelism?

The worker pool size is such a limiting factor. May be important for you.
If so, plan another throttling mechanism.

On Fri, Jul 3, 2020 at 7:03 AM Atakan Çolak 
wrote:

> Hiya everyone,
>
> I have a simple processor function that takes in data, processes it, and
> sends it to a fellow goroutine which will eventually write them to DB. My
> main concern is just to maximise current task/sec with a hard limit on CPU
> usage. Currently I'm using a worker pool model that share a task channel to
> receive their tasks. I think another alternative solution could be setting
> GOMAXPROCS and generate a new goroutine for each task, as Go is able to
> handle thousands of goroutines easily. I wanted to ask about ups and down
> of both of these approaches.
>
> Thank you.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/567ff4bd-a99e-4af5-bd46-7859d98575cbo%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQx4Wf5Czs8LmR2caOTYT5vVNZpzWNn0Yj6aUhhR9RHmFw%40mail.gmail.com.

Re: [go-nuts] Re: Speed up png.Decode

[Michael Jones]

An easy "comfort" exercise:

save 1000 images to a directory in png form.
decode with go and with several tools such as imagemagick
compare times

for format in jpeg, tiff, ...
convert all images to that format; do the test above anew

the result will be a comparison chart of Go existing image decoders vs X in
various image formats for your images.

if it is within a few percent, you can be comfortable
if it is much slower, be uncomfortable and report a performance bug.
if it is faster, be happy.

michael

On Sat, Jun 27, 2020 at 5:31 AM Robert Engels  wrote:

> Just because the bulk of the time is in the decode doesn’t mean the decode
> is inefficient or can be improved upon. It might be the most expensive
> stage in the process regardless of the implementation.
>
> On Jun 27, 2020, at 12:15 AM, Lee Armstrong  wrote:
>
> 
> Thanks Rob,
>
> Yes you are right these images are small. <100kb each with an example
> below. And as I mentioned I have millions of these.
>
> I am able to max the CPUs out with a worker pool but that is when I
> noticed a lot of the work was actually decoding the PNG images which made
> we wonder if if was able to reuse decoders at all and even if that would
> help!
>
>
> On Sat, 27 Jun 2020 at 01:30, Rob Pike  wrote:
>
>> Without information about the particular images - the data set you're
>> working with - it's hard to provide any accurate advice. If the images are
>> small, maybe it's all overhead. If the images are of a particular form of
>> PNG, maybe a fast path is missing. And so on.
>>
>> To get meaningful help for problems like this, the more information you
>> can provide, the better.
>>
>> -rob
>>
>>
>> On Fri, Jun 26, 2020 at 10:59 PM Lee Armstrong  wrote:
>>
>>> Thanks, I am already maxing out some servers but wondered if it could be
>>> sped up.
>>>
>>> I will give the bimg package a go!
>>>
>>> On Friday, June 26, 2020 at 1:55:40 PM UTC+1 ren...@ix.netcom.com wrote:
>>>
 Just parallelize in the cloud. At a minimum parallelize locally with
 multiple Go routines.

 On Jun 26, 2020, at 7:51 AM, howar...@gmail.com wrote:

 

 I don't know if the CGo transitions for 16 million images will
 completely swamp the speed gains, or how much post-processing you need to
 do to these images, but you might try https://github.com/h2non/bimg and
 see if it gives you any wins. It claims 'typically 4x faster' then the Go
 Image package. It is a CGO interface to libvips, which is an experimental
 branch of libspng, which is a performance focused alternative to libpng.

 Howard

 --
 You received this message because you are subscribed to the Google
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to golang-nuts...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com
 
 .

 --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>
>>
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/d694c7d7-8438-4d7c-93ab-46cccaf09b65n%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAC5LoL_yWxgfDUCs5UwStep-dOMoWovpF_RK0K_MuPNzsG6Unw%40mail.gmail.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/D4EB5766-01C2-4025-898E-0768A302C8DF%40ix.netcom.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an 

Re: [go-nuts] Re: Why not use F in generic？

[Michael Jones]

Many (nearly all?) computer languages cater to English readers who have
experience with certain noun/verb and noun/modifier ordering.

Globally, “bird blue” and “pizza pepperoni” may be more common in other
natural languages than blue bird and pepperoni pizza.

The proposed order, “the T type of...” is not so unusual, arguably more
usual in natural language.

Michael

On Wed, Jun 24, 2020 at 2:48 PM i3dmaster  wrote:

> Actually, this isn't too bad. Given Go already flips types and names in
> syntax, having the generic specification in front of the entity it
> decorates doesn't seem too alien looking. The too-mang-() concern has
> already brought up to the draft, I think it likely require some careful
> thinking... production code could be way more complicated than trivial
> examples, I hope bringing generics wouldn't kill the elegant looking and
> simplicity of Go code in large...
>
> On Monday, June 22, 2020 at 9:46:56 AM UTC-7, redst...@gmail.com wrote:
>>
>> I read the new generic draft. And I know F,F[T],F《T》 is discarded. I
>> think put the type paremeter in front of the function name may be better.
>> No ambiguous and more readable code.
>>
>> func Print(type T)(s []T) {}
>>
>> Print(int)([]int{1, 2, 3})
>>
>> func Print(s []T) {}
>>
>> Print([]int{1, 2, 3})
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/cf511869-052e-4964-a9c6-89b136f82da4o%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzyMcmXPcvFm6%2Byke9c2saEDHnyYQY-fXKsiLdE-nrsVg%40mail.gmail.com.

Re: [go-nuts] How I can translate this shell command to golang code?

[Michael Jones]

Bakul's answer is best... starting the shell, the shell pipeline, etc. is
not free in execution time and memory use. The "do it all in dmtxwrite by
sending input" is the fastest, lightest way of doing what you want.

On Mon, Jun 22, 2020 at 9:21 AM Franco Marchesini <
franco.marches...@gmail.com> wrote:

> Ciao,
>
> I had read this tutorial, but  my mistake was not having called the shell
> command
>
> Il giorno lunedì 22 giugno 2020 18:01:04 UTC+2, Michael Jones ha scritto:
>>
>> https://tutorialedge.net/golang/executing-system-commands-with-golang/
>>
>> On Mon, Jun 22, 2020 at 8:54 AM Franco Marchesini 
>> wrote:
>>
>>> Help,
>>>
>>> how I can translate this shell command :
>>>
>>> echo -n 123456 | dmtxwrite -s 16x48  -o image.png
>>>
>>> to golang
>>>
>>>  exec.Commad()
>>>
>>> Thanks in advance
>>> Franco
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/c6aaba94-453b-41dc-83a4-ffd4f3912494o%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/c6aaba94-453b-41dc-83a4-ffd4f3912494o%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>
>>
>> --
>>
>> *Michael T. jonesmichae...@gmail.com*
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/e84a97ec-d326-4ccc-8e31-bf25991ec897o%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/e84a97ec-d326-4ccc-8e31-bf25991ec897o%40googlegroups.com?utm_medium=email_source=footer>
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxrE34J%2B4WDMwgsVJvEp_oMRfLK8tDO-o3GHCeRJxvfjw%40mail.gmail.com.

Re: [go-nuts] How I can translate this shell command to golang code?

[Michael Jones]

https://tutorialedge.net/golang/executing-system-commands-with-golang/

On Mon, Jun 22, 2020 at 8:54 AM Franco Marchesini <
franco.marches...@gmail.com> wrote:

> Help,
>
> how I can translate this shell command :
>
> echo -n 123456 | dmtxwrite -s 16x48  -o image.png
>
> to golang
>
>  exec.Commad()
>
> Thanks in advance
> Franco
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/c6aaba94-453b-41dc-83a4-ffd4f3912494o%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyTyKibkm-r2u_3btrUpn2GsnOL6WDNL36%2BHq05Y_jE5g%40mail.gmail.com.

Re: [go-nuts] [generics] Infer types on return

[Michael Jones]

>
> > I'm wondering what fundamentally separates inference of arguments from
> inference of returns, FWIW. Why is it a problem if we try to infer the
> return-type when assigning to an interface, but not when we try to infer
> the parameter-type when passing one?



> The difference is the directionality.  Argument passing is like
> assignment.  When I write "a = b", and b has an interface type, then a
> also has to have an interface type.  So type inference for passing an
> argument means that passing an interface type (b) gives you an
> interface type (a).  But for results it's the other way around.  Now
> we know type a and we are trying to infer type b.  And when a has an
> interface type, b can be anything that implements that interface.  As
> you say, we can simply reject the case.  We pretty much have to.  But
> it might be confusing that we can do inference for some result type
> but not others.  And it's a kind of confusion that arises only for
> results, not for arguments.


It is a situation that seems easily explainable, so teachable.

The single possibility case "specific = specific" says this about inference:
unknown = specific, infer unknown matches specific
specific = unknown, infer unknown matches specific

The multiple possibility cases are as you say:
unknown = nonspecific, infer unknown matches nonspecific (identical
interfaces)
nonspecific = unknown, no inference possible; unknown may be any of
multiple possibilities.

The teachable maxim is: "can infer from single choice; cannot infer from
multiple choices"

On Fri, Jun 19, 2020 at 4:03 PM Ian Lance Taylor  wrote:

> On Fri, Jun 19, 2020 at 2:40 PM Axel Wagner
>  wrote:
> >
> > On Fri, Jun 19, 2020 at 10:37 PM Ian Lance Taylor 
> wrote:
> >>
> >> On Fri, Jun 19, 2020 at 12:44 PM Brandon Bennett 
> wrote:
> >> >
> >> > I was playing around with a, probably not good, idea of what a
> rust-like Return type would look like:
> https://go2goplay.golang.org/p/k0NKN9Q6Wc1
> >> >
> >> > It seems when we are returning a specific value like Result(int) from
> a function as in
> >> >
> >> > func myFunc(input string) Result(int) {
> >> > return Err(int)(ErrBadValue)
> >> > }
> >> >
> >> > That it would be nice if we didn't have to specify Err(int) and that
> it could be inferred from the return type instead.
> >>
> >> If I'm reading this right, you are suggesting that because the result
> >> of Err is being assigned to a value of type Result(int) (via the
> >> return statement) we should be able to use that fact to infer the type
> >> argument of Err.  I think that would be possible for cases where we
> >> are assigning to a variable of known type.  It's a little more
> >> confusing when using := to assign to a variable whose type is not yet
> >> known.  And it also seems unlikely to work when assigning to a
> >> variable of interface type.
> >
> >
> > I think the solution to this would probably be the same as if you use `x
> := nil` - a compiler-error stating that there can't be a type assigned to
> the RHS. I think it would be fine to just require type-annotations where we
> can't clearly and unambiguously infer a type, as long as those cases are
> understandable - the two examples seem easy to understand exceptions, so as
> long as not more come up, that would still be fine IMO.
> >
> > That being said, I assume you generally see the value in more
> type-inference and I trust you to come up with decent rules for this
> eventually :)
> >
> > I'm wondering what fundamentally separates inference of arguments from
> inference of returns, FWIW. Why is it a problem if we try to infer the
> return-type when assigning to an interface, but not when we try to infer
> the parameter-type when passing one?
>
> The difference is the directionality.  Argument passing is like
> assignment.  When I write "a = b", and b has an interface type, then a
> also has to have an interface type.  So type inference for passing an
> argument means that passing an interface type (b) gives you an
> interface type (a).  But for results it's the other way around.  Now
> we know type a and we are trying to infer type b.  And when a has an
> interface type, b can be anything that implements that interface.  As
> you say, we can simply reject the case.  We pretty much have to.  But
> it might be confusing that we can do inference for some result type
> buts not others.  And it's a kind of confusion that arises only for
> results, not for arguments.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVxC%3D1bBoL4JcmWG8ODkyK1UXJ8%2B8id-S_LmQOtcw%3DcUw%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the 

Re: [go-nuts] Go has confusing pointers

[Michael Jones]

this says "increment the data c points at" not "increment the pointer"

the function is returning an int instead of a Count

On Fri, Jun 19, 2020 at 10:23 AM  wrote:

> Hi,
> Please help me understand the below:
> I got this from "The Zoo of Go Functions"
>
> type Count int
>
> func (c *Count) Incr() int {
>  *c = *c + 1 //why use *c and not just c, based on (c *Count) c is already
> a pointer?
>  return int(*c) //why cast to int, c is already of type int?
> }
>
> Christof
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/47bb77be-9634-47cb-9899-3bb07328ee50o%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxCr0%2B2cuMP7deP4e9jOK0hDhS%3D%2Bbipd%3DRw0yAH1rWbFA%40mail.gmail.com.

[go-nuts] [generics] Thank you Go team

[Michael Jones]

I doubt this will help, but I have to try...

Really smart and accomplished people have worked for a year to refine the
generics approach--as the impressive updated design draft and the scholarly
FeatherweightGo paper both demonstrate. The design is accompanied with
tools to allow development and experience by any interested Go developer.
This is marvelous. Thank you Go team and helpers!

Let's post about substantial things in the design ("generic type switch
necessary at day one" maybe, whatever) and not only about parenthesis. I'm
going crazy here seeing all the comments about that. I mean, think of how
hard they are working on this; parenthesis posts read like *More Cowbell *to
me.

Sorry for the outburst,
Michael

P.S. I challenge you to fully read the Featherweight Go paper and the
design draft, as I did, and then come away thinking about the expression of
the idea rather than the idea. I just can't comprehend it -- it is the
ideas that hold power and beauty, that will transform our daily work.

-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyzv%3D4JxtJjpqC6R5u%3DkwjMtYH%3Die9H3KQYWyB3U%2B1Ybg%40mail.gmail.com.

Re: [go-nuts] function multiple returned values signature, compiler optimisation/feature question

[Michael Jones]

This is beyond most translation systems.

Interestingly, the exceptions are themselves old in origin.

The LISP compiler, emerging after interpretation was the universal
environment for LISP, was able to “execute” the program being compiled to
discover what could be evaluated to a static conclusion, or a simpler case
of the general one.

>From about the same time, the early symbolic algebra systems had this same
notion in their runtimes of iteritavely evaluating expressions until a
fixed point In code terms was reached. REDUCE and MACSYMA are in my
thoughts here. Mathematica has a sense of this too in some of its
programming modes, an organic rule-driven rewrite system. (It is brilliant
at the task)

Not something seen in general languages very much. Some flavor of it in C
metaprogramming.

On Wed, Jun 10, 2020 at 11:28 AM 'simon place' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> when you have functions that return multiple values, because its more
> efficient to generate in one go when you are using ALL returned values;
>
> like in math, where you have Sin(), Cos() and Sincos()
>
> and when you also don't use all the returned values (using '_') does the
> compiler optimise? does it compile multiple times so dead code is removed
> from the version used with some return values unneeded?
>
> i would guess it doesn't, but i wanted to check, because i seem to keep
> coming over this, probably the way i think, seems to me to reduce the
> solution space.
>
> also this seems to, in some ways, overlap with generics, particularly if
> there were new language features that let you specify some high
> level/simple code path changes, things specific to the problem that the
> compiler could never determine.
>
> i guess with code generation its possible, but that doesn't feel like the
> best way to go.
>
> i could have tested for this but am also interested in all
> architectures/future developments/other ways to achieve this.
>
> thanks.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3369daf2-2cbd-4390-875b-e62eb4fab73ao%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxHLbLcbs84rUc-09gR8j4nm-jQH7yqU9BrHSnC_iM4%3Dg%40mail.gmail.com.

Re: [go-nuts] Re: gzip.Reader.Read does not fill the given buffer

[Michael Jones]

Philosophically

On Wed, Jun 10, 2020 at 11:26 AM Michael Jones 
wrote:

> Philosophy, this is one of the Go library’s most beautiful designs. It
> allows the one in charge—the program you write—to say what should happen,
> while allowing everything subordinate to be guided on how it should happen.
>
> This allows code to be adaptive to device optimal read sizes, network
> happenstance, the size of various buffers in code that you can’t see, etc.
>
> Maximum flexibility, simple formulaic coding:
>
> While not done, Use this, Ask for more.
>
> On Wed, Jun 10, 2020 at 7:45 AM Ronny Bangsund 
> wrote:
>
>> Reading in small chunks is handy for progress displays too.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/115eb9de-b821-44ae-a91c-b8dde26c28beo%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/115eb9de-b821-44ae-a91c-b8dde26c28beo%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
> --
>
> *Michael T. jonesmichael.jo...@gmail.com *
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyYhEsr_%3DQUKr_GoBAiRcSCvGsZU%3DAoSm98Pih8n6mEbQ%40mail.gmail.com.

Re: [go-nuts] Re: gzip.Reader.Read does not fill the given buffer

[Michael Jones]

Philosophy, this is one of the Go library’s most beautiful designs. It
allows the one in charge—the program you write—to say what should happen,
while allowing everything subordinate to be guided on how it should happen.

This allows code to be adaptive to device optimal read sizes, network
happenstance, the size of various buffers in code that you can’t see, etc.

Maximum flexibility, simple formulaic coding:

While not done, Use this, Ask for more.

On Wed, Jun 10, 2020 at 7:45 AM Ronny Bangsund 
wrote:

> Reading in small chunks is handy for progress displays too.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/115eb9de-b821-44ae-a91c-b8dde26c28beo%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxja1C%2BEGRToUSkK2iBz2Hqna1dxB0%3DD0hyP-8v5Fr%2BwA%40mail.gmail.com.

Re: [go-nuts] Re: what is the complexity of regexp.MustCompile()?

[Michael Jones]

Ray, only the discussion is exponential.

On Mon, Jun 8, 2020 at 4:23 PM 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> On Tue, Jun 9, 2020 at 12:48 AM Kurtis Rader  wrote:
>
>> You're talking past each other. Robert is talking about limiting the
>> length of the regex, not the string(s) evaluated by the regex.
>>
>
> So am I. Note that e.g. a Code Search based on PCRE would break, even if
> you limit *both* (or rather, any limit causing it to not break would result
> in a completely useless piece of software).
>
> It should be possible to compile any regex of a reasonable length in a
>> matter of microseconds. Regardless of whether the application of the regex
>> to a given input is near linear (as in the case of the Go RE
>> implementation) or exponential (as in the case of PCRE).
>>
>
> This might be, where we talk past each other. I am using application as an
> example for concrete numbers on how quickly exponential growth can devolve.
> Of course, compilation of a regexp is fast - I am aware of that. As it
> turns out, so is application of a regexp, if you use RE2 (or Go's regexp
> package).
>
> What I take issue with are the statements that a) the question about the
> complexity of compiling a regexp is irrelevant, because b) limiting the
> algorithmic complexity of a function to counteract resource exhaustion
> attacks "never works". RE2 is an excellent example to show that it does
> work; it is carefully designed for linear complexity of the combined
> compilation+matching of a regular expression.
>
> I'm not saying compiling a regular expression is an exponential operation.
> I'm saying *if it was*, you could never reasonably build something like
> Code Search. And we can use the fact that *application* indeed is (in most
> engines) exponential in the combined input length of expression+search text
> as a good basis to make that case.
>
> Of course we don't even need this fantasy world to make the case - after
> all, saying "you can't build Code Search on PCRE, but you can on RE2" is
> just as effective to prove the effectiveness of lowering algorithmic
> complexity. It's just that OP's question was about compilation, so to talk
> about why OP's question *is* relevant, we need to talk about what would
> happen if the answer *wasn't* "it's linear".
>
> (I also genuinely don't understand the instinct to tell someone "why would
> you care?" instead of telling them the answer, FWIW. Especially in a case
> like this, where the answer is pretty simple)
>
> I'm pretty sure Robert is not arguing that the scaling problems of the
>> regex implementation used by Perl, and too many others, can be mitigated
>> simply by limiting the size of the string to be matched by the regex. If
>> compiling a regex of reasonable length takes a non-negligible amount of
>> time something is wrong.
>>
>> On Mon, Jun 8, 2020 at 3:22 PM Wojciech S. Czarnecki 
>> wrote:
>>
>>> Dnia 2020-06-08, o godz. 16:22:24
>>> Robert Engels  napisał(a):
>>>
>>> > it is trivial to limit the input size to something a user could input.
>>>
>>> With exponential complexity simple regex /(x+x+)+y/ blows up at input of
>>> 20 to 30 x-es.
>>> See: https://www.regular-expressions.info/catastrophic.html
>>>
>>> [Cut long explanations... Axel just posted most of what I was writing
>>> regarding trade-offs).
>>>
>>> Hope this helps,
>>>
>>> --
>>> Wojciech S. Czarnecki
>>>  << ^oo^ >> OHIR-RIPE
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/20200609002207.0a161adf%40xmint
>>> .
>>>
>>
>>
>> --
>> Kurtis Rader
>> Caretaker of the exceptional canines Junior and Hank
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD9sKeHnqUAqB61y5Ts-U_f%2BctVAuS4BC0ae8phhhcp1iw%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGnRrAt%3Dx3buNcfoOc9xGqVj_tfMEviU%3D7Amjw01e%3Df_w%40mail.gmail.com
> 

Re: [go-nuts] Re: Preventing SIGBUS errors when memmove-ing an unsafe.Pointer to multiple destinations

[Michael Jones]

Thank you. I now officially know that I don’t understand. Sorry.

On Sun, Jun 7, 2020 at 7:54 AM Viktor Kojouharov 
wrote:

> The pointer is being copied via typedmemmove, which itself calls memmove,
> which, according to its documentation, copies bytes from the source to the
> destination. Not sure why that would be impossible, considering it does
> work for some code (the source pointer preserves its data)
>
> Not sure what you mean by "copied via unsafe". Also, the source pointer
> never goes out of scope. It's part of a struct that is passed to the
> function that calls typedmemmove, and its lifetime is more or less static.
> So while the destination pointers go out of scope, the source one never
> does.
>
>
> On Sunday, June 7, 2020 at 4:45:40 PM UTC+3, Michael Jones wrote:
>>
>> Do you mean that you have a problem with the value of the pointer? That
>> is "copying the pointer." This seems impossible.
>>
>> Attempting to access through a pointer copied via unsafe is (generally)
>> inviting doom, and seems highly possible. The instant the last pointer to
>> that data goes out of scope the address range occupied by the formerly
>> pointed to items is formally inaccessible. Using unsafe to keep a shadow
>> copy of the address and then poking around is quite likely to fail, and
>> even when it does not, it is quite likely to be meaningless. (random data
>> from some other use).
>>
>> If I misunderstood, please forgive me.
>>
>> On Sun, Jun 7, 2020 at 6:15 AM Viktor Kojouharov 
>> wrote:
>>
>>> p.s. should such questions be posted in golang-dev, since it deals with
>>> runtime internals?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/71d7fb5c-3ef5-4611-b9ce-299f7b90945eo%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/71d7fb5c-3ef5-4611-b9ce-299f7b90945eo%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>
>>
>> --
>>
>> *Michael T. jonesmichae...@gmail.com*
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/fa18bf7e-8965-4917-9d81-00a8f43289c3o%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/fa18bf7e-8965-4917-9d81-00a8f43289c3o%40googlegroups.com?utm_medium=email_source=footer>
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyBHzkKdYQ_7HF%3DQhmg5na2PZfhi0QmMkOtXY-%3Dh5u3vw%40mail.gmail.com.

Re: [go-nuts] Re: Preventing SIGBUS errors when memmove-ing an unsafe.Pointer to multiple destinations

[Michael Jones]

Do you mean that you have a problem with the value of the pointer? That is
"copying the pointer." This seems impossible.

Attempting to access through a pointer copied via unsafe is (generally)
inviting doom, and seems highly possible. The instant the last pointer to
that data goes out of scope the address range occupied by the formerly
pointed to items is formally inaccessible. Using unsafe to keep a shadow
copy of the address and then poking around is quite likely to fail, and
even when it does not, it is quite likely to be meaningless. (random data
from some other use).

If I misunderstood, please forgive me.

On Sun, Jun 7, 2020 at 6:15 AM Viktor Kojouharov 
wrote:

> p.s. should such questions be posted in golang-dev, since it deals with
> runtime internals?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/71d7fb5c-3ef5-4611-b9ce-299f7b90945eo%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwf4wL3TPdh5yi0hU5ibJaN7-35SsQgGfGLTRAaEjihMA%40mail.gmail.com.

Re: [go-nuts] What Go devs think about issues described by Daniel Lemire in article "The Go compiler needs to be smarter"?

[Michael Jones]

When he writes "you expect a good, optimizing compiler to..." he is
implicitly stating that he wants more  than good optimizing compilation, he
wants (understandably, but let's be explicit) iterated compile-time
evaluation -- he wants fun() to be an integer value of 60.

It is true that the gc Go compiler lacks this. I like it as a feature, it
is like invoking "reduce()" in a symbolic algebra system. I hinted at it a
while back by asking if Go should notice when a function returns a pure
value and do substitutions -- clearly "x := math.Sqrt(4)" suggests "x :=
2.0" or "const x = 2.0" depending on context. But this is a non-trivial
step from compilation in the direction of metaprogramming via
Turing-complete macro processors. It will make the compiler bigger and
slower, and intellectually larger.

It is an open issue decision wise, and a great deal of work implementation
wise.

On Thu, Jun 4, 2020 at 1:20 PM Andy Balholm  wrote:

> I think by “at compile time” he means at JIT time (when converting
> bytecode to machine language).
>
> Andy
>
>
>
> On Jun 4, 2020, at 1:02 PM, Robert Engels  wrote:
>
> The author either doesn’t know Java or had significant editing errors -
> Java determines uses the runtime processor type to optimize - it is not
> done at compiler time. It has different implementations based on processor
> - like Go - and it does JIT optimizations based on processor as well.
>
> More importantly, these micro benchmarks with fractions of a percent are
> meaningless. All powerful systems are distributed to scale, not micro
> optimized.
>
>
>
> On Jun 4, 2020, at 2:34 PM, Igor Yemelianov  wrote:
>
> 
> Link
>  to
> the article.
>
> The question is - is it possible that the issues described in the article
> can be solved in for example next major version?
>
> Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7aef8f47-8834-4cd3-9ba0-67e51728a110o%40googlegroups.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/01446EC6-44C7-4920-971F-4537BD404414%40ix.netcom.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAD4A4F7-0C56-430F-AF54-86993C704655%40gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyKs8WQHx01pDOLoOM0PkPtua%2Bbcg%3DnxnEx%3DjidbnWVjA%40mail.gmail.com.

Re: [go-nuts] x, err = some_func(); if err != nil { } seems awkward

[Michael Jones]

Before you get an avalanche of emails about "that is an old question,
discussed in design documentation, mailing lists, and extension discussions
over a decade", let me ask you to consider a concept that your comment
implies.

For that to work, it means that a function call either succeeds or fails.
If it succeeds, then that's that. If it fails, then something special
happens. Your "#" means "do this only if the function call failed." In that
case itmust be true that functions have a notion of success and failure,
that the cause of failure is invisibly returned and available as an extra
"and here's why it failed" return value, and, new language concepts are
required to access the value of the invisible return inside that "#{ ... }"
block.

There is good precedent for the S (success) and F (failure) outcomes of
function calls--SNOBOL has that.

For everything else though, it has generally considered more awkward than
the present situation. If you want to contribute along these lines, think
about how to make the access to the invisible error return value feel
natural. (maybe think about it before reading the hundreds of pages of
discussion that await you.

Michael

On Thu, Jun 4, 2020 at 8:43 AM  wrote:

> ?? Why not a cleaner syntax e.g.  x = some_func ()  #{ }   .. symbol #
> arbitrarily chosen
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ea305494-c6ec-46f9-962d-9add01edc8bd%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQygvQtsfKCmaYgDnwbv5tSqcp7Zuqdh0wHtJ0WqkeMYbg%40mail.gmail.com.

Re: [go-nuts] Re: what is the complexity of regexp.MustCompile()?

[Michael Jones]

If you have thousands of fixed strings, a map is your friend. Remarkably
so.

On Wed, Jun 3, 2020 at 2:43 PM Ray Pereda  wrote:

> Typo fix, regexp#MatchString  and
> related Match functions are linear. That is an amazing guarantee and makes
> regular
> expressions 10x more useful than regexps in most other programming
> languages.
>
> Question: Does the *compile* of regular expressions also guaranteed
> linear runtime? I'm considering regular expressions with
> 1000s of keywords; very long regexps. I looked at the source and it was
> not obvious if that is the case. Intricate code.
>
> On Wednesday, June 3, 2020 at 10:07:12 AM UTC-7, Ray Pereda wrote:
>>
>> I believe that the complexity of regexp.MustCompile()
>>  is linear based on this
>> comment in the regexp package overview.
>> 
>>
>> "The regexp implementation provided by this package is guaranteed to run
>> in time linear in the size of the input"
>>
>>
>> What is the complexity of regexp.MustCompile()
>> ? Is it linear in the length
>> of the regular expression?
>>
>> -ray
>>
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/f51125e7-f66e-45e8-af3a-381f96071d9a%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQy9TBAJo4Oz_wYsWvacDW_Q_y49bj%3DUCuY1tcdqSR5Ouw%40mail.gmail.com.

Re: [go-nuts] Re: Incorrect arithmetic answers

[Michael Jones]

fixed, in light of the prior crucial comment...
https://play.golang.org/p/d-AwGEwj_YB

On Mon, Jun 1, 2020 at 12:20 PM  wrote:

> Floating point calculations are imprecise.
>
> https://0.30004.com/
>
> https://www.phys.uconn.edu/~rozman/Courses/P2200_15F/downloads/floating-point-guide-2015-10-15.pdf
>
> s
>
> On Monday, June 1, 2020 at 11:56:55 AM UTC-7, Saksham Saxena wrote:
>>
>> Hello,
>>
>> I was trying to implement Substring Search using a rolling hash function
>> (fairly simple implementation): https://play.golang.org/p/_fjgxPC06So
>>
>> This works correctly and as expected. However, if you change the const
>> `baseNumber` to 128, the program is unable to do the intermediate
>> arithmetic incorrectly (see the difference after running it).
>>
>> How could this be ? I've stared at it long enough and it doesn't seem
>> like it's overflowing (or is it really ? I thought float64 had great range
>> ?)
>>
>> Any ideas/comments are appreciated.
>>
>> Thanks!
>> Saksham
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d071b2be-9fc1-45e4-9da8-baecb9968ea7%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxs8vmTE89qb-pHGgO9w3kVxPB3qOQdcYRstDB0Tdw%3DNw%40mail.gmail.com.

Re: [go-nuts] Re: Is there a way to create a global variable inside go routine stack? so that i can access any time, without passing around between methods/functions. Crazy thought !!!..

[Michael Jones]

Variables are names associated with values, values that can *vary*, that
is, can be changed. (Unlike constants, which are *constant* values.)

In the beginning, all variables were global, and this was not good.


Then came local variables, ones *local* to a function, which came and went
as functions executed, and then block-scoped variables, ones local to a
particular begin...end or {...} block of code. This was seen as good, and
the people were urged to avoid global variables (and *goto* and other ways
of the past).

Now, every combination of ways has a home in some language's culture:
*static* in C is a function-local global variable, Go style embraces global
variables for flag values, and there are local variables in sophisticated
assembly languages that never go to memory.

Now, what problem is it exactly that you want to solve?

On Sun, May 24, 2020 at 11:06 PM  wrote:

> Yeah this works, but you can say this as workaround, what i really want
> is, does native go support? if not why?
>
> On Monday, May 25, 2020 at 7:57:17 AM UTC+5:30, tokers wrote:
>>
>> You may try to inspect this go package: https://github.com/jtolio/gls
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3e71bdf8-3b99-4f47-9412-68cc50f69e84%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzv8DpS%3DBG-peZj3%3D2oht%2BD%3DOoipyiK3gDGo7gaswQPDA%40mail.gmail.com.

Re: [go-nuts] Re: help understanding weird Benchmark results

[Michael Jones]

as was explained, the loop needs to be "for i:=0; i < b.N; i++"

as was mentioned, the compiler's dead code elimination efforts can
frustrate benchmarking. they way to make sure the test code survives is to
not let it be dead code. for example

// external var dummy

func
for i:=0; i < b.N; i++ {
  dummy += rand.Int63()
}

On Tue, May 19, 2020 at 10:22 AM Warren Bare  wrote:

> >You are supposed to run the loop b.N times, not some fixed constant.
>
> I understand.  This is a simulation of a single bigger task that takes a
> while.  I'm not trying to time the rand function inside the loop.  The loop
> is simply to burn time.  This simple function is a minimal example that
> demonstrates a problem I was having with my own real benchmark test.
>
> I don't know for sure that the compiler is NOT optimizing away this rand
> function, but I can tell you this problem was happening on my own code that
> can not be optimized away.
>
> Thanks.
>
>
>
> On Tuesday, May 19, 2020 at 12:39:33 PM UTC-4, Volker Dobler wrote:
>>
>> You are supposed to run the loop b.N times, not
>> some fixed constant. Also make sure the compiler
>> doesn't optimize away the whole function.
>>
>> V.
>>
>> On Tuesday, 19 May 2020 18:20:43 UTC+2, Warren Bare wrote:
>>>
>>> Hi Folks,
>>>
>>> I'm getting weird results from Benchmark.  Maybe someone can help me
>>> understand this.  I'm running on amd-64 (Threadripper 16 core 32 thread)
>>> Windows 10.  Go 1.14.3
>>>
>>> I have the benchmark below (main_test.go) on a minimum "hello world"
>>> main.go (just like playground).
>>>
>>> When I run the benchmark as it is below, I get the results included just
>>> below here.  Notice it reports 0.135 ns/op but the time is actually 135
>>> *ms* so it is off by a factor of 1 billion.  It is like it trying to
>>> report in seconds but did not change the label from ns to s.
>>>
>>> Further, if I increase the loop 10x from 10_000_000 to 100_000_000, then
>>> it prints Duration 1.349 seconds (good) and now the Benchmark time has
>>> increased by a factor of 10 *billion *and is now correctly reported as
>>> 1349224200 ns/op
>>>
>>> What am I missing here?
>>>
>>>
>>> BenchmarkMarshalSample-32   10   0.135 ns/op   
>>> 0 B/op  0 allocs/op
>>> --- BENCH: BenchmarkMarshalSample-32
>>> main_test.go:14: Duration 136.1221ms
>>> main_test.go:14: Duration 135.1214ms
>>> main_test.go:14: Duration 134.1763ms
>>> main_test.go:14: Duration 135.1217ms
>>> main_test.go:14: Duration 135.1298ms
>>> main_test.go:14: Duration 135.1217ms
>>> main_test.go:14: Duration 135.1218ms
>>> main_test.go:14: Duration 135.1213ms
>>> main_test.go:14: Duration 135.1298ms
>>> main_test.go:14: Duration 135.1216ms
>>> ... [output truncated]
>>> PASS
>>>
>>>
>>>
>>> package main
>>>
>>> import (
>>> "math/rand"
>>> "testing"
>>> "time"
>>> )
>>>
>>> func BenchmarkMarshalSample(b *testing.B) {
>>> start := time.Now()
>>> for i := 0; i < 10_000_000; i++ {
>>> rand.Int63()
>>> }
>>> b.Log("Duration", time.Now().Sub(start))
>>> }
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/2abb7390-3c3c-4172-aead-021dbf500bad%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwN%2Bou8hqYEi5R1RuC%2Bdvbzzemsha35QA5LwK87Q_whVA%40mail.gmail.com.

Re: [go-nuts] Image Resize and Crop

[Michael Jones]

I have an extremely elaborate resizing library, but it is so complex it
would not make sense as a standard tool for common uses. (Many convolution
kernels, separate windows, forward and backward mapping, separable
convolutions, upsampling first for Nyquist issues, strategy phase and then
concurrent filtering, etc. good work but over the top)

If nothing else, you could port Paul Heckbert’s filter program from days of
yore. Check graphics gems.

On Sun, May 10, 2020 at 9:36 AM Vivi  wrote:

> How do you advice to resize and crop JPEG and PNG or probably WebP images
> without rely on 3rd parties dependencies?
>
> It was hard to find a good snippet or could be useful to have basic API
> function in Go standard library since it's a common feature.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/89e32097-6aa0-48a4-b2ca-8d3756e26af3%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwOxRXsrB0%2B-oeAWB938A%2BOzCRppTQvrKENp9%3D8o3BcYQ%40mail.gmail.com.

Re: [go-nuts] Why I fear Go

[Michael Jones]

I have hired many people over the years, run companies, bought companies,
and make VC investment decisions now, which are almost the same task. In
every case there is the minimum qualification and the personal magic you
bring above or outside that. The desire for "senior" means "skills and
experience." It may well be met with skills and magic other than
experience, such as passion, intellect, and energy. Your email suggests
these, so be sure to bring them to your interviews!

On Sat, May 9, 2020 at 12:00 PM Chris Burkert 
wrote:

> Don't be afraid of the term "senior". The process prior to signing a
> contract is mutual application. The company advertises itself like the
> candidate advertises himself. Of course companies want the best developers
> so they attract these with "senior" and other terms. But sometimes you
> don't get what you want - company and applicant.
> Be proud and honest. Be open and listen. What else can a company want?
> And even if you get a different job then Go may have tought you some
> valuable things like simplicity, orthogonality, effectiveness and more.
> These will help you in many ways even if you don't use Go in the end. So go
> for it and learn Go!
> PS: and then approach the next hard thing in life :-)
>
> Am Sa., 9. Mai 2020 um 19:56 Uhr schrieb shammah Zealsham Agwor <
> shammahag...@gmail.com>:
>
>> I have been programming since 2016 but I have never really dived deep
>> into any language and haven't got a job with any . I decided to change all
>> that and focus with every soul in me on golang. But I have my fear as every
>> go job I have seen are looking for senior developers. And from what I
>> noticed in big corps and small ones alike ,people writing go there are
>> senior who wants to port existing large scale system to go . I'm afraid
>> I'll never be able to get an entry job with it and all my hard work will be
>> a waste and I'll have to put in such efforts again into another language .
>> Should I still go for it . I've been learning it for months now and I'm
>> scared .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/be5de5a5-b43b-4585-bfbc-c14fc8336044%40googlegroups.com
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CALWqRZrb8k86z8OheicHXG63dZfKJQF3OkB%3DVeYLFuM0-mfvTg%40mail.gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQw_BCUkjLpEUFioBGJ%3DAT4P%3D%2BHG9LFMCyYujjy%3DYOdSYA%40mail.gmail.com.

Re: [go-nuts] getting values from memory addresses within another process...

[Michael Jones]

The general dangerous ability to do this is why protected mode went into
the i368 and is the first and most essential promise to prevent of every OS
other than MS DOS, original MacOS, and practically the threads in shared
memory model of Smalltalk & MP Mathematica.

On Thu, Apr 30, 2020 at 7:13 PM Kurtis Rader  wrote:

> On Thu, Apr 30, 2020 at 6:59 PM Trig  wrote:
>
>> I'm attempting to read memory from another process.  I've installed
>> 'Cheat Engine' to do this, to make sure I'm pulling the correct value from
>> the address I'm attempting to; however, nothing I found works  I did find
>> this article:
>>
>>
>> https://stackoverflow.com/questions/37358478/read-random-memory-locations-with-golang
>>
>> I don't believe that is correct though, as using the address of the
>> location I'm attempting to read doesn't result in a value anywhere near
>> what 'Cheat Engine' is reporting.  I've looked at the 'unsafe' and
>> 'syscall' packages; however, there's very little information on them.
>> Also, searched many ways trying to find examples on how to do this.  I'm on
>> a Mac (and use Linux).  On Windows, I can do this fairly easy.
>>
>
> Really? I'd love to see your Go code that allows reading arbitrary memory
> on MS Windows.
>
> As Ian pointed out on UNIX, and most operating systems for that matter, do
> not allow a process to read the memory of other processes without using
> specialized operating system APIs meant for debugging; such as the
> `ptrace()` syscall.
>
> Note that the stackoverflow question you linked to is bollocks. The
> questioner apparently wants to read the virtual memory of other processes.
> Yet they accepted as correct an answer that does no such thing. The
> "answer" only reads arbitrary virtual memory of the Go process.
>
> --
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD9v%2BssEHjeZfV2Xb9QQYAkWgF5wskOYi2LkrfPqak4JrQ%40mail.gmail.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQw%3D1GtAw9CZVKJZ8eBhuyH2ufs%3Dg_fm6_QWU%2BSerAxYeQ%40mail.gmail.com.

Re: [go-nuts] Safe ways to call C with less overhead

[Michael Jones]

Function call intensity seems directly addressed by a tree or DAG like
chain of command buffers, not necessarily a full scene graph (with logic
and selection) but a call at the top and traverse tool to let you make just
a few cgo transitions to c per frame.

I’ve done this several ways myself (non-Go) and it works a charm.

On Thu, Apr 30, 2020 at 6:18 AM Constantine Shablya 
wrote:

> Thanks for reply, Ian
>
> To clear up, by safety I only mean presence of stack guards or, more
> generally,
> means of ensuring the program doesn't silently end up writing past the
> stack.
>
> From this I take my next step will be to make something between
> systemstack and
> asmcgocall so that I still run (subset of) Go while on systemstack but
> otherwise
> it would be as if it was a Cgo call, with minor difference that aligning
> stack
> and change of calling convention would happen at calls to C as opposed to
> at
> asmcgocall.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAEp866QQrqNUznSOFR4j_s6vW4X7ftEQVy48%3DH42UTBoe4DtXw%40mail.gmail.com
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQz6KT3zYibL7zG%2B8%3DnqQB1%3DHw0YJOcefU1_Y%3DQu6iP0AA%40mail.gmail.com.

Re: [go-nuts] Why isn't there "number plus" postfix syntax for more than or equal?

[Michael Jones]

Hi “Anon”,

A more serious answer (though just personal opinion) is that your idea
lacks the “grammar” that it implies.

Value Relationship Value “a >= 99”

is a “sentence” or fragment in most computer languages. It is a logical
assertion that is either true or false and can be composed into larger
sentences in that grammar.

99 <= a && a <= 123

What your “value range” notion is saying is not just “99 or more” but also
“is in the range of 99 or more”. The “in the range of” is implied in
addition to the + as “or more”.

a 99+

It feels empty of the relationship even if 99+ has the range meaning.
Compare:

a in 99+

That feels clearer to me. Both concepts are now made explicit. The language
Pascal has ranges and a membership clause. ADA also embraced some of it. We
could imagine:

a in 99..
a in 99..123

The “in” part feels important to me.

Michael


On Thu, Apr 23, 2020 at 6:32 PM anon notmyfault64 
wrote:

> In this context, this number plus syntax short-circuit more than or equal,
> so
>
> a 99+
>
> is same as
>
> a >= 99
>
>
>
> On Thursday, April 23, 2020 at 11:43:32 PM UTC+7, Ian Lance Taylor wrote:
>>
>> On Thu, Apr 23, 2020 at 8:48 AM anon notmyfault64 
>> wrote:
>> >
>> > Many times outside programming we use "number plus" postfix syntax to
>> denote more than or equal, for example:
>> >
>> > a 99+
>> >
>> > But why isn't there such syntax above in all programming languages,
>> including Go? That is, why does following code not compile with invalid
>> syntax error?
>> >
>> > var r int = 18
>> >
>> > if r 13+ {
>> > fmt.Println("Hooray! We are teen! We can do anything!")
>> > } else {
>> > fmt.Println("Oh No! We are still child, so we need parental
>> control!")
>> > }
>>
>> I don't see the advantage over writing r >= 13.
>>
>> It's not useful for a programming language to have multiple ways of
>> writing the exact same thing.  Of course, any language does have
>> multiple ways of doing some things, but there is should always be a
>> reason for it.  I don't see a reason for this one.
>>
>> For what it's worth, I'm not familiar with the "a 99+" notation.  I
>> would not know what that meant without an explanation.
>>
>> Ian
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/77d05b4d-6743-48ed-affa-b6fb66d13500%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwHxsc_xQ%3DU%2BhWFx5JHqVZ_%2B2GkAveLBpE6BYenG8%2Brng%40mail.gmail.com.

Re: [go-nuts] Why isn't there "number plus" postfix syntax for more than or equal?

[Michael Jones]

You could extend the notation:

If r 13+- {
  // if r is close to 13, in a handwavy sense
  :
}

On Thu, Apr 23, 2020 at 9:43 AM Ian Lance Taylor  wrote:

> On Thu, Apr 23, 2020 at 8:48 AM anon notmyfault64 
> wrote:
> >
> > Many times outside programming we use "number plus" postfix syntax to
> denote more than or equal, for example:
> >
> > a 99+
> >
> > But why isn't there such syntax above in all programming languages,
> including Go? That is, why does following code not compile with invalid
> syntax error?
> >
> > var r int = 18
> >
> > if r 13+ {
> > fmt.Println("Hooray! We are teen! We can do anything!")
> > } else {
> > fmt.Println("Oh No! We are still child, so we need parental
> control!")
> > }
>
> I don't see the advantage over writing r >= 13.
>
> It's not useful for a programming language to have multiple ways of
> writing the exact same thing.  Of course, any language does have
> multiple ways of doing some things, but there is should always be a
> reason for it.  I don't see a reason for this one.
>
> For what it's worth, I'm not familiar with the "a 99+" notation.  I
> would not know what that meant without an explanation.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWm5jaEa36FMqLaea22BTB_WwZOsZpwrW1ei1N_KN64eg%40mail.gmail.com
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzf754cub_eNsxtPXmbbnawFMc82-eiTqygBxW8231dcg%40mail.gmail.com.

Re: [go-nuts] Re: [ANN] Unik, a Go unikernel capable of running Gio GUI programs

[Michael Jones]

IBM VM/370 and much that followed is exactly this “VM is a hardware
abstraction” line of reasoning. Logging into a user session was to “IPL a
machine.” (Initial Program Load, imagine an IBM PC bios screen rushing by)

On Mon, Apr 13, 2020 at 2:06 PM Brian Candler  wrote:

> On Monday, 13 April 2020 19:42:17 UTC+1, Elias Naur wrote:
>>
>> The project is an experiment in a larger exploration of the question:
>> "what if the operating
>> system process abstraction were a virtual machine?
>>
>
> Interesting.
>
> BTW, have you seen gVisor ?  It takes a
> different approach to the problem: emulating the syscall interface on top
> of a running kernel, rather than on top of (real or emulated) bare metal.
> I was reminded of it since it also "pretends to be a Linux kernel" - and is
> written in Go.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/4b4e4279-19d6-4afc-b743-c1cfa3e9c6a0%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxDc7RPtHn2BzaGjhZXeh_UAZHRv1iQ5_TvnbGMfQU_3A%40mail.gmail.com.

Re: [go-nuts] Re: go run requires internet connection?

[Michael Jones]

Unable to reproduce the issue on MacOS Mojave 10.14.6, though I tried with
gusto.

On Thu, Apr 9, 2020 at 4:32 AM Tanmay Das  wrote:

> Thanks, everyone for your valuable comments. I think Brain is right. It
> might be an OS-related issue. I really like how active this group is. I
> look forward to coming back with more topics in the future. Stay Home. Stay
> Safe.
>
> On Wednesday, April 8, 2020 at 10:17:36 PM UTC+6, Tanmay Das wrote:
>>
>> Hey Gophers,
>> My very first post here.
>>
>> Today I faced an unexpected power outage and I wanted to tinker with Go a
>> little bit. I wrote a simple hello world program and ran
>> go run helloworld.go
>>
>> Strangely the code didn't run. In fact, the terminal prompt never exited.
>> I kept running the same command over and over again but no luck. I checked
>> all my configurations, my env vars, etc. and everything was ok. After
>> ruling out all the possibilities it suddenly hit me: what if Go actually
>> requires an internet connection to run a program for no apparent reason? I
>> waited for the electricity to come back and as soon as I was connected to
>> the internet I ran `go run` command again and voilà!
>>
>> Is this behavior expected? If it is, why did the go authors make such a
>> decision? I mean making the internet connectivity a dependency for the
>> execution of a program sounds counter-productive to me, honestly. :(
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/8fc3b552-f5c4-43de-94ba-4e0d4511f79a%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyTgSTRDROSmgmKQAtJCzACv-aHPn7165tPjg0QVR-Smg%40mail.gmail.com.

Re: [go-nuts] Bencharking issue with b.StartTimer and b.StopTimer. Was: Benchmark using b.StopTimer and b.StartTimer has unexpected behaviour - to me at least

[Michael Jones]

yes

On Tue, Mar 24, 2020 at 11:48 AM Orson Cart 
wrote:

> On Tuesday, 24 March 2020 18:31:29 UTC, Michael Jones wrote:
>>
>> You use them to stop the time charged against your benchmark.
>>
>> For example:
>>
>> bench:
>>   stop timer
>>   generate initial data
>>   start timer
>>   do test
>>
>
> Thanks Michael. What if the initial data has to be generated from scratch
> on each iteration? Is the following expected to work?
>
> bench:
> b.N Loop
> stop timer
> generate new data
> start timer
> do test
>
>
>
>
>>
>> On Tue, Mar 24, 2020 at 10:47 AM Orson Cart 
>> wrote:
>>
>>>
>>> On Tuesday, 24 March 2020 16:49:55 UTC, Robert Engels wrote:
>>>>
>>>> Can you please succinctly explain the problem?
>>>>
>>>
>>> Let's see. Benchmarks can yield incorrect results when b.StopTimer and
>>> b.StartTimer are used.
>>>
>>> My reasoning:
>>>
>>> 1/ I have a benchmark that calls a single function. The reported
>>> duration is T.
>>> 2/ I have another benchmark which invokes the same function twice. I'd
>>> expect the reported duration for this to be nominally 2T and within reason
>>> it appears to be the case.
>>> 3/ I have yet another benchmark which also invokes the same function
>>> twice but it stops the benchmark timer before the first call and then
>>> starts it after the the first call has completed. I'd expect the reported
>>> duration to be nominally T again. It isn't. It's closer to 3T on my
>>> hardware and I've seen much worse correlation on other hardware - almost 5T
>>>
>>> Now it's entirely possible that I'm misunderstanding something about how
>>> b.StopTimer and b.StartTimer are intended to be used, hence my post here :)
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>> On Mar 24, 2020, at 11:24 AM, Orson Cart  wrote:
>>>>
>>>> 
>>>> I posted this earlier but I realised that the code had a fundamental
>>>> error in it. I've corrected here it but the underlying problem still 
>>>> exists.
>>>>
>>>> I've recently started using go test's benchmarks support and I'm
>>>> particularly interested in understanding the benchmark timer functions.
>>>> I've been getting results that I found surprising and I was wondering if
>>>> anyone could explain what's going on here.
>>>>
>>>> The code below has three benchmarks that each invoke a single function
>>>> (foo). The implementation of foo isn't important, it's just there to
>>>> consume some time:
>>>> - foo is called once per iteration in Benchmark1.
>>>> - It's called twice per iteration in Benchmark2 so I'd expect
>>>> Benchmark2's duration to be nominally twice that of Benchmark1.
>>>> - It's also called twice per iteration in Benchmark3 but the first call
>>>> is wrapped in b.StopTimer and b.startTimer calls. Because of this I'd have
>>>> expected Benchmark3 to be about the same duration as Benchmark1
>>>>
>>>> Apologies for the length of the example but I didn't think it fair to
>>>> ask the question and leave anything out.
>>>>
>>>> package demo_test
>>>>
>>>> import (
>>>> "strconv"
>>>> "testing"
>>>> )
>>>>
>>>> var Foo1 []string
>>>> var Foo2 []string
>>>> var Count int = 32767
>>>>
>>>> func Benchmark1(b *testing.B) {
>>>> for i := 0; i < b.N; i++{
>>>> Foo1 = foo(Count)
>>>> }
>>>> }
>>>>
>>>> func Benchmark2(b *testing.B) {
>>>> for i := 0; i < b.N; i++{
>>>> Foo1 = foo(Count)
>>>> Foo2 = foo(Count)
>>>> }
>>>> }
>>>>
>>>> func Benchmark3(b *testing.B) {
>>>> for i := 0; i < b.N; i++{
>>>> b.StopTimer()
>>>> Foo1 = foo(Count)
>>>> b.StartTimer()
>>>> Foo2 = foo(Count)
>>>> }
>>>> }
>>>>
>>>> func foo(count int) []string{
>>>> testData := []string{}
>>>> for i:= 0; i < count; i++ {
>>>> testData = append(testData, strconv.Itoa(i))
>>>> }
>>>>
>>>> return testData
>>>> }

Re: [go-nuts] Bencharking issue with b.StartTimer and b.StopTimer. Was: Benchmark using b.StopTimer and b.StartTimer has unexpected behaviour - to me at least

[Michael Jones]

You use them to stop the time charged against your benchmark.

For example:

bench:
  stop timer
  generate initial data
  start timer
  do test

On Tue, Mar 24, 2020 at 10:47 AM Orson Cart 
wrote:

>
> On Tuesday, 24 March 2020 16:49:55 UTC, Robert Engels wrote:
>>
>> Can you please succinctly explain the problem?
>>
>
> Let's see. Benchmarks can yield incorrect results when b.StopTimer and
> b.StartTimer are used.
>
> My reasoning:
>
> 1/ I have a benchmark that calls a single function. The reported duration
> is T.
> 2/ I have another benchmark which invokes the same function twice. I'd
> expect the reported duration for this to be nominally 2T and within reason
> it appears to be the case.
> 3/ I have yet another benchmark which also invokes the same function twice
> but it stops the benchmark timer before the first call and then starts it
> after the the first call has completed. I'd expect the reported duration to
> be nominally T again. It isn't. It's closer to 3T on my hardware and I've
> seen much worse correlation on other hardware - almost 5T
>
> Now it's entirely possible that I'm misunderstanding something about how
> b.StopTimer and b.StartTimer are intended to be used, hence my post here :)
>
>
>
>
>
>
>
>>
>>
>> On Mar 24, 2020, at 11:24 AM, Orson Cart  wrote:
>>
>> 
>> I posted this earlier but I realised that the code had a fundamental
>> error in it. I've corrected here it but the underlying problem still exists.
>>
>> I've recently started using go test's benchmarks support and I'm
>> particularly interested in understanding the benchmark timer functions.
>> I've been getting results that I found surprising and I was wondering if
>> anyone could explain what's going on here.
>>
>> The code below has three benchmarks that each invoke a single function
>> (foo). The implementation of foo isn't important, it's just there to
>> consume some time:
>> - foo is called once per iteration in Benchmark1.
>> - It's called twice per iteration in Benchmark2 so I'd expect
>> Benchmark2's duration to be nominally twice that of Benchmark1.
>> - It's also called twice per iteration in Benchmark3 but the first call
>> is wrapped in b.StopTimer and b.startTimer calls. Because of this I'd have
>> expected Benchmark3 to be about the same duration as Benchmark1
>>
>> Apologies for the length of the example but I didn't think it fair to ask
>> the question and leave anything out.
>>
>> package demo_test
>>
>> import (
>> "strconv"
>> "testing"
>> )
>>
>> var Foo1 []string
>> var Foo2 []string
>> var Count int = 32767
>>
>> func Benchmark1(b *testing.B) {
>> for i := 0; i < b.N; i++{
>> Foo1 = foo(Count)
>> }
>> }
>>
>> func Benchmark2(b *testing.B) {
>> for i := 0; i < b.N; i++{
>> Foo1 = foo(Count)
>> Foo2 = foo(Count)
>> }
>> }
>>
>> func Benchmark3(b *testing.B) {
>> for i := 0; i < b.N; i++{
>> b.StopTimer()
>> Foo1 = foo(Count)
>> b.StartTimer()
>> Foo2 = foo(Count)
>> }
>> }
>>
>> func foo(count int) []string{
>> testData := []string{}
>> for i:= 0; i < count; i++ {
>> testData = append(testData, strconv.Itoa(i))
>> }
>>
>> return testData
>> }
>>
>>
>> When the benchmarks are run the results are as follows:
>>
>> Benchmark1-4 351   3345215 ns/op
>> Benchmark2-4 166   7206582 ns/op
>> Benchmark3-4 334   3457907 ns/op
>> PASS
>> ok  bar.com/benchmarks  6.881s
>>
>> OK benchmark3 is a little slower than Benchmark1 but that's not what's
>> bothering me. It's this: if I now change Count to something much smaller
>> the results are a surprise, at least to me. Here are the results when Count
>> = 8:
>>
>> Benchmark1-4 2706196   442 ns/op
>> Benchmark2-4 1357482   873 ns/op
>> Benchmark3-4  840729  1387 ns/op
>> PASS
>> ok  bar.com/benchmarks  23.547s
>>
>> The ratio of timings for Benchmark1 and Benchmark2 are roughly in line
>> with expectations but I was surprised to see that the timings for
>> Benchmark3 are now larger than those for Benchmark2.
>>
>> Can anyone explain this?
>>
>> TIA
>> Orson
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golan...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/38f10940-a770-4fc5-86a9-19cc1371152a%40googlegroups.com
>> 
>> .
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/1d291cef-420a-4f80-b719-e0654585c25d%40googlegroups.com
> 

Re: [go-nuts] Re: Mem-Leak in Go method

[Michael Jones]

hi. get the time at the start, check the elapsed time in your infinite
loop, and trigger the write/exit after a minute, 10 minutes, 100 minutes,
...

On Fri, Mar 13, 2020 at 5:45 AM Nitish Saboo 
wrote:

> Hi Michael,
>
> Thanks for your response.
>
> That code looks wrong. I see the end but not the start. Look here and copy
> carefully:
>
> >>Since I did not want cpu profiling I omitted the start of the code and
> just added memory profiling part.
>
> Call at end, on way out.
>
> >>Oh yes, I missed that.I have to call memory profiling code at the end on
> the way out.But the thing is that it runs as a service in infinite for loop.
>
> func main() {
> flag.Parse()
> if *cpuprofile != "" {
> f, err := os.Create(*cpuprofile)
> if err != nil {
> fmt.Println("could not create CPU profile: ", err)
> }
> defer f.Close() // error handling omitted for example
> if err := pprof.StartCPUProfile(f); err != nil {
> fmt.Print("could not start CPU profile: ", err)
> }
> defer pprof.StopCPUProfile()
> }
>
> A_chan := make(chan bool)
> B_chan := make(chan bool)
> go util.A(A_chan)
> go util.B(B_chan)
> (..Rest of the code..)
>
> for {
> select {
> case <-A_chan:
> continue
> case <-B_chan:
> continue
>
> }
> }
>
> }
>
> What would be the correct way to add the memprofile code changes, since it
> is running in an infinite for loop ?
>
> Also, as shared by others above, there are no promises about how soon the
> dead allocations go away, The speed gets faster and faster version to
> version, and is impressive indeed now, so old versions are not the best to
> use, ubt even so, if the allocation feels small to th GC the urgency to
> free it will be low. You need to loop in allocating and see if the memory
> grows and grows.
>
> >> Yes, got it.I will try using the latest version of Go and check the
> behavior.
>
> Thanks,
> Nitish
>
> On Fri, Mar 13, 2020 at 6:20 AM Michael Jones 
> wrote:
>
>> That code looks wrong. I see the end but not the start. Look here and
>> copy carefully:
>> https://golang.org/pkg/runtime/pprof/
>>
>> Call at end, on way out.
>>
>> Also, as shared by others above, there are no promises about how soon the
>> dead allocations go away, The speed gets faster and faster version to
>> version, and is impressive indeed now, so old versions are not the best to
>> use, ubt even so, if the allocation feels small to th GC the urgency to
>> free it will be low. You need to loop in allocating and see if the memory
>> grows and grows.
>>
>> On Thu, Mar 12, 2020 at 9:22 AM Nitish Saboo 
>> wrote:
>>
>>> Hi,
>>>
>>> I have compiled my Go binary against go version 'go1.7 linux/amd64'.
>>> I added the following code change in the main function to get the memory
>>> profiling of my service
>>>
>>> var memprofile = flag.String("memprofile", "", "write memory profile to
>>> `file`")
>>>
>>> func main() {
>>> flag.Parse()
>>> if *memprofile != "" {
>>> f, err := os.Create(*memprofile)
>>> if err != nil {
>>> fmt.Println("could not create memory profile: ", err)
>>> }
>>> defer f.Close() // error handling omitted for example
>>> runtime.GC() // get up-to-date statistics
>>> if err := pprof.WriteHeapProfile(f); err != nil {
>>> fmt.Println("could not write memory profile: ", err)
>>> }
>>> }
>>> ..
>>> ..
>>> (Rest code to follow)
>>>
>>> I ran the binary with the following command:
>>>
>>> nsaboo@ubuntu:./main -memprofile=mem.prof
>>>
>>> After running the service for couple of minutes, I stopped it and got
>>> the file 'mem.prof'
>>>
>>> 1)mem.prof contains the following:
>>>
>>> nsaboo@ubuntu:~/Desktop/memprof$ vim mem.prof
>>>
>>> heap profile: 0: 0 [0: 0] @ heap/1048576
>>>
>>> # runtime.MemStats
>>> # Alloc = 761184
>>> # TotalAlloc = 1160960
>>> # Sys = 3149824
>>> # Lookups = 10
>>> # Mallocs = 8358
>>> # Frees = 1981
>>> # HeapAlloc = 761184
>>> # HeapSys = 1802240
>>> # HeapIdle = 499712
>>> # HeapInuse = 1302528
>>> # HeapReleased = 0
>>> # HeapObjects = 6377
>>> # Stack = 294912 / 294912
>>> # MSpan = 22560 / 32768
>>> # MCache = 2400 / 16384
>>> # BuckHashSys = 2727
>>> # NextGC = 4194304
>>> # Pau

Re: [go-nuts] Re: Mem-Leak in Go method

[Michael Jones]

That code looks wrong. I see the end but not the start. Look here and copy
carefully:
https://golang.org/pkg/runtime/pprof/

Call at end, on way out.

Also, as shared by others above, there are no promises about how soon the
dead allocations go away, The speed gets faster and faster version to
version, and is impressive indeed now, so old versions are not the best to
use, ubt even so, if the allocation feels small to th GC the urgency to
free it will be low. You need to loop in allocating and see if the memory
grows and grows.

On Thu, Mar 12, 2020 at 9:22 AM Nitish Saboo 
wrote:

> Hi,
>
> I have compiled my Go binary against go version 'go1.7 linux/amd64'.
> I added the following code change in the main function to get the memory
> profiling of my service
>
> var memprofile = flag.String("memprofile", "", "write memory profile to
> `file`")
>
> func main() {
> flag.Parse()
> if *memprofile != "" {
> f, err := os.Create(*memprofile)
> if err != nil {
> fmt.Println("could not create memory profile: ", err)
> }
> defer f.Close() // error handling omitted for example
> runtime.GC() // get up-to-date statistics
> if err := pprof.WriteHeapProfile(f); err != nil {
> fmt.Println("could not write memory profile: ", err)
> }
> }
> ..
> ..
> (Rest code to follow)
>
> I ran the binary with the following command:
>
> nsaboo@ubuntu:./main -memprofile=mem.prof
>
> After running the service for couple of minutes, I stopped it and got the
> file 'mem.prof'
>
> 1)mem.prof contains the following:
>
> nsaboo@ubuntu:~/Desktop/memprof$ vim mem.prof
>
> heap profile: 0: 0 [0: 0] @ heap/1048576
>
> # runtime.MemStats
> # Alloc = 761184
> # TotalAlloc = 1160960
> # Sys = 3149824
> # Lookups = 10
> # Mallocs = 8358
> # Frees = 1981
> # HeapAlloc = 761184
> # HeapSys = 1802240
> # HeapIdle = 499712
> # HeapInuse = 1302528
> # HeapReleased = 0
> # HeapObjects = 6377
> # Stack = 294912 / 294912
> # MSpan = 22560 / 32768
> # MCache = 2400 / 16384
> # BuckHashSys = 2727
> # NextGC = 4194304
> # PauseNs = [752083 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
> # NumGC = 1
> # DebugGC = false
>
> 2)When I tried to open the file using the following command, it just goes
> into interactive mode and shows nothing
>
> a)Output from go version go1.7 linux/amd64 for mem.prof
>
> nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof mem.prof
> Entering interactive mode (type "help" for commands)
> (pprof) top
> profile is empty
> (pprof)
>
> b)Output from go version go1.12.4 linux/amd64 for mem.prof
>
> nsaboo@ubuntu:~/Desktop/memprof$ go tool pprof mem.prof
> Type: space
> No samples were found with the default sample value type.
> Try "sample_index" command to analyze different sample values.
> Entering interactive mode (type "help" for commands, "o" for options)
> (pprof) o
>   call_tree = false
>   compact_labels= true
>   cumulative= flat //: [cum | flat]
>   divide_by = 1
>   drop_negative = false
>   edgefraction  = 0.001
>   focus = ""
>   granularity   = functions//: [addresses |
> filefunctions | files | functions | lines]
>   hide  = ""
>   ignore= ""
>   mean  = false
>   nodecount = -1   //: default
>   nodefraction  = 0.005
>   noinlines = false
>   normalize = false
>   output= ""
>   prune_from= ""
>   relative_percentages  = false
>   sample_index  = space//: [objects | space]
>   show  = ""
>   show_from = ""
>   tagfocus  = ""
>   taghide   = ""
>   tagignore = ""
>   tagshow   = ""
>   trim  = true
>   trim_path = ""
>   unit  = minimum
> (pprof) space
> (pprof) sample_index
> (pprof) top
> Showing nodes accounting for 0, 0% of 0 total
>   flat  flat%   sum%cum   cum%
>
>
> 3)Please let me know if it is this the correct way of getting the memory
> profiling ?
>
> 4)Can we deduce something from this memory stats that points us to
> increase in memory usage?
>
> 5)I am just thinking out loud, since I am using go1.7, can that be the
> reason for the issue of increase in memory usage that might get fixed with
> latest go versions ?
>
> Thanks,
> Nitish
>
> On 

Re: [go-nuts] Re: An important proposal will fail without your support

[Michael Jones]

I wrote some very efficient math software in Go. To accomplish my goals, I
used make and awk programs that generate assembly code tuned to the exact
demands of command line make configuration. Then it builds in go, then
runs.

This is not mainstream, but has worked well since Go 1.0. It never seemed
to me that it was something everyone needs, so that means it being “worse”
for me but simpler for a million others might be a reasonable trade off.

Is there anything of this kind at play here? Can it not be done perfectly
using m4, or cpp, or ...

On Tue, Mar 10, 2020 at 11:21 AM Brian Candler  wrote:

> Sorry, somehow I scrolled over the definition of func #gettext.  So: this
> function emits literal source code text to be inserted into the compilation
> point.
>
> There are interesting semantic implications here: the args can't be
> evaluated at compile time (unless they are constant expressions), but need
> to be passed as strings representing the expression plus a type.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/5024a878-1392-4f1e-ac48-799f0cf5eee8%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyo3D10Tut9nJ10m3p3sJ9fExH9KxWOYWJj9_%2B-itDV1g%40mail.gmail.com.

Re: [go-nuts] How to handle EINTR from syscall.Dup2

[Michael Jones]

...and if true, this notion of “it’s atomic unless it’s not” deserves a
name: the “subatomic operation.”

On Sun, Mar 1, 2020 at 6:56 AM Ian Lance Taylor  wrote:

> On Sat, Feb 29, 2020 at 9:45 PM Philip Boampong 
> wrote:
> >
> > > And dup2 is not documented to return EINTR
> >
> > Yes, it is. In the same man page you quoted [1].
> > |  EINTR  The dup2() or dup3() call was interrupted by a signal; see
> > | signal(7).
> >
> > > and the Linux kernel code shown above does not have any path that
> > > returns EINTR.
> >
> > That's a relief, but I would be more comfortable if the man page was
> accurate.
> >
> > > dup2 is documented to atomically close newfd and duplicate oldfd
> > > onto it.  That means that either newfd is untouched, or oldfd is
> > > duplicated onto it.
> >
> > If that is true, there is indeed no problem in retrying on EINTR, but
> > (as I explained two times already) someone disagrees with your
> > premise, notably the python standard library implementation [2].
> >
> > From PEP 475 [3]:
> > | os.close, close() methods and os.dup2() are a special case: they will
> > | ignore EINTR instead of retrying. The reason is complex but involves
> > | behaviour under Linux and the fact that the file descriptor may really
> > | be closed even if EINTR is returned.
> >
> > "the file descriptor may really be closed even if EINTR is returned",
> > hence the alleged race if dup2 is retried.
> >
> > libuv is of the same opinion [4].
> >
> > That's why I was asking for more evidence about the exact meaning of
> > "performed atomically". It could just mean that no one can "steal"
> > newfd between close and duplication, even though your stronger
> > interpretation makes sense and seems to reflect the actual kernel
> > code.
> >
> > Maybe the python folks are wrongly assuming that errors from the
> > implicit close are reported by dup2 (the man clearly says that they
> > are not).
> >
> > [1] http://man7.org/linux/man-pages/man2/dup2.2.html
> > [2]
> https://github.com/python/cpython/blob/6e02691f300c9918ac5806dafa1f2ecef451d733/Modules/posixmodule.c#L8730
> > [3] https://www.python.org/dev/peps/pep-0475/
> > [4] https://github.com/libuv/libuv/issues/462
>
>
> If dup2 can 1) close newfd; 2) receive a signal before duping oldfd to
> newfd; 3) return EINTR leaving newfd closed, then dup2 requires
> considerable care in any multi-threaded program.  It requires that if
> one thread is calling dup2, no other thread is permitted to open a
> file or socket or other file descriptor.  That seems both unfortunate
> and unbelievable.  I would like to see hard evidence before believing
> that kernel developers for any OS would create a system with such a
> bug.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUbftML8sERv1B9CMa0tKRi0y2Nf%2BRJgP%2B7zQhJ7ZsXzA%40mail.gmail.com
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxK5%3DErxB8L1YPKdW4Ca5_D3Zq0T68vB1O1EsJsdPeE1Q%40mail.gmail.com.

Re: [go-nuts] Re: Lot's of test errors in package zmq4 with Go version 1.14, no errors with earlier versions

[Michael Jones]

Ian, I guess not. I’d not thought of that indirectness. I withdraw my
musings.

On Wed, Feb 26, 2020 at 8:36 PM Robert Engels  wrote:

> The problem is that Go designers are taking the position that any sys call
> should be able to be interrupted. This is invalid. For the vast majority or
> “unix” os an interrupt is a very rare condition and so they treat it as an
> error. If you issue interrupts continually you are creating an unexpected
> context.
>
> > On Feb 26, 2020, at 8:39 PM, Ian Lance Taylor  wrote:
> >
> > On Wed, Feb 26, 2020 at 5:51 PM Michael Jones 
> wrote:
> >>
> >> Sorry...I meant the go system signal interface could loop if desired.
> (Not recommending, just saying that panicky people could be coddled if
> desired)
> >
> > Ah, I see.  Except, no, I don't.  Could we really do that?  Even if
> > the signal arrived while executing some function written in C and
> > called via cgo?
> >
> > Ian
> >
> >
> >>> On Wed, Feb 26, 2020 at 5:48 PM Ian Lance Taylor 
> wrote:
> >>>
> >>> On Wed, Feb 26, 2020 at 5:42 PM Michael Jones 
> wrote:
> >>>>
> >>>> There is the BSD notion of sa_restart, a convenience to loop for the
> caller as appropriate.
> >>>>
> >>>> https://www.freebsd.org/cgi/man.cgi?query=sigaction
> >>>>
> >>>> Go could adopt such a notion if desired.
> >>>
> >>> We already do.  We install all signal handlers with the SA_RESTART
> flag set.
> >>>
> >>> Unfortunately if you peruse "man 7 signal" on a GNU/Linux system you
> >>> will see a list of system calls that return EINTR even if the handler
> >>> has SA_RESTART set.
> >>>
> >>> Ian
> >>>
> >>>> On Wed, Feb 26, 2020 at 5:14 PM Ian Lance Taylor 
> wrote:
> >>>>>
> >>>>> On Wed, Feb 26, 2020 at 9:11 AM Manlio Perillo <
> manlio.peri...@gmail.com> wrote:
> >>>>>>
> >>>>>> On Wednesday, February 26, 2020 at 4:14:38 PM UTC+1, Ian Lance
> Taylor wrote:
> >>>>>>>
> >>>>>>> On Wed, Feb 26, 2020 at 7:11 AM Manlio Perillo <
> manlio...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>> On Wednesday, February 26, 2020 at 3:51:54 PM UTC+1, Peter
> Kleiweg wrote:
> >>>>>>>>>
> >>>>>>>>> Op woensdag 26 februari 2020 13:05:40 UTC+1 schreef Manlio
> Perillo:
> >>>>>>>>>>
> >>>>>>>>>> On Wednesday, February 26, 2020 at 12:33:05 PM UTC+1, Peter
> Kleiweg wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> With Go version 1.14 I get a lot of errors when I run:
> >>>>>>>>>>>
> >>>>>>>>>>>go test -v github.com/pebbe/zmq4
> >>>>>>>>>>>
> >>>>>>>>>>> I didn't see this with Go 1.13.8 or any earlier version.
> >>>>>>>>>>>
> >>>>>>>>>>> Is this a problem with Go 1.14, or am I doing something wrong
> and just got lucky until now?
> >>>>>>>>>>>
> >>>>>>>>>>> How do I debug this? The errors are different for each run.
> Below is a sample of some errors.
> >>>>>>>>>>> Line numbers are not always accurate, because I inserted some
> calls to test.Log().
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> The errors are probably caused by
> https://golang.org/doc/go1.14#runtime.
> >>>>>>>>>>
> >>>>>>>>>> The solution is to update zmq4  to explicitly handle
> interrupted system calls.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Often the program freezes before I get an interrupted system
> call. It hangs inside a ZeroMQ C++ library function.
> >>>>>>>>> zmq4 is just a wrapper for ZeroMQ. I can't "fix" ZeroMQ to make
> it work with Go.
> >>>>>>>>>
> >>>>>>>>> Is there a way to stop Go from interrupting my system calls? It
> happens rather randomly all over the place.
> >>>>>>>>
> >>>>>>>&g

Re: [go-nuts] Re: Lot's of test errors in package zmq4 with Go version 1.14, no errors with earlier versions

[Michael Jones]

Sorry...I meant the go system signal interface could loop if desired. (Not
recommending, just saying that panicky people could be coddled if desired)

On Wed, Feb 26, 2020 at 5:48 PM Ian Lance Taylor  wrote:

> On Wed, Feb 26, 2020 at 5:42 PM Michael Jones 
> wrote:
> >
> > There is the BSD notion of sa_restart, a convenience to loop for the
> caller as appropriate.
> >
> > https://www.freebsd.org/cgi/man.cgi?query=sigaction
> >
> > Go could adopt such a notion if desired.
>
> We already do.  We install all signal handlers with the SA_RESTART flag
> set.
>
> Unfortunately if you peruse "man 7 signal" on a GNU/Linux system you
> will see a list of system calls that return EINTR even if the handler
> has SA_RESTART set.
>
> Ian
>
> > On Wed, Feb 26, 2020 at 5:14 PM Ian Lance Taylor 
> wrote:
> >>
> >> On Wed, Feb 26, 2020 at 9:11 AM Manlio Perillo <
> manlio.peri...@gmail.com> wrote:
> >> >
> >> > On Wednesday, February 26, 2020 at 4:14:38 PM UTC+1, Ian Lance Taylor
> wrote:
> >> >>
> >> >> On Wed, Feb 26, 2020 at 7:11 AM Manlio Perillo 
> wrote:
> >> >> >
> >> >> > On Wednesday, February 26, 2020 at 3:51:54 PM UTC+1, Peter Kleiweg
> wrote:
> >> >> >>
> >> >> >> Op woensdag 26 februari 2020 13:05:40 UTC+1 schreef Manlio
> Perillo:
> >> >> >>>
> >> >> >>> On Wednesday, February 26, 2020 at 12:33:05 PM UTC+1, Peter
> Kleiweg wrote:
> >> >> >>>>
> >> >> >>>> With Go version 1.14 I get a lot of errors when I run:
> >> >> >>>>
> >> >> >>>> go test -v github.com/pebbe/zmq4
> >> >> >>>>
> >> >> >>>> I didn't see this with Go 1.13.8 or any earlier version.
> >> >> >>>>
> >> >> >>>> Is this a problem with Go 1.14, or am I doing something wrong
> and just got lucky until now?
> >> >> >>>>
> >> >> >>>> How do I debug this? The errors are different for each run.
> Below is a sample of some errors.
> >> >> >>>> Line numbers are not always accurate, because I inserted some
> calls to test.Log().
> >> >> >>>
> >> >> >>>
> >> >> >>> The errors are probably caused by
> https://golang.org/doc/go1.14#runtime.
> >> >> >>>
> >> >> >>> The solution is to update zmq4  to explicitly handle interrupted
> system calls.
> >> >> >>
> >> >> >>
> >> >> >> Often the program freezes before I get an interrupted system
> call. It hangs inside a ZeroMQ C++ library function.
> >> >> >> zmq4 is just a wrapper for ZeroMQ. I can't "fix" ZeroMQ to make
> it work with Go.
> >> >> >>
> >> >> >> Is there a way to stop Go from interrupting my system calls? It
> happens rather randomly all over the place.
> >> >> >
> >> >> >
> >> >> >
> https://stackoverflow.com/questions/36040547/zeromq-how-to-react-on-different-signal-types-on-eintr
> >> >> >
> >> >> > ZeroMQ may return an EINTR error , but zmq4 does not list it in
> errors.go.
> >> >> > ZeroMQ asks the caller to handle EINTR, so zmq4 should handle it
> internally or return it to the caller.
> >> >> >
> >> >> > https://golang.org/doc/go1.14#runtime should have mentioned that
> not only programs that use packages like syscall or golang.org/x/sys/unix
> will see more slow system calls fail with EINTR errors, but also programs
> that use Cgo.
> >> >>
> >> >> I don't know ZeroMQ.  If the ZeroMQ calls correspond closely to
> system
> >> >> calls, then it could work for them to return EINTR.  In that case the
> >> >> fix is going to be for the Go wrapper around ZeroMQ to check whether
> >> >> the error returned is syscall.EINTR, and to retry the call if it is.
> >> >>
> >> >
> >> > Unfortunately it is not that simple:
> >> >
> >> > http://250bpm.com/blog:12
> >> > https://alobbs.com/post/54503240599/close-and-eintr
> >> > http://man7.org/linux/man-pages/man7/signal.7.html
> >> > https://github.com/golang/go/issues/11180
> >> > https://www.python.org/dev/peps/pep-0475/
&

Re: [go-nuts] Re: Lot's of test errors in package zmq4 with Go version 1.14, no errors with earlier versions

[Michael Jones]

There is the BSD notion of sa_restart, a convenience to loop for the caller
as appropriate.

https://www.freebsd.org/cgi/man.cgi?query=sigaction

Go could adopt such a notion if desired.

On Wed, Feb 26, 2020 at 5:14 PM Ian Lance Taylor  wrote:

> On Wed, Feb 26, 2020 at 9:11 AM Manlio Perillo 
> wrote:
> >
> > On Wednesday, February 26, 2020 at 4:14:38 PM UTC+1, Ian Lance Taylor
> wrote:
> >>
> >> On Wed, Feb 26, 2020 at 7:11 AM Manlio Perillo 
> wrote:
> >> >
> >> > On Wednesday, February 26, 2020 at 3:51:54 PM UTC+1, Peter Kleiweg
> wrote:
> >> >>
> >> >> Op woensdag 26 februari 2020 13:05:40 UTC+1 schreef Manlio Perillo:
> >> >>>
> >> >>> On Wednesday, February 26, 2020 at 12:33:05 PM UTC+1, Peter Kleiweg
> wrote:
> >> 
> >>  With Go version 1.14 I get a lot of errors when I run:
> >> 
> >>  go test -v github.com/pebbe/zmq4
> >> 
> >>  I didn't see this with Go 1.13.8 or any earlier version.
> >> 
> >>  Is this a problem with Go 1.14, or am I doing something wrong and
> just got lucky until now?
> >> 
> >>  How do I debug this? The errors are different for each run. Below
> is a sample of some errors.
> >>  Line numbers are not always accurate, because I inserted some
> calls to test.Log().
> >> >>>
> >> >>>
> >> >>> The errors are probably caused by
> https://golang.org/doc/go1.14#runtime.
> >> >>>
> >> >>> The solution is to update zmq4  to explicitly handle interrupted
> system calls.
> >> >>
> >> >>
> >> >> Often the program freezes before I get an interrupted system call.
> It hangs inside a ZeroMQ C++ library function.
> >> >> zmq4 is just a wrapper for ZeroMQ. I can't "fix" ZeroMQ to make it
> work with Go.
> >> >>
> >> >> Is there a way to stop Go from interrupting my system calls? It
> happens rather randomly all over the place.
> >> >
> >> >
> >> >
> https://stackoverflow.com/questions/36040547/zeromq-how-to-react-on-different-signal-types-on-eintr
> >> >
> >> > ZeroMQ may return an EINTR error , but zmq4 does not list it in
> errors.go.
> >> > ZeroMQ asks the caller to handle EINTR, so zmq4 should handle it
> internally or return it to the caller.
> >> >
> >> > https://golang.org/doc/go1.14#runtime should have mentioned that not
> only programs that use packages like syscall or golang.org/x/sys/unix
> will see more slow system calls fail with EINTR errors, but also programs
> that use Cgo.
> >>
> >> I don't know ZeroMQ.  If the ZeroMQ calls correspond closely to system
> >> calls, then it could work for them to return EINTR.  In that case the
> >> fix is going to be for the Go wrapper around ZeroMQ to check whether
> >> the error returned is syscall.EINTR, and to retry the call if it is.
> >>
> >
> > Unfortunately it is not that simple:
> >
> > http://250bpm.com/blog:12
> > https://alobbs.com/post/54503240599/close-and-eintr
> > http://man7.org/linux/man-pages/man7/signal.7.html
> > https://github.com/golang/go/issues/11180
> > https://www.python.org/dev/peps/pep-0475/
> >
> > The second entry about close and EINTR is enlightening.
>
> Thanks for the links.  Note that these issues don't really have
> anything to do with Go.  For certain system calls, you need to handle
> EINTR one way or another.  The Go runtime does as much as it can to
> avoid these problems, but on Unix systems it is impossible to avoid
> them entirely.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXGcdfpPAz8t8adfqGodKPjZNxKzkTUyB0b4L1zysVFSQ%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzADfr-DxiAMkPN8JGpPiFux8tJX%2Bf2vB6E-HLMe%2Bsmuw%40mail.gmail.com.

Re: [go-nuts] Fuchsia Programming Language Policy

[Michael Jones]

Actually, you should read the whole note -- it's fun. Half of the languages
are bad because of memory leaks, the other half are bad because of having
GC; half are bad because of difficult asynchronism, the other half are bad
because of having a runtime. etc.

It reads like an imperiously-worded tautology about
safety/power/convenience coming at a cost.

On Tue, Feb 25, 2020 at 9:51 AM Mohamed Yousif  wrote:

> It seems they are betting high on Dart/flutter and their front end is
> already written with flutter. The assessment seems to be pretty much the
> same as for Dart.
>
> Dart won with the ui side, while go was competing with C.
>
> On Tue, 25 Feb 2020 at 7:22 PM, Jon Conradt  wrote:
>
>> The Fuchsia Programming Language Policy
>> 
>>  gives
>> some insight into the experience the Fuchsia team has had with Go, and it
>> doesn't sound good.
>>
>> "The Fuchsia Platform Source Tree has had negative implementation
>> experience using Go. The system components the Fuchsia project has built in
>> Go have used more memory and kernel resources than their counterparts (or
>> replacements) the Fuchsia project has built using C++ or Rust."
>>
>>
>> The Fuchsia Platform Source tree is defined as "The *Fuchsia Platform
>> Source Tree* is the source code hosted on fuchsia.googlesource.com."
>>
>> Their conclusion, and each language has some issues is pretty severe.
>>
>>- Go is not approved, with the following exceptions:
>>   - *netstack*. Migrating netstack to another language would require
>>   a significant investment. In the fullness of time, we should migrate
>>   netstack to an approved language.
>>- All other uses of Go in Fuchsia for production software on the
>>target device must be migrated to an approved language.
>>
>>  That's a shame. I was hoping that Fuchsia would provide a way for Go to
>> have a nice GUI.
>>
>> Two of the issues listed as cons include the toolchain producing 'large
>> binaries' and the related issue of their being a 'substantial runtime.' It
>> seems to me that both of these issues can be addressed through some of the
>> techniques used to build tiny Docker images from Go, but I suspect they
>> would like to have a much simpler route, e.g. a go build flag.
>>
>> Jon
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/7778a387-f1f5-4ed0-8453-5b811bac4a6d%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAHrL7wHqJnDnEVe4%3D--%3DcSW9oA-eYxcbKagESdZHgSkrdLutpA%40mail.gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQx0mMnuKQHOrp9ptxM1iN5y9%2B3rXYDZ4TaOYK%3DfXmq4Sg%40mail.gmail.com.

Re: [go-nuts] Go without garbage collector

[Michael Jones]

To me it seems the issue of concurrency and dynamic ownership of memory are
so deeply connected to Go’s programming methodology that the “no GC”
comparison is biased.

In particular, coding to do it yourself but as perfectly as the GC across
many concurrent routines is hard. Doing it better than the GC is hard.
Caution encourages use of the tuned GC.

Agree with posts above: preallocation is fastest. Hard real time from the
80s lesson.

On Wed, Feb 12, 2020 at 1:07 PM  wrote:

> I'm very familiar with this paper. It's not the first one that uses
> oracular memory management for comparison, the earlier one used ML as its
> langauge.
>
> The problem with these papers is that they're using very artificial
> benchmarks, not really representative of real workloads. They additionally
> use languages that are very heap-oriented, with very few value objects.
>
> GCs also have not radically improved since then, if anything they are
> worse now in massively-parallel environment than on single-core CPUs of
> yore.
>
> On Tuesday, February 11, 2020 at 8:54:29 PM UTC-8, robert engels wrote:
>>
>> Here is a paper from 2005
>> https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf that proves
>> otherwise.
>>
>> GC techniques have radically improved since then, some with hardware
>> support, so much so that it is no longer a contest.
>>
>> To reiterate though, if you don’t have dynamic memory management - which
>> is essentially allocate and forget - that will “probably" be faster (many
>> GC systems have an extra level of indirection).
>>
>> You can write robust systems without dynamic memory, but it is very very
>> difficult - beyond the skills of most developers.
>>
>> So most developers resort to dynamic memory at some point - and once you
>> do that - GC will crush your manual memory management techniques.
>>
>> On Feb 11, 2020, at 10:31 PM, alex.b...@gmail.com wrote:
>>
>> Actually, it was not proven. And in practice manual memory management
>> seems to be outperforming GC in majority of cases.
>>
>> On Tuesday, February 11, 2020 at 5:59:26 PM UTC-8, robert engels wrote:
>>>
>>> It’s been PROVEN that GC outperforms all manual memory management except
>>> in EXTREMELY isolated cases (very non-traditional allocation or
>>> deallocation patterns).
>>>
>>> It’s all about constraints and tolerances.
>>>
>>> You design a “system” that takes both into account - if not, you’re not
>>> engineering, you're guessing.
>>>
>>> On Feb 11, 2020, at 4:29 AM, deat...@gmail.com wrote:
>>>
>>> What about #vlang ? https://vlang.io/
>>>
>>> On Sunday, 17 June 2012 22:40:30 UTC+2, nsf wrote:

 On Sun, 17 Jun 2012 11:48:53 -0700 (PDT)
 ⚛ <0xe2.0...@gmail.com> wrote:

 > > You can't have Go syntax without a garbage collector.
 > >
 >
 > I wouldn't be so sure about it.
 >

 Let me rephrase myself. When someone says "I want Go without garbage
 collection" it means a person wants a feel he has with Go, but at the
 same time without garbage collection. At least that's my case. I wanted
 exactly that. And you can't have that. You can build a language similar
 to Go without GC, but you won't get a feel of Go. At least, I couldn't
 do it. And maybe it's kind of obvious, but when there is a need to
 manage memory, that factor alone creates a different programmer
 mindset.
 And in my opinion what Go does so well for a programmer is establishing
 its own mindset that gives a very nice and smooth development process.
 What we call "a feel of Go".

 That's actually very same mistake that leads to talks like "where is my
 feature X? I want feature X in your language". And the problem here is
 that a language is not just a collection of features, it's a
 composition of features. You can't just stick something in and make it
 better (see C++) and you can't throw something out. Every feature
 addition/removal affects the language as a whole, mutating it to a
 different state. And in my opinion GC is a critical feature that allows
 you to have memory safety and (well, let's put it that way) memory
 safety is one of the major features in Go.

 So.. think about it. "I want Go with templates" and "I want Go without
 garbage collection" are very similar things. Both hide the desire of
 improving/changing something without realization that this will affect
 other areas dramatically.

 And to make a summary: I tried that, I did that mistake thinking you
 can build something out of Go just by taking parts you like and mixing
 them in some weird way. I was stupid (to make it clear, I'm not
 implying that anyone is). Hopefully what I said makes some sense.


 Offtopic:

 Btw. Thanks for your work on GC precision, I really hope those patches
 will get into Go. One of the areas where I want to apply Go is desktop
 applications. And for these you need a precise GC, because some 

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Michael Jones]

forgot the code...final test is what generates the report above.

float128.go: https://play.golang.org/p/2MzjllLC3qT
float128_test.go: https://play.golang.org/p/ZQwsFh0_Apt

On Sat, Feb 1, 2020 at 7:32 PM Michael Jones 
wrote:

> As promised, some careful results from one of the tests in my
> doubled-precision library.
>
> Rows are iteration's of Jean-Michel Muller's roundoff-intolerant Œ()
> function:
>
> http://perso.ens-lyon.fr/jean-michel.muller/
>
> Columns are 32-bit floating point,64-bit floating point, my fast 128-bit
> library, Go big float, and Go rationals (printed as float)
>
> Answers are good for a while, and then fail as they must. compare always
> to the right column.
>
> Also, Dr. Kahan on the topic -- always read his every word!
>
> How Futile are Mindless Assessments of Roundoff In Floating-Point
> Computation ?
> Professor W. Kahan
> § 5, page 14
> https://www.cs.berkeley.edu/~wkahan/Mindless.pdf
>
>
> === RUN   TestMuller
>   0:   4.0   4.00
> +4.000e+00 4.000
> 4.00…
>   1:   4.25000   4.25
> +4.250e+00 4.250
> 4.25…
>   2:   4.470588684   4.470588235294115975
> +4.4705882352941176470588235294117e+00 4.4705882352941176470588235294118
> 4.4705882352941176470588235294117647058823529411764705882352941176470588…
>   3:   4.644744873   4.644736842105217534
> +4.6447368421052631578947368421047e+00 4.6447368421052631578947368421053
> 4.6447368421052631578947368421052631578947368421052631578947368421052632…
>   4:   4.770706177   4.770538243625082941
> +4.7705382436260623229461756373816e+00 4.7705382436260623229461756373938
> 4.7705382436260623229461756373937677053824362606232294617563739376770538…
>   5:   4.859214783   4.855700712568562949
> +4.8557007125890736342042755341875e+00 4.8557007125890736342042755344418
> 4.8557007125890736342042755344418052256532066508313539192399049881235154…
>   6:   4.983123779   4.910847498660629640
> +4.9108474990827932004402592585550e+00 4.9108474990827932004402592637887
> 4.9108474990827932004402592637886755533814357343769108474990827932004403…
>   7:   6.395431519   4.945537395530507752
> +4.9455374041239167247733836966321e+00 4.9455374041239167247733838031676
> 4.9455374041239167247733838031676461798983962546070325729654348042633728…
>   8:  27.632629395   4.966962408040998866
> +4.9669625817627005987119363335665e+00 4.9669625817627005987119384872579
> 4.9669625817627005987119384872578590383346845054961655244646085209448470…
>   9:  86.993759155   4.980042204293013697
> +4.9800457013556311612685646399287e+00 4.9800457013556311612686079942904
> 4.9800457013556311612686079942903718963021236734644222853921922457735370…
>  10:  99.255508423   4.987909232795786352
> +4.9879794484783922601392624051427e+00 4.9879794484783922601401328939769
> 4.987979448478392260140132893976940072111033913186330660946103656976…
>  11:  99.962585449   4.991362641314552206
> +4.9927702880620680974721416957309e+00 4.9927702880620680974895925483283
> 4.9927702880620680974895925483282696604561239860070565468956074766812844…
>  12:  99.998130798   4.967455095552267608
> +4.9956558915066340262745164361610e+00 4.9956558915066340266240282615671
> 4.9956558915066340266240282615670560447223264138375322338595040105640829…
>  13:  99.08447   4.429690498308829660
> +4.9973912683813441058976817599037e+00 4.9973912683813441128938690216426
> 4.9973912683813441128938690216425944791795065582105346913888017869771418…
>  14: 100.0  -7.817236578459315410
> +4.9984339439448167790186456309858e+00 4.9984339439448169190138971781902
> 4.9984339439448169190138971781902382881448247682045125641413593279228615…
>  15: 100.0 168.939167671064581100
> +4.9990600719708910670530314075958e+00 4.9990600719708938678168396062869
> 4.9990600719708938678168396062869249162956010827138675312945769521384463…
>  16: 100.0 102.039963152059272034
> +4.9994359371467831224113039612204e+00 4.9994359371468391479978508864998
> 4.9994359371468391479978508864997899689960802031068986343509970885820809…
>  17: 100.0 100.099947516249699220
> +4.9996615241026469022949297281691e+00 4.9996615241037675377868879857271
> 4.9996615241037675377868879857270921122953326315959962489248369318086283…
>  18: 100.0 100.004992040972439327
> +4.9997969006910037174292363662054e+00 4.9997969007134179126629930746330
> 4.9997969007134179126629930746330014204806141590496819671038715851326606…
>  19: 100.0 100.000249579237305397
> +4.999878135029629

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Michael Jones]

As promised, some careful results from one of the tests in my
doubled-precision library.

Rows are iteration's of Jean-Michel Muller's roundoff-intolerant Œ()
function:

http://perso.ens-lyon.fr/jean-michel.muller/

Columns are 32-bit floating point,64-bit floating point, my fast 128-bit
library, Go big float, and Go rationals (printed as float)

Answers are good for a while, and then fail as they must. compare always to
the right column.

Also, Dr. Kahan on the topic -- always read his every word!

How Futile are Mindless Assessments of Roundoff In Floating-Point
Computation ?
Professor W. Kahan
§ 5, page 14
https://www.cs.berkeley.edu/~wkahan/Mindless.pdf


=== RUN   TestMuller
  0:   4.0   4.00
+4.000e+00 4.000
4.00…
  1:   4.25000   4.25
+4.250e+00 4.250
4.25…
  2:   4.470588684   4.470588235294115975
+4.4705882352941176470588235294117e+00 4.4705882352941176470588235294118
4.4705882352941176470588235294117647058823529411764705882352941176470588…
  3:   4.644744873   4.644736842105217534
+4.6447368421052631578947368421047e+00 4.6447368421052631578947368421053
4.6447368421052631578947368421052631578947368421052631578947368421052632…
  4:   4.770706177   4.770538243625082941
+4.7705382436260623229461756373816e+00 4.7705382436260623229461756373938
4.7705382436260623229461756373937677053824362606232294617563739376770538…
  5:   4.859214783   4.855700712568562949
+4.8557007125890736342042755341875e+00 4.8557007125890736342042755344418
4.8557007125890736342042755344418052256532066508313539192399049881235154…
  6:   4.983123779   4.910847498660629640
+4.9108474990827932004402592585550e+00 4.9108474990827932004402592637887
4.9108474990827932004402592637886755533814357343769108474990827932004403…
  7:   6.395431519   4.945537395530507752
+4.9455374041239167247733836966321e+00 4.9455374041239167247733838031676
4.9455374041239167247733838031676461798983962546070325729654348042633728…
  8:  27.632629395   4.966962408040998866
+4.9669625817627005987119363335665e+00 4.9669625817627005987119384872579
4.9669625817627005987119384872578590383346845054961655244646085209448470…
  9:  86.993759155   4.980042204293013697
+4.9800457013556311612685646399287e+00 4.9800457013556311612686079942904
4.9800457013556311612686079942903718963021236734644222853921922457735370…
 10:  99.255508423   4.987909232795786352
+4.9879794484783922601392624051427e+00 4.9879794484783922601401328939769
4.987979448478392260140132893976940072111033913186330660946103656976…
 11:  99.962585449   4.991362641314552206
+4.9927702880620680974721416957309e+00 4.9927702880620680974895925483283
4.9927702880620680974895925483282696604561239860070565468956074766812844…
 12:  99.998130798   4.967455095552267608
+4.9956558915066340262745164361610e+00 4.9956558915066340266240282615671
4.9956558915066340266240282615670560447223264138375322338595040105640829…
 13:  99.08447   4.429690498308829660
+4.9973912683813441058976817599037e+00 4.9973912683813441128938690216426
4.9973912683813441128938690216425944791795065582105346913888017869771418…
 14: 100.0  -7.817236578459315410
+4.9984339439448167790186456309858e+00 4.9984339439448169190138971781902
4.9984339439448169190138971781902382881448247682045125641413593279228615…
 15: 100.0 168.939167671064581100
+4.9990600719708910670530314075958e+00 4.9990600719708938678168396062869
4.9990600719708938678168396062869249162956010827138675312945769521384463…
 16: 100.0 102.039963152059272034
+4.9994359371467831224113039612204e+00 4.9994359371468391479978508864998
4.9994359371468391479978508864997899689960802031068986343509970885820809…
 17: 100.0 100.099947516249699220
+4.9996615241026469022949297281691e+00 4.9996615241037675377868879857271
4.9996615241037675377868879857270921122953326315959962489248369318086283…
 18: 100.0 100.004992040972439327
+4.9997969006910037174292363662054e+00 4.9997969007134179126629930746330
4.9997969007134179126629930746330014204806141590496819671038715851326606…
 19: 100.0 100.000249579237305397
+4.9998781350296295179302235821482e+00 4.9998781354779312492317320300204
4.9998781354779312492317320300204045157850179321411177471298083841044938…
 20: 100.0 100.12478620163847
+4.268705383513445350585201744e+00 4.268795045999044664529810255
4.268795045999044664529810255297667014507019629318651153480676099193…
 21: 100.0 100.00623921607712
+4.559477336189227766623157740e+00 4.561270611577381190152822945
4.561270611577381190152822944886054388799381889725655444194135741792…
 22: 100.0 100.00031195796169
+4.700894239992550386653856177e+00 4.736760057124445790151493568

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Michael Jones]

Your expectations are wrong in the sense of being overly optimistic.

This is a chaotic attractor, under any form of rounding or precision it
will “fail” in the way you mean (and “succeed” in proving the power of a
strong attractor!)

I have this as a test at home that does each step in 32, 64, doubled
precision (~120 bit fp at 1/4 of full speed), and big. I’ll send it later
today.

No matter what you choose to do, the chaotic attractor will win.

Michael

On Sat, Feb 1, 2020 at 2:02 PM  wrote:

> Perhaps I'm doing something wrong or using the library outside of its
> intended purpose, but I found that this library doesn't handle Muller's
> Recurrence correctly. For those not familiar, Muller's Recurrence is 108
> - (815-1500/z)/y
>
> https://play.golang.org/p/sePTgjZzHeY
>
> See
> https://latkin.org/blog/2014/11/22/mullers-recurrence-roundoff-gone-wrong/
>
> On Sunday, January 26, 2020 at 8:46:16 AM UTC-8, Robert Engels wrote:
>>
>> Which is exactly what github.com/robaho/fixed and many others do!
>>
>> On Jan 26, 2020, at 10:34 AM, Michael Jones  wrote:
>>
>> ...thus the virtue of scaled integers. scaling by 100 makes cents whole,
>> scaling by 1*100 gives four decimal places beyond that. There is
>> nothing bad about floating point despite the reputation, it's just not the
>> number system from algebra; nor is binary floating point the same as
>> decimal floating point. The problems all start with false presumptions.
>>
>> On Sun, Jan 26, 2020 at 8:20 AM Robert Engels 
>> wrote:
>>
>>> Just an FYI, often that is not correct. Many financial systems require
>>> fractional pennies due to the volume of transactions. Think about taxing
>>> stock exchanges the pennies add up quickly at any tax rate, so they use
>>> fractional pennies to reduce the size of the error bucket.
>>>
>>> On Jan 26, 2020, at 8:50 AM, Pat Farrell  wrote:
>>>
>>> never use floating point if you are trying to represent money, say
>>> dollars and cents or decimal values of the euro.
>>> Store the money as integer number of pennies.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/cc852ce3-6f88-40fd-8b19-877c76deec10%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/cc852ce3-6f88-40fd-8b19-877c76deec10%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/AF9827F5-C849-4F4E-8229-005D6C9A0E03%40ix.netcom.com
>>> <https://groups.google.com/d/msgid/golang-nuts/AF9827F5-C849-4F4E-8229-005D6C9A0E03%40ix.netcom.com?utm_medium=email_source=footer>
>>> .
>>>
>>
>>
>> --
>>
>> *Michael T. jonesmichae...@gmail.com*
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/46cadc14-9c8e-4c3a-9c6b-d0af7b621061%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/46cadc14-9c8e-4c3a-9c6b-d0af7b621061%40googlegroups.com?utm_medium=email_source=footer>
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzsFhP%3DwxVwAkp38HyQWDEH7ys90XSm%2BLOe%3Dct5EwhV3w%40mail.gmail.com.

Re: [go-nuts] Go Tool to convert svg to png/jpg

[Michael Jones]

Just to be clear: PNG is a description of pixel values comprising an image
(RGBa, RGBa, RGBa, ...), SVG is a program for creating an image (set color
to blue, draw a circle, change to red, draw a line, ...). Going from SVG
(scalable vector graphics) to pixels is to render an image by executing the
simple program of graphical operation codes. (As said above)

Solutions depending on situation: most every browser has code to rasterize
SVG. Can you invoke one to produce your image? There are numerous free
tools to do the same--can you bundle one of those and invoke it as an
external program? Cairo can do it. (these are in addition to what you
really seem to want, which is a fully Go SVG -> PNG renderer.)

Good luck in your project,
michael

On Fri, Jan 31, 2020 at 6:03 PM robert engels  wrote:

> There is no cross-platform graphics library included in Go.
>
> Most likely you’ll need a C binding to Qt or similar to perform the
> rasterization.
>
> You might be able to put one together using something like
> https://github.com/hajimehoshi/ebiten
>
>
> On Jan 31, 2020, at 7:31 PM, maithri.fri...@gmail.com wrote:
>
> I'm looking for one too but can't find anything online either..
>
> On Monday, March 9, 2015 at 11:35:51 PM UTC-7, will wrote:
>>
>> Hi Gophers,
>>
>> Is there a Golang based tool (or library) tool to convert svg to png or
>> jpg?
>>
>> regards,
>>
>> Will
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d03ad514-1e56-42a3-b7c6-798346a76ca1%40googlegroups.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/BF136C90-3061-4E01-890B-A6FB38E94E07%40ix.netcom.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQw_8%3D3AkkrmZNG9Xrzq37TaauFF77wJUGjkp-Mx3%3DrUPQ%40mail.gmail.com.

Re: [go-nuts] Why is go too slow?

[Michael Jones]

Following up on this, consider how long it actually takes to compute F(40)
in the normal iterative way, single threaded -- this is the natural
amount of work under discussion.

iterative:
BenchmarkFibonacci40-36 5197482821.9 ns/op   0 B/op   0
allocs/op

22 nanoseconds, about a 52 millionth of a second. Doing 40 evaluations
takes 40 times longer (single threaded) but let's just go with the single
evaluation for now.

recursive (after changing commented-out call to FibonacciR):
BenchmarkFibonacci40-36   3 353790912 ns/op   0 B/op   0
allocs/op

353,790,912 nanoseconds, about a third of a second. The program is not
doing more essential work down the Fibonacci path, but is doing a great
deal more redundant work because of the many redundant calls with the same
arguments.

f(40) is 102,334,155 and requires 39 additions the first way and
204,668,309 recursive function calls (2 f(n) - 1) and 102,334,154 additions
the second way (all but the near-leaf elements of which have an addition)
When we divide the 353,790,912 ns/op by the 204,668,309 function calls per
op (up from one in the iterative case) we get 1.73 ns per function call,
including the 102,334,154 required additions.

Now I realize that the original question is about C performance vs Go
performance under this storm of redundant function calls and additions, and
that question further complicated by various ways and means of concurrent
evaluation. Still, in any benchmark it is important to understand what is
actually happening, what is being tested, and especially for this kind of
microbenchmark, to understand all of this well enough to decode the meaning
of it all as it might apply to a real program.

Here is my opinion of the meaning as one might understand from this single
line of testing:

We can do more than 578 million real (not inlined) stack-pushing and
popping function calls per second because they take 1.72 ns each even with
the if tests and the addition of stack values. This is a lot of
function-call intensity and it is hard to imagine anything other than
recursive Fibonacci or Ackerman function evaluation that would do so many
each doing so little.



If it is true that any other language is really 2x faster here (or 10x or
whatever), then when the body of the function is more than an if-test and
an add, say an FFT or a sin() calculation, or writing to memory, or heaven
forbid an I/O or OS interaction, or anything else "real" that implicitly
takes 100 or 100k, or 100m the time of the function call, then the relative
speeds will be immeasurably close to 1.

This is the problem with the microbenchmark. It is important, but mostly to
the compiler writers and CPU architects. For people who write programs to
do things, the real benchmark is timing those things on machine A vs
machine B, or language A vs language B, or whatever.

Also, and perhaps not obvious, is that for so-called cloud providers and
operators of huge data centers for internal purposes, the real benchmark is
even more indirect, generally timing of those things * watts per second or
$ per second. (Both across millions of CPUs in some cases of personal
knowledge.) It is just not possible to understand such things by
extrapolating from microbenchmarks. Not a defense of Go just a word to the
wise.

Michael


On Tue, Jan 28, 2020 at 8:02 AM nilsocket  wrote:

> Thanks a lot for your answer.
>
> Thank you.
>
> On Tuesday, January 28, 2020 at 9:15:58 PM UTC+5:30, Ian Lance Taylor
> wrote:
>>
>> On Tue, Jan 28, 2020 at 7:33 AM nilsocket  wrote:
>>
>>> Attached two files main.c and main.go,
>>> both try to compute fib(40), 20 times on `x` no.of threads/goroutines.
>>> fib(40) is recursive.
>>>
>>> *Prerequisites for C:*
>>>
>>>1. libuv (https://github.com/libuv/libuv)
>>>
>>>
>>>
>>> *Compile*:
>>>
>>>- *C :*
>>>   - *Unoptimized:*
>>>
>>> * gcc main.c -lpthread -luv *
>>>
>>>
>>>- *Optimized:*
>>>   gcc-O3 main.c -lpthread -luv
>>>
>>>
>>>- *Go :*
>>>   - go build
>>>
>>>
>>> *Run:*
>>>
>>>-
>>>
>>> *C:time UV_THREADPOOL_SIZE=x ./a.out // substitute `x` with physical
>>>core count *
>>>-
>>>
>>> *GO:./temp x // substitute `x` with physical core count *
>>>
>>>
>>> Results:
>>>
>>> Thread CountC (Optimized)C (Unoptimized)GO
>>> 1 4.38s 11.36s 11.7s
>>> 4 1.12s 3.1s 2.9s
>>> 8 1.1s 2.9s 2.7s
>>>
>>>
>>> Laptop with 4 physical cores(8 logical cores).
>>>
>>> Why can't go provide Optimization flag?
>>> I understand go developers want compiler to be simple, but the
>>> difference seems too big to leave it out.
>>>
>>> But isn't there any possibility to abstract the complexity and provide
>>> optimization flag?
>>>
>>
>>
>> Go doesn't provide an optimization flag because it always optimizes.
>>
>> Your program amounts to a microbenchmark of the tail recursion
>> optimization.  The Go compiler doesn't optimize tail recursion, because it
>> confuses stack traces and breaks runtime.Callers.  The C compiler has no
>> such 

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Michael Jones]

Python, to its credit, has the nice inclusive property of extensible
interpreters of being friendly to "hang things" onto just like ornaments on
trees. By linking with C/C++-libraries and adding glue/shim code, anything
can be used from Python. This facility and interpretive execution (as Eric
Johnson observed above) makes it a natural for discovery and trying of
things.

These virtues may seem less so in eventual deployments, at scale, and in
other cases.

On Mon, Jan 27, 2020 at 8:48 AM 'Eric Johnson' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

>
> On Sunday, January 26, 2020 at 10:27:35 PM UTC-8, pentel...@gmail.com
> wrote:
>>
>>
>> IMHO, golang didn't make a dent on key areas to become a language of
>> choice like big data (analytics, complex event processing, etc.) and
>> consequently, hot topics like artificial intelligence. Exactly areas where
>> python excels today.
>
>
> At least when it comes to analytics, ML, AI, there are various languages
> at play in this space. Python isn't replacing "R" either. I don't think we
> should expect that it would either. One of Python's intended strengths for
> AI / ML / analytics involves REPL, which is specifically not a targeted
> feature of Go. On the other hand, when it comes to situations like the
> execution of AI / ML models in performance critical environments, I have
> heard / read of anecdotal evidence of teams opting for Go.
>
> The question isn't whether Go making an impact for a specific area, it is
> whether it is making an impact where it is appropriate. For that, from what
> I see, the answer is overwhelmingly *yes*.
>
> Eric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/1c8579d4-27b5-470b-a49d-1bca548bc8f7%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzCEWPWhcnv6rt0e-BjhA3%3DRrswL%2B10CgAv0sL6cOgRbw%40mail.gmail.com.

Re: [go-nuts] Re: keep just 2 decimal places in a float64

[Michael Jones]

...thus the virtue of scaled integers. scaling by 100 makes cents whole,
scaling by 1*100 gives four decimal places beyond that. There is
nothing bad about floating point despite the reputation, it's just not the
number system from algebra; nor is binary floating point the same as
decimal floating point. The problems all start with false presumptions.

On Sun, Jan 26, 2020 at 8:20 AM Robert Engels  wrote:

> Just an FYI, often that is not correct. Many financial systems require
> fractional pennies due to the volume of transactions. Think about taxing
> stock exchanges the pennies add up quickly at any tax rate, so they use
> fractional pennies to reduce the size of the error bucket.
>
> On Jan 26, 2020, at 8:50 AM, Pat Farrell  wrote:
>
> 
> never use floating point if you are trying to represent money, say dollars
> and cents or decimal values of the euro.
> Store the money as integer number of pennies.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/cc852ce3-6f88-40fd-8b19-877c76deec10%40googlegroups.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/AF9827F5-C849-4F4E-8229-005D6C9A0E03%40ix.netcom.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQy7m4-%3DJVQHpGf-kmH3T13gYQGe4%3DGd4%2BBBQT1pGtFDpA%40mail.gmail.com.

Re: [go-nuts] Re: How to convert time interval in days to human readable form?

[Michael Jones]

well, if you have the start and end dates (sy/sm/sd, ey/em/ed), then you
can do it directly.

the number of complete years is ey-sy+1 if em>sm or em==sm && ed>=sd, one
less otherwise
(i think this captures the edge conditions correctly)

same for months (irrespective of the number of days in each), and same for
days, simplified since hours, minutes, and seconds are not being used.

you do not need to do a calendrical base conversion (with the issues
Christian shared) now that we realize that you have the start and end
details.

On Fri, Jan 24, 2020 at 9:20 AM Constantine Vassilev 
wrote:

> I've the start-end date, then how to do that?
>
> On Friday, January 24, 2020 at 9:06:12 AM UTC-8, Constantine Vassilev
> wrote:
>>
>> I have the following task:
>>
>> 1) a number representing days.
>> 2) how to convert it to human readable form?
>>
>> Like this:
>> 11 years 3 months 5 days
>>
>> The actual calculations are:
>>
>> 4095 days = 136.5 months = 11.375 years
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/70cc09a4-0c57-4683-b7b1-de45afcdff6d%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxwGPGq6vkOFCNqANQMt-t3L36hpOLNqW-BtnwGO_ctGg%40mail.gmail.com.

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

[Michael Jones]

How global mindshare develops is something that I know quite a bit about
through leadership and engineering experience in multiple billion user
projects.

One key lesson for me was that you reach a point where the audience you
originally wanted to serve (or refocused to serve) has been served. That’s
when the debate of “more for this group” or “something for other groups”
starts with vigor.

There is a natural desire to grow but my advice here, after looking back
honestly, is that the way to move forward is to be so excellent at some
aspect(s) that users become effective missionaries. This is the only
scaling mechanism at scale (other than force in unusually controlled
scenarios).

Looking forward (i.e. guessing) maybe Go needs new greatness in what it
already is and has by way of an “encyclopedia” of well-loved solutions.
Imagine a guide to 100 top uses with links to tools, samples for each, and
lots of details so that anyone wanting to build a static or dynamic web
server, ftp server, ssh client, mail processor, ... would have complete
guides from start to finish.

Maybe existing solutions are sufficient or maybe they could be better. If
they have room for improvement then my guess is that this kind of beginner
hand holding might be the most effective investment for user growth.

Just a guess,
Michael

On Thu, Jan 16, 2020 at 7:33 AM Amnon Baron Cohen  wrote:

> Go was originally conceived as a niche language. And if it does what we
> need, then I don't think
> we need to be particularly bothered if other languages are more "popular".
>
> But when looking at language popularity, I am not sure that the number of
> google searches is the most meaningful metric.
>
> Lines of code on github could be more interesting.
>
> FWIW: Githubs octoverse shows shows a 147% growth in Go usage last year.
>
> And more interesting growth stats can be found on the Go blog
> https://blog.golang.org/8years
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/28b8066f-876b-41f6-b249-94dc4f255347%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQy-c1eH-bCH1vmTyMdf3_NMt31v6cWSdD8abha1OT%2BF%3Dw%40mail.gmail.com.

Re: [go-nuts] Quantum mechanics mindshare is low & ~flat, per Google Trends

[Michael Jones]



On Wed, Jan 15, 2020 at 1:49 PM Dan Kortschak  wrote:

> Google Trends graph showing past 5y of Mechanic, Quantum mechanics
>
>
> https://trends.google.com/trends/explore?date=today%205-y=%2Fm%2F03f_s3,%2Fm%2F069dx
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/0c3eba037a2697b9d5d39bf502a171da98c4113c.camel%40kortschak.io
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQx1bERdDwcP97bmLkQ5h%2B%2BxYGGJdxV6JcYHPbB%3DegE_ug%40mail.gmail.com.

Re: [go-nuts] distributed runtime

[Michael Jones]

Go is a shared memory system. Your challenge would be to understand a
pointer that came to you from a different machine (i.e., remotely, the R in
RPC).

On Fri, Jan 3, 2020 at 2:31 PM  wrote:

> Hi all and Happy New Year,
>
> I was daydreaming the other day and I was wondering if it was possible to
> create some alternate runtime package.
> The point would be to have the scheduler schedule goroutine not only on
> different CPU, but also on different CPU on different machines.
> The system would be a binary that you could run as the scheduler or as a
> worker, and the scheduler would distribute among workers via some RPC. Of
> course, it would take cache optimisation to schedule it on the correct
> machine so they could share the same CPU when needed.
> Is it a stupid idea? A very difficult one? A naive approach that would
> just have networking bandwidth and latency issues?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/5a05e457-4d9d-45b2-9822-d1accb48d7af%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQw06TmJO5qET%3DdfjfTFQwNqZwjUhxv57q%3DcjRrch1LDrA%40mail.gmail.com.

Re: [go-nuts] Simple worker pool in golnag

[Michael Jones]

I often choose worker pools when there is working storage to be allocated
as part of the work to be done. This way means that such data and
processing structures are naturally built and torn down right there before
and after the work-channel loop in the worker, with no repeated allocations
during runtime, and no need for locked central data or data pools. It feels
clean to me and stays clean even with work-stealing or worker-subdivision
schemes.

Other ways make sense too, so not arguing for only one approach. Just
saying that it works for me very well.

Michael

On Sat, Dec 28, 2019 at 8:17 AM Ian Lance Taylor  wrote:

> On Sat, Dec 28, 2019 at 6:11 AM Robert Engels 
> wrote:
> >
> > Spinning up a Go routine when for each piece of work may be idiomatic
> but it is far from the best practice for many situations - mainly because
> of cpu cache invalidation. Most HPC systems create worker pools by type and
> then isolate them to cpus/cores - something you can’t do in Go.
> >
> > I believe there are outstanding proposals for grouping routines and core
> affinity.
>
> I think that today CPU cache invalidation is more or less independent
> of spinning up separate goroutines.  CPU cache invalidation is
> something to consider for threads, but, as you say, goroutines have no
> affinity to threads.  Whether you use a worker pool or not, Go doesn't
> currently give you any way to control how goroutines are assigned to
> threads.  So in general I don't see any reason to think that a worker
> pool would be better or worse with regard to CPU cache invalidation.
>
> If Go introduces some way to associate goroutines with threads and
> some way to associate threads with CPUs, that might well have an
> effect on whether it is better to use a worker pool.
>
> Ian
>
>
> > On Dec 28, 2019, at 7:02 AM, Agniva De Sarker <
> agniva.quicksil...@gmail.com> wrote:
> >
> > 
> > > (the task was to limit the whole thing to about 10% of cores)
> >
> > I still don't think you needed a worker pool here. Like OP mentioned
> above, you could just limit the number of goroutines executed to 10% of
> total cores.
> >
> >
> > On Saturday, 28 December 2019 18:02:08 UTC+5:30, Chris Burkert wrote:
> >>
> >> There are Pros and Cons for everything in life. Some time ago I wrote a
> database tool which does something per table where the runtime largely
> depends on the table size. I started with one goroutine per table because
> it was easy. But that put a high load on the database (the task was to
> limit the whole thing to about 10% of cores) with out of memory situations.
> So I switched to a worker pool which solved that. However now the overall
> runtime was unpredictable. When the large tables were handled at the end
> their runtime defined the overall runtime. So I to feed the pool with large
> tables first. This again lead to out of memory situations so I reordered
> the tables such that large tables are mixed with smaller tables.
> >>
> >> Brian Candler  schrieb am Sa. 28. Dez. 2019 um
> 11:09:
> >>>
> >>> On Friday, 27 December 2019 16:30:48 UTC, Bruno Albuquerque wrote:
> 
>  This might be useful too you, in any case:
> 
>  https://git.bug-br.org.br/bga/workerpool
> 
> >>>
> >>> I think the point from Bryan Mills' video is, "worker pool" is
> something of an anti-pattern in go.  goroutines are so cheap that you might
> as well start a goroutine for each piece of work you have to do, and let it
> terminate when that piece of work is done.
> >>>
> >>> Apart from the startup cost, the other reason for having a "worker
> pool" is to limit the number of concurrent tasks being executed, and there
> are better ways of doing that in go (also shown in the video).
> >>>
> >>> --
> >>> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> an email to golan...@googlegroups.com.
> >>> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/f840beee-748f-42b6-809f-4c7505208aee%40googlegroups.com
> .
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ecddafbc-eb73-45e1-8a5a-f738e88c6821%40googlegroups.com
> .
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/8C1153D5-25E7-4A06-8626-1FE61D9015D5%40ix.netcom.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.

Re: [go-nuts] cgo & long jump

[Michael Jones]

The longjmp() facility is a wild, dangerous, improper, and popular
mechanism. It's premise is that the app can go back in time, like you
waking up last week with no knowledge of what happened since then. If you
can follow that analogy, then imagine your surprise when external things
you did last week--ordering from Amazon or getting married--start showing
up in your life by surprise or you fail to show up as you should in theirs,
through ignorance, with consequences either way. So it will be with the
relationship between C and Go. You'll leave many things hanging or
abandoned when you longump() back to your application's youth.

(I imagine this is colorful terms, like getting a text on my phone reading
"setjmp() returned 1" and having a paroxysm of imagined crises like "maybe
I got married, had a daughter, and she's waiting for me now at daycare!
OMG, I just don't know.")

On Mon, Dec 9, 2019 at 10:30 AM  wrote:

> On Monday 09 December 2019 10:18:26 Ian Lance Taylor wrote:
> > On Mon, Dec 9, 2019 at 8:57 AM  wrote:
> > >
> > > I would like to ask some technical question about cgo and long jumps.
> > >
> > > In Go I have written shared library which exports C API and therefore
> > > can be called by other C applications. This my library is compiling
> with
> > > cgo, more precisely via: go build -buildmode=c-shared -compiler gc
> > >
> > > Now I would like to call from my Go library some C function which may
> > > call longjmp(). And setjmp() was already called by application which is
> > > using my Go library.
> > >
> > > It is possible to call from Go C function which calls longjmp() which
> > > jumps outside of that my Go library? What would happen with Go garbage
> > > collector and "defer" code?
> > >
> > > To imagine, whole stack looks like this:
> > >
> > >   +--+
> > >   | C application|
> > >   | main():  |
> > >   |   call setjmp()  |
> > >   |   call f1() from Go library  |
> > >   |   ...|
> > >   +--+
> > >   | Go library with exported C f1() function |
> > >   | f1():|
> > >   |   do something in Go |
> > >   |   call f2() from C library   |
> > >   |   ...|
> > >   |   return from f1()   |
> > >   +--+
> > >   | C library with exported f2() function|
> > >   | f2():|
> > >   |   do something in C  |
> > >   |   if error call longjmp()|
> > >   |   else return from f2()  |
> > >   +--+
> > >
> > > And if longjmp() is called then it jumps to main() where setjmp() was
> > > called. So effectively f1() function (in Go) does not return.
> >
> > I'm pretty sure that won't work at all.  Sorry.
>
> Hi Ian! Thank you for your answer.
>
> I was trying to find any resource on Internet about long jumps and
> "import C" but neither in official Go documentation nor in any other
> blog post is written about it.
>
> There are many resources how Go is (fully) compatible with C and can
> call C functions, but absolutely zero information how it is implemented
> nor what would happen with such core feature of C language as long jumps
> or signals.
>
> > Certainly no Go deferred functions will be run.  I don't think this
> > would break the garbage collector as such, but I expect that it would
> > break the Go scheduler.  In the best case you'll be left with a
> > dangling goroutine that will never be run and never go away.
>
> So should I interpret your answer as long jumps are not supported and
> caller of C function from Go must ensure that called C function would
> not call longjmp()?
>
>
> And another interesting question, it is possible to call longjmp() from
> Go code? To throw C exception back to the main C application.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/20191209183039.khrexxnpedzitepy%40pali
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzJ9hJXv%3DSmnsXHuiQtHHghBRwZcZMQBBQOD%2BfJADBrZA%40mail.gmail.com.

Re: [go-nuts] Re: Why Iran is banned by Google?

[Michael Jones]

Hello parisaparvizi1994,

Your frustration feels natural and understandable to me. Jan just answered
you clearly, but since my words did not help, I'm thinking to try again.
Maybe I can do better this time. First though, I must be clear that *I no
longer speak for Google*--well, at least not officially
--though
what I said back then is still true now.

Every country has rules concerning its citizens' actions and potential
interactions with other countries.

Every company, from Mobarakeh Steel Company to Coca-Cola Inc., has a main
office in its home country, such as Iran or the USA. In every case, the
company is a "legal person" responsible to obey the laws of its home
country as well as any other countries where it does business.

When a pair of countries have some disagreement or debate, and it takes the
form of restrictions on free trade, then there will be laws shaping,
restricting, or preventing commerce, import, export, or even visiting. Many
pairs of countries have at least some form of this and such cases, citizens
and local businesses are limited in their actions.

The United States law includes: *"Title 31 → Subtitle B → Chapter V → Part
560 → Subpart B → §560.204: Prohibited exportation, reexportation, sale, or
supply of goods, technology, or services to Iran
."*
These
and other regulations are the barriers you feel; barriers that do not mean
the companies or people obeying the law are happy about it. Maybe Mobarakeh
Steel would like to sell steel in the USA, and in much the same way,
Google's stated desire is to share information helpfully with everyone
everywhere.

Hopefully you can see that it is Google being banned from freely sharing to
Iran and not Google banning Iran or its people from information.

Michael

P.S. Thank you Jan

On Mon, Dec 9, 2019 at 9:34 AM Jan Mercl <0xj...@gmail.com> wrote:

> On Mon, Dec 9, 2019 at 5:57 PM  wrote:
>
> > So in simpler words, Google has banned Iran, right?
>
> Wrong. Michael was, 5 years ago, very precise and clear about the
> facts, which are exactly the opposite.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAA40n-X_FnrLAoVnL10QgDz_3KDytCD8KXrN%3DK5wmN3xySdxTA%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQz8hZdqafRJuU%3Dx5QgdpgTfRESgrZ-3sApZfOj-7Md%3DBg%40mail.gmail.com.

Re: [go-nuts] Re: Where is the middle of Brazil?

[Michael Jones]

...now *this* is something that I know a great deal about (resulting in
label placement in Google Maps and Google Earth, discussions with
colleagues in USGS, the US Census Bureau, DoD, the United Nations GGIM
experts group, UK Ordnance Survey, Survey of India, etc.)

Indeed most of the issues Andrey shared are significant, though the final
note about "which map projection to use" is very odd. The "geographic
center" however one might define it, is a geographic concept concerning
place and not a cartographic concept concerning boundaries after
projection. The computation only has its natural meaning on the globe.

The typical choice is the lat-lon midpoint of the contiguous landmass
(easy) or major aggregate areas (more handwavy) such as would be the case
in Michigan

in the United States. Note that the link shows the two major areas as well
as a large Isle Royale National Park to the north. The question being to
include it or not. The same issue arises with Stewart Island in New Zealand.

There is a pragmatic issue as well--the center may not be in the geographic
entity at all! Imagine a 'C' shaped land. Such a land mass will have a
center, computed by any reasonable means, that is in the empty center and
not in the surrounding ring. Does this happen? Yes, thousands of times
.
Note the placement of "Molokini Shoal Marine Life Conservation District"
compared to the "Molokini" point of interest label--any "center" is along
the line segment between these yet most are offshore. Same in Ecuador with
its Galápagos Islands, where the midpoint is in the ocean. (The same thing
happens with 'L' shaped regions, such as Florida in the United States, and
hook or curved places including Cuba, Laos, Vietnam, and Norway.)

The story of Richard Feynman's travels to the center of Asia
 is chronicled
in Ralph Leighton's delightful book "Tuva or Bust.
"
(Note that at least three places are claimed as the center of Asia.)

On Sat, Dec 7, 2019 at 8:38 AM andrey mirtchovski 
wrote:

> this is quickly becoming off-topic. however, from
> https://en.wikipedia.org/wiki/Geographical_centre:
>
> "As noted in a USGS document "There is no generally accepted
> definition of geographic center, and no completely satisfactory method
> for determining it."[1]
>
> In general, there is room for debate around various details such as
> whether or not to include islands and similarly, large bodies of
> water, how best to handle the curvature of the Earth (a more
> significant factor with larger regions) and closely related to that
> issue, which map projection to use."
> --
> [1]: https://pubs.er.usgs.gov/publication/70039437
>
> On Sat, Dec 7, 2019 at 9:32 AM JuciÊ Andrade  wrote:
> >
> > If one wants to improve precision she may use a bigger map.
> > The resulting position would be the same, only more accurate.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/270e41e1-5dd5-4351-be8a-1ba683bdd6a8%40googlegroups.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAK4xykV5zOMW%2Bbv3DWb1pTXad-rieqnYdoop6dpJwmsnkkGx2w%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzm1V3FUaQ0PYfaKwY0wkk1rwRZcZjub_08hgdfkLJFrw%40mail.gmail.com.

Re: [go-nuts] Inconsistent rounding with float printf ?

[Michael Jones]

Agree with Ian.

Solutions are: change expectations, use decimal floating point, or use a
base-independent decimal representation. The latter implies scaled integers.

Quick, ugly, and typed on one hand from bed, but here it is:
https://play.golang.org/p/fBztRY6qHP0

999000/1000 = 999.0
999050/1000 = 999.1
999100/1000 = 999.1
999150/1000 = 999.2
999200/1000 = 999.2
999250/1000 = 999.3
999300/1000 = 999.3
999350/1000 = 999.4
999400/1000 = 999.4
999450/1000 = 999.5
999500/1000 = 999.5
999550/1000 = 999.6
999600/1000 = 999.6
999650/1000 = 999.7
999700/1000 = 999.7
999750/1000 = 999.8
999800/1000 = 999.8
999850/1000 = 999.9
00/1000 = 999.9
50/1000 = 1000.0
-50/1000 = -1000.0
-00/1000 = -999.9
-999850/1000 = -999.9
-999800/1000 = -999.8
-999750/1000 = -999.8
-999700/1000 = -999.7
-999650/1000 = -999.7
-999600/1000 = -999.6
-999550/1000 = -999.6
-999500/1000 = -999.5
-999450/1000 = -999.5
-999400/1000 = -999.4
-999350/1000 = -999.4
-999300/1000 = -999.3
-999250/1000 = -999.3
-999200/1000 = -999.2
-999150/1000 = -999.2
-999100/1000 = -999.1
-999050/1000 = -999.1


On Fri, Dec 6, 2019 at 2:31 AM Ian Davis  wrote:

> On Fri, 6 Dec 2019, at 9:25 AM, Christophe Meessen wrote:
>
> I have noticed that printf performs an apparently inconsistent rounding of
> floating point values.
>
> I divide a big number by 1000 and printf the resulting value with "%.1f".
> Here is the code: https://play.golang.org/p/e7dD3c6IHq2
>
>
> I think you are just seeing the usual problems of floating point
> representation.
>
> You may wonder why 999450/1000=999.45, 999500/1000=999.50 and
> 999550/1000=999.55 all format as 999.5. The answer is that the internal
> representation of the three results cannot correspond to the mathematical
> result you expect.
>
> This link shows the internal representation of each answer:
> https://play.golang.org/p/bBTNCdsAttR
>
> You can see that 999550/1000 = 999.5499954525264911353588104248046875
> which is printed as 999.5
>
>
> I would expect the rounding rule to be "round away from zero" as defined
> here: https://math.stackexchange.com/a/2252888/33796
> In this case 0.5 is rounded to 1 (or 0.05 to 0.1) and -0.5 to -1 (or -0.05
> to -0.1).
>
>
> The strconv and fmt packages use round to even as a rule. Use math.Round
> to round away from zero.
>
> -- Ian
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ee94624c-5485-4daf-98ad-8e59055056dd%40www.fastmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQy%3DMCWgb69GgsZwAv2GT4ar%3DWrdvn212sFK1PfGES1ijw%40mail.gmail.com.

Re: [go-nuts] CGO - Passing pointer to C

[Michael Jones]

Speaking for the Rick Hudson Fan Club, thank you for your wonderful,
amazing work on the Go GC...

https://blog.golang.org/ismmkeynote

...and for this reminder that we don't want programming practice to
preclude continuing innovations.

On Wed, Dec 4, 2019 at 9:52 AM Rick Hudson  wrote:

> Breaking the Go CGO pointer rules comes up periodically and the rules
> have not changed. Applications have lived with the rules simply
> because breaking them results in revisiting the application code
> every time a new Go release comes out. Did the compiler improve and
> some object is now allocated on the stack instead of the heap? Did the
> runtime borrow some MESH [1] virtual memory page fragmentation
> techniques but improve them for Go by updating pointers to reducing
> TLB pressure? Is there value in moving objects from NVRAM to DRAM
> and updating pointers? And so forth and so on. Nobody knows if any of
> this will ever happen but the Go CGO pointer rules leave open the
> possibility.
>
>
> [1] Bobby Powers, David Tench, Emery D. Berger, and Andrew McGregor. 2019.
> Mesh: Compacting Memory Management for C/C++ Applications. In
> Proceedings of the 40th ACM SIGPLAN Conference on Programming Language
> Design and Implementation (PLDI ’19), June 22ś26, 2019, Phoenix, AZ, USA.
> ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/3314221.3314582
>
>
> On Wednesday, December 4, 2019 at 11:14:17 AM UTC-5, Ian Lance Taylor
> wrote:
>>
>> On Wed, Dec 4, 2019 at 6:48 AM Robert Johnstone 
>> wrote:
>> >
>> > Thanks for the quick reply.  I had not considered the write barriers,
>> but if the Go objects are "live" longer than the held in C, it would work.
>>
>> The write barriers do not look only at the pointer being stored, they
>> also look at the contents of memory being stored into.  That is why C
>> code must never store a Go pointer into Go memory.
>>
>>
>> > I definitely agree that there are risks associated with this approach.
>> We are giving up some of the safety of Go.  Unfortunately, we are using cgo
>> for more than some computation, so we have live objects in C as well, and
>> so we cannot completely escape the manual memory management required.
>> >
>> > What is the state of plans for a moving garbage collector?  This would
>> definitely wreck havoc if any pointers to Go memory were held in C.
>>
>> There are no current plans for a moving garbage collector.
>>
>> I cannot promise that no other changes will break this approach.
>> Obviously we won't consider bug reports for code that breaks the cgo
>> rules.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/23885bd0-4802-40c7-b4c9-52541e92029f%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwZiyh7wjL5OokVKoPkw-8tjdm-uu7zJt75z%2B_H-9AwjA%40mail.gmail.com.

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

 reviewing doesn’t your code
> not detect the case where a value was skipped ? (... not that it could
> happen - but for completeness of the robust test)
>
> On Dec 1, 2019, at 10:44 PM, Michael Jones 
> wrote:
>
> not necessary as the testing and updating is only done in one place by one
> the main goroutine.
>
> On Sun, Dec 1, 2019 at 7:46 PM Robert Engels 
> wrote:
>
>> The updating of the bit array if shared needs to atomic as well, probably
>> with a read and cas.
>>
>> On Dec 1, 2019, at 9:19 PM, Liam  wrote:
>>
>> 
>> Oh you've allocated a bit array for every value in the test range, then
>> checked for gaps in it?
>>
>> On Sunday, December 1, 2019 at 2:21:55 PM UTC-8, Michael Jones wrote:
>>>
>>> Oh! That's just a bit per integer in the test range 0..total-1. Since Go
>>> (and everything else) lacks a bit type, I just type such code
>>> automatically. Bytes hold 8 bits. Array size must be rounded up, so
>>>
>>> a := make([]byte, (total+8-1)/8)
>>>
>>> array index for test integer n is n/8, so "n>>3"
>>>
>>> bit index for the j-th bit, counting up from 0 for the 1's place is
>>> "1<>>
>>> j is n%8, so "n&(8-1)"
>>>
>>> if mask=1<<(n&(8-1)) then one can test if the bit is set with
>>>
>>> a[n>>3] & mask != 0
>>>
>>> to set it is
>>>
>>> a[n>>3] |= mask
>>>
>>> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using
>>> 64-bit ints they become 6 and 64.
>>>
>>> On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:
>>>
>>>> I wrote a less-sophisticated version of your test, then realized I'd
>>>> misspent my time; all I needed was to change the atomic.Add*() to a
>>>> mutex-protected counter, and see whether my app still failed; it did.
>>>>
>>>> But since you took the trouble, I read your code, and would like to
>>>> understand your collision detector. Could you explain this bit?
>>>>
>>>> for _, v := range a {
>>>>   mask := byte(1 << (v & (8 - 1)))
>>>>   index := v >> 3
>>>>
>>>>   if tally[index] != 0 { ... }
>>>>   ...
>>>> }
>>>>
>>>> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>>>>
>>>>> As a follow-up, some more timing:
>>>>>
>>>>> *47088064 atomic increments/sec (my original email above for heavy
>>>>> synchronization conflict incrementing)*
>>>>>
>>>>> 142049067 atomic increments/sec when each goroutine has its own atomic
>>>>> update target. (Not testing global synchronization/mutex, just the
>>>>> overhead of congested vs not.)
>>>>>
>>>>> 426232527 ordinary "x++" increments in the workers.
>>>>>
>>>>> General idea to remember:
>>>>>
>>>>> Atomic increment is ~3x slower than simple add when uncontested.
>>>>> Highly contested atomic increment is ~3x closer than uncontested,
>>>>> therefore ~9x-10x slower than simple add.
>>>>>
>>>>> 10x is not insignificant, but is nevertheless remarkable for a
>>>>> reliable atomic operation. This was once, "back in the day", a
>>>>> remarkably expensive operation, an a feat of genius to accomplish 
>>>>> (Dekker's
>>>>> Algorithm <https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That
>>>>> it is now just a number-of-fingers cycles is fantastic progress!
>>>>>
>>>>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones 
>>>>> wrote:
>>>>>
>>>>>> Liam,
>>>>>>
>>>>>> I just wrote a little stress test program for you. Maybe it will make
>>>>>> you less stressed. ;-)
>>>>>> https://play.golang.org/p/5_7Geyczd1V
>>>>>>
>>>>>> 4 CPU 2016 MacBook Pro:
>>>>>>
>>>>>> *celeste:atom mtj$ go run main.go*
>>>>>> *32 concurrent workers*
>>>>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>>>>> *2.850 seconds elapsed, 47088064 atomic increments/sec*
>>>>>> *0 collisions*
>>>>>>
>>>>>>
>>>>>> 1

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

"Oh you've allocated a bit array for every value in the test range, then
checked for gaps in it?"

Yes. What I should have said. (Though the test looks not for gaps but for
two pigeons in one hole, but the same idea.)

On Sun, Dec 1, 2019 at 8:44 PM Michael Jones 
wrote:

> not necessary as the testing and updating is only done in one place by one
> the main goroutine.
>
> On Sun, Dec 1, 2019 at 7:46 PM Robert Engels 
> wrote:
>
>> The updating of the bit array if shared needs to atomic as well, probably
>> with a read and cas.
>>
>> On Dec 1, 2019, at 9:19 PM, Liam  wrote:
>>
>> 
>> Oh you've allocated a bit array for every value in the test range, then
>> checked for gaps in it?
>>
>> On Sunday, December 1, 2019 at 2:21:55 PM UTC-8, Michael Jones wrote:
>>>
>>> Oh! That's just a bit per integer in the test range 0..total-1. Since Go
>>> (and everything else) lacks a bit type, I just type such code
>>> automatically. Bytes hold 8 bits. Array size must be rounded up, so
>>>
>>> a := make([]byte, (total+8-1)/8)
>>>
>>> array index for test integer n is n/8, so "n>>3"
>>>
>>> bit index for the j-th bit, counting up from 0 for the 1's place is
>>> "1<>>
>>> j is n%8, so "n&(8-1)"
>>>
>>> if mask=1<<(n&(8-1)) then one can test if the bit is set with
>>>
>>> a[n>>3] & mask != 0
>>>
>>> to set it is
>>>
>>> a[n>>3] |= mask
>>>
>>> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using
>>> 64-bit ints they become 6 and 64.
>>>
>>> On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:
>>>
>>>> I wrote a less-sophisticated version of your test, then realized I'd
>>>> misspent my time; all I needed was to change the atomic.Add*() to a
>>>> mutex-protected counter, and see whether my app still failed; it did.
>>>>
>>>> But since you took the trouble, I read your code, and would like to
>>>> understand your collision detector. Could you explain this bit?
>>>>
>>>> for _, v := range a {
>>>>   mask := byte(1 << (v & (8 - 1)))
>>>>   index := v >> 3
>>>>
>>>>   if tally[index] != 0 { ... }
>>>>   ...
>>>> }
>>>>
>>>> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>>>>
>>>>> As a follow-up, some more timing:
>>>>>
>>>>> *47088064 atomic increments/sec (my original email above for heavy
>>>>> synchronization conflict incrementing)*
>>>>>
>>>>> 142049067 atomic increments/sec when each goroutine has its own atomic
>>>>> update target. (Not testing global synchronization/mutex, just the
>>>>> overhead of congested vs not.)
>>>>>
>>>>> 426232527 ordinary "x++" increments in the workers.
>>>>>
>>>>> General idea to remember:
>>>>>
>>>>> Atomic increment is ~3x slower than simple add when uncontested.
>>>>> Highly contested atomic increment is ~3x closer than uncontested,
>>>>> therefore ~9x-10x slower than simple add.
>>>>>
>>>>> 10x is not insignificant, but is nevertheless remarkable for a
>>>>> reliable atomic operation. This was once, "back in the day", a
>>>>> remarkably expensive operation, an a feat of genius to accomplish 
>>>>> (Dekker's
>>>>> Algorithm <https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That
>>>>> it is now just a number-of-fingers cycles is fantastic progress!
>>>>>
>>>>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones 
>>>>> wrote:
>>>>>
>>>>>> Liam,
>>>>>>
>>>>>> I just wrote a little stress test program for you. Maybe it will make
>>>>>> you less stressed. ;-)
>>>>>> https://play.golang.org/p/5_7Geyczd1V
>>>>>>
>>>>>> 4 CPU 2016 MacBook Pro:
>>>>>>
>>>>>> *celeste:atom mtj$ go run main.go*
>>>>>> *32 concurrent workers*
>>>>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>>>>> *2.850 seconds elapsed, 47088064 atomic increments/sec*
>>>>>> *0 collisions*
>>>

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

not necessary as the testing and updating is only done in one place by one
the main goroutine.

On Sun, Dec 1, 2019 at 7:46 PM Robert Engels  wrote:

> The updating of the bit array if shared needs to atomic as well, probably
> with a read and cas.
>
> On Dec 1, 2019, at 9:19 PM, Liam  wrote:
>
> 
> Oh you've allocated a bit array for every value in the test range, then
> checked for gaps in it?
>
> On Sunday, December 1, 2019 at 2:21:55 PM UTC-8, Michael Jones wrote:
>>
>> Oh! That's just a bit per integer in the test range 0..total-1. Since Go
>> (and everything else) lacks a bit type, I just type such code
>> automatically. Bytes hold 8 bits. Array size must be rounded up, so
>>
>> a := make([]byte, (total+8-1)/8)
>>
>> array index for test integer n is n/8, so "n>>3"
>>
>> bit index for the j-th bit, counting up from 0 for the 1's place is "1<>
>> j is n%8, so "n&(8-1)"
>>
>> if mask=1<<(n&(8-1)) then one can test if the bit is set with
>>
>> a[n>>3] & mask != 0
>>
>> to set it is
>>
>> a[n>>3] |= mask
>>
>> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using
>> 64-bit ints they become 6 and 64.
>>
>> On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:
>>
>>> I wrote a less-sophisticated version of your test, then realized I'd
>>> misspent my time; all I needed was to change the atomic.Add*() to a
>>> mutex-protected counter, and see whether my app still failed; it did.
>>>
>>> But since you took the trouble, I read your code, and would like to
>>> understand your collision detector. Could you explain this bit?
>>>
>>> for _, v := range a {
>>>   mask := byte(1 << (v & (8 - 1)))
>>>   index := v >> 3
>>>
>>>   if tally[index] != 0 { ... }
>>>   ...
>>> }
>>>
>>> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>>>
>>>> As a follow-up, some more timing:
>>>>
>>>> *47088064 atomic increments/sec (my original email above for heavy
>>>> synchronization conflict incrementing)*
>>>>
>>>> 142049067 atomic increments/sec when each goroutine has its own atomic
>>>> update target. (Not testing global synchronization/mutex, just the
>>>> overhead of congested vs not.)
>>>>
>>>> 426232527 ordinary "x++" increments in the workers.
>>>>
>>>> General idea to remember:
>>>>
>>>> Atomic increment is ~3x slower than simple add when uncontested.
>>>> Highly contested atomic increment is ~3x closer than uncontested,
>>>> therefore ~9x-10x slower than simple add.
>>>>
>>>> 10x is not insignificant, but is nevertheless remarkable for a reliable
>>>> atomic operation. This was once, "back in the day", a remarkably expensive
>>>> operation, an a feat of genius to accomplish (Dekker's Algorithm
>>>> <https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That it is now
>>>> just a number-of-fingers cycles is fantastic progress!
>>>>
>>>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones 
>>>> wrote:
>>>>
>>>>> Liam,
>>>>>
>>>>> I just wrote a little stress test program for you. Maybe it will make
>>>>> you less stressed. ;-)
>>>>> https://play.golang.org/p/5_7Geyczd1V
>>>>>
>>>>> 4 CPU 2016 MacBook Pro:
>>>>>
>>>>> *celeste:atom mtj$ go run main.go*
>>>>> *32 concurrent workers*
>>>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>>>> *2.850 seconds elapsed, 47088064 atomic increments/sec*
>>>>> *0 collisions*
>>>>>
>>>>>
>>>>> 18 CPU 2019 iMacPro:
>>>>>
>>>>> *plum:atom mtj$ go run main.go*
>>>>> *32 concurrent workers*
>>>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>>>> *2.730 seconds elapsed, 49167382 atomic increments/sec*
>>>>> *0 collisions*
>>>>>
>>>>>
>>>>> Exhaustive demonstration is no proof, but changing the parameters here
>>>>> may increase your comfort.
>>>>>
>>>>> Michael
>>>>>
>>>>> On Sat, Nov 30, 2019 at 1:02 PM Robert Engels 

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

agree, makes sense. also if you trust the compiler, change to use the mod
function, etc.

On Sun, Dec 1, 2019 at 2:31 PM robert engels  wrote:

> I’d prefer v / 8 over v >> 3 - provides more context in my opinion. The
> compiler will change to right shift if more efficient anyway.
>
> On Dec 1, 2019, at 4:21 PM, Michael Jones  wrote:
>
> Oh! That's just a bit per integer in the test range 0..total-1. Since Go
> (and everything else) lacks a bit type, I just type such code
> automatically. Bytes hold 8 bits. Array size must be rounded up, so
>
> a := make([]byte, (total+8-1)/8)
>
> array index for test integer n is n/8, so "n>>3"
>
> bit index for the j-th bit, counting up from 0 for the 1's place is "1<
> j is n%8, so "n&(8-1)"
>
> if mask=1<<(n&(8-1)) then one can test if the bit is set with
>
> a[n>>3] & mask != 0
>
> to set it is
>
> a[n>>3] |= mask
>
> the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using
> 64-bit ints they become 6 and 64.
>
> On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:
>
>> I wrote a less-sophisticated version of your test, then realized I'd
>> misspent my time; all I needed was to change the atomic.Add*() to a
>> mutex-protected counter, and see whether my app still failed; it did.
>>
>> But since you took the trouble, I read your code, and would like to
>> understand your collision detector. Could you explain this bit?
>>
>> for _, v := range a {
>>   mask := byte(1 << (v & (8 - 1)))
>>   index := v >> 3
>>
>>   if tally[index] != 0 { ... }
>>   ...
>> }
>>
>> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>>
>>> As a follow-up, some more timing:
>>>
>>> *47088064 atomic increments/sec (my original email above for heavy
>>> synchronization conflict incrementing)*
>>>
>>> 142049067 atomic increments/sec when each goroutine has its own atomic
>>> update target. (Not testing global synchronization/mutex, just the
>>> overhead of congested vs not.)
>>>
>>> 426232527 ordinary "x++" increments in the workers.
>>>
>>> General idea to remember:
>>>
>>> Atomic increment is ~3x slower than simple add when uncontested.
>>> Highly contested atomic increment is ~3x closer than uncontested,
>>> therefore ~9x-10x slower than simple add.
>>>
>>> 10x is not insignificant, but is nevertheless remarkable for a reliable
>>> atomic operation. This was once, "back in the day", a remarkably expensive
>>> operation, an a feat of genius to accomplish (Dekker's Algorithm
>>> <https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That it is now
>>> just a number-of-fingers cycles is fantastic progress!
>>>
>>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones 
>>> wrote:
>>>
>>>> Liam,
>>>>
>>>> I just wrote a little stress test program for you. Maybe it will make
>>>> you less stressed. ;-)
>>>> https://play.golang.org/p/5_7Geyczd1V
>>>>
>>>> 4 CPU 2016 MacBook Pro:
>>>>
>>>> *celeste:atom mtj$ go run main.go*
>>>> *32 concurrent workers*
>>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>>> *2.850 seconds elapsed, 47088064 atomic increments/sec*
>>>> *0 collisions*
>>>>
>>>>
>>>> 18 CPU 2019 iMacPro:
>>>>
>>>> *plum:atom mtj$ go run main.go*
>>>> *32 concurrent workers*
>>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>>> *2.730 seconds elapsed, 49167382 atomic increments/sec*
>>>> *0 collisions*
>>>>
>>>>
>>>> Exhaustive demonstration is no proof, but changing the parameters here
>>>> may increase your comfort.
>>>>
>>>> Michael
>>>>
>>>> On Sat, Nov 30, 2019 at 1:02 PM Robert Engels 
>>>> wrote:
>>>>
>>>>> If this was broken I think a lot of things would break.
>>>>>
>>>>> On Nov 30, 2019, at 1:56 PM, Liam  wrote:
>>>>>
>>>>> 
>>>>> The stress test for my app fails frequently with what looks like a
>>>>> collision in atomic.AddUint64() results, so I wondered whether I had
>>>>> misunderstood atomic-add.
>>>>>
>>>>> So far I can't reproduce it with a small

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

Oh! That's just a bit per integer in the test range 0..total-1. Since Go
(and everything else) lacks a bit type, I just type such code
automatically. Bytes hold 8 bits. Array size must be rounded up, so

a := make([]byte, (total+8-1)/8)

array index for test integer n is n/8, so "n>>3"

bit index for the j-th bit, counting up from 0 for the 1's place is "1<>3] & mask != 0

to set it is

a[n>>3] |= mask

the values 3 and 8 here are from 8 bits in a byte and 8 = 2**3. if using
64-bit ints they become 6 and 64.

On Sat, Nov 30, 2019 at 7:06 PM Liam  wrote:

> I wrote a less-sophisticated version of your test, then realized I'd
> misspent my time; all I needed was to change the atomic.Add*() to a
> mutex-protected counter, and see whether my app still failed; it did.
>
> But since you took the trouble, I read your code, and would like to
> understand your collision detector. Could you explain this bit?
>
> for _, v := range a {
>   mask := byte(1 << (v & (8 - 1)))
>   index := v >> 3
>
>   if tally[index] != 0 { ... }
>   ...
> }
>
> On Saturday, November 30, 2019 at 5:33:50 PM UTC-8, Michael Jones wrote:
>>
>> As a follow-up, some more timing:
>>
>> *47088064 atomic increments/sec (my original email above for heavy
>> synchronization conflict incrementing)*
>>
>> 142049067 atomic increments/sec when each goroutine has its own atomic
>> update target. (Not testing global synchronization/mutex, just the
>> overhead of congested vs not.)
>>
>> 426232527 ordinary "x++" increments in the workers.
>>
>> General idea to remember:
>>
>> Atomic increment is ~3x slower than simple add when uncontested.
>> Highly contested atomic increment is ~3x closer than uncontested,
>> therefore ~9x-10x slower than simple add.
>>
>> 10x is not insignificant, but is nevertheless remarkable for a reliable
>> atomic operation. This was once, "back in the day", a remarkably expensive
>> operation, an a feat of genius to accomplish (Dekker's Algorithm
>> <https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That it is now
>> just a number-of-fingers cycles is fantastic progress!
>>
>> On Sat, Nov 30, 2019 at 3:38 PM Michael Jones 
>> wrote:
>>
>>> Liam,
>>>
>>> I just wrote a little stress test program for you. Maybe it will make
>>> you less stressed. ;-)
>>> https://play.golang.org/p/5_7Geyczd1V
>>>
>>> 4 CPU 2016 MacBook Pro:
>>>
>>> *celeste:atom mtj$ go run main.go*
>>> *32 concurrent workers*
>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>> *2.850 seconds elapsed, 47088064 atomic increments/sec*
>>> *0 collisions*
>>>
>>>
>>> 18 CPU 2019 iMacPro:
>>>
>>> *plum:atom mtj$ go run main.go*
>>> *32 concurrent workers*
>>> *128 batches of 1048576 atomic increments, 134217728 total increments*
>>> *2.730 seconds elapsed, 49167382 atomic increments/sec*
>>> *0 collisions*
>>>
>>>
>>> Exhaustive demonstration is no proof, but changing the parameters here
>>> may increase your comfort.
>>>
>>> Michael
>>>
>>> On Sat, Nov 30, 2019 at 1:02 PM Robert Engels 
>>> wrote:
>>>
>>>> If this was broken I think a lot of things would break.
>>>>
>>>> On Nov 30, 2019, at 1:56 PM, Liam  wrote:
>>>>
>>>> 
>>>> The stress test for my app fails frequently with what looks like a
>>>> collision in atomic.AddUint64() results, so I wondered whether I had
>>>> misunderstood atomic-add.
>>>>
>>>> So far I can't reproduce it with a small program, so I've probably
>>>> misunderstood my app :-)
>>>>
>>>> On Friday, November 29, 2019 at 6:41:39 PM UTC-8, Kurtis Rader wrote:
>>>>>
>>>>> On Fri, Nov 29, 2019 at 6:21 PM Liam  wrote:
>>>>>
>>>>>> Does atomic.AddInt32(, 1) always yield unique values for concurrent
>>>>>> callers?
>>>>>>
>>>>>> I'm guessing not, because (I think) I'm seeing that two callers get
>>>>>> x+2, neither gets x+1.
>>>>>>
>>>>>
>>>>> That shouldn't happen, AFAICT. Can you share the code where the
>>>>> incorrect behavior is occurring? Or, preferably, a simple reproducer
>>>>> program?
>>>>>
>>>>>
>>>>>> Is there a way to generate uni

Re: [go-nuts] Where is the middle of Brazil?

[Michael Jones]

Sigh. I overlooked the riddle and took it as a straight question. Typical
for me all my life. (once considered making a T-shirt saying "answers
rhetorical questions"). Mild chagrin here... (P.S. sorry for bad typing.
Keyboard's B and SHIFT keys are roken broken.)

On Sat, Nov 30, 2019 at 5:48 PM andrey mirtchovski 
wrote:

> i think JuciÊ wants us to crack the md5. i'm fresh off a CTF
> competition so i don't have any more resources to throw at warming the
> universe and increasing entropy, unfortunately...
>
> On Sat, Nov 30, 2019 at 6:43 PM Michael Jones 
> wrote:
> >
> >
> > My answer is this place.
> > 14°35'03.5"S 53°03'51.3"W
> > -14.584305, -53.064239
> >
> > Not an official national answer, just my own. (That said, I invented
> Google Earth's technology and was the CTO of Google's Geo team from its
> inception...so I have some minor street cred in such things. ;-)
> >
> > On Sat, Nov 30, 2019 at 5:30 PM JuciÊ Andrade  wrote:
> >>
> >> When I was a kid I asked my teacher why my country capital had been
> moved from Rio de Janeiro to Brasilia. She said the reason was Brasilia is
> right in the middle of our territory, that way our president could take
> care of our entire country more effectively. I accepted that answer. Many
> years later, contemplating a map, I doubted Brasilia is right in the middle.
> >>
> >>
> >> Where is the middle of Brazil?
> >>
> >>
> >> MD5 SHA-1
> >>
> >> 
> >>
> >> 4c021557d057327f2977dd739b67da6b
> b3913154ca0c5f48f3555c536fc987322169e607 ihavetheanswer.txt
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> >> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/850ccd64-5c85-4c91-9438-bd28c4320b8a%40googlegroups.com
> .
> >
> >
> >
> > --
> > Michael T. Jones
> > michael.jo...@gmail.com
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CALoEmQwBJudMkah7eWRG9mhfCLjuJM%2B42LGdnP1BEAayH57EiA%40mail.gmail.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAK4xykVTe6avmw7%3Dn3WV6PEp2eh1MRN-E%3DVLLMCq3R1RA0PsLw%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxffWUG6-mTUojvCFan5vu7uCTu_Npp0%3DOaH7xmSZUb0w%40mail.gmail.com.

Re: [go-nuts] Where is the middle of Brazil?

[Michael Jones]

My answer is this place

.
14°35'03.5"S 53°03'51.3"W
-14.584305, -53.064239

Not an official national answer, just my own. (That said, I invented Google
Earth's technology and was the CTO of Google's Geo team from its
inception...so I have some minor street cred in such things. ;-)

On Sat, Nov 30, 2019 at 5:30 PM JuciÊ Andrade  wrote:

> When I was a kid I asked my teacher why my country capital had been moved
> from Rio de Janeiro to Brasilia. She said the reason was Brasilia is right
> in the middle of our territory, that way our president could take care of
> our entire country more effectively. I accepted that answer. Many years
> later, contemplating a map, I doubted Brasilia is right in the middle.
>
>
> Where is the middle of Brazil?
>
>
> MD5 SHA-1
>
> --
>
> 4c021557d057327f2977dd739b67da6b b3913154ca0c5f48f3555c536fc987322169e607
> ihavetheanswer.txt
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/850ccd64-5c85-4c91-9438-bd28c4320b8a%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwBJudMkah7eWRG9mhfCLjuJM%2B42LGdnP1BEAayH57EiA%40mail.gmail.com.

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

As a follow-up, some more timing:

*47088064 atomic increments/sec (my original email above for heavy
synchronization conflict incrementing)*

142049067 atomic increments/sec when each goroutine has its own atomic
update target. (Not testing global synchronization/mutex, just the
overhead of congested vs not.)

426232527 ordinary "x++" increments in the workers.

General idea to remember:

Atomic increment is ~3x slower than simple add when uncontested.
Highly contested atomic increment is ~3x closer than uncontested, therefore
~9x-10x slower than simple add.

10x is not insignificant, but is nevertheless remarkable for a reliable
atomic operation. This was once, "back in the day", a remarkably expensive
operation, an a feat of genius to accomplish (Dekker's Algorithm
<https://en.wikipedia.org/wiki/Dekker%27s_algorithm>). That it is now just
a number-of-fingers cycles is fantastic progress!

On Sat, Nov 30, 2019 at 3:38 PM Michael Jones 
wrote:

> Liam,
>
> I just wrote a little stress test program for you. Maybe it will make you
> less stressed. ;-)
> https://play.golang.org/p/5_7Geyczd1V
>
> 4 CPU 2016 MacBook Pro:
>
> *celeste:atom mtj$ go run main.go*
> *32 concurrent workers*
> *128 batches of 1048576 atomic increments, 134217728 total increments*
> *2.850 seconds elapsed, 47088064 atomic increments/sec*
> *0 collisions*
>
>
> 18 CPU 2019 iMacPro:
>
> *plum:atom mtj$ go run main.go*
> *32 concurrent workers*
> *128 batches of 1048576 atomic increments, 134217728 total increments*
> *2.730 seconds elapsed, 49167382 atomic increments/sec*
> *0 collisions*
>
>
> Exhaustive demonstration is no proof, but changing the parameters here may
> increase your comfort.
>
> Michael
>
> On Sat, Nov 30, 2019 at 1:02 PM Robert Engels 
> wrote:
>
>> If this was broken I think a lot of things would break.
>>
>> On Nov 30, 2019, at 1:56 PM, Liam  wrote:
>>
>> 
>> The stress test for my app fails frequently with what looks like a
>> collision in atomic.AddUint64() results, so I wondered whether I had
>> misunderstood atomic-add.
>>
>> So far I can't reproduce it with a small program, so I've probably
>> misunderstood my app :-)
>>
>> On Friday, November 29, 2019 at 6:41:39 PM UTC-8, Kurtis Rader wrote:
>>>
>>> On Fri, Nov 29, 2019 at 6:21 PM Liam  wrote:
>>>
>>>> Does atomic.AddInt32(, 1) always yield unique values for concurrent
>>>> callers?
>>>>
>>>> I'm guessing not, because (I think) I'm seeing that two callers get
>>>> x+2, neither gets x+1.
>>>>
>>>
>>> That shouldn't happen, AFAICT. Can you share the code where the
>>> incorrect behavior is occurring? Or, preferably, a simple reproducer
>>> program?
>>>
>>>
>>>> Is there a way to generate unique values with pkg atomic, or is a mutex
>>>> required?
>>>>
>>>
>>> Keep in mind that atomic.AddInt32() has the usual two's-complement
>>> overflow semantics. If all you want is a generation counter you really
>>> should be using a uint32 and atomic.AddUint32(). Also, depending on your
>>> preferences and performance considerations you might find it preferable to
>>> use a channel that holds a single int, or small number of ints, that is fed
>>> by a producer goroutine and consumed by any context needing a uniq ID. That
>>> makes it easier to abstract the generation of "unique" ints so that they
>>> satisfy other constraints (e.g., they must be even, odd, prime, etc.).
>>>
>>> --
>>> Kurtis Rader
>>> Caretaker of the exceptional canines Junior and Hank
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/4f62dfff-6895-4aaa-9f0d-b635d5ba7ea7%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/4f62dfff-6895-4aaa-9f0d-b635d5ba7ea7%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/C7B99DEA-D183-44EF-

Re: [go-nuts] C variadic macro equivalent in Golang?

[Michael Jones]

You might refer to...

https://blog.learngoprogramming.com/golang-variadic-funcs-how-to-patterns-369408f19085

...to better appreciate Robert's advice. As he suggested, a variadic Go
function accepting a slice of the special, magic "empty interface" type
("interface{}") can indeed do anything. The downside is that you have to do
the anything yourself...validating the types of the unknown and mysterious
arguments at run time rather than the compiler validating it for you at
compile time.

For moral support, fmt.Printf is a popular example of this API structure.

On Sat, Nov 30, 2019 at 4:20 PM  wrote:

> Thanks, but then you would have to define the interface beforehand for any
> argument type. And there are a lot:
> format strings, strings, characters, integers of different size,
> floating-pont numbers, file-ids
>
> Meanwhile I found this discussion:
> https://github.com/golang/go/issues/18605
> which hints that comma-separated variadic arguments might come with Go 2.x
> in some unknown future.
>
>
> Am Samstag, 30. November 2019 22:00:52 UTC+1 schrieb Robert Engels:
>
>> Make the type interface{} and you can pass anything and use reflect
>>
>> On Nov 30, 2019, at 2:08 PM, minf...@arcor.de wrote:
>>
>> 
>> C allows comma-separated argument lists via variadic macros. Whereas
>> AFAIK golang allows only variadic arguments of the same type.
>> (Please excuse if I am wrong, I am considering myself a golang newbie)
>>
>> C demo program that prints:   -- -- 1 and 1 1 and 1
>>
>> // ## C variadic macro test
>>
>> #include 
>>
>> #define PR(...) if(df){printf(__VA_ARGS__);\
>> if(ef)fprintf(stderr,__VA_ARGS__);}
>>
>> // Flags
>> int df=1, ef=1;
>>
>> int main(void) {
>>   PR("-- ");
>>   PR("%d and %d ",df,ef);
>> }
>>
>> // ## End
>>
>> How would one implement an equivalent PR(...) "function" in Go ???
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golan...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/e36eb032-ffbc-4b26-8c41-f76aa6dcdd00%40googlegroups.com
>> 
>> .
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/181a2df8-f5a2-4a35-9d34-6645d2b1356a%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxDG%2BSXeatp9CRB9O1x6woiGoHsFFYFrxmMaXktYWzpsg%40mail.gmail.com.

Re: [go-nuts] atomic.Add*() doesn't produce unique values

[Michael Jones]

Liam,

I just wrote a little stress test program for you. Maybe it will make you
less stressed. ;-)
https://play.golang.org/p/5_7Geyczd1V

4 CPU 2016 MacBook Pro:

*celeste:atom mtj$ go run main.go*
*32 concurrent workers*
*128 batches of 1048576 atomic increments, 134217728 total increments*
*2.850 seconds elapsed, 47088064 atomic increments/sec*
*0 collisions*


18 CPU 2019 iMacPro:

*plum:atom mtj$ go run main.go*
*32 concurrent workers*
*128 batches of 1048576 atomic increments, 134217728 total increments*
*2.730 seconds elapsed, 49167382 atomic increments/sec*
*0 collisions*


Exhaustive demonstration is no proof, but changing the parameters here may
increase your comfort.

Michael

On Sat, Nov 30, 2019 at 1:02 PM Robert Engels  wrote:

> If this was broken I think a lot of things would break.
>
> On Nov 30, 2019, at 1:56 PM, Liam  wrote:
>
> 
> The stress test for my app fails frequently with what looks like a
> collision in atomic.AddUint64() results, so I wondered whether I had
> misunderstood atomic-add.
>
> So far I can't reproduce it with a small program, so I've probably
> misunderstood my app :-)
>
> On Friday, November 29, 2019 at 6:41:39 PM UTC-8, Kurtis Rader wrote:
>>
>> On Fri, Nov 29, 2019 at 6:21 PM Liam  wrote:
>>
>>> Does atomic.AddInt32(, 1) always yield unique values for concurrent
>>> callers?
>>>
>>> I'm guessing not, because (I think) I'm seeing that two callers get x+2,
>>> neither gets x+1.
>>>
>>
>> That shouldn't happen, AFAICT. Can you share the code where the incorrect
>> behavior is occurring? Or, preferably, a simple reproducer program?
>>
>>
>>> Is there a way to generate unique values with pkg atomic, or is a mutex
>>> required?
>>>
>>
>> Keep in mind that atomic.AddInt32() has the usual two's-complement
>> overflow semantics. If all you want is a generation counter you really
>> should be using a uint32 and atomic.AddUint32(). Also, depending on your
>> preferences and performance considerations you might find it preferable to
>> use a channel that holds a single int, or small number of ints, that is fed
>> by a producer goroutine and consumed by any context needing a uniq ID. That
>> makes it easier to abstract the generation of "unique" ints so that they
>> satisfy other constraints (e.g., they must be even, odd, prime, etc.).
>>
>> --
>> Kurtis Rader
>> Caretaker of the exceptional canines Junior and Hank
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/4f62dfff-6895-4aaa-9f0d-b635d5ba7ea7%40googlegroups.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/C7B99DEA-D183-44EF-9EDA-0B1841AB9DE5%40ix.netcom.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxm4LHW4n5Gs223X2UzVQ0QqBD6ZAx48_jYwve-QjXwmg%40mail.gmail.com.

Re: [go-nuts] Change slice type

[Michael Jones]

The general rule -- if there can be a general rule for risk behavior -- is
that CPUs like having addresses aligned on an integer multiple of the data
element size. So:

access-as-byte data may be on any address (address&(1-1)==0),
access as 2-byte data on a multiple of two address (address&(2-1)==0),
access as 4-byte data on a multiple of four address (address&(4-1)==0),
access as 8-byte data on a multiple of eight address (address&(8-1)==0),
access as 16-byte data on a multiple of sixteen address
(address&(16-1)==0),
:

(How this might extend to "SIMD vector registers" of various kinds depends
on the machine--is it based on the individual component's size, the
aggregate size, etc.) There may be an upper limit to this logic of
alignment restriction, so maybe 32-bits or 64-bits is always sufficient.

Based on this thinking, the safest way to abuse Go's implicit type safety
is to tell Go's allocator that you want something with large data elements
(uint64, say) and then use unsafe to view this as 8x as many bytes or 4x as
many uint16s.

this what Ian meant.

On Wed, Nov 27, 2019 at 9:26 AM  wrote:

> Hi Ian, thanks for the help... quick question about alignment because i
> may be doing something wrong but it shows me that it doesn't matter for the
> slice type, everything is alignment the same way. Can you comment on this?
>
> On 32 bit it'll always return 4 and for 64 bit - 8... even for char or
> boolean slices
> https://play.golang.org/p/wLbt41sISft
>
> Another question, i create []byte slice, then i create []int slice, in the
> middle of []byte by changing
>
> arr := [10]byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
> size := len(arr)
> p := uintptr(unsafe.Pointer())
>
> var data []int
> sh := (*reflect.SliceHeader)(unsafe.Pointer())
> sh.Data = p + some_alignment_offset_
> sh.Len = size
> sh.Cap = size
>
> Now data points at memory used by arr, when arr will become unreachable...
> will underlying memory be collected by GC? Should i call some function to
> inform GC to not collect the memory?
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7430d6a7-909e-41fd-a292-5de2be9f9733%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwkP%3D%2Bfi8%2BRNQz-YtE5CM9WX4LnfZijfv37N63RxzmzSQ%40mail.gmail.com.

Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

[Michael Jones]

Agree. Essentially I'm saying the "channel aspect" is not an issue.

On Thu, Nov 21, 2019 at 12:12 PM Robert Engels 
wrote:

> He stated "each request takes 2 secs to process" - what's involved in that
> is the important aspect imo.
>
> -----Original Message-
> From: Michael Jones
> Sent: Nov 21, 2019 2:06 PM
> To: Robert Engels
> Cc: Sankar , golang-nuts
> Subject: Re: [go-nuts] golang multiple go routines reading from a channel
> and performance implications
>
> In my (past) benchmarking, I got ~3M channel send/receive operations per
> second on my MacBook Pro. It is faster on faster computers. 2k requests/src
> is much less than 3M, clearly, and the 1/1000 duty cycle suggests that
> you'll have 99.9% non-overhead to do your processing. This is back of the
> envelope thinking, but what I go through for every path explored. What
> should it be? How is in in fact? What explains the difference? ... that
> kind of thing.
>
> On Thu, Nov 21, 2019 at 11:25 AM Robert Engels 
> wrote:
>
>> You need to determine how well they parallelize and what the resource
>> consumption of a request is. For example, if every request can run
>> concurrently at 100% (not possible btw because of switching overhead), and
>> each request takes 0.5 secs of CPU, and 1.5 secs of IO, for a total wall
>> time of 2 secs). At 2k request/per sec, you need a machine with 1000 CPUs.
>> IO can run concurrently on most modern setups, so you can essentially
>> factor this out, less so if most of the operations are writes.
>>
>> Your local CPU requirements may be less if the request is then handled by
>> a cluster (over the network, database etc), but you will still need 1000
>> cpus in the cluster (probably a lot more due to the network overhead).
>>
>> You can look at github.com/robaho/go-trader for an example of very high
>> CPU based processing using Go and channels (and other concurrency structs).
>>
>>
>>
>> -Original Message-
>> From: Sankar
>> Sent: Nov 21, 2019 12:30 PM
>> To: golang-nuts
>> Subject: [go-nuts] golang multiple go routines reading from a channel and
>> performance implications
>>
>> We have a setup where we have a producer goroutine pumping in a few
>> thousand objects into a channel (Approximately 2k requests per second).
>> There are a configurable number of goroutines that work as consumers,
>> consuming from this single channel. If none of the consumer threads could
>> receive the message, the message gets discarded. Each consumer go routine
>> takes about 2 seconds for each work to be completed, by which they will
>> come back to read the next item in the channel. The channel is sized such
>> that it can hold up to 10,000 messages.
>>
>> The code is roughly something like:
>>
>> producer.go:
>> func produce() {
>>  ch <- item
>> }
>>
>> func consumer() {
>>  for i:=0; i < NumberOfWorkers; i ++ {
>>go func() {
>>   for _, item := range ch {
>>  // process item
>>   }
>>} ()
>>  }
>> }
>>
>> With this above setup, we are seeing about 40% of our messages getting
>> dropped.
>>
>> So my questions are:
>>
>> 1) In such a high velocity incoming data, will this above design work ?
>> (Producer, Consumer Worker Threads)
>> 2) We did not go for an external middleware for saving the message and
>> processing data later, as we are concerned about latency for now.
>> 3) Are channels bad for such an approach ? Are there any other alternate
>> performant mechanism to achieve this in the go way ?
>> 4) Are there any sample FOSS projects that we can refer to see such
>> performant code ? Any other book, tutorial, video or some such for these
>> high performance Golang application development guidelines ?
>>
>> I am planning to do some profiling of our system to see where the
>> performance is getting dropped, but before that I wanted to ask here, in
>> case there are any best-known-methods that I am missing out on. Thanks.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/f8b5d9fb-d9b7-44b8-bc50-70dcb5f10cd0%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/f8b5d9fb-d9b7-44b8-bc50-70dcb5f10cd0%40googlegroups.com?utm_medium=email_source

Re: [go-nuts] golang multiple go routines reading from a channel and performance implications

[Michael Jones]

In my (past) benchmarking, I got ~3M channel send/receive operations per
second on my MacBook Pro. It is faster on faster computers. 2k requests/src
is much less than 3M, clearly, and the 1/1000 duty cycle suggests that
you'll have 99.9% non-overhead to do your processing. This is back of the
envelope thinking, but what I go through for every path explored. What
should it be? How is in in fact? What explains the difference? ... that
kind of thing.

On Thu, Nov 21, 2019 at 11:25 AM Robert Engels 
wrote:

> You need to determine how well they parallelize and what the resource
> consumption of a request is. For example, if every request can run
> concurrently at 100% (not possible btw because of switching overhead), and
> each request takes 0.5 secs of CPU, and 1.5 secs of IO, for a total wall
> time of 2 secs). At 2k request/per sec, you need a machine with 1000 CPUs.
> IO can run concurrently on most modern setups, so you can essentially
> factor this out, less so if most of the operations are writes.
>
> Your local CPU requirements may be less if the request is then handled by
> a cluster (over the network, database etc), but you will still need 1000
> cpus in the cluster (probably a lot more due to the network overhead).
>
> You can look at github.com/robaho/go-trader for an example of very high
> CPU based processing using Go and channels (and other concurrency structs).
>
>
>
> -Original Message-
> From: Sankar
> Sent: Nov 21, 2019 12:30 PM
> To: golang-nuts
> Subject: [go-nuts] golang multiple go routines reading from a channel and
> performance implications
>
> We have a setup where we have a producer goroutine pumping in a few
> thousand objects into a channel (Approximately 2k requests per second).
> There are a configurable number of goroutines that work as consumers,
> consuming from this single channel. If none of the consumer threads could
> receive the message, the message gets discarded. Each consumer go routine
> takes about 2 seconds for each work to be completed, by which they will
> come back to read the next item in the channel. The channel is sized such
> that it can hold up to 10,000 messages.
>
> The code is roughly something like:
>
> producer.go:
> func produce() {
>  ch <- item
> }
>
> func consumer() {
>  for i:=0; i < NumberOfWorkers; i ++ {
>go func() {
>   for _, item := range ch {
>  // process item
>   }
>} ()
>  }
> }
>
> With this above setup, we are seeing about 40% of our messages getting
> dropped.
>
> So my questions are:
>
> 1) In such a high velocity incoming data, will this above design work ?
> (Producer, Consumer Worker Threads)
> 2) We did not go for an external middleware for saving the message and
> processing data later, as we are concerned about latency for now.
> 3) Are channels bad for such an approach ? Are there any other alternate
> performant mechanism to achieve this in the go way ?
> 4) Are there any sample FOSS projects that we can refer to see such
> performant code ? Any other book, tutorial, video or some such for these
> high performance Golang application development guidelines ?
>
> I am planning to do some profiling of our system to see where the
> performance is getting dropped, but before that I wanted to ask here, in
> case there are any best-known-methods that I am missing out on. Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/f8b5d9fb-d9b7-44b8-bc50-70dcb5f10cd0%40googlegroups.com
> 
> .
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/225583457.2024.1574364299085%40wamui-scooby.atl.sa.earthlink.net
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwNmToO6SBUCem7pWQ24nMkra7z8gLr_Mn0QV6fwbPHwA%40mail.gmail.com.

Re: [go-nuts] pprof cpu profile doesn't write sample to file

[Michael Jones]

If your program is endless, then you'll need to have a timer that says
"it's been long enough, write the profile now"

On Thu, Nov 14, 2019 at 9:35 AM bln prasad  wrote:

> My applications main starts few go routines and runs indefinitely. its
> like pprof.StopCPUProfile() may never get called.
>
> Thanks,
> BLN
>
> On Wednesday, 13 November 2019 23:30:47 UTC+5:30, Ian Lance Taylor wrote:
>>
>> On Wed, Nov 13, 2019 at 9:37 AM bln prasad  wrote:
>> >
>> > I'm running go application as systemd service. I've enabled cpu profile
>> using pprof as specified in documents. When service ran, it's creating file
>> but no samples are getting written to file.
>> > Does it required any special build arguments?
>> >
>> >f, err := os.Create("/tmp/cpu.prof")
>> > if err != nil {
>> > log.Fatal("could not create CPU profile: ", err)
>> > }
>> > defer f.Close()
>> >
>> > if err := pprof.StartCPUProfile(f); err != nil {
>> > log.Fatal("could not start CPU profile: ", err)
>> > }
>> > defer pprof.StopCPUProfile()
>>
>> It's impossible to tell without seeing more of your program.  The
>> program has to actually do something between the calls to
>> StartCPUProfile and StopCPUProfile.  In particular, samples are taken
>> by default every 10 milliseconds, so if your program runs for a much
>> shorter amount of time then it would be normal to not see any samples.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/44a1f5ef-f289-499c-b35c-c56c5f1bf936%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQyz%3Dcc-EejPyx-ZCWfKN_hFD%2BGZiKwZkRjFEjr-WPZCeA%40mail.gmail.com.

Re: [go-nuts] strict type assignability to prevent arbitrary values

[Michael Jones]

...job for a getter function.

On Fri, Nov 8, 2019 at 9:31 AM Jake Montgomery  wrote:

> The inability to have a type safe enum in Go has bothered me as well.
>
> While using a struct, as Peter suggests, does prevent accidental use of
> literals, it also prevents you from making your enum items constant.This
> means that the values can be changed, accidentally, or intentionally, in
> another package. For example:
>
> package enum
>
> import "fmt"
>
> type Status struct {
> s string
> }
>
> var One = Status{"one"}
> var Two = Status{"two"}
>
> func PrintIt(stat Status) {
> fmt.Println("Status is", stat.s)
> }
>
> But a client of your package can then do:
>
> func Foo() {
>
> enum.One = enum.Two
> enum.PrintIt(enum.One)
>
> }
>
> Which will print "two", and perminantly alter the meaning of enum.One.
>
>
> On Thursday, November 7, 2019 at 8:15:20 AM UTC-5, speter wrote:
>>
>> Hi bsr,
>>
>> I'd suggest to use a struct type with a single string field. It will
>> prevent conversion from untyped string constant "by mistake".
>> Moreover, if you make the string field unexported, you can limit new
>> instance creation to the declaring package, allowing to enforce predefined
>> values.
>> Unlike with some other languages, there is no memory or runtime overhead
>> due to encapsulating the string within a struct.
>>
>> HTH,
>> Peter
>>
>> On Thu, Nov 7, 2019 at 7:58 PM bsr  wrote:
>>
>>> Hello,
>>>
>>> I am a long time user of go, but I always had the impression that below
>>> code would not work as string and Status are different type.
>>> I thought I need to explicitly convert as ```exec(Status("abc"))``` it
>>> to work.
>>>
>>> I think, this part of the spec may be the reason
>>> https://golang.org/ref/spec#Assignability
>>>
>>>- x is an untyped constant 
>>>representable  by a
>>>value of type T.
>>>
>>> Is there a way I can prevent this behavior.
>>> I am using Status like an enum, and only predefined status values should
>>> be allowed.
>>>
>>>
>>>
>>>
>>> https://play.golang.org/p/4zsb7KtPBC6
>>>
>>> package main
>>>
>>> import (
>>> "fmt"
>>> )
>>>
>>> type Status string
>>>
>>> func main() {
>>> exec("abc")
>>> }
>>>
>>> func exec(s Status) {
>>> fmt.Printf("Hello, %s", s)
>>> }
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/a20a7034-19c3-410a-bc86-25deff38534f%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/9df8504c-4309-43ab-86ee-518b52eb2e2f%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQw2WZQjcmpRzue8xpX4JKOc3PsR_5TAcoWnaaS%3DxkR5CQ%40mail.gmail.com.

Re: [go-nuts] Preprocessing "go" statements with preserved semantics

[Michael Jones]

Alas. Thus the need for and glory of macros, hold/uneval, and backtick in
LISP. (Problems solved in the 1970s)

On Fri, Nov 8, 2019 at 9:08 AM André Eriksson  wrote:

> That works in simple cases, but does not work when the expression is an
> untyped constant, like 1 or nil. In the case of 1 the variable will get a
> concrete type of int, while fn may accept a float32, or even a private type
> that cannot be named in the current package.
>
> On Friday, November 8, 2019 at 5:51:10 PM UTC+1, Michael Jones wrote:
>>
>> If expr was evaluable in the original code then why not rewrite in place
>> after assigning temporaries?
>>
>> go fn(e1,e2)
>>
>> {
>> t1,t2 := e1,e2
>> go func() {
>>   defer instrument()
>>   fn(t1,t2)
>> }
>>
>>
>> On Fri, Nov 8, 2019 at 8:38 AM André Eriksson  wrote:
>>
>>> I am working on a type of Go preprocessor that rewrites source code to
>>> add additional instrumentation to certain types of statements.
>>>
>>> One such statement is the go statement. I would like to instrument the
>>> newly created goroutine, injecting some instrumentation code at the start
>>> and finish of the goroutine.
>>>
>>> In the simple case, the rewrite is straightforward:
>>>
>>> go fn()
>>>
>>> becomes
>>>
>>> go func() {
>>> defer instrument()()
>>> fn()
>>> }()
>>>
>>> However this approach does not work when fn takes parameters.
>>> If we were to rewrite go fn(expr) into the equivalent form above:
>>>
>>> go func() {
>>> defer instrument()()
>>> fn(expr)
>>> }()
>>>
>>>
>>> the semantics change, since in the rewrite expr gets evaluated inside
>>> the newly created goroutine, which can change the behavior and introduce
>>> data races.
>>>
>>> My attempts to address this have not been particularly fruitful.
>>>
>>> One cannot pass in expr as an argument to the closure, because the type
>>> of the expression may not have a valid name in the current package (for
>>> example if expr evaluates to a private type in some other package).
>>>
>>> Similarly, if expr is a constant expression (like 1 or nil) the type
>>> may depend on the corresponding parameter in fn’s signature.
>>>
>>> The only semantics-preserving rewrite I can think of revolves around
>>> using package reflect, and rewriting like so:
>>>
>>> go func(fn reflect.Value, vals …reflect.Value) {
>>> defer instrument()
>>> fn.Call(vals)
>>> }(reflect.ValueOf(fn), reflect.ValueOf(expr))
>>>
>>> As far as I understand, this should be semantics-preserving, although
>>> with a slight performance cost. (Though I imagine the cost of a
>>> reflection-based call is dwarfed by the cost of spawning a goroutine.)
>>>
>>> Unfortunately this also comes with a major downside: the rewritten code
>>> does not typecheck identically to the original code. Ideally I would like
>>> the rewritten form to cause identical typechecking failures to the old
>>> code, so that these errors are caught at compile time without requiring a
>>> separate typechecking pass for the original code.
>>>
>>> Am I correct in the above reasoning? Can anyone think of a way to do
>>> this sort of rewrite in a semantics-preserving and typechecking-preserving
>>> way?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/a92641f3-2eda-4d4a-ab02-d2b40e3bde75%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/a92641f3-2eda-4d4a-ab02-d2b40e3bde75%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>
>>
>> --
>>
>> *Michael T. jonesmichae...@gmail.com*
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ec41a345-163f-4a8a-a24f-b868def081a0%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/ec41a345-163f-4a8a-a24f-b868def081a0%40googlegroups.com?utm_medium=email_source=footer>
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxe--frSeHfKo822bKhWStJoQdJpkNdAYM6zTPLUuZ%2BAg%40mail.gmail.com.

Re: [go-nuts] Preprocessing "go" statements with preserved semantics

[Michael Jones]

If expr was evaluable in the original code then why not rewrite in place
after assigning temporaries?

go fn(e1,e2)

{
t1,t2 := e1,e2
go func() {
  defer instrument()
  fn(t1,t2)
}


On Fri, Nov 8, 2019 at 8:38 AM André Eriksson  wrote:

> I am working on a type of Go preprocessor that rewrites source code to add
> additional instrumentation to certain types of statements.
>
> One such statement is the go statement. I would like to instrument the
> newly created goroutine, injecting some instrumentation code at the start
> and finish of the goroutine.
>
> In the simple case, the rewrite is straightforward:
>
> go fn()
>
> becomes
>
> go func() {
> defer instrument()()
> fn()
> }()
>
> However this approach does not work when fn takes parameters.
> If we were to rewrite go fn(expr) into the equivalent form above:
>
> go func() {
> defer instrument()()
> fn(expr)
> }()
>
>
> the semantics change, since in the rewrite expr gets evaluated inside the
> newly created goroutine, which can change the behavior and introduce data
> races.
>
> My attempts to address this have not been particularly fruitful.
>
> One cannot pass in expr as an argument to the closure, because the type
> of the expression may not have a valid name in the current package (for
> example if expr evaluates to a private type in some other package).
>
> Similarly, if expr is a constant expression (like 1 or nil) the type may
> depend on the corresponding parameter in fn’s signature.
>
> The only semantics-preserving rewrite I can think of revolves around using
> package reflect, and rewriting like so:
>
> go func(fn reflect.Value, vals …reflect.Value) {
> defer instrument()
> fn.Call(vals)
> }(reflect.ValueOf(fn), reflect.ValueOf(expr))
>
> As far as I understand, this should be semantics-preserving, although with
> a slight performance cost. (Though I imagine the cost of a reflection-based
> call is dwarfed by the cost of spawning a goroutine.)
>
> Unfortunately this also comes with a major downside: the rewritten code
> does not typecheck identically to the original code. Ideally I would like
> the rewritten form to cause identical typechecking failures to the old
> code, so that these errors are caught at compile time without requiring a
> separate typechecking pass for the original code.
>
> Am I correct in the above reasoning? Can anyone think of a way to do this
> sort of rewrite in a semantics-preserving and typechecking-preserving way?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/a92641f3-2eda-4d4a-ab02-d2b40e3bde75%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQx4n-PLJ_scPHfcqNtTnyCo2y1RG_enpV-Rqe4FtxwaQg%40mail.gmail.com.

Re: [go-nuts] Channels may not be the best solution in Go

[Michael Jones]

Travis, glad to hear that you are exploring Harshad Numbers. It is an area
where I have done more than a decade of work and I did not know that anyone
else even cared about them. If you ever want to know how many thousand
digit (or whatever) base 10 (or whatever) numbers have the Harshad
property, and of those, how many of their digits are 3 or 11 (or whatever),
let me know. I have a Go-language program that uses channels that can
answer the question. ;-)

Michael

On Thu, Oct 3, 2019 at 12:45 PM burak serdar  wrote:

> On Thu, Oct 3, 2019 at 11:59 AM JuciÊ Andrade  wrote:
> >
> > Burak, feel free to correct me if I am wrong, but now I think I
> understood the heart of the matter:
> >
> > Your approach to software development is different from mine. Nothing
> wrong with that.
> >
> > . you normally write sequential code, and uses concurrent code where it
> fits best. That is fine.
> > . I use to write concurrent code, and use sequential code where it fits
> best. That is fine as well.
> >
> > Concurrency mechanisms in Go are so easy to use that it allows me to
> take that approach.
> > With a little bit of caution to not create a big ball of mud[1], you can
> write clean concurrent code.
> > You said there is a synchronization overhead when a program uses
> channels. That is true.
> > On the other hand, when we divide the load among several cores we end up
> with a net gain.
> > Depending on the task at hand the difference can be staggering! I mean
> 15x faster or more!
> >
> > If we consider that nearly all CPU is multicore these days [2], we will
> soon conclude that writing concurrent code even for simple tasks makes
> sense, in order to leverage that processing power.
> >
> > Your concurrent code will run keep running well in newer CPUs. Your
> single threaded code won't.
>
> Not all programs benefit from concurrency. Writing concurrent code for
> essentially sequential programs will not benefit from multiple cores,
> like generating prime numbers. Do not forget that concurrency includes
> overhead for context switch and memory barriers. Using channels in a
> sequential program is I think misuse of channels. There is no
> performance gain in having a sequence of goroutines, each waiting for
> the previous one the complete.
>
>
> >
> > Again, nothing wrong with your approach, ok? To each its own.
> >
> > [1] http://laputan.org/mud/
> > [2] https://www.amd.com/en/products/cpu/amd-epyc-7742
> >
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/41a107de-e426-456a-abab-d0b058e39373%40googlegroups.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAMV2RqqGSQd61EE_UfdMYKYAWbZ1mXK8mTPDLv18igE%2BA2R0OQ%40mail.gmail.com
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQxhWppWEJ4EjdzNEZE2MH7pvuE2573LLn2X3i-0wFGH%2BQ%40mail.gmail.com.

Re: [go-nuts] Code for Phi = arctan(y/x) + psi, psi = 0 or pi

[Michael Jones]

https://golang.org/pkg/math/#Atan2

On Fri, Sep 13, 2019 at 5:40 AM SATEESH KANDUKURI <
p20170...@hyderabad.bits-pilani.ac.in> wrote:

> Can anyone help me to write the code for Phi = arctan(y/x) + psi, psi = 0
> or pi and (x, y) are the spatial coordinates in the film plane with the
> origin at the center of a circular disk of radius 50nm.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d1cad1c5-37c7-4dcd-aabd-e00ece8f87fc%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQzc6SM8UJyC1h9zc8jCCN8o57R5ta7P4WAqOgRHbixMfw%40mail.gmail.com.

Re: [go-nuts] By passing go type check when performing bitwise operator

[Michael Jones]

Int(uint(a) >> uint(b))

Is always ok, costs nothing, not a type safety issue.

On Sun, Sep 1, 2019 at 11:56 AM Steven Hartland 
wrote:

> This has been changed in the upcoming 1.13 release, so you might want to
> try the latest release candidate.
>
>
> On 01/09/2019 19:03, Albert Tedja wrote:
>
>  I am trying to perform some bitwise operators, but Go is not letting me
> because the mask is an uint and the values are int. Setting the mask to int
> will break the code since I am shifting bits right and left and prefer the
> prefix 0 when shifting right, and if I am not mistaken, Go does not have
> >>>.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/a211253c-eec1-4efb-8f4c-5e94f04e6681%40googlegroups.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/0cd00fab-a296-823d-61bc-1d6b873db761%40multiplay.co.uk
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwibN2T4ux%3DLP8zgKuxN_CnbqA5cRFYJawfOzoMA9Rqgg%40mail.gmail.com.

Re: [go-nuts] sync.Mutex encounter large performance drop when goroutine contention more than 3400

[Michael Jones]

This is a perennial issue.

We all want to know and understand; it is the sign of true intellect to
feel a need to own the details. It is also true that measuring “out of
context” is literally to measure something different and often that
difference is bigger than the measurement. It can be very difficult to map
results from the measured situation back to the real world.

This has grown to be especially true over the last decade. As the size of
caches increase, the levels increase, CPU speeds outpace RAM, with NUMA
architectures at many levels, with on chip connection inconsistencies,
multiple thread instruction dispatch, wide execution units, and hardware
hazard stalls — all of these are in play and any one can make
micro-benchmarks moot.

“Back in the old days”, so to speak, a cycle was a cycle and timing was
simple. We’re 100x to 1000x faster now, but not by making the old model
1000x faster, but by doing 100 things at once, each 10x faster. That’s the
problem with micro-benchmarks, they do one thing, back to back.

This is obvious in CPU benchmarks. Integer divide is very slow compared to
everything else. It is also rare in the instruction stream compared to
other things. In real code, it is rarely the bottleneck; in back to back
divide micro-benchmarks, it always is. Both are true, but only one case is
meaningful to understand the design choices or estimate performance.



On Mon, Aug 26, 2019 at 3:34 PM Robert Engels  wrote:

> I said in my very first response to you, that the mechanisms of the
> implementation are different, with the in-kernel futex of the channel
> implementation faster that the Go. Much of this is probably because the
> thread is dedicated at this point. All that means is that up to a certain
> point - the CAS works, but then due to contention that path no longer works.
>
> So you get better far performance up to N routines and slightly worse
> performance after. Seems like a decent design decision for a lot of
> workloads.
>
> Still, you keep ignoring this aspect - in the context of actual workloads
> the difference is negligible.
>
> -Original Message-
> From: changkun
> Sent: Aug 26, 2019 4:08 PM
> To: golang-nuts
> Subject: Re: [go-nuts] sync.Mutex encounter large performance drop when
> goroutine contention more than 3400
>
> Based on the pprof graph, I would rather believe that the massive
> performance drop happens because of the `semacquire1` implementation.
> When the number of goroutines is small, most of the `semacquire1` success
> in the `cansemacquire ` fast path, or a middle path where a lock was
> required but then `cansemacquire` success again.
> The drop happens in the case that goroutines are failed for fast path and
> middle path, and therefore needs to be parked, which involves runtime
> schedule costs.
> How do you refute to this argument?
>
> On Monday, August 26, 2019 at 10:56:21 PM UTC+2, changkun wrote:
>>
>> I also tested many times with `go tool pprof`, and it
>> reproducible reports the following difference:
>>
>> Here is for 2400 goroutines:
>>
>> [image: 2400.png]
>>
>> Here is for 4800 goroutines:
>>
>> [image: 4800.png]
>>
>> The difference here is: 4800 goroutines heavily call `gopark` and  2400
>> goroutines heavily calls runtime.procyield, have you notice this
>> difference? Are they normal?
>> In attachment, you find the SVG graphs.
>>
>> On Monday, August 26, 2019 at 10:41:42 PM UTC+2, Robert Engels wrote:
>>>
>>> You might want to try 'perf mem' to report the access delays - it may be
>>> contention on the memory controller as well.
>>>
>>> Thinking about it again, I wouldn't expect a large jump if things were
>>> fair - for example, if at 100 they all fit in the cache, at 110, some are
>>> still in the cache, but some operations are slower, etc. so I would expect
>>> a jump but not as large as you see.
>>>
>>> Still, most linux context switches are 3-4 us, and you are talking about
>>> 300 ns, so you're still doing pretty good, and at approx 40 ns, there are
>>> so many aspects that come into play, i'm not sure you or anyone has the
>>> time to figure out - maybe the HFT guys are interested...
>>>
>>> Like I said, on my OSX machine the times are very similar with both
>>> approaches, so it is OS dependent, and probably OS and hardware
>>> configuration dependent - so I think I've probably reached the end of being
>>> able to help.
>>>
>>> And finally, it probably doesn't matter at all - if the Go routine is
>>> doing anything of value, 300 ns is probably an insignificant cost.
>>>
>>>
>>> -Original Message-
>>> From: changkun
>>> Sent: Aug 26, 2019 3:15 PM
>>> To: golang-nuts
>>> Subject: Re: [go-nuts] sync.Mutex encounter large performance drop when
>>> goroutine contention more than 3400
>>>
>>> Did I do anything wrong, the cache hint ratio decrease linearly, is it
>>> an expected result? I thought the cache hint ratio would have a significant
>>> drop:
>>>
>>> [image: chart.png]
>>> Raw data:
>>>
>>> #goroutines 

[go-nuts] Re: [Go tool capable of hiding any file within an image.

[Michael Jones]

Thank you for sharing this.

On Fri, Aug 2, 2019 at 11:45 AM Dimitar Petrov 
wrote:

> Okay, thanks for clarification!
>
> петък, 2 август 2019 г., 21:42:23 UTC+3, Bryan C. Mills написа:
>>
>> This list is for discussion of the development of the Go project.
>>
>> For announcements of third-party tools and libraries written in Go,
>> please use the golang-nuts list instead.
>>
>>
>> On Friday, August 2, 2019 at 12:51:35 PM UTC-4, buldoz...@gmail.com
>> wrote:
>>>
>>> stegify  is a simple command
>>> line tool capable of fully transparent hiding any file within an image. It
>>> works pretty good and it is very useful for hiding any secrets in slightly
>>> extraordinary way. :) (https://github.com/DimitarPetrov/stegify)
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-dev/bcd07698-3737-42d8-af74-65a8a6bb2e24%40googlegroups.com
> 
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwbqW7diNOX%2Bd1cJCFKZTs1NKuwtcNfk5%3Da1yOnW14XPA%40mail.gmail.com.

Re: [go-nuts] prevent alteration of binaries once distributed in the wild?

[Michael Jones]

One more thought ... just to expand your thinking beyond the excellent
responses already given, is to rephrase the goal as "how do I prevent a
modified binary (executable) from causing problems?" This is a weaker goal
but can in some contexts be more manageable.

I can't share full details, but I'm involved in engineering at Niantic,
with Pokémon GO and Harry Potter Wizards Unite as example executables.
Among the 100M+ players are more than a few cheaters who want to hack the
OS or app to gain unfair advantage. How do we fight this? We'd like to
prevent modification and we'd like to have the phone not be rooted, but it
is a big world with many moving parts that we don't control. However, we do
control the world the servers create for the app, the scoring function that
rewards player activity, and the memory aspect that accumulates player's
scores. Here's the general idea: the huge player base is the training set
for "how good can people do here" all over the world, how fast can they
walk right here, how far can they see right here, etc. If a player does
better than 40k people have done at this park this month, that's a flag. If
the player matches Santa Claus in terms of travel speed (Paris, Da Nang,
Cape Town, all in a few minutes) that's an obvious flag, but so is shaving
a minute here and there, if you know how to look. When we see signs we can
inspect, journal all activity, roll back cheater updates, and apply them to
a lonely cheater world where your false glory is not sharable, we can do
many other things too -- even though we can't necessarily stop
modifications.

Just sharing this other way in case it helps you. Obviously better to
prevent modification, have key code in a trusted enclave, etc.

On Tue, Jul 23, 2019 at 12:57 PM Tom Mitchell  wrote:

>
> On Tue, Jul 23, 2019 at 11:51 AM clement auger 
> wrote:
>
>> Hi,
>>
>> I m looking for a technique to prevent binary alteration once distributed
>> in the wild.
>>
>> I have no clue what i m asking for.
>>
>
> The best current solutions are package manager oriented.
> Decide on the platform you want to work on then look at the package
> manager
> tools.   As well as the access control and audit tools on the platform so
> it is installed
> in a safe and secure way.
>
> Today most systems have the option of installing mandatory access control
> system services.
>
> Some package managers have verify and repair options that can give someone
> a warm fuzzy.
>
> Start with keeping a log of the cryptographic quality check sum and other
> metadata for your program.  An example: I can download a golang package
> and verify that the checksum is as expected before installing.  This is
> valuable when the fastest
> download is a mirror and the primary metadata is on a far or  slow primary
> distribution
> machine.
>
> Each platform will have different features.
>
> --
>   T o mM i t c h e l l ( o n   N i f t y E g g )
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAAMy4URxAx2C60qRucBiRpiLrYOtU0%2BZ4mO7vnyy9boNBQGcnA%40mail.gmail.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQwZ-JTQRQMeWuz9BRwgMNWqose3w%2BqCCE9xrD-MqXxN%2BA%40mail.gmail.com.

Re: [go-nuts] P-local/M-local storage for goroutines?

[Michael Jones]

The simple, common way--if I understand you need correctly--is to launch a
method goroutine.

type CacheManager struct {
// things a worker needs to know, such as the global cache, the specific
worker's local cache, etc.
}

func master() {
  for i := 0; i < workers; i++ {
  m := new(CacheManager)
  m.x = y // set up your thread local storage
:
  go m.Worker()
  }
}


Unfortunately this does not seem to be in any intro guides, which pushes
people to complicated workarounds.

On Tue, Jul 23, 2019 at 10:22 AM Zihan Yang 
wrote:

> I am trying to implement an LRU cache. Several global lru lists could be
> accessed concurrently by multiple goroutines, which could be a disaster in
> a machine with 24 or more cores.
>
>
> Therefore, it would be great if I can add the item to P-local storage and
> flush the batched item into the lru list as a whole. This should greatly
> reduce the contention for the global lru list.
>
>
> How can I do it? I saw some related github issues, #8281
>  and #21355
> , which leads me to a project
> called gls , but the code seems too much
> to integrate into my project (actually I'd better not include any
> third-party package to avoid potential law issues). Is there any built-in
> way to achieve this?
>
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/c79d3801-2f03-43fd-8dd8-35904b481341%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQz12W0vAnU-CrXw9Epb0nTLdyZEr%2Bwvv-%2Bh7iv1Pn8S%2Bw%40mail.gmail.com.

Re: [go-nuts] Windows vs Linux ReadString

[Michael Jones]

show some code.

On Sat, Jul 20, 2019 at 6:15 PM Allan Edwards 
wrote:

> Wow, thanks for responding guys.  So I ran my exe on linux without goland
> and bam the readstring blocked as needed.  For some reason the ReadString
> call is not blocking  inside of goland.  It seems like a great IDE but I
> found a weird bug.  On the Windows side the ReadString blocks inside of
> golandon Windows.  It just does not work correctly on Linux.
>
> On Saturday, July 20, 2019 at 3:52:39 PM UTC-5, Ian Lance Taylor wrote:
>>
>> On Sat, Jul 20, 2019 at 1:48 PM  wrote:
>> >
>> > I started a go project on windows that leveraged readstring to read
>> command line input.  It blocked on Windows.  I just ran my code on Linux
>> and the readstring does not block at all.  Can someone explain this to me?
>>
>> We will be able to help you better if you show us your code and tell
>> us precisely what you did.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/675c8964-f2eb-475b-b042-20e01af50e99%40googlegroups.com
> 
> .
>


-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQys5ecJXUTaQH8LH0ypOyAfhkm_%3Dmt14WFnKC52y0gT%2BA%40mail.gmail.com.

Re: [go-nuts] Does reducing capacity of a slice return memory?

[Michael Jones]

Yes! That always works at the application level. Write an “inspector” that
examines things, duplicating and copying the live parts.

I do that in several of my long running math apps, where my pool of
big.Ints end up with 50-100 thousand digit allocations in a big because of
one giant multiply...I examine them as I recycle and manage this. Makes a
big difference.

But doing it when you know the situation is easy. It’s having the compiler
“just know” that is so challenging. (Unless you box everything like
Java...and that’s an expensive path)

On Fri, Jul 19, 2019 at 12:43 PM Andrey Tcherepanov <
xnow4fippy...@sneakemail.com> wrote:

> My suggestion was not to make "the kitchen" work harder by marking parts
> of the hamburger to be good for the consumption, no. My suggestion was to
> recycle bitten hamburger out and put just the bitten piece aside in a doggy
> bag, sorry.
>
> A.
>
> On Friday, July 19, 2019 at 1:08:51 PM UTC-6, Michael Jones wrote:
>>
>> There is a difference in the meanings of terms in this discussion, maybe
>> that’s confusing to some. (That is, Ian went right to the heart of the
>> matter but maybe a simpler fact needs to be made plain to others.)
>>
>> When you think of the memory used by a slice and of maybe using less of
>> it and “returning” the rest, like putting uneaten food away for tomorrow,
>> you’re making several big demands.
>>
>> You demand severabity, that half a TV set is a reasonable thing to
>> discuss, like half a pie. In fact that backing array was created by the
>> allocator as “256 bytes you can use, with other info you don’t know about
>> somewhere.” To chop off half and “give it back” is not part of the
>> contract. The contract is: use it until you’re done with it. (The whole TV,
>> because of wires and glass bits).
>>
>> One alternative is to make 256 allocations of the small parts, then
>> allocate an array as pointers to each little part. Then you can reshape the
>> array and free the extra parts. This freedom pays a tax in many ways, and
>> always pays it in the normal case.
>>
>> But in the efficient single group allocation style (Go, C, C++, ...) the
>> desire to return part of a monolithic allocation is a major burden. Ian
>> went right to the elements of that burden. But since the follow ups still
>> wanted the burden I thought maybe a reminder of why it is hard would help.
>>
>> On Fri, Jul 19, 2019 at 7:36 AM Ian Lance Taylor 
>> wrote:
>>
>>> I'm not sure what to say.  If someone wants to write up and work on a
>>> detailed implementation proposal, that is fine.  Go for it.  As I said
>>> earlier, I think it would be hard.  I'll add that I don't think that
>>> the difficulty would pay off in terms of the amount of real code that
>>> it would help.  There is a lot of other hard work that could be done
>>> in the compiler that would, I think, help many more programs.
>>>
>>> Ian
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVvDz1kRzpoKNtw5MDH%2BrFpsW6fLSqrT%3DS83JGhM_Y-%2BA%40mail.gmail.com
>>> .
>>>
>> --
>>
>> *Michael T. jonesmichae...@gmail.com*
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d371646c-1f7c-4df1-8077-14e40c25ae04%40googlegroups.com
> <https://groups.google.com/d/msgid/golang-nuts/d371646c-1f7c-4df1-8077-14e40c25ae04%40googlegroups.com?utm_medium=email_source=footer>
> .
>
-- 

*Michael T. jonesmichael.jo...@gmail.com *

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALoEmQy%3DRtfLzFqUnnzng-dba2mm0kx0o7hc2pTUDT_RTb1UpA%40mail.gmail.com.