Re: [go-nuts] Re: is it possible to speed up type assertion?

[Chris Lu]

Thanks for the detailed thoughtful answer!

Statistically you are correct. 95% of Go features are reasonably fast, and
will get 99% job done very fast. But can we call the benchmark on the rest
5% slow part micro benchmarking?

This can easily become an objective discussion based on different
experiences. We can get 99% people supporting that this is a micro
benchmarking. However that would not help anything. Indeed this seems an
ostrich approach.

We are talking about Go language here, not any specific application. A fair
benchmark can be measuring differences against other languages, showing the
gap for Go to catch up.

Anything we feel slow should be tracked so that we can see the performance
improvement over time and newbies can have a correct expectation of the
cost.

Chris


On Thu, Feb 2, 2017 at 3:26 PM, Michael Jones 
wrote:

> Insight here feels tenuous.
>
> It is rare that a well-written "real" program would be measurably
> influenced by something this small. As it happens, I often write such rare
> programs (where a 2x faster math.Log() really might make the whole program
> run 1.9x faster). Even as a poster child for it, I think it is very
> uncommon. To get here you can never read or write data, transcode data,
> communicate with devices, processes, or users, or do anything else so
> typical of real programs.
>
> Benchmarks that measure one single thing in the absence of everything else
> are often misleading. They hide the effect of the "rest of the time" as
> outlined above and worse, they are immune to the good and bad of the
> reality of real computers. For example, integer divide is slow, but is also
> often infrequent. If you do such a divide every 100 cycles, the nature of
> modern CPUs is that the delay will be hidden in overlapped execution of the
> instruction stream and at worst, other instruction streams. A nothing but
> integer divide micro benchmark will block on the 19-cycle or whatever
> completion rate. That's the good. On the other hand, a "write over and over
> to the same memory" micro benchmark will run fast  thanks to multiple
> layers of caching while the same thing on a real-program scale will have
> cache contention effects that could be 1/20th the throughput. (This is the
> same as measuring highway drive time at 3am vs 8am.)
>
> The most meaningful benchmark measures a real use case. In this situation
> the resulting measurements directly interpret application performance. In
> any synthetic benchmark, however, you can often be unsure how to apply the
> result to the real world. The smaller and more focused the benchmark, the
> less easy it is to learn from the result.
>
> Even simple statistical inference is subtly difficult; consider the fact
> that the average person has less than two legs. If this is hard for people,
> then how much harder to properly understand the whole-program meaning of a
> 10x slowdown in type assertions?
>
> Michael
>
> On Thu, Feb 2, 2017 at 1:55 PM, Chris Lu  wrote:
>
>> Cool! Upgrading to 1.8.rc3 shows great improvement! I wish all problems
>> can be resolved by upgrading. :) Here is the before and after result.
>>
>> chris$ go test -bench=.
>> testing: warning: no tests to run
>> BenchmarkAssertion-8 2 9.57 ns/op
>> BenchmarkAssertionOK-8   2 9.02 ns/op
>> BenchmarkBare-8   10 2.09 ns/op
>> BenchmarkIface-8 500027.0 ns/op
>> BenchmarkReflect-8   2 9.00 ns/op
>> PASS
>> ok   _/Users/chris/tmp/test 11.980s
>>
>> chris$ go version
>> go version go1.8rc3 darwin/amd64
>>
>> chris$ go test -bench=.
>> BenchmarkAssertion-8 10 2.67 ns/op
>> BenchmarkAssertionOK-8   10 2.32 ns/op
>> BenchmarkBare-8   10 2.10 ns/op
>> BenchmarkIface-8 500026.0 ns/op
>> BenchmarkReflect-8   2 8.80 ns/op
>> PASS
>> ok   _/Users/chris/tmp/test 11.761s
>>
>>
>> The time taken of the original type casting code shrinks from 8 seconds
>> to 474ms!
>>
>> count=1073741824 time taken=474.843325ms
>> count=1073741824 time taken=308.870452ms
>>
>>
>> On Thu, Feb 2, 2017 at 1:31 PM, Ian Lance Taylor  wrote:
>>
>>> On Thu, Feb 2, 2017 at 1:04 PM, Chris Lu  wrote:
>>> >
>>> > I am trying to build a generic distributed map reduce system similar to
>>> > Spark. Without generics, the APIs pass data via interface{}. For
>>> example, a
>>> > reducer is written this way:
>>> >
>>> > func sum(x, y interface{}) (interface{}, error) {
>>> >
>>> > return x.(uint64) + y.(uint64), nil
>>> >
>>> > }
>>> >
>>> >
>>> > To be more generic, this framework also support LuaJIT.
>>> >
>>> > There is a noticeable difference in terms of performance difference.
>>> LuaJIT
>>> > is faster than pure Go. The profiling of pure Go showed the assertE2T
>>> and
>>> > assertI2T cost a non-trivial amount of time.
>>>
>>> Note 

[go-nuts] Go Fonts: request for feedback

[Nigel Tao]

Bigelow & Holmes, the designers of the Go Fonts, are preparing an
update to those fonts. The changes so far are:

• adjusted box/chart/shade/split-integral to align vertically
• adjusted shade characters to align vertically & horizontally (more
or less aesthetically)
• merged the contours of ring of Aring U+00C5 with the A
• renamed U+02C9 -> uni02C9 in Bold and Bold Italic to conform to name
in Regular and Italic and avoid name conflict with "macron"
• added U+FFFD "Replacement" character

If you have any other bug reports, feature requests, or just like to
comment in general, please reply to the discussion group here, and
I'll pass it on to B

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] json: cannot unmarshal array into Go value of type main error

[Dan Kortschak]

You have handed json.Unmarshal a non-slice/non-array type.

Try this https://play.golang.org/p/Zl5G_Rkt26

On Thu, 2017-02-02 at 15:15 -0800, Rejoy wrote:
> I 'd like to unmarshal database records into a struct type. I get the
> error "json: 
> cannot unmarshal array into Go value of type main..".
> 
> Working example https://play.golang.org/p/keY-3z7lyA
>  
> I can unmarshal the data to []map[String]string, but cannot reference
> it 
> using key value pairs in the template.
> 
> The snippet for code in the template:
>  
>  {{ range $key, $value := . }} 
>     {{ $key }}: {{ $value }} 
>     {{ end }} 
>     
> 
> The output that I get on the webserver:
> 
> - *0*: map[quantity:100 image:1Appleiphone7.jpeg pname:iphone7
> price:7 
> puid:d6742e4e-2ad6-43c5-97f4-e8a7b00684e2] 
> - *1*: map[image:2SamsungGalaxys7.jpeg pname:SamsungGalaxy7
> price:7 
> puid:12d3d8fc-66b6-45f9-a91b-d400b91c32aa quantity:100]
> 
> 
> 
> I wanted to display the data as a table by ranging over the key-
> value 
> pairs. But am not able to figure out how.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How do you take a cipher.StreamWriter and pass it to a io.Reader

[Ilya Kostarev]

You can  use io.TeeReader to connect them all. Also to simultaneously 
read from one file, encrypt, and write to another file on the fly you 
seems to need a bit of concurrency


func (b2h *B2Handler) EncyptandUpload(iv []byte, fileName string, 
outputFileName string) {

key := []byte("example key 1234")

inFile, err := os.Open(fileName)
if err != nil {
panic(err)
}
defer inFile.Close()

block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}

// If the key is unique for each ciphertext, then it's ok to use a zero
// IV.
//var iv [aes.BlockSize]byte
stream := cipher.NewOFB(block, iv[:])

// outFile, err := os.OpenFile(outputFileName, 
os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)

// if err != nil {
// panic(err)
// }
// defer outFile.Close()
pReader, pWriter := io.Pipe()
writer := {S: stream, W: pWriter}
// Copy the input file to the output file, encrypting as we go.
// if _, err := io.Copy(writer, inFile); err != nil {
// panic(err)
// }

//writer.Write(inFile) // cannot use inFile.Read (type func([]byte) 
(int, error)) as type []byte in argument to writer.Write


teeReader := io.TeeReader(inFile, writer)  //construct TeeReader
go ioutil.ReadAll(teeReader)   //use it 
concurrently, err check omitted for simplicity


metadata := make(map[string]string)
_, err3 := b2h.currentBucket.UploadFile(outputFileName, metadata, pReader)

if err3 != nil {
fmt.Println(err3.Error)
}
or you can do sequentially allocating in-memory  buffers for data
in, err := ioutil.ReadAll(inFile)
var buf bytes.Buffer
writer := {S: stream, W: buf}
_, err = writer.Write(in)
_, err3 := b2h.currentBucket.UploadFile(outputFileName, metadata, buf)

On 02/03/2017 01:18 AM, Justin C wrote:
I think your on the right track I just can't get this to work. It seem 
like I'm missing something simple to do with GOs readers and writters.



func (b2h *B2Handler) EncyptandUpload(iv []byte, fileName string, 
outputFileName string) {

key := []byte("example key 1234")

inFile, err := os.Open(fileName)
if err != nil {
panic(err)
}
defer inFile.Close()

block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}

// If the key is unique for each ciphertext, then it's ok to use a zero
// IV.
//var iv [aes.BlockSize]byte
stream := cipher.NewOFB(block, iv[:])

// outFile, err := os.OpenFile(outputFileName, 
os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)

// if err != nil {
// panic(err)
// }
// defer outFile.Close()
pReader, pWriter := io.Pipe()
writer := {S: stream, W: pWriter}
// Copy the input file to the output file, encrypting as we go.
// if _, err := io.Copy(writer, inFile); err != nil {
// panic(err)
// }

writer.Write(inFile) // cannot use inFile.Read (type func([]byte) 
(int, error)) as type []byte in argument to writer.Write

metadata := make(map[string]string)
_, err3 := b2h.currentBucket.UploadFile(outputFileName, metadata, pReader)

if err3 != nil {
fmt.Println(err3.Error)
}

I'm still getting a error on reading the file into the writer.Write(). 
 *cannot use inFile.Read (type func([]byte) (int, error)) as type 
[]byte in argument to writer.Write* It seems like you should be able 
to write it and upload it but i might have to write it out to a temp 
file and then upload it.


Here is the encyption example https://golang.org/pkg/crypto/cipher/#NewOFB
here is the library i am using to upload 
it https://github.com/kothar/go-backblaze

If that helps.


On Thursday, February 2, 2017 at 2:39:19 PM UTC-7, Uvelichitel wrote:

Try to construct an io.Pipe maybe
pReader, pWriter := io.Pipe()
writer:= {S: stream, W: pWriter}
UploadFile(outputFileName, metadata, pReader)
writer.Write(src)
On 02/02/2017 10:27 PM, Justin C wrote:

I have a encryption function that gives a  cipher.StreamWriter

writer := {S: stream, W: outFile}

I am trying to upload it to using a library that has a function
that is uploadFile that requires a IO.reader is there a way to
pass the writer to the io.reader so the file is in crypted then
uploaded?

UploadFile(outputFileName, metadata, io.Reader)

My code looks like this


writer := {S: stream, W: outFile}
// Copy the input file to the output file, encrypting as we go.

metadata := make(map[string]string)
b2h.currentBucket.UploadFile(outputFileName, metadata, writer)

I get this error.
cannot use writer (type *cipher.StreamWriter) as type io.Reader
in argument to b2h.currentBucket.UploadFile: *cipher.StreamWriter
does not implement io.Reader (missing Read method)

I'm having a little trouble understanding how this is suppose to
work together.
Does anyone have any ideas?
Thank you!
-- 
You received this message because you are subscribed to the

Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to golang-nuts...@googlegroups.com .
For more options, visit 

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Michael Jones]

Insight here feels tenuous.

It is rare that a well-written "real" program would be measurably
influenced by something this small. As it happens, I often write such rare
programs (where a 2x faster math.Log() really might make the whole program
run 1.9x faster). Even as a poster child for it, I think it is very
uncommon. To get here you can never read or write data, transcode data,
communicate with devices, processes, or users, or do anything else so
typical of real programs.

Benchmarks that measure one single thing in the absence of everything else
are often misleading. They hide the effect of the "rest of the time" as
outlined above and worse, they are immune to the good and bad of the
reality of real computers. For example, integer divide is slow, but is also
often infrequent. If you do such a divide every 100 cycles, the nature of
modern CPUs is that the delay will be hidden in overlapped execution of the
instruction stream and at worst, other instruction streams. A nothing but
integer divide micro benchmark will block on the 19-cycle or whatever
completion rate. That's the good. On the other hand, a "write over and over
to the same memory" micro benchmark will run fast  thanks to multiple
layers of caching while the same thing on a real-program scale will have
cache contention effects that could be 1/20th the throughput. (This is the
same as measuring highway drive time at 3am vs 8am.)

The most meaningful benchmark measures a real use case. In this situation
the resulting measurements directly interpret application performance. In
any synthetic benchmark, however, you can often be unsure how to apply the
result to the real world. The smaller and more focused the benchmark, the
less easy it is to learn from the result.

Even simple statistical inference is subtly difficult; consider the fact
that the average person has less than two legs. If this is hard for people,
then how much harder to properly understand the whole-program meaning of a
10x slowdown in type assertions?

Michael

On Thu, Feb 2, 2017 at 1:55 PM, Chris Lu  wrote:

> Cool! Upgrading to 1.8.rc3 shows great improvement! I wish all problems
> can be resolved by upgrading. :) Here is the before and after result.
>
> chris$ go test -bench=.
> testing: warning: no tests to run
> BenchmarkAssertion-8 2 9.57 ns/op
> BenchmarkAssertionOK-8   2 9.02 ns/op
> BenchmarkBare-8   10 2.09 ns/op
> BenchmarkIface-8 500027.0 ns/op
> BenchmarkReflect-8   2 9.00 ns/op
> PASS
> ok   _/Users/chris/tmp/test 11.980s
>
> chris$ go version
> go version go1.8rc3 darwin/amd64
>
> chris$ go test -bench=.
> BenchmarkAssertion-8 10 2.67 ns/op
> BenchmarkAssertionOK-8   10 2.32 ns/op
> BenchmarkBare-8   10 2.10 ns/op
> BenchmarkIface-8 500026.0 ns/op
> BenchmarkReflect-8   2 8.80 ns/op
> PASS
> ok   _/Users/chris/tmp/test 11.761s
>
>
> The time taken of the original type casting code shrinks from 8 seconds to
> 474ms!
>
> count=1073741824 time taken=474.843325ms
> count=1073741824 time taken=308.870452ms
>
>
> On Thu, Feb 2, 2017 at 1:31 PM, Ian Lance Taylor  wrote:
>
>> On Thu, Feb 2, 2017 at 1:04 PM, Chris Lu  wrote:
>> >
>> > I am trying to build a generic distributed map reduce system similar to
>> > Spark. Without generics, the APIs pass data via interface{}. For
>> example, a
>> > reducer is written this way:
>> >
>> > func sum(x, y interface{}) (interface{}, error) {
>> >
>> > return x.(uint64) + y.(uint64), nil
>> >
>> > }
>> >
>> >
>> > To be more generic, this framework also support LuaJIT.
>> >
>> > There is a noticeable difference in terms of performance difference.
>> LuaJIT
>> > is faster than pure Go. The profiling of pure Go showed the assertE2T
>> and
>> > assertI2T cost a non-trivial amount of time.
>>
>> Note that in the 1.8 release assertE2T and assertI2T no longer exist.
>> They were removed by https://golang.org/cl/32313.  That should speed
>> up these cases; assertE2T and assertI2T were trivial, but in some
>> cases they did call typedmemmove.  When your interface values store
>> non-pointers, and the code is inlined as it is in 1.8, the calls to
>> typedmemmove disappear.
>>
>> You might want to retry your benchmarks with the 1.8 release candidate
>> to see if you can observe any real difference.
>>
>> Ian
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Michael T. Jones
michael.jo...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and 

[go-nuts] json: cannot unmarshal array into Go value of type main error

[Rejoy]

I 'd like to unmarshal database records into a struct type. I get the error 
"json: 
cannot unmarshal array into Go value of type main..".

Working example https://play.golang.org/p/keY-3z7lyA
 
I can unmarshal the data to []map[String]string, but cannot reference it 
using key value pairs in the template.

The snippet for code in the template:
 
 {{ range $key, $value := . }} 
{{ $key }}: {{ $value }} 
{{ end }} 


The output that I get on the webserver:

- *0*: map[quantity:100 image:1Appleiphone7.jpeg pname:iphone7 price:7 
puid:d6742e4e-2ad6-43c5-97f4-e8a7b00684e2] 
- *1*: map[image:2SamsungGalaxys7.jpeg pname:SamsungGalaxy7 price:7 
puid:12d3d8fc-66b6-45f9-a91b-d400b91c32aa quantity:100]



I wanted to display the data as a table by ranging over the key-value 
pairs. But am not able to figure out how.


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How do you take a cipher.StreamWriter and pass it to a io.Reader

[Justin C]

I think your on the right track I just can't get this to work. It seem like 
I'm missing something simple to do with GOs readers and writters. 


func (b2h *B2Handler) EncyptandUpload(iv []byte, fileName string, 
outputFileName string) {
key := []byte("example key 1234")

inFile, err := os.Open(fileName)
if err != nil {
panic(err)
}
defer inFile.Close()

block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}

// If the key is unique for each ciphertext, then it's ok to use a zero
// IV.
//var iv [aes.BlockSize]byte
stream := cipher.NewOFB(block, iv[:])

// outFile, err := os.OpenFile(outputFileName, 
os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
// if err != nil {
// panic(err)
// }
// defer outFile.Close()
pReader, pWriter := io.Pipe()
writer := {S: stream, W: pWriter}
// Copy the input file to the output file, encrypting as we go.
// if _, err := io.Copy(writer, inFile); err != nil {
// panic(err)
// }

writer.Write(inFile) // cannot use inFile.Read (type func([]byte) (int, 
error)) as type []byte in argument to writer.Write
metadata := make(map[string]string)
_, err3 := b2h.currentBucket.UploadFile(outputFileName, metadata, pReader)

if err3 != nil {
fmt.Println(err3.Error)
}

I'm still getting a error on reading the file into the writer.Write(). 
 *cannot use inFile.Read (type func([]byte) (int, error)) as type []byte in 
argument to writer.Write* It seems like you should be able to write it and 
upload it but i might have to write it out to a temp file and then upload 
it. 

Here is the encyption example https://golang.org/pkg/crypto/cipher/#NewOFB
here is the library i am using to upload 
it https://github.com/kothar/go-backblaze
If that helps. 


On Thursday, February 2, 2017 at 2:39:19 PM UTC-7, Uvelichitel wrote:
>
> Try to construct an io.Pipe maybe
> pReader, pWriter := io.Pipe()
> writer := {S: stream, W: pWriter}
> UploadFile(outputFileName, metadata, pReader)
> writer.Write(src)
> On 02/02/2017 10:27 PM, Justin C wrote:
>
> I have a encryption function that gives a  cipher.StreamWriter  
>
> writer := {S: stream, W: outFile}
>
> I am trying to upload it to using a library that has a function that is 
> uploadFile that requires a IO.reader is there a way to pass the writer to 
> the io.reader so the file is in crypted then uploaded?
>
> UploadFile(outputFileName, metadata, io.Reader)
>
> My code looks like this
>
>
> writer := {S: stream, W: outFile}
> // Copy the input file to the output file, encrypting as we go.
>
> metadata := make(map[string]string)
> b2h.currentBucket.UploadFile(outputFileName, metadata, writer)
>
> I get this error.
> cannot use writer (type *cipher.StreamWriter) as type io.Reader in 
> argument to b2h.currentBucket.UploadFile: *cipher.StreamWriter does not 
> implement io.Reader (missing Read method)
>
> I'm having a little trouble understanding how this is suppose to work 
> together.
> Does anyone have any ideas? 
> Thank you!
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Chris Lu]

Cool! Upgrading to 1.8.rc3 shows great improvement! I wish all problems can
be resolved by upgrading. :) Here is the before and after result.

chris$ go test -bench=.
testing: warning: no tests to run
BenchmarkAssertion-8 2 9.57 ns/op
BenchmarkAssertionOK-8   2 9.02 ns/op
BenchmarkBare-8   10 2.09 ns/op
BenchmarkIface-8 500027.0 ns/op
BenchmarkReflect-8   2 9.00 ns/op
PASS
ok   _/Users/chris/tmp/test 11.980s

chris$ go version
go version go1.8rc3 darwin/amd64

chris$ go test -bench=.
BenchmarkAssertion-8 10 2.67 ns/op
BenchmarkAssertionOK-8   10 2.32 ns/op
BenchmarkBare-8   10 2.10 ns/op
BenchmarkIface-8 500026.0 ns/op
BenchmarkReflect-8   2 8.80 ns/op
PASS
ok   _/Users/chris/tmp/test 11.761s


The time taken of the original type casting code shrinks from 8 seconds to
474ms!

count=1073741824 time taken=474.843325ms
count=1073741824 time taken=308.870452ms


On Thu, Feb 2, 2017 at 1:31 PM, Ian Lance Taylor  wrote:

> On Thu, Feb 2, 2017 at 1:04 PM, Chris Lu  wrote:
> >
> > I am trying to build a generic distributed map reduce system similar to
> > Spark. Without generics, the APIs pass data via interface{}. For
> example, a
> > reducer is written this way:
> >
> > func sum(x, y interface{}) (interface{}, error) {
> >
> > return x.(uint64) + y.(uint64), nil
> >
> > }
> >
> >
> > To be more generic, this framework also support LuaJIT.
> >
> > There is a noticeable difference in terms of performance difference.
> LuaJIT
> > is faster than pure Go. The profiling of pure Go showed the assertE2T and
> > assertI2T cost a non-trivial amount of time.
>
> Note that in the 1.8 release assertE2T and assertI2T no longer exist.
> They were removed by https://golang.org/cl/32313.  That should speed
> up these cases; assertE2T and assertI2T were trivial, but in some
> cases they did call typedmemmove.  When your interface values store
> non-pointers, and the code is inlined as it is in 1.8, the calls to
> typedmemmove disappear.
>
> You might want to retry your benchmarks with the 1.8 release candidate
> to see if you can observe any real difference.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] How do you take a cipher.StreamWriter and pass it to a io.Reader

[Ilya Kostarev]

Try to construct an io.Pipe maybe
pReader, pWriter := io.Pipe()
writer:= {S: stream, W: pWriter}
UploadFile(outputFileName, metadata, pReader)
writer.Write(src)
On 02/02/2017 10:27 PM, Justin C wrote:

I have a encryption function that gives a  cipher.StreamWriter

writer := {S: stream, W: outFile}

I am trying to upload it to using a library that has a function that 
is uploadFile that requires a IO.reader is there a way to pass the 
writer to the io.reader so the file is in crypted then uploaded?


UploadFile(outputFileName, metadata, io.Reader)

My code looks like this


writer := {S: stream, W: outFile}
// Copy the input file to the output file, encrypting as we go.

metadata := make(map[string]string)
b2h.currentBucket.UploadFile(outputFileName, metadata, writer)

I get this error.
cannot use writer (type *cipher.StreamWriter) as type io.Reader in 
argument to b2h.currentBucket.UploadFile: *cipher.StreamWriter does 
not implement io.Reader (missing Read method)


I'm having a little trouble understanding how this is suppose to work 
together.

Does anyone have any ideas?
Thank you!
--
You received this message because you are subscribed to the Google 
Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to golang-nuts+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Ian Lance Taylor]

On Thu, Feb 2, 2017 at 1:04 PM, Chris Lu  wrote:
>
> I am trying to build a generic distributed map reduce system similar to
> Spark. Without generics, the APIs pass data via interface{}. For example, a
> reducer is written this way:
>
> func sum(x, y interface{}) (interface{}, error) {
>
> return x.(uint64) + y.(uint64), nil
>
> }
>
>
> To be more generic, this framework also support LuaJIT.
>
> There is a noticeable difference in terms of performance difference. LuaJIT
> is faster than pure Go. The profiling of pure Go showed the assertE2T and
> assertI2T cost a non-trivial amount of time.

Note that in the 1.8 release assertE2T and assertI2T no longer exist.
They were removed by https://golang.org/cl/32313.  That should speed
up these cases; assertE2T and assertI2T were trivial, but in some
cases they did call typedmemmove.  When your interface values store
non-pointers, and the code is inlined as it is in 1.8, the calls to
typedmemmove disappear.

You might want to retry your benchmarks with the 1.8 release candidate
to see if you can observe any real difference.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Add section about use of underscores in file names to Go Code Review Comments

[konrad . reiche]

What both? What's the alternative?

On Wednesday, February 1, 2017 at 4:07:30 PM UTC-8, Dave Cheney wrote:
>
> I think both are symptomatic of following a Javaesq style of one type per 
> file.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Chris Lu]

Original poster here. Please do not call it micro benchmarking. My real
life use case is this.

I am trying to build a generic distributed map reduce system similar to
Spark. Without generics, the APIs pass data via interface{}. For example, a
reducer is written this way:

func sum(x, y interface{}) (interface{}, error) {

return x.(uint64) + y.(uint64), nil

}


To be more generic, this framework also support LuaJIT.

There is a noticeable difference in terms of performance difference. LuaJIT
is faster than pure Go. The profiling of pure Go showed the assertE2T
and assertI2T
cost a non-trivial amount of time.

In the original post, you can see pure Go with type casting took 8 seconds,
while LuaJIT only used 900 milliseconds to get the same amount of job done.

btw: Java took 30 ms to do the same. https://play.golang.org/p/FSnvLb2uxA
10% of time of pure go without any time casting.

Chris
See more about the framework on https://github.com/chrislusf/gleam


On Thu, Feb 2, 2017 at 10:53 AM, 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> Making the gap smaller means making the compiler dumber. And I disagree
> with you about microbenchmarks. There is a difference between benchmarks
> and microbenchmarks and using the latter as the primary means of evaluating
> performance is plain wrong. Nothing wrong with artificial benchmarks in
> itself; if you benchmark how quickly a json-decoder decodes some artificial
> set of data is fine, it's reasonably close to real-world to give you some
> indication of performance. Benchmarking some code that will compile to only
> a handful of instructions will give you simply zero information.
>
> On Thu, Feb 2, 2017 at 7:46 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:
>
>> The compiler field hasn't so far reached a state of sophistication in
>> handling microbenchmarks like the human mind does handle them. Until that
>> state is reached in compiler technology, it is natural for people to keep
>> using microbenchmarks as one of the primary performance indicators.
>>
>> The observed difference is in [how a human mind would execute the
>> microbenchmark in order to execute it in the shortest amount of time
>> possible] compared to [how the microbenchmark is being executed on a real
>> CPU running instructions generated by a real compiler].
>>
>> Rational path for compiler technology is to make the observed gap
>> gradually smaller.
>>
>> On Thursday, February 2, 2017 at 6:56:09 PM UTC+1, Axel Wagner wrote:
>>>
>>> The point is, that (as repeatedly pointed out) microbenchmarks just, in
>>> general, don't tell you anything really useful, as any effects you measure
>>> could be due to anything ranging from random noise, over compiler
>>> optimizations to code-alignments. Avoiding type assertions based on
>>> anything contained in this thread is pretty unreasonable; I see zero
>>> evidence here, that they are, at all, a problem.
>>>
>>> Go gives you pretty amazing tools to find the actual bottlenecks in your
>>> code. Again: Write your code as it's most convenient, then run it with
>>> production data on production hardware and *if* it isn't fast enough,
>>> use pprof and similar tools to find the actual bottlenecks. They very
>>> likely won't be type-assertions.
>>>
>>> On Thu, Feb 2, 2017 at 5:16 PM, Jan Ziak <0xe2.0x...@gmail.com> wrote:
>>>
 A large difference between an implementable highly-optimized run-time
 performance of feature X and the physically implemented run-time
 performance of feature X causes programmers to avoid X.

 On Thursday, February 2, 2017 at 3:32:16 PM UTC+1, Marvin Renich wrote:
>
> * Marvin Renich  [170202 09:22]:
> >
> > BenchmarkIface is testing the wrong thing; the value is swamped by
> the
> > implicit conversion of d (type T) to the function argument of type
> I.
>
> Or, maybe it is testing the correct thing.  That is the problem with
> microbenchmarking.  Try benchmarking your actual code instead.  :-P
>
> ...Marvin
>
> --
 You received this message because you are subscribed to the Google
 Groups "golang-nuts" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to golang-nuts...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/golang-nuts/Og8s9Y-Kif4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> 

[go-nuts] Re: Pass variables to bash

[Rich]

Thanks everyone for your input... I tried HowardCShaw's method before 
posting, but I went back and experimented more with it and found that the 
output was putting all the variables on one line. I separated them all into 
separate lines and it works! 
What I was doing:
exportVar=`
export var1="Foo Bar"
export var2="Bar Foo"
export var3="Far Boo"
...
export var34="ooB raF"
`
fmt.Println(exportVar)

In bash:
myVars=`goSetVar`
eval myVars

myVars would equal all of the lines above on one line.  What I did was just 
put every line with fmt.Println and that seems to work.


On Thursday, February 2, 2017 at 11:23:49 AM UTC-5, Rich wrote:
>
> Hi All,
>
> My dev team did a very bad thing and issued lots of scripts that they 
> wrote source a file that is in clear text which contains usernames / 
> passwords etc.  Without having to re-write a ton of existing bash scripts, 
> I wanted to use Go and have that set the usernames / passwords and 
> introduce some user validation etc. Is there a way for Go to set 
> environment variables that persist to the script that ran the go program? 
> For example:
>
> Go Program SetPass:
>
> package main
>
> import (
> "fmt"
> "os"
>
> )
>
> func main() {
>
>  os.Setenv("MyPass","Abc.1234")
>
> }
>
>
>
> Bash Script:
>
> #!/bin/bash
>
> /usr/local/bin/SetPass
>
> echo $MyPass
>
>
> This is a simplistic example, in the real script I'd do things like look 
> at the user / groups that are running the program to ensure that they've 
> got the rights to run.
>
> Thanks,
>
> Rich
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

Thanks! I'm sure I'll never forget this experience :)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

Sorry guys. False alarm. The real issue was that we needed to use 
covermode=atomic instead of covermode=count in our CI.

On Thursday, February 2, 2017 at 12:43:09 PM UTC-5, Aaron Wood wrote:
>
> Hi all,
>
> I'm seeing a very strange issue that I'm not quite sure how to fix:
>
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80
>
> This causes a data race, and of course we only see it when running on a 
> beefy server with many cores :) I've never dealt with handling concurrency 
> and channels/selects when they're built dynamically using reflection. I've 
> tried surrounding the access on line 25 with a mutex but that does not 
> solve anything. Looking more into reflect.ValueOf() I see that it always 
> escapes to the heap as well as works with unsafe pointers in the 
> unpackEface() method 
> https://golang.org/src/reflect/value.go?s=61558:61591#L2073
>
> What is the proper way to handle the data race here?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

Oh man, it looks like you can't use covermode=count with race 
detection? https://github.com/golang/go/issues/12118

On Thursday, February 2, 2017 at 12:43:09 PM UTC-5, Aaron Wood wrote:
>
> Hi all,
>
> I'm seeing a very strange issue that I'm not quite sure how to fix:
>
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80
>
> This causes a data race, and of course we only see it when running on a 
> beefy server with many cores :) I've never dealt with handling concurrency 
> and channels/selects when they're built dynamically using reflection. I've 
> tried surrounding the access on line 25 with a mutex but that does not 
> solve anything. Looking more into reflect.ValueOf() I see that it always 
> escapes to the heap as well as works with unsafe pointers in the 
> unpackEface() method 
> https://golang.org/src/reflect/value.go?s=61558:61591#L2073
>
> What is the proper way to handle the data race here?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Dynamic select/channel causes data race

[Kale Blankenship]

Ah, right. You need to use `-covermode=atomic` if you want it to work with
the race detector, otherwise the coverage profiling itself can cause races.

On Thu, Feb 2, 2017 at 11:49 AM, Aaron Wood  wrote:

> Okay, so what's interesting is that the races only show up when I run
> go test -race -covermode=count -coverprofile=count_tmp.out
>
> Running either
> go test -race
> or
> go test -race -cover
> does not report any races. Does something seem fishy here?
>
> On Thursday, February 2, 2017 at 12:43:09 PM UTC-5, Aaron Wood wrote:
>
>> Hi all,
>>
>> I'm seeing a very strange issue that I'm not quite sure how to fix:
>>
>> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
>> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80
>>
>> This causes a data race, and of course we only see it when running on a
>> beefy server with many cores :) I've never dealt with handling concurrency
>> and channels/selects when they're built dynamically using reflection. I've
>> tried surrounding the access on line 25 with a mutex but that does not
>> solve anything. Looking more into reflect.ValueOf() I see that it always
>> escapes to the heap as well as works with unsafe pointers in the
>> unpackEface() method https://golang.org/src/
>> reflect/value.go?s=61558:61591#L2073
>>
>> What is the proper way to handle the data race here?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

Okay, so what's interesting is that the races only show up when I run
go test -race -covermode=count -coverprofile=count_tmp.out

Running either
go test -race
or
go test -race -cover
does not report any races. Does something seem fishy here?

On Thursday, February 2, 2017 at 12:43:09 PM UTC-5, Aaron Wood wrote:
>
> Hi all,
>
> I'm seeing a very strange issue that I'm not quite sure how to fix:
>
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80
>
> This causes a data race, and of course we only see it when running on a 
> beefy server with many cores :) I've never dealt with handling concurrency 
> and channels/selects when they're built dynamically using reflection. I've 
> tried surrounding the access on line 25 with a mutex but that does not 
> solve anything. Looking more into reflect.ValueOf() I see that it always 
> escapes to the heap as well as works with unsafe pointers in the 
> unpackEface() method 
> https://golang.org/src/reflect/value.go?s=61558:61591#L2073
>
> What is the proper way to handle the data race here?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

I am actually! I didn't realize that reporting on coverage messes with the 
output.
Let me run the tests without coverage enabled...

On Thursday, February 2, 2017 at 2:35:11 PM UTC-5, Kale B wrote:
>
> Are you running your test with coverage enabled? The coverage rewriting 
> causes the race detector to indicate the wrong lines in the source file.
>
> On Thu, Feb 2, 2017 at 11:33 AM, Aaron Wood  > wrote:
>
>> I can't unfortunately since it's private right now :( I can show you the 
>> entire back off source file here if you'd like, I just can't link you to 
>> anything right now.
>>
>> On Thursday, February 2, 2017 at 2:29:37 PM UTC-5, James Bardin wrote:
>>>
>>> Can you point to the actual source you're using? The filenames and line 
>>> numbers in the stack traces don't line up with the file you linked 
>>> originally. 
>>>
>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Dynamic select/channel causes data race

[Kale Blankenship]

Are you running your test with coverage enabled? The coverage rewriting
causes the race detector to indicate the wrong lines in the source file.

On Thu, Feb 2, 2017 at 11:33 AM, Aaron Wood  wrote:

> I can't unfortunately since it's private right now :( I can show you the
> entire back off source file here if you'd like, I just can't link you to
> anything right now.
>
> On Thursday, February 2, 2017 at 2:29:37 PM UTC-5, James Bardin wrote:
>>
>> Can you point to the actual source you're using? The filenames and line
>> numbers in the stack traces don't line up with the file you linked
>> originally.
>>
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

I can't unfortunately since it's private right now :( I can show you the 
entire back off source file here if you'd like, I just can't link you to 
anything right now.

On Thursday, February 2, 2017 at 2:29:37 PM UTC-5, James Bardin wrote:
>
> Can you point to the actual source you're using? The filenames and line 
> numbers in the stack traces don't line up with the file you linked 
> originally. 
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: Dynamic select/channel causes data race

[James Bardin]

Can you point to the actual source you're using? The filenames and line
numbers in the stack traces don't line up with the file you linked
originally.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] How do you take a cipher.StreamWriter and pass it to a io.Reader

[Justin C]

I have a encryption function that gives a  cipher.StreamWriter 

writer := {S: stream, W: outFile}

I am trying to upload it to using a library that has a function that is 
uploadFile that requires a IO.reader is there a way to pass the writer to 
the io.reader so the file is in crypted then uploaded?

UploadFile(outputFileName, metadata, io.Reader)

My code looks like this


writer := {S: stream, W: outFile}
// Copy the input file to the output file, encrypting as we go.

metadata := make(map[string]string)
b2h.currentBucket.UploadFile(outputFileName, metadata, writer)

I get this error.
cannot use writer (type *cipher.StreamWriter) as type io.Reader in argument 
to b2h.currentBucket.UploadFile: *cipher.StreamWriter does not implement 
io.Reader (missing Read method)

I'm having a little trouble understanding how this is suppose to work 
together.
Does anyone have any ideas? 
Thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

Also, the tests that trigger this case look like this:

// Verify that the burst notifier both returns correctly and provides the 
right data from its channel.
func TestBurstNotifier(t *testing.T) {
tokens := BurstNotifier(5, 5*time.Second, 5*time.Second, make(chan 
struct{}))

if reflect.TypeOf(tokens) != reflect.TypeOf(make(<-chan struct{})) {
t.Fatal("Burst notifier returns incorrect channel type")
}
if reflect.TypeOf(<-tokens) != reflect.TypeOf(*new(struct{})) {
t.Fatal("Token is of the wrong type")
}
}

// Verify that the notifier both returns correctly and provides the right 
data from its channel.
func TestNotifier(t *testing.T) {
tokens := Notifier(5*time.Second, 5*time.Second, make(chan struct{}))

if reflect.TypeOf(tokens) != reflect.TypeOf(make(<-chan struct{})) {
t.Fatal("Burst notifier returns incorrect channel type")
}
if reflect.TypeOf(<-tokens) != reflect.TypeOf(*new(struct{})) {
t.Fatal("Token is of the wrong type")
}
}




On Thursday, February 2, 2017 at 1:43:12 PM UTC-5, James Bardin wrote:
>
> Can you post the actual output from the race detector?
>
>
> On Thursday, February 2, 2017 at 12:43:09 PM UTC-5, Aaron Wood wrote:
>>
>> Hi all,
>>
>> I'm seeing a very strange issue that I'm not quite sure how to fix:
>>
>> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
>> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80
>>
>> This causes a data race, and of course we only see it when running on a 
>> beefy server with many cores :) I've never dealt with handling concurrency 
>> and channels/selects when they're built dynamically using reflection. I've 
>> tried surrounding the access on line 25 with a mutex but that does not 
>> solve anything. Looking more into reflect.ValueOf() I see that it always 
>> escapes to the heap as well as works with unsafe pointers in the 
>> unpackEface() method 
>> https://golang.org/src/reflect/value.go?s=61558:61591#L2073
>>
>> What is the proper way to handle the data race here?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Dynamic select/channel causes data race

[Aaron Wood]

==
WARNING: DATA RACE
Read at 0x00641d50 by goroutine 8:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:80 +0x55

Previous write at 0x00641d50 by goroutine 7:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:80 +0x6f

Goroutine 8 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9

Goroutine 7 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9
==
==
WARNING: DATA RACE
Read at 0x00641d54 by goroutine 9:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:85 +0xd1

Previous write at 0x00641d54 by goroutine 7:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:85 +0xeb

Goroutine 9 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9

Goroutine 7 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9
==
==
WARNING: DATA RACE
Read at 0x00641d54 by goroutine 8:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:85 +0xd1

Previous write at 0x00641d54 by goroutine 7:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:85 +0xeb

Goroutine 8 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9

Goroutine 7 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9
==
==
WARNING: DATA RACE
Read at 0x00641d54 by goroutine 11:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:85 +0xd1

Previous write at 0x00641d54 by goroutine 7:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:85 +0xeb

Goroutine 11 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9

Goroutine 7 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 +0x1d1
  mesos-go/backoff.TestBurstNotifier()
  /go/src/mesos-go/backoff/backoff_test.go:11 +0x96
  testing.tRunner()
  /usr/local/go/src/testing/testing.go:610 +0xc9
==
==
WARNING: DATA RACE
Read at 0x00641d68 by goroutine 11:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:97 +0x3df

Previous write at 0x00641d68 by goroutine 8:
  mesos-go/backoff.Notifier.func1()
  mesos-go/backoff/_test/_obj_test/backoff.go:97 +0x3f9

Goroutine 11 (running) created at:
  mesos-go/backoff.Notifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:118 +0x1d3
  mesos-go/backoff.BurstNotifier()
  mesos-go/backoff/_test/_obj_test/backoff.go:25 

Re: [go-nuts] Re: is it possible to speed up type assertion?

['Axel Wagner' via golang-nuts]

Making the gap smaller means making the compiler dumber. And I disagree
with you about microbenchmarks. There is a difference between benchmarks
and microbenchmarks and using the latter as the primary means of evaluating
performance is plain wrong. Nothing wrong with artificial benchmarks in
itself; if you benchmark how quickly a json-decoder decodes some artificial
set of data is fine, it's reasonably close to real-world to give you some
indication of performance. Benchmarking some code that will compile to only
a handful of instructions will give you simply zero information.

On Thu, Feb 2, 2017 at 7:46 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:

> The compiler field hasn't so far reached a state of sophistication in
> handling microbenchmarks like the human mind does handle them. Until that
> state is reached in compiler technology, it is natural for people to keep
> using microbenchmarks as one of the primary performance indicators.
>
> The observed difference is in [how a human mind would execute the
> microbenchmark in order to execute it in the shortest amount of time
> possible] compared to [how the microbenchmark is being executed on a real
> CPU running instructions generated by a real compiler].
>
> Rational path for compiler technology is to make the observed gap
> gradually smaller.
>
> On Thursday, February 2, 2017 at 6:56:09 PM UTC+1, Axel Wagner wrote:
>>
>> The point is, that (as repeatedly pointed out) microbenchmarks just, in
>> general, don't tell you anything really useful, as any effects you measure
>> could be due to anything ranging from random noise, over compiler
>> optimizations to code-alignments. Avoiding type assertions based on
>> anything contained in this thread is pretty unreasonable; I see zero
>> evidence here, that they are, at all, a problem.
>>
>> Go gives you pretty amazing tools to find the actual bottlenecks in your
>> code. Again: Write your code as it's most convenient, then run it with
>> production data on production hardware and *if* it isn't fast enough,
>> use pprof and similar tools to find the actual bottlenecks. They very
>> likely won't be type-assertions.
>>
>> On Thu, Feb 2, 2017 at 5:16 PM, Jan Ziak <0xe2.0x...@gmail.com> wrote:
>>
>>> A large difference between an implementable highly-optimized run-time
>>> performance of feature X and the physically implemented run-time
>>> performance of feature X causes programmers to avoid X.
>>>
>>> On Thursday, February 2, 2017 at 3:32:16 PM UTC+1, Marvin Renich wrote:

 * Marvin Renich  [170202 09:22]:
 >
 > BenchmarkIface is testing the wrong thing; the value is swamped by
 the
 > implicit conversion of d (type T) to the function argument of type I.

 Or, maybe it is testing the correct thing.  That is the problem with
 microbenchmarking.  Try benchmarking your actual code instead.  :-P

 ...Marvin

 --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Jan Ziak]

The compiler field hasn't so far reached a state of sophistication in 
handling microbenchmarks like the human mind does handle them. Until that 
state is reached in compiler technology, it is natural for people to keep 
using microbenchmarks as one of the primary performance indicators.

The observed difference is in [how a human mind would execute the 
microbenchmark in order to execute it in the shortest amount of time 
possible] compared to [how the microbenchmark is being executed on a real 
CPU running instructions generated by a real compiler].

Rational path for compiler technology is to make the observed gap gradually 
smaller.

On Thursday, February 2, 2017 at 6:56:09 PM UTC+1, Axel Wagner wrote:
>
> The point is, that (as repeatedly pointed out) microbenchmarks just, in 
> general, don't tell you anything really useful, as any effects you measure 
> could be due to anything ranging from random noise, over compiler 
> optimizations to code-alignments. Avoiding type assertions based on 
> anything contained in this thread is pretty unreasonable; I see zero 
> evidence here, that they are, at all, a problem.
>
> Go gives you pretty amazing tools to find the actual bottlenecks in your 
> code. Again: Write your code as it's most convenient, then run it with 
> production data on production hardware and *if* it isn't fast enough, use 
> pprof and similar tools to find the actual bottlenecks. They very likely 
> won't be type-assertions.
>
> On Thu, Feb 2, 2017 at 5:16 PM, Jan Ziak <0xe2.0x...@gmail.com 
> > wrote:
>
>> A large difference between an implementable highly-optimized run-time 
>> performance of feature X and the physically implemented run-time 
>> performance of feature X causes programmers to avoid X.
>>
>> On Thursday, February 2, 2017 at 3:32:16 PM UTC+1, Marvin Renich wrote:
>>>
>>> * Marvin Renich  [170202 09:22]: 
>>> > 
>>> > BenchmarkIface is testing the wrong thing; the value is swamped by the 
>>> > implicit conversion of d (type T) to the function argument of type I. 
>>>
>>> Or, maybe it is testing the correct thing.  That is the problem with 
>>> microbenchmarking.  Try benchmarking your actual code instead.  :-P 
>>>
>>> ...Marvin 
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Dynamic select/channel causes data race

[James Bardin]

Can you post the actual output from the race detector?


On Thursday, February 2, 2017 at 12:43:09 PM UTC-5, Aaron Wood wrote:
>
> Hi all,
>
> I'm seeing a very strange issue that I'm not quite sure how to fix:
>
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
> https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80
>
> This causes a data race, and of course we only see it when running on a 
> beefy server with many cores :) I've never dealt with handling concurrency 
> and channels/selects when they're built dynamically using reflection. I've 
> tried surrounding the access on line 25 with a mutex but that does not 
> solve anything. Looking more into reflect.ValueOf() I see that it always 
> escapes to the heap as well as works with unsafe pointers in the 
> unpackEface() method 
> https://golang.org/src/reflect/value.go?s=61558:61591#L2073
>
> What is the proper way to handle the data race here?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: HTTP requests bottlenecks

[emartinez1847]

Ben, you are correct the dial thing, not sure how that ended up commented.

Anyway, I've removed the custom dial (and tried with the timeout enabled as 
well), it did increase r/s a lil bit (5%-10% or so) but it increased the 
amounts of timeouts on the remote urls as well.

El jueves, 2 de febrero de 2017, 14:11:29 (UTC-3), James Bardin escribió:
>
> First things I notice are that you're overriding the default dialer with 
> one that doesn't timeout, and you've commented out ReadTimeout in the 
> client. Both of those could indefinitely hold up client connections 
> regardless of the DoTimeout call, which just ensures that the Do function 
> returns before the deadline. 
>
> Otherwise, I think you're going to have to instrument the code a little 
> better to see what is holding you up. 
>
>
> On Thursday, February 2, 2017 at 11:27:38 AM UTC-5, emarti...@gmail.com 
> wrote:
>>
>> Thanks for the answer.
>>
>> Yes, it seems to be blocking, I just fixed it with: 
>> http://blog.sgmansfield.com/2016/01/the-hidden-dangers-of-default-rand/
>>
>> After that change my code is working a lil bit better but still I see a 
>> ton of timeouts + high latency on responses. Maybe the code is locking on 
>> another part of the code?
>>
>> El jueves, 2 de febrero de 2017, 4:09:07 (UTC-3), land...@gmail.com 
>> escribió:
>>>
>>> func randInt(min int, max int) int {
>>> rand.Seed(int64(time.Now().Nanosecond()))
>>> return min + rand.Intn(max-min)
>>> }
>>>
>>> Is the culprit. the default rand locks globally for concurrent access. 
>>>  You need to create a new rand in each goroutine you want to use it in, for 
>>> maximum speed.
>>>
>>> On Wednesday, February 1, 2017 at 8:26:07 PM UTC-6, emarti...@gmail.com 
>>> wrote:

 Hello,

 I'm writing a POC for a future RTB platform. Basically I'm doing stress 
 tests by receiving HTTP requests and performing HTTP GET requests in the 
 background.

 Issue I face is that when i try to scale up, URLs starts to timeout. I 
 am trying to find what our bottleneck is but so far no luck (we aren't 
 running out of ephemeral ports).

 Remote URLs takes about 500ms-1000ms to respond. We are stress testing 
 it with wrk with 5000 concurrent requests. (5000 incoming requests which 
 translated into 30 remote URL requests = 150k requests)

 Here is our code:

 package main

 import (
  "net/http"
  "time"
   "runtime"
   "io"
   "net"
   "github.com/valyala/fasthttp"
   "math/rand"
 )

 type HttpResponse struct {
   url  string
   response *http.Response
   err  error
 }

 var urls = []string{
 //lots of urls
 }


 //func asyncHttpGets(urls []string) []*HttpResponse {

 var(
  clients []fasthttp.Client
  total_clients int
  max_urls int
 )

 func init(){
 max_urls=30
   clients =append(clients, create_client() )
 }

 func create_client() fasthttp.Client{

 return fasthttp.Client{
  Dial: func(addr string) (net.Conn, error) {
 var dialer = net.Dialer{}
 return dialer.Dial("tcp", addr)
 },

 MaxIdleConnDuration:30*time.Second,
 MaxConnsPerHost:2024,
  //ReadTimeout: 1*time.Second,
 }
 }

 func randInt(min int, max int) int {
 rand.Seed(int64(time.Now().Nanosecond()))
 return min + rand.Intn(max-min)
 }


 func asyncHttpGets(urls []string) []string{
  ch:=make(chan string,max_urls)
 var responses  []string

   cl:=0
   for i := 0; i <= max_urls; i++ {
 url:=urls[i]
   go func(url string) {

req := fasthttp.AcquireRequest()
  req.SetRequestURI(url)
  req.Header.Add("Connection", "keep-alive")
  resp := fasthttp.AcquireResponse()
  clients[cl].DoTimeout(req, resp,1*time.Second)

 bodyBytes := resp.Body()
   ch <- string(bodyBytes)

   fasthttp.ReleaseRequest(req)
   fasthttp.ReleaseResponse(resp)
   }(url)
   }

   for {
   select {
   case r := <-ch:
   responses = append(responses, r)
   if len(responses) == max_urls {
   return responses
   }
   }
   }
   return responses
 }

 func hello(w http.ResponseWriter, r *http.Request) {
   results := asyncHttpGets(urls)
   for _, result := range results {
io.WriteString(w,"%s status: %s" + " " + result + "\n")
   }
 }


  func main() {
 runtime.GOMAXPROCS(0)

 server8000 := http.NewServeMux()
 server8000.HandleFunc("/", hello)
 http.ListenAndServe(":8001", 

Re: [go-nuts] Re: is it possible to speed up type assertion?

['Axel Wagner' via golang-nuts]

The point is, that (as repeatedly pointed out) microbenchmarks just, in
general, don't tell you anything really useful, as any effects you measure
could be due to anything ranging from random noise, over compiler
optimizations to code-alignments. Avoiding type assertions based on
anything contained in this thread is pretty unreasonable; I see zero
evidence here, that they are, at all, a problem.

Go gives you pretty amazing tools to find the actual bottlenecks in your
code. Again: Write your code as it's most convenient, then run it with
production data on production hardware and *if* it isn't fast enough, use
pprof and similar tools to find the actual bottlenecks. They very likely
won't be type-assertions.

On Thu, Feb 2, 2017 at 5:16 PM, Jan Ziak <0xe2.0x9a.0...@gmail.com> wrote:

> A large difference between an implementable highly-optimized run-time
> performance of feature X and the physically implemented run-time
> performance of feature X causes programmers to avoid X.
>
> On Thursday, February 2, 2017 at 3:32:16 PM UTC+1, Marvin Renich wrote:
>>
>> * Marvin Renich  [170202 09:22]:
>> >
>> > BenchmarkIface is testing the wrong thing; the value is swamped by the
>> > implicit conversion of d (type T) to the function argument of type I.
>>
>> Or, maybe it is testing the correct thing.  That is the problem with
>> microbenchmarking.  Try benchmarking your actual code instead.  :-P
>>
>> ...Marvin
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: [ANN] template-compiler: compile go templates to go code

[mhhcbon]

HI!

Some updates here, again!

Down the road, i realized my benchmarks where not so correct,
kind of erratic with sometimes huge improvements, 
sometimes minor improvements.

I got this sorted out, and found the root of evil to be 
`template.HTMLEscapeString`

So i worked this out, and now i m very happy to show you the latest 
benchmarks,
as they really are encouraging,

  $ go test -bench=. -benchmem
  BenchmarkRenderWithCompiledTemplateA-42000   78.9 ns/op   
  48 B/op  1 allocs/op
  BenchmarkRenderWithJitTemplateA-4  300   668 ns/op   
   96 B/op  2 allocs/op
  BenchmarkRenderWithCompiledTemplateB-42000   82.4 ns/op   
  48 B/op  1 allocs/op
  BenchmarkRenderWithJitTemplateB-4  300   603 ns/op   
   96 B/op  2 allocs/op
  BenchmarkRenderWithCompiledTemplateC-4  50  2530 ns/op   
  192 B/op  6 allocs/op
  BenchmarkRenderWithJitTemplateC-45 38245 ns/op   
 3641 B/op 82 allocs/op
  BenchmarkRenderWithCompiledTemplateD-42000   114 ns/op   
   48 B/op  1 allocs/op
  BenchmarkRenderWithJitTemplateD-4  300   809 ns/op   
  144 B/op  3 allocs/op

  // the next 2 benchmarks are particularly encouraging
  // as they involve 2k html string escaping
  BenchmarkRenderWithCompiledTemplateE-4   1103929 ns/op   
  160 B/op  2 allocs/op
  BenchmarkRenderWithJitTemplateE-4  300   6207912 ns/op   
   656598 B/op  18012 allocs/op
  BenchmarkRenderWithCompiledTemplateF-4   1111047 ns/op   
  160 B/op  2 allocs/op
  BenchmarkRenderWithJitTemplateF-4  200   5836766 ns/op   
   657000 B/op  18024 allocs/op

The trick was to get ride of calls to `template.HTMLEscapeString`

See its code here
https://golang.org/src/text/template/funcs.go?s=13003:13041#L505
The problem in this function is that it trades an allocation for doubled 
iteration on the string to escape.
Which makes sense for the std package as it does a lots of allocations,
but in the case of compiled templates it does not make sense as the 
allocations count 
is drastically reduced.

Now, one could say, right, lets just invoke `template.HTMLEscape(writer, 
[]byte(string))`
instead of `template.HTMLEscapeString`.
I gone down that road too, and its not totally satisfying because it 
actually creates
as many new allocations as there are calls to HTMLEscape (see the 
[]byte(string) conversion).

In the end, i decided to use a buffer, it creates a static extra allocation,
at beginning of every template function, but avoid so many possible and 
future
string to byte conversion.

Finally, html escaping string looks like this,

func fnaTplaTpl0(t parse.Templater, w io.Writer, indata interface { }) 
error { 
var bw bytes.Buffer
// ...
 bw.WriteString(somestringhere)
 template.HTMLEscape(w, bw.Bytes())
 bw.Reset()
}

Doing so the allocation count stays very low stable, and it s still 
possible to reduce cpu usage.


I looked into the std packages, i could not find an obvious way to 
implement that optimization, 
maybe you have some ideas ?

Anyway, there s still a lot to do on this compiler!
The next steps will be 
- to remove duplicated std package, i made my mind to not even try to patch 
core code.
- to implement more optimizations
- to improve the end user programmatic interface

That's it!

On Saturday, January 21, 2017 at 3:58:53 PM UTC+1, mhh...@gmail.com wrote:
>
> Hi,
>
> I have been working on a package to compile go templates to regular go 
> code.
> Today i can announce an early release of it.
>
> Find it here 
> https://github.com/mh-cbon/template-compiler
>
> There is still have a lot to test and comment, 
> and to detect some edge cases.
> But the general idea and structure exists, does work,
>  and shows a way to make that happen.
>
> If you are interested to get this production ready, 
> i d like very much to see your questions and PR!
>
> Being in a hurry, this announce is very short, 
> I expect a more detailed announce later.
>
> This said, i d like to ask the go team if the template package
> could have a new structure to register a func as a template,
> see 1 
> ,
>  
> 2 
> 
>  
> and 3 
> 
>
> Finally, i want to take chance of this message to wish
> to the golang, core team, contributors, users
> an happy new year and all the best for 2017!
>
> Happy coding!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

[go-nuts] Dynamic select/channel causes data race

[Aaron Wood]

Hi all,

I'm seeing a very strange issue that I'm not quite sure how to fix:

https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L25
https://github.com/mesos/mesos-go/blob/next/backoff/backoff.go#L80

This causes a data race, and of course we only see it when running on a 
beefy server with many cores :) I've never dealt with handling concurrency 
and channels/selects when they're built dynamically using reflection. I've 
tried surrounding the access on line 25 with a mutex but that does not 
solve anything. Looking more into reflect.ValueOf() I see that it always 
escapes to the heap as well as works with unsafe pointers in the 
unpackEface() 
method https://golang.org/src/reflect/value.go?s=61558:61591#L2073

What is the proper way to handle the data race here?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: HTTP requests bottlenecks

[James Bardin]

First things I notice are that you're overriding the default dialer with 
one that doesn't timeout, and you've commented out ReadTimeout in the 
client. Both of those could indefinitely hold up client connections 
regardless of the DoTimeout call, which just ensures that the Do function 
returns before the deadline. 

Otherwise, I think you're going to have to instrument the code a little 
better to see what is holding you up. 


On Thursday, February 2, 2017 at 11:27:38 AM UTC-5, emarti...@gmail.com 
wrote:
>
> Thanks for the answer.
>
> Yes, it seems to be blocking, I just fixed it with: 
> http://blog.sgmansfield.com/2016/01/the-hidden-dangers-of-default-rand/
>
> After that change my code is working a lil bit better but still I see a 
> ton of timeouts + high latency on responses. Maybe the code is locking on 
> another part of the code?
>
> El jueves, 2 de febrero de 2017, 4:09:07 (UTC-3), land...@gmail.com 
> escribió:
>>
>> func randInt(min int, max int) int {
>> rand.Seed(int64(time.Now().Nanosecond()))
>> return min + rand.Intn(max-min)
>> }
>>
>> Is the culprit. the default rand locks globally for concurrent access. 
>>  You need to create a new rand in each goroutine you want to use it in, for 
>> maximum speed.
>>
>> On Wednesday, February 1, 2017 at 8:26:07 PM UTC-6, emarti...@gmail.com 
>> wrote:
>>>
>>> Hello,
>>>
>>> I'm writing a POC for a future RTB platform. Basically I'm doing stress 
>>> tests by receiving HTTP requests and performing HTTP GET requests in the 
>>> background.
>>>
>>> Issue I face is that when i try to scale up, URLs starts to timeout. I 
>>> am trying to find what our bottleneck is but so far no luck (we aren't 
>>> running out of ephemeral ports).
>>>
>>> Remote URLs takes about 500ms-1000ms to respond. We are stress testing 
>>> it with wrk with 5000 concurrent requests. (5000 incoming requests which 
>>> translated into 30 remote URL requests = 150k requests)
>>>
>>> Here is our code:
>>>
>>> package main
>>>
>>> import (
>>>  "net/http"
>>>  "time"
>>>   "runtime"
>>>   "io"
>>>   "net"
>>>   "github.com/valyala/fasthttp"
>>>   "math/rand"
>>> )
>>>
>>> type HttpResponse struct {
>>>   url  string
>>>   response *http.Response
>>>   err  error
>>> }
>>>
>>> var urls = []string{
>>> //lots of urls
>>> }
>>>
>>>
>>> //func asyncHttpGets(urls []string) []*HttpResponse {
>>>
>>> var(
>>>  clients []fasthttp.Client
>>>  total_clients int
>>>  max_urls int
>>> )
>>>
>>> func init(){
>>> max_urls=30
>>>   clients =append(clients, create_client() )
>>> }
>>>
>>> func create_client() fasthttp.Client{
>>>
>>> return fasthttp.Client{
>>>  Dial: func(addr string) (net.Conn, error) {
>>> var dialer = net.Dialer{}
>>> return dialer.Dial("tcp", addr)
>>> },
>>>
>>> MaxIdleConnDuration:30*time.Second,
>>> MaxConnsPerHost:2024,
>>>  //ReadTimeout: 1*time.Second,
>>> }
>>> }
>>>
>>> func randInt(min int, max int) int {
>>> rand.Seed(int64(time.Now().Nanosecond()))
>>> return min + rand.Intn(max-min)
>>> }
>>>
>>>
>>> func asyncHttpGets(urls []string) []string{
>>>  ch:=make(chan string,max_urls)
>>> var responses  []string
>>>
>>>   cl:=0
>>>   for i := 0; i <= max_urls; i++ {
>>> url:=urls[i]
>>>   go func(url string) {
>>>
>>>req := fasthttp.AcquireRequest()
>>>  req.SetRequestURI(url)
>>>  req.Header.Add("Connection", "keep-alive")
>>>  resp := fasthttp.AcquireResponse()
>>>  clients[cl].DoTimeout(req, resp,1*time.Second)
>>>
>>> bodyBytes := resp.Body()
>>>   ch <- string(bodyBytes)
>>>
>>>   fasthttp.ReleaseRequest(req)
>>>   fasthttp.ReleaseResponse(resp)
>>>   }(url)
>>>   }
>>>
>>>   for {
>>>   select {
>>>   case r := <-ch:
>>>   responses = append(responses, r)
>>>   if len(responses) == max_urls {
>>>   return responses
>>>   }
>>>   }
>>>   }
>>>   return responses
>>> }
>>>
>>> func hello(w http.ResponseWriter, r *http.Request) {
>>>   results := asyncHttpGets(urls)
>>>   for _, result := range results {
>>>io.WriteString(w,"%s status: %s" + " " + result + "\n")
>>>   }
>>> }
>>>
>>>
>>>  func main() {
>>> runtime.GOMAXPROCS(0)
>>>
>>> server8000 := http.NewServeMux()
>>> server8000.HandleFunc("/", hello)
>>> http.ListenAndServe(":8001", server8000)
>>>  }
>>>
>>>
>>> Any help is really appreciated.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Pass variables to bash

[Shawn Milochik]

You can set environment variables in an exec.Cmd:
https://golang.org/pkg/os/exec/#Cmd

% cat example.sh
#!/usr/bin/env bash

echo "The password is $password."

% export password=old_password

% bash example.sh
The password is old_password.


% cat example.go
package main

import (
"fmt"
"os"
"os/exec"
)

func main() {
c := exec.Command("bash", "example.sh")
c.Env = []string{"password=monkey"}
c.Stdout = os.Stdout
c.Stderr = os.Stderr
if err := c.Run(); err != nil {
fmt.Printf("Command failed: %s\n", err)
}
}

% go run example.go
The password is monkey.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Pass variables to bash

[howardcshaw]

I don't think you can do what you want to do, precisely.

But, I do think you can solve your problem!

Instead of actually writing to your environment in the Go code (which can 
only be seen by *child* processes *of the Go program*!), just print to 
stdout.

That is, instead of os.Setenv("MyPass","Abc.1234"), you would just do 
fmt.Printf("export %s=%s\n", envVarName, envVarValue),

so the output of the program is lines like 

export MyPass=Abc.1234
export somethingelse=thisotherthing

Then take that output and eval it in the shell script.

Example:

osm@Beast:~/test$ echo "#!/bin/sh
> echo export TEST=TRUE
> echo export TEST2=FALSE" > test.sh

osm@Beast:~/test$ cat test.sh
#!/bin/sh
echo export TEST=TRUE
echo export TEST2=FALSE

osm@Beast:~/test$ env | grep TEST
osm@Beast:~/test$ ./test.sh
export TEST=TRUE
export TEST2=FALSE
osm@Beast:~/test$ env | grep TEST
osm@Beast:~/test$ eval "$(./test.sh)"
osm@Beast:~/test$ env | grep TEST
TEST=TRUE
TEST2=FALSE

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Pass variables to bash

[mhhcbon]

this : 
http://craigwickesser.com/2015/02/golang-cmd-with-custom-environment/ ?


On Thursday, February 2, 2017 at 5:23:49 PM UTC+1, Rich wrote:
>
> Hi All,
>
> My dev team did a very bad thing and issued lots of scripts that they 
> wrote source a file that is in clear text which contains usernames / 
> passwords etc.  Without having to re-write a ton of existing bash scripts, 
> I wanted to use Go and have that set the usernames / passwords and 
> introduce some user validation etc. Is there a way for Go to set 
> environment variables that persist to the script that ran the go program? 
> For example:
>
> Go Program SetPass:
>
> package main
>
> import (
> "fmt"
> "os"
>
> )
>
> func main() {
>
>  os.Setenv("MyPass","Abc.1234")
>
> }
>
>
>
> Bash Script:
>
> #!/bin/bash
>
> /usr/local/bin/SetPass
>
> echo $MyPass
>
>
> This is a simplistic example, in the real script I'd do things like look 
> at the user / groups that are running the program to ensure that they've 
> got the rights to run.
>
> Thanks,
>
> Rich
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: HTTP requests bottlenecks

[emartinez1847]

Thanks for the answer.

Yes, it seems to be blocking, I just fixed it with: 
http://blog.sgmansfield.com/2016/01/the-hidden-dangers-of-default-rand/

After that change my code is working a lil bit better but still I see a ton 
of timeouts + high latency on responses. Maybe the code is locking on 
another part of the code?

El jueves, 2 de febrero de 2017, 4:09:07 (UTC-3), land...@gmail.com 
escribió:
>
> func randInt(min int, max int) int {
> rand.Seed(int64(time.Now().Nanosecond()))
> return min + rand.Intn(max-min)
> }
>
> Is the culprit. the default rand locks globally for concurrent access. 
>  You need to create a new rand in each goroutine you want to use it in, for 
> maximum speed.
>
> On Wednesday, February 1, 2017 at 8:26:07 PM UTC-6, emarti...@gmail.com 
> wrote:
>>
>> Hello,
>>
>> I'm writing a POC for a future RTB platform. Basically I'm doing stress 
>> tests by receiving HTTP requests and performing HTTP GET requests in the 
>> background.
>>
>> Issue I face is that when i try to scale up, URLs starts to timeout. I am 
>> trying to find what our bottleneck is but so far no luck (we aren't running 
>> out of ephemeral ports).
>>
>> Remote URLs takes about 500ms-1000ms to respond. We are stress testing it 
>> with wrk with 5000 concurrent requests. (5000 incoming requests which 
>> translated into 30 remote URL requests = 150k requests)
>>
>> Here is our code:
>>
>> package main
>>
>> import (
>>  "net/http"
>>  "time"
>>   "runtime"
>>   "io"
>>   "net"
>>   "github.com/valyala/fasthttp"
>>   "math/rand"
>> )
>>
>> type HttpResponse struct {
>>   url  string
>>   response *http.Response
>>   err  error
>> }
>>
>> var urls = []string{
>> //lots of urls
>> }
>>
>>
>> //func asyncHttpGets(urls []string) []*HttpResponse {
>>
>> var(
>>  clients []fasthttp.Client
>>  total_clients int
>>  max_urls int
>> )
>>
>> func init(){
>> max_urls=30
>>   clients =append(clients, create_client() )
>> }
>>
>> func create_client() fasthttp.Client{
>>
>> return fasthttp.Client{
>>  Dial: func(addr string) (net.Conn, error) {
>> var dialer = net.Dialer{}
>> return dialer.Dial("tcp", addr)
>> },
>>
>> MaxIdleConnDuration:30*time.Second,
>> MaxConnsPerHost:2024,
>>  //ReadTimeout: 1*time.Second,
>> }
>> }
>>
>> func randInt(min int, max int) int {
>> rand.Seed(int64(time.Now().Nanosecond()))
>> return min + rand.Intn(max-min)
>> }
>>
>>
>> func asyncHttpGets(urls []string) []string{
>>  ch:=make(chan string,max_urls)
>> var responses  []string
>>
>>   cl:=0
>>   for i := 0; i <= max_urls; i++ {
>> url:=urls[i]
>>   go func(url string) {
>>
>>req := fasthttp.AcquireRequest()
>>  req.SetRequestURI(url)
>>  req.Header.Add("Connection", "keep-alive")
>>  resp := fasthttp.AcquireResponse()
>>  clients[cl].DoTimeout(req, resp,1*time.Second)
>>
>> bodyBytes := resp.Body()
>>   ch <- string(bodyBytes)
>>
>>   fasthttp.ReleaseRequest(req)
>>   fasthttp.ReleaseResponse(resp)
>>   }(url)
>>   }
>>
>>   for {
>>   select {
>>   case r := <-ch:
>>   responses = append(responses, r)
>>   if len(responses) == max_urls {
>>   return responses
>>   }
>>   }
>>   }
>>   return responses
>> }
>>
>> func hello(w http.ResponseWriter, r *http.Request) {
>>   results := asyncHttpGets(urls)
>>   for _, result := range results {
>>io.WriteString(w,"%s status: %s" + " " + result + "\n")
>>   }
>> }
>>
>>
>>  func main() {
>> runtime.GOMAXPROCS(0)
>>
>> server8000 := http.NewServeMux()
>> server8000.HandleFunc("/", hello)
>> http.ListenAndServe(":8001", server8000)
>>  }
>>
>>
>> Any help is really appreciated.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Pass variables to bash

[Rich]

Hi All,

My dev team did a very bad thing and issued lots of scripts that they wrote 
source a file that is in clear text which contains usernames / passwords 
etc.  Without having to re-write a ton of existing bash scripts, I wanted 
to use Go and have that set the usernames / passwords and introduce some 
user validation etc. Is there a way for Go to set environment variables 
that persist to the script that ran the go program? For example:

Go Program SetPass:

package main

import (
"fmt"
"os"

)

func main() {

 os.Setenv("MyPass","Abc.1234")

}



Bash Script:

#!/bin/bash

/usr/local/bin/SetPass

echo $MyPass


This is a simplistic example, in the real script I'd do things like look at 
the user / groups that are running the program to ensure that they've got 
the rights to run.

Thanks,

Rich

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Jan Ziak]

A large difference between an implementable highly-optimized run-time 
performance of feature X and the physically implemented run-time 
performance of feature X causes programmers to avoid X.

On Thursday, February 2, 2017 at 3:32:16 PM UTC+1, Marvin Renich wrote:
>
> * Marvin Renich  [170202 09:22]: 
> > 
> > BenchmarkIface is testing the wrong thing; the value is swamped by the 
> > implicit conversion of d (type T) to the function argument of type I. 
>
> Or, maybe it is testing the correct thing.  That is the problem with 
> microbenchmarking.  Try benchmarking your actual code instead.  :-P 
>
> ...Marvin 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: $GOPTAH/bin and $PATH

[Dmitri Shuralyov]

Yes, it is.

As far as I know, everybody does it and/or expects you to. Unless you have 
some specific reason not to.


On Thursday, February 2, 2017 at 5:31:32 AM UTC-5, adrian...@hushmail.com 
wrote:
>
> Gophers, 
>
> Is it idiomatic to add $GOPTAH/bin to $PATH in order access a Go 
> executable across my filesystem?
>
> Aidy 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Rene Kaufmann]

BenchmarkAssertion-4 3 4.08 ns/op
BenchmarkAssertionOK-4   5 3.03 ns/op
BenchmarkBare-4   5 3.01 ns/op
BenchmarkIface-4 300055.1 ns/op
BenchmarkReflect-4   112.8 ns/op

CPU: Intel(R) Core(TM) i5 CPU   M 540  @ 2.53GHz

On Thu, Feb 2, 2017 at 10:51 AM Dave Cheney  wrote:

> 0.44ns/op is about 2.2.ghz, the compiler has optimised away your
> microbenchmark.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: XML unmarshal woes

[Henrik Johansson]

Ahh fantastic thanks a lot!

tors 2 feb. 2017 kl 15:11 skrev :

> Here is a playground link: https://play.golang.org/p/_wJEBd1L9x
>
> ABStuff string `xml:",innerxml"`
>
> You can capture the innerxml with an appropriate tag; you can also, as my
> third example in the above link demonstrates, capture multiple repeated
> tags like that to a list of values. If, as I think your comment implies,
> you need to preserve the order of a's and b's AND their order relative to
> each other, that obviously does not help, so the first example demonstrates
> the ,innerxml capture
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Marvin Renich]

* Marvin Renich  [170202 09:22]:
> 
> BenchmarkIface is testing the wrong thing; the value is swamped by the
> implicit conversion of d (type T) to the function argument of type I.

Or, maybe it is testing the correct thing.  That is the problem with
microbenchmarking.  Try benchmarking your actual code instead.  :-P

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Marvin Renich]

* T L  [170202 04:20]:
> 
> 
> On Thursday, February 2, 2017 at 4:58:32 PM UTC+8, Axel Wagner wrote:
> >
> > Hi,
> >
> > I can not really reproduce your results. I rewrote your code to use the 
> > builtin benchmarking: http://sprunge.us/IfQc
> > Giving, on my laptop:
> >
> > BenchmarkAssertion-4 10 2.89 ns/op
> > BenchmarkAssertionOK-4   5 2.66 ns/op
> > BenchmarkBare-4   10 2.22 ns/op
> > BenchmarkIface-4 500030.0 ns/op
> > BenchmarkReflect-4   2 9.74 ns/op
> >
> > Note, that a) yes, there is an overhead of the type-assertion, but b) it's 
> > pretty small, especially compared to the other things you're trying and c) 
> > it can be further reduced by using the two-value form (so that there is 
> > never a need to consider stack-unwinding).
> >
> > Overall, this smells like a micro-benchmark. I wouldn't worry too much 
> > about it until you have specific evidence that it's slowing down a real 
> > program.
> >
> 
> The result on my machine for your test:
> BenchmarkAssertion-4 517.9 ns/op
> BenchmarkAssertionOK-4   517.9 ns/op
> BenchmarkBare-4  20 3.93 ns/op
> BenchmarkIface-4 186.7 ns/op
> BenchmarkReflect-4   515.6 ns/op

BenchmarkIface is testing the wrong thing; the value is swamped by the
implicit conversion of d (type T) to the function argument of type I.
Try:

func BenchmarkIface(b *testing.B) {
count := 0
var t T = 1
d := I(t)
for i := 0; i < b.N; i++ {
Iface(, d)
}
}

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: XML unmarshal woes

[howardcshaw]

Here is a playground link: https://play.golang.org/p/_wJEBd1L9x

ABStuff string `xml:",innerxml"`

You can capture the innerxml with an appropriate tag; you can also, as my 
third example in the above link demonstrates, capture multiple repeated 
tags like that to a list of values. If, as I think your comment implies, 
you need to preserve the order of a's and b's AND their order relative to 
each other, that obviously does not help, so the first example demonstrates 
the ,innerxml capture

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: $GOPTAH/bin and $PATH

[Manlio Perillo]

Il giorno giovedì 2 febbraio 2017 11:31:32 UTC+1, adrian...@hushmail.com ha 
scritto:
>
> Gophers, 
>
> Is it idiomatic to add $GOPTAH/bin to $PATH in order access a Go 
> executable across my filesystem?
>
>
On my system I have set $GOBIN to ~/.local/bin and added ~/.local/bin to 
$PATH 


Manlio

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] XML unmarshal woes

[Henrik Johansson]

I have xml that looks like this:


  sadasd
  gfdfg
  ...
  ...


Where a and be can appear in any order and this order needs to be preserved.

I have managed to Marshal xml decently using a struct such as:

type Holder {
  A A
  B A
}

type A string
type B string

using a MarshalXML method on the "node" type but unmarshalling is
apparently more tricky.

Right now I would just settle for very crude hand parsing if I could just
get the the whole sub tree xml into a string in the corresponding
UnmarshalXML method.

Any ideas? I can't change this unfortunately which I would if I could. It
is hurting us elsewhere as well but it's just too much work atm.

Thx,

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Interfaces and sync.Mutex

[Jens Hausherr]

Thanks for the insights.

Am Donnerstag, 12. Januar 2017 21:02:21 UTC+1 schrieb Dave Cheney:
>
> If you had this
>
> type I interface { ... }
> type T struct {
> sync.Mutex
> }
> var t T
> var i I = t
>
> This is a mistake because a *copy* of the contents of t is assigned to i. 
>
> Every assignment in Go is a copy, there are no exceptions. 
>
> To solve this assignment the *address* of t to I.
>
> var i I = 
>
> Or more suscinctly
>
> var i I = {  }
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[mhhcbon]

is there a paper to read that introduces and explains benchmark pitfalls 
and BP for naive developers ?

there are papers out there which talks about go bench, it is sparse and 
often they pretext an obvious 
bad code to introduce the reader to the benchmarks go tooling .
Which is good, don't take me wrong, but that did not help me to avoid some 
pitfalls i faced IRL.

I suspect the topic is hairy to explain to beginners so they can be 
autonomous and confident,
in my experience and feeling, if you don t have a benchmarking background 
in other languages,
you are left to take the wrong decision because you can t produce the right 
understanding of the situation.

On Thursday, February 2, 2017 at 11:41:25 AM UTC+1, Axel Wagner wrote:
>
> I want to re-emphasize that all of these are micro-benchmarks. They say 
> nothing useful at all, as proven by this thread; in some circumstances, the 
> added cost may be significant, in others it isn't. The same rule that has 
> been repeated lots of times on this list still applies: Write your program 
> to be simple and readable, *if* it is too slow, benchmark and improve.​ 
> Basing your code on what any of these Benchmarks says is just ridiculous, 
> base it on the bottlenecks you measure in your actual, real-world program 
> running on the actual production hardware with actual production data.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] $GOPTAH/bin and $PATH

[Dave Cheney]

Yes

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 6:01:19 PM UTC+8, T L wrote:
>
> I found that it is much faster if the dynamic values are pointers instead 
> of non-pointer.
>

By looking the code of function assertE2T in runtime/iface.go,
ot looks memmove(to, from unsafe.Pointer, n uintptr) is slow for values 
with size <= one word.
Currently, official gc doesn't make optimizations for types with size <= 
one word.
Pointers are treated as special value in interface.
 

>
> package main
>
> import (
> "testing"
> )
>
> func AssertInt(c *int, d interface{}) {
> *c += d.(int)
> }
>
> func BenchmarkAssertionInt(b *testing.B) {
> count := 0
> var t int = 1
> d := (interface{})(t)
> for i := 0; i < b.N; i++ {
> AssertInt(, d)
> }
> }
>
> func AssertPtr(c *int, d interface{}) {
> *c += *d.(*int)
> }
>
> func BenchmarkAssertionPtr(b *testing.B) {
> count := 0
> var t int = 1
> d := (interface{})()
> for i := 0; i < b.N; i++ {
> AssertPtr(, d)
> }
> }
>
> BenchmarkAssertionInt-4   515.3 ns/op
> BenchmarkAssertionPtr-4   20 3.09 ns/op
>
>
>
> On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>>
>> Go's type assertion seems quite slow. The added cost is too much if it 
>> has to be in a tight loop. Here are the time taken on my laptop for the 
>> following code.
>>
>> https://play.golang.org/p/cA96miTkx_
>>
>>
>> chris$ time ./p
>> count=1073741824 time taken=7.899207181s
>> count=1073741824 time taken=300.601453ms
>>
>> real 0m8.205s
>> user 0m8.163s
>> sys 0m0.029s
>>
>> chris$ time luajit -e "count = 0
>> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
>> > print(count)"
>> 1073741824
>>
>> real 0m0.900s
>> user 0m0.891s
>> sys 0m0.005s
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

['Axel Wagner' via golang-nuts]

I want to re-emphasize that all of these are micro-benchmarks. They say
nothing useful at all, as proven by this thread; in some circumstances, the
added cost may be significant, in others it isn't. The same rule that has
been repeated lots of times on this list still applies: Write your program
to be simple and readable, *if* it is too slow, benchmark and improve.​
Basing your code on what any of these Benchmarks says is just ridiculous,
base it on the bottlenecks you measure in your actual, real-world program
running on the actual production hardware with actual production data.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] $GOPTAH/bin and $PATH

[Sander van Harmelen]

Don’t know if I would call it idiomatic, but (IMO) it makes things easier  for 
a few use cases so I’ve also added $GOPATH/bin to my $PATH :)

Sander


On 2 Feb 2017, at 11:31: 32, adrian_le...@hushmail.com wrote:

Gophers, 

Is it idiomatic to add $GOPTAH/bin to $PATH in order access a Go executable 
across my filesystem?

Aidy 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com 
.
For more options, visit https://groups.google.com/d/optout 
.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] $GOPTAH/bin and $PATH

[adrian_lewis]

Gophers, 

Is it idiomatic to add $GOPTAH/bin to $PATH in order access a Go executable 
across my filesystem?

Aidy 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: is it possible to speed up type assertion?

[T L]

I found that it is much faster if the dynamic values are pointers instead 
of non-pointer.

package main

import (
"testing"
)

func AssertInt(c *int, d interface{}) {
*c += d.(int)
}

func BenchmarkAssertionInt(b *testing.B) {
count := 0
var t int = 1
d := (interface{})(t)
for i := 0; i < b.N; i++ {
AssertInt(, d)
}
}

func AssertPtr(c *int, d interface{}) {
*c += *d.(*int)
}

func BenchmarkAssertionPtr(b *testing.B) {
count := 0
var t int = 1
d := (interface{})()
for i := 0; i < b.N; i++ {
AssertPtr(, d)
}
}

BenchmarkAssertionInt-4   515.3 ns/op
BenchmarkAssertionPtr-4   20 3.09 ns/op



On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>
> Go's type assertion seems quite slow. The added cost is too much if it has 
> to be in a tight loop. Here are the time taken on my laptop for the 
> following code.
>
> https://play.golang.org/p/cA96miTkx_
>
>
> chris$ time ./p
> count=1073741824 time taken=7.899207181s
> count=1073741824 time taken=300.601453ms
>
> real 0m8.205s
> user 0m8.163s
> sys 0m0.029s
>
> chris$ time luajit -e "count = 0
> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
> > print(count)"
> 1073741824
>
> real 0m0.900s
> user 0m0.891s
> sys 0m0.005s
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Dave Cheney]

0.44ns/op is about 2.2.ghz, the compiler has optimised away your 
microbenchmark. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Steven Hartland]

On 02/02/2017 09:27, Ian Davis wrote:


On Thu, 2 Feb 2017, at 09:20 AM, T L wrote:



On Thursday, February 2, 2017 at 4:58:32 PM UTC+8, Axel Wagner wrote:

Hi,

I can not really reproduce your results. I rewrote your code to
use the builtin benchmarking: http://sprunge.us/IfQc
Giving, on my laptop:

BenchmarkAssertion-4 102.89 ns/op
BenchmarkAssertionOK-4 52.66 ns/op
BenchmarkBare-4 102.22 ns/op
BenchmarkIface-4 5000   30.0 ns/op
BenchmarkReflect-4 29.74 ns/op

Note, that a) yes, there is an overhead of the type-assertion,
but b) it's pretty small, especially compared to the other things
you're trying and c) it can be further reduced by using the
two-value form (so that there is never a need to consider
stack-unwinding).

Overall, this smells like a micro-benchmark. I wouldn't worry too
much about it until you have specific evidence that it's slowing
down a real program.


The result on my machine for your test:
BenchmarkAssertion-4 517.9 ns/op
BenchmarkAssertionOK-4 517.9 ns/op
BenchmarkBare-4 20 3.93 ns/op
BenchmarkIface-4 186.7 ns/op
BenchmarkReflect-4 515.6 ns/op


What version of Go and what OS/hardware?



Similar results here:
BenchmarkAssertion-24   1   14.4 ns/op
BenchmarkAssertionOK-24 1   14.0 ns/op
BenchmarkBare-2410   2.81 ns/op
BenchmarkIface-24   300040.2 ns/op
BenchmarkReflect-24 1   13.2 ns/op

go version go1.7.5 freebsd/amd64

CPU: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz (2500.06-MHz K8-class CPU)

Regards
Steve

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 5:28:26 PM UTC+8, Ian Davis wrote:
>
>
> On Thu, 2 Feb 2017, at 09:20 AM, T L wrote:
>
>
>
> On Thursday, February 2, 2017 at 4:58:32 PM UTC+8, Axel Wagner wrote:
>
> Hi,
>
> I can not really reproduce your results. I rewrote your code to use the 
> builtin benchmarking: http://sprunge.us/IfQc
> Giving, on my laptop:
>
> BenchmarkAssertion-4 10 2.89 ns/op
> BenchmarkAssertionOK-4   5 2.66 ns/op
> BenchmarkBare-4   10 2.22 ns/op
> BenchmarkIface-4 500030.0 ns/op
> BenchmarkReflect-4   2 9.74 ns/op
>
> Note, that a) yes, there is an overhead of the type-assertion, but b) it's 
> pretty small, especially compared to the other things you're trying and c) 
> it can be further reduced by using the two-value form (so that there is 
> never a need to consider stack-unwinding).
>
> Overall, this smells like a micro-benchmark. I wouldn't worry too much 
> about it until you have specific evidence that it's slowing down a real 
> program.
>
>
> The result on my machine for your test:
> BenchmarkAssertion-4 517.9 ns/op
> BenchmarkAssertionOK-4   517.9 ns/op
> BenchmarkBare-4  20 3.93 ns/op
> BenchmarkIface-4 186.7 ns/op
> BenchmarkReflect-4   515.6 ns/op
>
>
> What version of Go and what OS/hardware?
>
>
$ go version
go version go1.7.5 linux/amd64
$ cat /proc/cpuinfo | grep 'model name' | uniq
model name: Intel(R) Core(TM) i3-2350M CPU @ 2.30GHz
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[Ian Davis]

On Thu, 2 Feb 2017, at 09:20 AM, T L wrote:

> 

> 

> On Thursday, February 2, 2017 at 4:58:32 PM UTC+8, Axel Wagner wrote:
>> Hi,

>> 

>> I can not really reproduce your results. I rewrote your code to use
>> the builtin benchmarking: http://sprunge.us/IfQc
>> Giving, on my laptop:

>> 

>> BenchmarkAssertion-4  10  2.89 ns/op

>> BenchmarkAssertionOK-45  2.66 ns/op

>> BenchmarkBare-4   10  2.22 ns/op

>> BenchmarkIface-4  5000 30.0 ns/op

>> BenchmarkReflect-42  9.74 ns/op

>> 

>> Note, that a) yes, there is an overhead of the type-assertion, but b)
>> it's pretty small, especially compared to the other things you're
>> trying and c) it can be further reduced by using the two-value form
>> (so that there is never a need to consider stack-unwinding).
>> 

>> Overall, this smells like a micro-benchmark. I wouldn't worry too
>> much about it until you have specific evidence that it's slowing down
>> a real program.
> 

> The result on my machine for your test:

> BenchmarkAssertion-4 517.9 ns/op

> BenchmarkAssertionOK-4   517.9 ns/op

> BenchmarkBare-4  20 3.93 ns/op

> BenchmarkIface-4 186.7 ns/op

> BenchmarkReflect-4   515.6 ns/op



What version of Go and what OS/hardware?




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 5:20:01 PM UTC+8, T L wrote:
>
>
>
> On Thursday, February 2, 2017 at 4:58:32 PM UTC+8, Axel Wagner wrote:
>>
>> Hi,
>>
>> I can not really reproduce your results. I rewrote your code to use the 
>> builtin benchmarking: http://sprunge.us/IfQc
>> Giving, on my laptop:
>>
>> BenchmarkAssertion-4 10 2.89 ns/op
>> BenchmarkAssertionOK-4   5 2.66 ns/op
>> BenchmarkBare-4   10 2.22 ns/op
>> BenchmarkIface-4 500030.0 ns/op
>> BenchmarkReflect-4   2 9.74 ns/op
>>
>> Note, that a) yes, there is an overhead of the type-assertion, but b) 
>> it's pretty small, especially compared to the other things you're trying 
>> and c) it can be further reduced by using the two-value form (so that there 
>> is never a need to consider stack-unwinding).
>>
>> Overall, this smells like a micro-benchmark. I wouldn't worry too much 
>> about it until you have specific evidence that it's slowing down a real 
>> program.
>>
>
> The result on my machine for your test:
> BenchmarkAssertion-4 517.9 ns/op
> BenchmarkAssertionOK-4   517.9 ns/op
> BenchmarkBare-4  20 3.93 ns/op
> BenchmarkIface-4 186.7 ns/op
> BenchmarkReflect-4   515.6 ns/op
>
>
The difference after inclining the functions for iface is quite large:
https://play.golang.org/p/9PGknpwxPj

BenchmarkAssertion-4 514.8 ns/op
BenchmarkAssertionOK-4   514.9 ns/op
BenchmarkBare-4  20 0.44 ns/op
BenchmarkIface-4 20 0.44 ns/op
BenchmarkReflect-4   1010.6 ns/op

I guess CPU make some cache. 


 
>
>>
>> On Thu, Feb 2, 2017 at 9:40 AM, T L  wrote:
>>
>>> Type assertion is even slower than reflect: 
>>> https://play.golang.org/p/zvUTEKDfiL
>>>
>>> assert: count=33554432 time taken=499.061188ms
>>> direct: count=33554432 time taken=14.981847ms
>>> method: count=33554432 time taken=176.977503ms
>>> reflect: count=33554432 time taken=383.905004ms
>>>
>>> On Thursday, February 2, 2017 at 4:25:05 PM UTC+8, T L wrote:
>>>


 On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>
> Go's type assertion seems quite slow. The added cost is too much if it 
> has to be in a tight loop. Here are the time taken on my laptop for the 
> following code.
>
> https://play.golang.org/p/cA96miTkx_
>
>
> chris$ time ./p
> count=1073741824 time taken=7.899207181s
> count=1073741824 time taken=300.601453ms
>
> real 0m8.205s
> user 0m8.163s
> sys 0m0.029s
>
> chris$ time luajit -e "count = 0
> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
> > print(count)"
> 1073741824
>
> real 0m0.900s
> user 0m0.891s
> sys 0m0.005s
>
>
>
 Type assertion is even slower than call dynamic method: 
 https://play.golang.org/p/jUrazcbB9h
 Some surprised.
  

>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 4:58:32 PM UTC+8, Axel Wagner wrote:
>
> Hi,
>
> I can not really reproduce your results. I rewrote your code to use the 
> builtin benchmarking: http://sprunge.us/IfQc
> Giving, on my laptop:
>
> BenchmarkAssertion-4 10 2.89 ns/op
> BenchmarkAssertionOK-4   5 2.66 ns/op
> BenchmarkBare-4   10 2.22 ns/op
> BenchmarkIface-4 500030.0 ns/op
> BenchmarkReflect-4   2 9.74 ns/op
>
> Note, that a) yes, there is an overhead of the type-assertion, but b) it's 
> pretty small, especially compared to the other things you're trying and c) 
> it can be further reduced by using the two-value form (so that there is 
> never a need to consider stack-unwinding).
>
> Overall, this smells like a micro-benchmark. I wouldn't worry too much 
> about it until you have specific evidence that it's slowing down a real 
> program.
>

The result on my machine for your test:
BenchmarkAssertion-4 517.9 ns/op
BenchmarkAssertionOK-4   517.9 ns/op
BenchmarkBare-4  20 3.93 ns/op
BenchmarkIface-4 186.7 ns/op
BenchmarkReflect-4   515.6 ns/op

 

>
> On Thu, Feb 2, 2017 at 9:40 AM, T L  
> wrote:
>
>> Type assertion is even slower than reflect: 
>> https://play.golang.org/p/zvUTEKDfiL
>>
>> assert: count=33554432 time taken=499.061188ms
>> direct: count=33554432 time taken=14.981847ms
>> method: count=33554432 time taken=176.977503ms
>> reflect: count=33554432 time taken=383.905004ms
>>
>> On Thursday, February 2, 2017 at 4:25:05 PM UTC+8, T L wrote:
>>
>>>
>>>
>>> On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:

 Go's type assertion seems quite slow. The added cost is too much if it 
 has to be in a tight loop. Here are the time taken on my laptop for the 
 following code.

 https://play.golang.org/p/cA96miTkx_


 chris$ time ./p
 count=1073741824 time taken=7.899207181s
 count=1073741824 time taken=300.601453ms

 real 0m8.205s
 user 0m8.163s
 sys 0m0.029s

 chris$ time luajit -e "count = 0
 > for i=1, 1024*1024*1024, 1 do count = count + 1 end
 > print(count)"
 1073741824

 real 0m0.900s
 user 0m0.891s
 sys 0m0.005s



>>> Type assertion is even slower than call dynamic method: 
>>> https://play.golang.org/p/jUrazcbB9h
>>> Some surprised.
>>>  
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: is it possible to speed up type assertion?

['Axel Wagner' via golang-nuts]

Hi,

I can not really reproduce your results. I rewrote your code to use the
builtin benchmarking: http://sprunge.us/IfQc
Giving, on my laptop:

BenchmarkAssertion-4 10 2.89 ns/op
BenchmarkAssertionOK-4   5 2.66 ns/op
BenchmarkBare-4   10 2.22 ns/op
BenchmarkIface-4 500030.0 ns/op
BenchmarkReflect-4   2 9.74 ns/op

Note, that a) yes, there is an overhead of the type-assertion, but b) it's
pretty small, especially compared to the other things you're trying and c)
it can be further reduced by using the two-value form (so that there is
never a need to consider stack-unwinding).

Overall, this smells like a micro-benchmark. I wouldn't worry too much
about it until you have specific evidence that it's slowing down a real
program.

On Thu, Feb 2, 2017 at 9:40 AM, T L  wrote:

> Type assertion is even slower than reflect: https://play.golang.org/p/
> zvUTEKDfiL
>
> assert: count=33554432 time taken=499.061188ms
> direct: count=33554432 time taken=14.981847ms
> method: count=33554432 time taken=176.977503ms
> reflect: count=33554432 time taken=383.905004ms
>
> On Thursday, February 2, 2017 at 4:25:05 PM UTC+8, T L wrote:
>
>>
>>
>> On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>>>
>>> Go's type assertion seems quite slow. The added cost is too much if it
>>> has to be in a tight loop. Here are the time taken on my laptop for the
>>> following code.
>>>
>>> https://play.golang.org/p/cA96miTkx_
>>>
>>>
>>> chris$ time ./p
>>> count=1073741824 time taken=7.899207181s
>>> count=1073741824 time taken=300.601453ms
>>>
>>> real 0m8.205s
>>> user 0m8.163s
>>> sys 0m0.029s
>>>
>>> chris$ time luajit -e "count = 0
>>> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
>>> > print(count)"
>>> 1073741824
>>>
>>> real 0m0.900s
>>> user 0m0.891s
>>> sys 0m0.005s
>>>
>>>
>>>
>> Type assertion is even slower than call dynamic method:
>> https://play.golang.org/p/jUrazcbB9h
>> Some surprised.
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: is it possible to speed up type assertion?

[T L]

Type assertion is even slower than reflect: 
https://play.golang.org/p/zvUTEKDfiL

assert: count=33554432 time taken=499.061188ms
direct: count=33554432 time taken=14.981847ms
method: count=33554432 time taken=176.977503ms
reflect: count=33554432 time taken=383.905004ms

On Thursday, February 2, 2017 at 4:25:05 PM UTC+8, T L wrote:
>
>
>
> On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>>
>> Go's type assertion seems quite slow. The added cost is too much if it 
>> has to be in a tight loop. Here are the time taken on my laptop for the 
>> following code.
>>
>> https://play.golang.org/p/cA96miTkx_
>>
>>
>> chris$ time ./p
>> count=1073741824 time taken=7.899207181s
>> count=1073741824 time taken=300.601453ms
>>
>> real 0m8.205s
>> user 0m8.163s
>> sys 0m0.029s
>>
>> chris$ time luajit -e "count = 0
>> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
>> > print(count)"
>> 1073741824
>>
>> real 0m0.900s
>> user 0m0.891s
>> sys 0m0.005s
>>
>>
>>
> Type assertion is even slower than call dynamic method: 
> https://play.golang.org/p/jUrazcbB9h
> Some surprised.
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 4:25:05 PM UTC+8, T L wrote:
>
>
>
> On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>>
>> Go's type assertion seems quite slow. The added cost is too much if it 
>> has to be in a tight loop. Here are the time taken on my laptop for the 
>> following code.
>>
>> https://play.golang.org/p/cA96miTkx_
>>
>>
>> chris$ time ./p
>> count=1073741824 time taken=7.899207181s
>> count=1073741824 time taken=300.601453ms
>>
>> real 0m8.205s
>> user 0m8.163s
>> sys 0m0.029s
>>
>> chris$ time luajit -e "count = 0
>> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
>> > print(count)"
>> 1073741824
>>
>> real 0m0.900s
>> user 0m0.891s
>> sys 0m0.005s
>>
>>
>>
> Type assertion is even slower than call dynamic method: 
> https://play.golang.org/p/jUrazcbB9h
> Some surprised.
>

The result:
assert: count=5242880 time taken=79.262474ms
direct: count=5242880 time taken=2.291717ms
method: count=5242880 time taken=27.456853ms
 

>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>
> Go's type assertion seems quite slow. The added cost is too much if it has 
> to be in a tight loop. Here are the time taken on my laptop for the 
> following code.
>
> https://play.golang.org/p/cA96miTkx_
>
>
> chris$ time ./p
> count=1073741824 time taken=7.899207181s
> count=1073741824 time taken=300.601453ms
>
> real 0m8.205s
> user 0m8.163s
> sys 0m0.029s
>
> chris$ time luajit -e "count = 0
> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
> > print(count)"
> 1073741824
>
> real 0m0.900s
> user 0m0.891s
> sys 0m0.005s
>
>
>
Type assertion is even slower than call dynamic method: 
https://play.golang.org/p/jUrazcbB9h
Some surprised.
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: is it possible to speed up type assertion?

[T L]

On Thursday, February 2, 2017 at 2:04:23 PM UTC+8, ChrisLu wrote:
>
> Go's type assertion seems quite slow. The added cost is too much if it has 
> to be in a tight loop. Here are the time taken on my laptop for the 
> following code.
>
> https://play.golang.org/p/cA96miTkx_
>
>
> chris$ time ./p
> count=1073741824 time taken=7.899207181s
> count=1073741824 time taken=300.601453ms
>
> real 0m8.205s
> user 0m8.163s
> sys 0m0.029s
>
> chris$ time luajit -e "count = 0
> > for i=1, 1024*1024*1024, 1 do count = count + 1 end
> > print(count)"
> 1073741824
>
> real 0m0.900s
> user 0m0.891s
> sys 0m0.005s
>
>
Can't believe it, type assertion is even slow than call interface method: 
https://play.golang.org/p/3if6xECkVk

count=5242880 time taken=76.27149ms
count=5242880 time taken=2.289666ms
count=5242880 time taken=27.244965ms

 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.