[go-nuts] Re: HTTP requests bottlenecks

func randInt(min int, max int) int { rand.Seed(int64(time.Now().Nanosecond())) return min + rand.Intn(max-min) } Is the culprit. the default rand locks globally for concurrent access. You need to create a new rand in each goroutine you want to use it in, for maximum speed. On Wednesday,

[go-nuts] is it possible to speed up type assertion?

Go's type assertion seems quite slow. The added cost is too much if it has to be in a tight loop. Here are the time taken on my laptop for the following code. https://play.golang.org/p/cA96miTkx_ chris$ time ./p count=1073741824 time taken=7.899207181s count=1073741824 time taken=300.601453ms

[go-nuts] Re: Heap fragmentation: HeapProfile reported memory vs Memstats.HeapInUse

yup, that's what I meant, sorry. Looks like the svg: lines are missing, have you filtered those out. If you have the net/http/pprof endpoint enabled, /debug/prof/heap?debug=1 will show you the memory stats from the allocator. Those values are accurate and will tell you the amount of memory

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

Thanks for the info! I will for sure take a look before submitting my proposal On Wednesday, February 1, 2017 at 5:34:53 PM UTC-8, Keith Randall wrote: > > &$*@^! Google! Apparently "publish to the web" doesn't mean publish to > the web when you're inside Google. > > But I can publish to a pdf

[go-nuts] HTTP requests bottlenecks

Hello, I'm writing a POC for a future RTB platform. Basically I'm doing stress tests by receiving HTTP requests and performing HTTP GET requests in the background. Issue I face is that when i try to scale up, URLs starts to timeout. I am trying to find what our bottleneck is but so far no

[go-nuts] Re: Heap fragmentation: HeapProfile reported memory vs Memstats.HeapInUse

Thanks for the reply. I am using Debian 8, go version 1.7.3 I assume that you requested for gctrace=1 MemStats: http://pastebin.com/iCYuvbGA GCtrace: http://pastebin.com/M7VpehUE Please find the attached heap profile svg as well. The process RSS ~ 36G -- Thanks, Sarath On Thursday,

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

This discussion reminds me of another technique I've seen, which is to use tagged pointers to store strings of max length 7 with no allocations at all. Apologies if this has already been discussed to death here. For background: tagged pointers take advantage of the fact that most architectures

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

&$*@^! Google! Apparently "publish to the web" doesn't mean publish to the web when you're inside Google. But I can publish to a pdf and then attach it... On Wed, Feb 1, 2017 at 5:24 PM, Caleb Spare wrote: > Your document is not accessible to me. (Google-internal?) > > On

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

Your document is not accessible to me. (Google-internal?) On Wed, Feb 1, 2017 at 5:18 PM, 'Keith Randall' via golang-nuts wrote: > I wrote up a proto-proposal for something like this a while ago. > >

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

I wrote up a proto-proposal for something like this a while ago. https://docs.google.com/a/google.com/document/d/18nu6QTr-ACYr5AiyP6x31SkXvuJZZaZ12lnvPTEQfgQ/pub It has a few numbers worth looking at in it. This proposal was before the fully precise GC we have today. It needs to have special

Re: [go-nuts] about sorting array

Wim, Ahh, I understand now. Thanks a lot for the explanation, Nestor On Wednesday, February 1, 2017 at 5:05:03 PM UTC-8, Wim Lewis wrote: > > On Feb 1, 2017, at 4:56 PM, Néstor Flórez > wrote: > > OK, thanks for the clarification on the size being immutable.(I am trying >

Re: [go-nuts] about sorting array

On Feb 1, 2017, at 4:56 PM, Néstor Flórez wrote: > OK, thanks for the clarification on the size being immutable.(I am trying to > teach myself Go) > > Still I want to know what happens when this statement is executed > sort.Ints(scores[:]) > Sort creates a slice > The slice is

Re: [go-nuts] about sorting array

Not quite. The slice shares storage with the array, so changing an element of the slice is the same as changing an element of the array. Slicing in Go (unlike some other languages) does not copy the elements. Thomas On Wed, Feb 1, 2017 at 4:56 PM Néstor Flórez wrote: > OK,

Re: [go-nuts] about sorting array

OK, thanks for the clarification on the size being immutable.(I am trying to teach myself Go) Still I want to know what happens when this statement is executed sort.Ints(scores[:]) - Sort creates a slice - The slice is sorted - Sort copies the slice into the array Is this

Re: [go-nuts] about sorting array

array contents are not immutable, just their size. On Wed, Feb 1, 2017 at 4:13 PM, Néstor Flórez wrote: > The below code sorts an array but it does it an by creating a slice, > correct? > What happened to the original array? > Arrays are inmutable > >

[go-nuts] about sorting array

The below code sorts an array but it does it an by creating a slice, correct? What happened to the original array? Arrays are inmutable -- package main import ("fmt" "sort")

[go-nuts] Re: Migrating from a classic OO pattern

Superb advice, seconded. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit

[go-nuts] Add section about use of underscores in file names to Go Code Review Comments

I think both are symptomatic of following a Javaesq style of one type per file. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to

[go-nuts] Re: Migrating from a classic OO pattern

With all due respect, Julien, but this reminds me of somebody trying to use reins in an automobile instead of a steering wheel. Go as a language offers a feature set that brings new possibilities. Allow yourself some time to assimilate the language idioms. Read the standard library. It has

[go-nuts] Add section about use of underscores in file names to Go Code Review Comments

Hey everyone, this is something which has came up now a couple of times in the Gophers Slack so I am posting it here: Would it make sense to add a section about how file names should look like? In particular, I would suggest that underscores should not be used unless for prefixing test files,

Re: [go-nuts] Question on decrypting a using StreamReader

Here's an example that I use (from my github): https://github.com/ShawnMilo/crypto/blob/master/crypto.go I wonder if it has to do with not padding the password length (see my fixLength) function. See if the same problem occurs when you use my functions instead. Incidentally, if anyone has any

[go-nuts] Question on decrypting a using StreamReader

I am trying to Decrypt and Encrypt a file downloaded as a stream using some sample code from golang. I am encrypting like this. // read content from your file plaintext, err := ioutil.ReadFile(fileName) if err != nil { panic(err.Error()) } // this is a key key :=

[go-nuts] Re: Heap fragmentation: HeapProfile reported memory vs Memstats.HeapInUse

Can you please run your program with GODEBUG=1 and paste the output. btw, which version of Go and which OS? On Thursday, 2 February 2017 02:26:45 UTC+11, Sarath Lakshman wrote: > > I have tried waiting long enough for the process to release memory back to > OS as well as debug.FreeOSMemory(). >

[go-nuts] netlink + wifi: packages to perform WiFi device interactions using nl80211 on Linux

Hi all, I built out a couple of packages recently that I figured may be interesting for some on this list. I decided that I wanted to figure out how to interact with WiFi devices from Go (without cgo), and that lead me to writing packages to work with netlink, generic netlink, and nl80211 on

Re: [go-nuts] Re: Is there a reason go doesn't use the small string optomization

On Wed, Feb 1, 2017 at 10:38 AM, Eliot Hedeman wrote: > Ok, I'm going to try to restate the issues raised and answer them one by > one. If I misunderstand, please let me know. > > 1. The GC needs to know if a pointer is actually a pointer. > Solution: Check if the high

[go-nuts] Re: Is there a reason go doesn't use the small string optomization

Ok, I'm going to try to restate the issues raised and answer them one by one. If I misunderstand, please let me know. 1. The GC needs to know if a pointer is actually a pointer. Solution: Check if the high bit is set. If it is, you know for sure that it is not a pointer. It actually doesn't

Re: [go-nuts] About 64bits alignment

On Thursday, February 2, 2017 at 12:11:39 AM UTC+8, Jan Mercl wrote: > > On Wed, Feb 1, 2017 at 5:04 PM T L > wrote: > > > But what does an allocated struct or slice means? A struct or slice > allocated on heap, not stack? > > There's no heap nor stack from the POV of the

[go-nuts] Re: Migrating from a classic OO pattern

Thanks for the input. I'm in the process of doing our homework for migrating to golang so we appreciate all that input to be able to avoid common misuse of the language. I'm far from having perfect understanding on the language but I'm starting to get the differences with a «typical» OO

Re: [go-nuts] About 64bits alignment

On Wed, Feb 1, 2017 at 5:04 PM T L wrote: > But what does an allocated struct or slice means? A struct or slice allocated on heap, not stack? There's no heap nor stack from the POV of the language spec. Allocated means appropriate amount of storage is now used by the

Re: [go-nuts] Request for very simple cgo example to check compiler environment

Thx Ian. That nails it. Compiler setup seems to be ok! Now I'll try and find reason for the errors in the other cgo project. THX! 2017-02-01 16:08 GMT+01:00 Ian Lance Taylor : > On Wed, Feb 1, 2017 at 6:57 AM, Arie van Wingerden > wrote: > > In another thread

[go-nuts] Re: Heap fragmentation: HeapProfile reported memory vs Memstats.HeapInUse

I have tried waiting long enough for the process to release memory back to OS as well as debug.FreeOSMemory(). The heapdump14 tool doesn't seem to work properly with heapdump from go1.7. Is there some way I can collect statistics about the memory fragmentation ? Thanks, Sararth On Wednesday,

Re: [go-nuts] Request for very simple cgo example to check compiler environment

On Wed, Feb 1, 2017 at 6:57 AM, Arie van Wingerden wrote: > In another thread I asked for help with Mingw64. > Following Andrew Gerrands advice I now have installed TDM-GCC-64. > > Still I get errors when compiling a quite big project of someone else. But I > am not sure

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

On Wed, Feb 1, 2017 at 6:35 AM, Ian Lance Taylor wrote: > On Tue, Jan 31, 2017 at 10:26 PM, wrote: >> It would know at string creation time because strings in go are immutable. >> Really these are two types, with duck typing between them. (the programmer >>

[go-nuts] Request for very simple cgo example to check compiler environment

In another thread I asked for help with Mingw64. Following Andrew Gerrands advice I now have installed TDM-GCC-64. Still I get errors when compiling a quite big project of someone else. But I am not sure whether that is because of wrong (gcc) compiler setup on my side or of the project itself.

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

Ok that clarifies it for me :) ons 1 feb. 2017 kl 15:29 skrev Ian Lance Taylor : > On Tue, Jan 31, 2017 at 10:08 PM, Henrik Johansson > wrote: > > What makes strings harder than for example []byte? > > Sorry, I'm not sure who you are asking, or, really

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

On Tue, Jan 31, 2017 at 10:26 PM, wrote: > It would know at string creation time because strings in go are immutable. > Really these are two types, with duck typing between them. (the programmer > unless using unsafe or reflection sees them the same) > You have a shortstring

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

On Tue, Jan 31, 2017 at 10:08 PM, Henrik Johansson wrote: > What makes strings harder than for example []byte? Sorry, I'm not sure who you are asking, or, really what you are asking. []byte doesn't have a small-slice-optimization either. Ian > On Wed, Feb 1, 2017, 06:15

Re: [go-nuts] internal compiler error: name too long: *struct {

On Wed, Feb 1, 2017 at 4:58 AM, wrote: > > Go 1.7.1, 1.7.5 does not compile program with a large struct defintion. > Version 1.6.4 and some other previous versions do compile. > It could be a problem of the 1.7 compiler. > > The declaration of the struct type is

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

>From what i read the top 16 bits in a pointer are generally not completely ignored by the cpu in x86-64 virtual address space: AMD64 Architecture Programmer’s Manual Volume 1 - 2.2.2 64-Bit Canonical Addresses: "Although implementations might not use all 64 bits of the virtual address, they

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

Strings are easier than []byte because they are immutable. On Wednesday, February 1, 2017 at 12:08:36 AM UTC-6, Henrik Johansson wrote: > > What makes strings harder than for example []byte? > > On Wed, Feb 1, 2017, 06:15 Ian Lance Taylor > wrote: > >> On Tue, Jan 31, 2017

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

It would know at string creation time because strings in go are immutable. Really these are two types, with duck typing between them. (the programmer unless using unsafe or reflection sees them the same) You have a shortstring type and a string type, what determines their type is the length of

[go-nuts] internal compiler error: name too long: *struct {

Go 1.7.1, 1.7.5 does not compile program with a large struct defintion. *Version 1.6.4 and some other previous versions do compile.* It could be a problem of the 1.7 compiler. The declaration of the struct type is about 70 Kbytes (70k of characters ) large and it is also complex, contains

[go-nuts] Re: Should golang.org/x/crypto/ssh.ParsePrivateKey() be extended to parse encrypted keys?

In case anyone else gets to this post but has an ssh-agent running, you can use this: import ( "net" "os" "golang.org/x/crypto/ssh/agent" ) if aconn, err := net.Dial("unix", os.Getenv("SSH_AUTH_SOCK")); err == nil { signerFn := agent.NewClient(aconn).Signers signers, err := signerFn()

[go-nuts] Re: Is there a "strings.Clone(string) string" alike function?

Is it a good idea to make the builtin copy function support one parameter? So that we can use it to deep copy string and interface values. On Wednesday, February 1, 2017 at 5:18:56 PM UTC+8, T L wrote: > > Sometimes, I want to create a new string which will not share the > underlying bytes with

Re: [go-nuts] Is there a reason go doesn't use the small string optomization

One problem of using the high bits are that they are only available on 64bit systems. There are however classical ways of remedying this problem. Note that I'm writing this on the top of my mind without any reference to look up, so things might be a bit off. I do, however think the gist of the

Re: [go-nuts] Is there a "strings.Clone(string) string" alike function?

On Wed, Feb 1, 2017 at 10:59 AM T L wrote: > It is obviously that both strings.Repeat and string([]byte(s)) make two 1M memory allocations. The observation is correct. However, stack allocating of non-leaking items is capped in size and I guess 1M is out of limits. The

Re: [go-nuts] Re: Wrestling with Mingw64 on Windows

@tim I tried that. So, I have installed the tdm-gcc-64 compiler, which should be capable of compiling 32 and 64 bit binaries. QUESTION: Does anybody have a short example of a project needing cgo to check if my installation is correct? (must compile to either 32 or 64 bit binary) Because at the

Re: [go-nuts] Is there a "strings.Clone(string) string" alike function?

On Wednesday, February 1, 2017 at 5:22:51 PM UTC+8, Jan Mercl wrote: > > On Wed, Feb 1, 2017 at 10:19 AM T L > wrote: > > > newStr := string([]byte(oldStr)) is simple but will make two memory > allocations, and one of them is a waste. > > Is the above an assumption or have

Re: [go-nuts] Is there a "strings.Clone(string) string" alike function?

On Wednesday, February 1, 2017 at 5:22:51 PM UTC+8, Jan Mercl wrote: > > On Wed, Feb 1, 2017 at 10:19 AM T L > wrote: > > > newStr := string([]byte(oldStr)) is simple but will make two memory > allocations, and one of them is a waste. > > Is the above an assumption or have

[go-nuts] Re: About "Declaring Empty Slices" of CodeReviewComments

On Tuesday, January 31, 2017 at 1:15:40 PM UTC+8, Keiji Yoshida wrote: > > Hi, > > "Declaring Empty Slices" of CodeReviewComments ( > https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices > ) says as below: > > ``` > When declaring a slice, use > > var t []string > >

[go-nuts] Re: About "Declaring Empty Slices" of CodeReviewComments

On Tuesday, January 31, 2017 at 1:15:40 PM UTC+8, Keiji Yoshida wrote: > > Hi, > > "Declaring Empty Slices" of CodeReviewComments ( > https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices > ) says as below: > > ``` > When declaring a slice, use > > var t []string > >

Re: [go-nuts] Is there a "strings.Clone(string) string" alike function?

On Wed, Feb 1, 2017 at 10:19 AM T L wrote: > newStr := string([]byte(oldStr)) is simple but will make two memory allocations, and one of them is a waste. Is the above an assumption or have you verified that? It would not surprise me that nowadays the compiler can infer the

[go-nuts] Is there a "strings.Clone(string) string" alike function?

Sometimes, I want to create a new string which will not share the underlying bytes with other strings. For example, to avoid the substring memory leak problem. newStr := string([]byte(oldStr)) is simple but will make two memory allocations, and one of them is a waste. There are two solutions