Re: [go-nuts] Unexpected evaluation order in a return statement with multiple operands

2020-01-15 Thread Robert Engels
or) { >x := { > Foo: f, >} >return x, x.intialize() > } > > You can say that you shouldn't do something like this, but it's IMO > reasonable code to write and it expects different behavior. > > However, as per my previous e-mail, I would stil

Re: [go-nuts] Unexpected evaluation order in a return statement with multiple operands

2020-01-15 Thread Robert Engels
: > > func identity(i int) int { > return i > } > > func modify(i *int) error { > *i = 1 > return nil > } > > func f() (int, error) { > i := 0 > return identity(i), modify() // returns 0, nil > } > > >&g

Re: [go-nuts] Unexpected evaluation order in a return statement with multiple operands

2020-01-15 Thread Robert Engels
I think the way to look at it is “what would be the behavior of if you were inline creating and initializing a struct containing both values” - clearly this behavior is not what you would want or expect. > On Jan 15, 2020, at 1:25 PM, Paul Jolly wrote: > >  >> >> "when evaluating the

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

2020-01-15 Thread Robert Engels
Please look deeper into how these "trends" are calculated.For example, if everyone that uses product Y can't figure out how feature X works, and they search for it. Product Y will be showing growth...-Original Message- From: Liam Sent: Jan 15, 2020 4:18 PM To: golang-nuts Subject:

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

2020-01-16 Thread Robert Engels
Well said. > On Jan 16, 2020, at 9:58 AM, Michael Jones wrote: > >  > How global mindshare develops is something that I know quite a bit about > through leadership and engineering experience in multiple billion user > projects. > > One key lesson for me was that you reach a point where

Re: [go-nuts] Re: Go mindshare is low & ~flat, per Google Trends

2020-01-16 Thread Robert Engels
Porsche’s sales are a tiny fraction of auto sales, but most in the industry consider it to be the best platform - and only wish they could emulate / afford to be in that conversation. > On Jan 16, 2020, at 2:10 PM, Liam wrote: > >  > Open source is a rapidly growing movement across

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread Robert Engels
This is solved pretty easily in Java using soft references and a hard memory cap. Similar techniques may work here. > On Jan 20, 2020, at 11:22 AM, Christian Mauduit wrote: > > Hi, > > That is a generic question and I think that if you want to keep an approach > with a "global indicator

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread robert engels
, but maybe they have an Unsafe/CGO way. I haven’t really researched the WeakRef packages for Go. > On Jan 20, 2020, at 4:58 PM, Eric S. Raymond wrote: > > Robert Engels : >> This is solved pretty easily in Java using soft references and a hard memory >> cap. > > That'd

Re: [go-nuts] Locking goroutine to "main" thread

2020-01-03 Thread robert engels
Also, even simpler - just remove the time.Atfter case() and use a default: - but the problem is you will spin a single thread at 100% cpu - you really need to use a blocking sdl.WaitEvent() > On Jan 3, 2020, at 6:04 PM, robert engels wrote: > > You only need a single thread locked

Re: [go-nuts] Locking goroutine to "main" thread

2020-01-03 Thread robert engels
in the thread that initialized the video > subsystem." > > > On Friday, January 3, 2020 at 1:58:29 PM UTC-8, Robert Engels wrote: > Even if you could I don’t think you would want to do it this way. > > Have a go routine sleep on a channel. Post to the channel from the na

Re: [go-nuts] Locking goroutine to "main" thread

2020-01-03 Thread Robert Engels
Even if you could I don’t think you would want to do it this way. Have a go routine sleep on a channel. Post to the channel from the native code. Let your command loop run on any thread and synchronize via a channel the calls to/from native. The os event loop doesn’t need to run on main -

Re: [go-nuts] Locking goroutine to "main" thread

2020-01-03 Thread robert engels
spawn this from the init() but I think that might make things harder. You can sync the calls so only a single routine/thread will ever be created. > On Jan 3, 2020, at 6:04 PM, robert engels wrote: > > You only need a single thread locked to the UI thread. > > Use one Go

Re: [go-nuts] Locking goroutine to "main" thread

2020-01-03 Thread robert engels
g two goroutines to the UI thread, I could split this > loop into two separate loops and remove the time.After, I *think*. > > On Friday, January 3, 2020 at 2:59:42 PM UTC-8, robert engels wrote: > You can definitely run the event loop for a process on a thread other than > main. The main thr

Re: [go-nuts] Re: net.conn TCP connection

2019-12-31 Thread Robert Engels
All of the source is there... but in general it is a bit more complex under the covers than most - as it uses select/poll to know when a socket is ready for IO then schedules the routine in the Read() to perform the IO. So Go has a bit of its own kernel code than you would see in typical

Re: [go-nuts] Re: net.conn TCP connection

2019-12-31 Thread Robert Engels
The important factors: 1) the larger the buffer the fewer system calls that need to be made 2) the larger the buffer the more memory use which can be significant with many simultaneous connections You need to remember that with correct socket options the kernel is already buffering, and adding

Re: [go-nuts] Re: net.conn TCP connection

2019-12-31 Thread Robert Engels
One other note, if you have a request / response type protocol with fairly defined lengths, you don’t need a buffer larger than the largest message if you don’t allow concurrent requests from the same client. > On Dec 31, 2019, at 9:35 AM, Robert Engels wrote: > >  > The impor

Re: [go-nuts] Re: net.conn TCP connection

2019-12-31 Thread Robert Engels
fast, so using a messaging library may help. > On Dec 31, 2019, at 10:37 AM, Robert Engels wrote: > >  > All of the source is there... but in general it is a bit more complex under > the covers than most - as it uses select/poll to know when a socket is ready > for

Re: [go-nuts] Simple worker pool in golnag

2019-12-26 Thread robert engels
toCancelChan chan string) > { > defer wg.Done() > autoCancelIds := getAutoCancelIdsFromSource2() > for autoCancelId := range autoCancelIds { > autoCancelChan <- autoCancelId > } > } > > Now does this code makes some sense? > > >

Re: [go-nuts] Simple worker pool in golnag

2019-12-26 Thread robert engels
Yes, the code doesn’t work :) - it will only ever produce 2 items - unless that was expected - even so, you want the N workers doing work, and probably a constant number sending to Kafka - but a lot depends on your “serial needs”. In your case you only have 2 workers producing work, and N

Re: [go-nuts] Simple worker pool in golnag

2019-12-29 Thread Robert Engels
easy create too many routines and then you are paying overhead switching costs for no reason. > On Dec 28, 2019, at 10:17 AM, Ian Lance Taylor wrote: > > On Sat, Dec 28, 2019 at 6:11 AM Robert Engels wrote: >> >> Spinning up a Go routine when for each piece of

Re: [go-nuts] Re: net.conn TCP connection

2019-12-29 Thread Robert Engels
Use read and expand the buffer as needed (or write the chunks to a file). If at the end it is all going to be in memory, you might as well start with the very large buffer. There is nothing special about Go in this regard - it’s standard IO processing. > On Dec 29, 2019, at 9:21 AM, Ron

Re: [go-nuts] Simple worker pool in golnag

2019-12-30 Thread Robert Engels
Right, but the overhead is not constant nor free. So if you parallelize the CPU bound task into 100 segments and you only have 10 cores, the contention on the internal locking structures (scheduler, locks in channels) will be significant and the entire process will probably take far longer -

Re: [go-nuts] Re: net.conn TCP connection

2019-12-30 Thread Robert Engels
ReadAll reads until buffer length or EOF. > On Dec 30, 2019, at 11:04 AM, Jake Montgomery wrote: > >  > It sounds like maybe you have some misconceptions about TCP. It is a stream > protocol, there are no data boundaries that are preserved. If send 20 bytes > via TCP in a single call, it is

Re: [go-nuts] Re: net.conn TCP connection

2019-12-30 Thread Robert Engels
That option requires proprietary protocols not standard tcp/udp. > On Dec 30, 2019, at 12:04 PM, Bruno Albuquerque wrote: > >  > But, to complicate things, you can create what is basically a TCp connection > with packet boundaries using SOCK_SEQPACKET (as opposed to SOCK_STREAM or >

Re: [go-nuts] Simple worker pool in golnag

2019-12-30 Thread Robert Engels
on the underlying structures. So blindly creating go routines does not achieve optimum performance for many workloads (even when the number of OS threads is capped).rengels@rengels:~/gotest$ go run main_bench.go 2.261805812s1.311269725s6.341378965s-Original Message- From: Robert Engels Sent: Dec 30, 2019 9

Re: [go-nuts] Simple worker pool in golnag

2019-12-30 Thread Robert Engels
as it is parallelizing the "all the clients work" - the Go program runs concurrently with other OS programs, but is parallelizing its own work.-Original Message- From: Jesper Louis Andersen Sent: Dec 30, 2019 3:41 PM To: Robert Engels Cc: Brian Candler , golang-nuts Subject: Re: [go-nu

Re: [go-nuts] Re: net.conn TCP connection

2019-12-30 Thread Robert Engels
t;> On Mon, Dec 30, 2019 at 12:23 PM Robert Engels wrote: >> That option requires proprietary protocols not standard tcp/udp. >> >>>> On Dec 30, 2019, at 12:04 PM, Bruno Albuquerque wrote: >>>> >>>  >>> But, to complicate things, you can

Re: [go-nuts] Re: net.conn TCP connection

2019-12-30 Thread Robert Engels
Oh, and I don’t think SCTP is natively supported on Windows yet. So your interoperability May vary... > On Dec 30, 2019, at 4:17 PM, Robert Engels wrote: > >  > Im pretty sure I’m correct. It is a socket type not an option on TCP, which > equates to a different proto

Re: [go-nuts] A question !

2020-01-04 Thread robert engels
Please everyone just benchmark this page https://www.techempower.com/benchmarks/ Rant ON... Python is not suitable for any high volume server application. It is designed for simple scripting. Rant OFF. > On Jan 4, 2020, at 5:41 PM, Justin Israel

Re: [go-nuts] A question !

2020-01-04 Thread robert engels
Well, don’t benchmark it, bookmark it :) > On Jan 4, 2020, at 5:51 PM, robert engels wrote: > > Please everyone just benchmark this page > https://www.techempower.com/benchmarks/ > <https://www.techempower.com/benchmarks/> > > Rant ON... Python is not suitable

Re: [go-nuts] Re: distributed runtime

2020-01-06 Thread Robert Engels
I think you want to use “nats streaming”. It does throttling via ACKs. > On Jan 6, 2020, at 7:35 AM, Jason E. Aten wrote: > >  >>> On Mon, Jan 6, 2020 at 6:24 AM Robert Engels wrote: >>> I’m pretty sure Nats.io has all of the same delivery guarantees as

Re: [go-nuts] Re: distributed runtime

2020-01-06 Thread Robert Engels
I’m pretty sure Nats.io has all of the same delivery guarantees as rabbitmq. What design is not accommodated by mats? > On Jan 6, 2020, at 4:56 AM, ffm2...@web.de wrote: > >  > >> >> However, if you want to apply supervision for distributed Go channels you >> have to change from channels to

Re: [go-nuts] Re: net.conn TCP connection

2019-12-27 Thread Robert Engels
You need a termination point. In the case of ReadString it is the line terminator. For an arbitrary read it is either a length or EOF - or you can read until the underlying socket has no more data but this is generally useless unless you are doing higher level buffering and protocol parsing.

Re: [go-nuts] Re: net.conn TCP connection

2019-12-27 Thread Robert Engels
The standard read will early return as soon as some of read is satisfied and a subsequent block would occur (or timeout) - so you have to decide when you want to stop reading... >> On Dec 27, 2019, at 9:48 PM, Robert Engels wrote: >  > You need a termination point. In the case o

Re: [go-nuts] Simple worker pool in golnag

2019-12-28 Thread Robert Engels
Spinning up a Go routine when for each piece of work may be idiomatic but it is far from the best practice for many situations - mainly because of cpu cache invalidation. Most HPC systems create worker pools by type and then isolate them to cpus/cores - something you can’t do in Go. I believe

[go-nuts] performance improvements

2020-01-07 Thread robert engels
Shout out to the Go team on a job well done. I took the time to re-run some benchmarks, here are the results (both 1.9.7 and 1.13.5 re-run on same hardware & OS). >From fixed iMac:tmp robertengels$ benchcmp bench_1_9_7.txt bench_1_13_15.txt benchmark

Re: [go-nuts] How to reduce the fluctuation of micro benchmark results

2020-01-07 Thread robert engels
Take a look at https://github.com/golang/go/issues/24735 You probably want to increase benchtime to a larger value, even 10x. > On Jan 7, 2020, at 8:44 PM, Xiangdong JI wrote: > > BTW. 'drop cache' is made, while setting cpu governor is not

Re: [go-nuts] Add "not null" parameter check and const variables/pointers

2020-03-11 Thread Robert Engels
receiver” in Go has always struck me as a bit “loose” and in the wild many times creates either performance issues or race conditions. Something like this might tighten it up a bit. > On Mar 11, 2020, at 12:20 AM, Ian Lance Taylor wrote: > > On Tue, Mar 10, 2020 at 10:08 PM robe

Re: [go-nuts] Add "not null" parameter check and const variables/pointers

2020-03-10 Thread robert engels
I think what the OP was trying to say, is that with ‘const’ the compiler could safely use a pointer receiver rather than a copy, and also enforce that const are not modified, and only passed as const args (may require a heap allocation though, so maybe only if struct is of a certain size based

Re: [go-nuts] Add "not null" parameter check and const variables/pointers

2020-03-11 Thread Robert Engels
Thinking about this over morning coffee it is more complex. It is not just sequential consistency that affects it - placing the struct in a map or some other reference container would also have to force a copy. Less and less utility. > On Mar 11, 2020, at 3:33 AM, Robert Engels wr

Re: [go-nuts] How to obtain complete call chain of runtime functions in pprof, like 'mcall'

2020-03-10 Thread Robert Engels
PC:0 goroutines, which were discussed in > #29103, but seems no further details are available, are they considered > irrelevant to perf. analysis? > >> On Thursday, March 5, 2020 at 8:09:18 PM UTC+8, Robert Engels wrote: >> You might be interested in github.co

Re: [go-nuts] How to find goroutines during debugging - aka goroutine labeling

2020-03-05 Thread Robert Engels
This is what I was referring to by performance. I think it needs to be "always on", and should be included in stack-traces - but we need exceptions first :)-Original Message- From: Jesper Louis Andersen Sent: Mar 4, 2020 7:29 AM To: Florin Pățan Cc: golang-nuts Subject: Re: [go-nuts]

Re: [go-nuts] Is this code in package unix assuming machine endianess?

2020-03-12 Thread Robert Engels
No disrespect to Rob but it’s a bit more complex than that. Almost certainly the code was older and written in C and it used fixed length linked records, so back when machines were a lot slower it was far more efficient to point to the start of a struct in memory and read/write directly - they

Re: [go-nuts] Re: How to know if interface{} data is nil w/o reflecting?

2020-04-11 Thread Robert Engels
I agree with the OP. The usefulness of nil interfaces is pretty limited. Show me a useful case that cant easily be implemented with non-nil interfaces. I would argue that allowing nil interfaces causes more subtle latent bugs and makes it harder to reason about the correctness of code when

Re: [go-nuts] testing code that uses ioutil.ReadDir?

2020-04-13 Thread Robert Engels
I don’t think littering your code with state variable and branches just to test is in anyway a good (or sustainable approach) approach. I have never seen any large scale project that did this - but could be my ignorance. > On Apr 13, 2020, at 12:13 PM, 'K Richard Pixley' via golang-nuts >

Re: [go-nuts] Intercepting field access and method call

2020-04-12 Thread robert engels
You probably want to use interfaces. You can do a lot of “seemingly dynamic” programming with interfaces. There are statically typed languages that pretty dynamic - Java with reflection and proxies - but Go is also statically compiled which makes the options limited - which is usually a good

Re: [go-nuts] memory leak of C functions with pprof WriteHeapProfile

2020-04-20 Thread robert engels
see https://www.gnu.org/software/libc/manual/html_node/Allocation-Debugging.html for C memory debugging > On Apr 20, 2020, at 1:19 PM, Pavan wrote: > > Hi, I am debugging memory leak in a go application. It has

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-16 Thread Robert Engels
Sounds like it. Probably in the C code. You need to check that your release/free code is correct. You can try a similar C program that instantiates and frees the structure to check for similar behavior. > On Mar 16, 2020, at 8:25 AM, Nitish Saboo wrote: > >  > Hi, > > I upgraded the go

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-16 Thread Robert Engels
Yes, you have a shared global variable you need to synchronize. > On Mar 16, 2020, at 9:35 AM, Nitish Saboo wrote: > >  > Hi, > > Are you saying it is working as expected? > > Thanks, > Nitish > >> On Mon, Mar 16, 2020 at 7:42 PM Volker Dobler >> wrote: >>> On Monday, 16 March 2020

Re: [go-nuts] Mem-Leak in Go method

2020-03-16 Thread Robert Engels
Looks like pattern_db is a global. Are you sure you don’t have multiple Go routines calling the load? The global changes may not be visible- so the free is doing nothing. You probably need synchronization or pinning the access routine to a thread. > On Mar 16, 2020, at 9:03 AM, Nitish Saboo

Re: [go-nuts] Mem-Leak in Go method

2020-03-16 Thread Robert Engels
I would also use pprof and ensure your Go code is not leaking memory. There are multiple tutorials on using memory profilers to detect memory leaks. > On Mar 16, 2020, at 9:12 AM, Robert Engels wrote: > >  > Looks like pattern_db is a global. Are you sure you don’t have

Re: [go-nuts] Mem-Leak in Go method

2020-03-16 Thread Robert Engels
to a thread. > > >>Why we need to pin routine to a thread ? > > Thanks, > Nitish > >> On Mon, Mar 16, 2020 at 7:41 PM Robert Engels wrote: >> Looks like pattern_db is a global. Are you sure you don’t have multiple Go >> routines calling the lo

Re: [go-nuts] Suggestions for dynamic per object lock

2020-04-04 Thread Robert Engels
The code (and even similar using channels) is highly inefficient. You need to understand the expected collision rate because if it’s low (expected) paying the lock cost on every operation is wasteful. Better to partition the work based on the token (hash N) and avoid the lock operations. You

Re: [go-nuts] Why golang allocated memory space increases?

2020-03-25 Thread robert engels
If the pool is a sync.Pool: Any item stored in the Pool may be removed automatically at any time without 18 // notification. If the Pool holds the only reference when this happens, the 19 // item might be deallocated. So placing an object in the pool does not guarantee it won’t be

Re: [go-nuts] http.client.Do(request) panics in highly concurrent apps (Go 1.14.1)

2020-04-02 Thread Robert Engels
Does your code use any unsafe or cgo? > On Apr 2, 2020, at 11:38 AM, 'hamid.gha...@live.com' via golang-nuts > wrote: > >  > Hi folks, > > An issue with the latest version of Go (1.14.1). My application panics when > making concurrent http GET requests to another API. > > Unfortunately

Re: [go-nuts] http.client.Do(request) panics in highly concurrent apps (Go 1.14.1)

2020-04-03 Thread Robert Engels
ote: > >  > The stack trace is identical to https://github.com/golang/go/issues/3 > > Le jeudi 2 avril 2020 23:23:02 UTC+2, Robert Engels a écrit : >> >> Does your code use any unsafe or cgo? >> >>>> On Apr 2, 2020, at 11:38 AM, 'hamid...@live.com'

Re: [go-nuts] http.client.Do(request) panics in highly concurrent apps (Go 1.14.1)

2020-04-03 Thread Robert Engels
Sorry, got the versions wrong. Looks like the fix in 1.14.1 might need fixing. > On Apr 3, 2020, at 6:12 AM, Robert Engels wrote: > >  > Which is a duplicate of https://github.com/golang/go/issues/37449 > so I would comment/reopen that issue as it appears either the fix was

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-30 Thread Robert Engels
el the way I gathered the mem >>>> profiling was not correct ..is it ? >>>> Please let me know where am I going wrong? >>>> >>>> Thanks, >>>> Nitish >>>> >>>>> On Tue, Mar 24, 2020 at 5:32 PM Nitish

Re: [go-nuts] Deleting map entry from the top level of a nested map doesn't clean the underlying memory

2020-04-28 Thread Robert Engels
Also, it may just be that the runtime is better off allocating more and not doing a GC based on available memory and CPU usage. The “max heap” feature in development may help here. > On Apr 28, 2020, at 3:18 PM, 'Kevin Chowski' via golang-nuts > wrote: > >  > Guessing based on your latest

Re: [go-nuts] Concurrent io.Copy(tcpConn, file) and tcpConn.Write(...)

2020-04-28 Thread Robert Engels
Depends on how the file descriptor is implemented. But the end result probably has the same performance unless the network card is doing the TLS - which is possible. > On Apr 28, 2020, at 2:21 PM, Tamás Gulácsi wrote: > >  > TLS needs encyption, not jost "shoveling the bytes" to the

Re: [go-nuts] getting values from memory addresses within another process...

2020-04-30 Thread Robert Engels
This can be done fairly easily if you run the Go process as root and read the /proc/$pid/mem pseudo file. > On Apr 30, 2020, at 10:01 PM, Michael Jones wrote: > >  > The general dangerous ability to do this is why protected mode went into the > i368 and is the first and most essential

Re: [go-nuts] Re: About text-based user interface for Windows and Linux

2020-05-04 Thread Robert Engels
Look at github.com/robaho/go-trader It has a nice example of using gcui > On May 3, 2020, at 11:06 PM, 洪嘉鴻 wrote: > >  > I see. > Actually I only want to make a simple application. > The application is about segmenting two text-boxes for input and output. > I've tried this and it seems good,

Re: [go-nuts] tracing latency issue

2020-05-04 Thread Robert Engels
You might want to look at github.com/robaho/goanalyzer which I think is more useful when doing latency analysis. > On May 4, 2020, at 10:13 AM, yafimk wrote: > > Hey all, > I've been trying to optimize a simple dns resolver i've built. > Its intended to run on relatively small linux based

Re: [go-nuts] About text-based user interface for Windows and Linux

2020-05-04 Thread robert engels
There is no file algo.go in the repo so I am not sure what you are doing. I just tested cmd/algo with Intellij and it ran fine. You would be interested in the file cmd/client/main.go to see how to use gocui. > On May 4, 2020, at 10:15 PM, 洪嘉鴻 wrote: > > I've downloaded from git clone. >

Re: [go-nuts] Re: Is the following funciton always safe to convert a string to a slice?

2020-04-22 Thread Robert Engels
But it’s a data race regardless. Even if the runtime did interrupt the go routine mid write and change the stack it would be forced to eventually correct the value to have the write be valid - but if another routine is reading that value, including the GC, there must be synchronization added.

Re: [go-nuts] pprof samples

2020-04-22 Thread Robert Engels
pprof is not only cpu profiling - it is the same tracing infrastructure used in the go analyzer. > On Apr 22, 2020, at 2:28 PM, David Finkel wrote: > >  > > >> On Mon, Apr 20, 2020 at 8:35 PM asaxena via golang-nuts >> wrote: >> Hi, >> >> I am trying CPU profiling a program that runs

Re: [go-nuts] Image Resize and Crop

2020-05-11 Thread Robert Engels
That’s assuming the image is displayed. It is easier to keep portions of the image on disk with certain formats (eg tiled). > On May 11, 2020, at 2:40 PM, Robert Engels wrote: > > That means there is a memory leak. Once an image is decoded it takes the > same amount of

Re: [go-nuts] Measuring cpu usage for a tree of goroutines

2020-05-11 Thread Robert Engels
Look at pprof labels. > On May 11, 2020, at 6:29 PM, Steven Canfield wrote: > >  > Hi, > > I have an RPC server which has heterogenous requests, e.g. some calls hit > cache and are cheaper to serve while others need to compute a result. > > Is there any way to keep track of the cpu used

Re: [go-nuts] Image Resize and Crop

2020-05-11 Thread Robert Engels
That means there is a memory leak. Once an image is decoded it takes the same amount of memory based on resolution and bit depth. > On May 11, 2020, at 11:59 AM, Vivi wrote: > > It will make sense when you change PNG to JPG and loop 100x to see the > actual memory consumption with grtme -v

Re: [go-nuts] Measuring cpu usage for a tree of goroutines

2020-05-12 Thread robert engels
that Go can't do some things, but being opaque about it isn't > helpful. > > Thanks for your help both of you! > -Steve > > On Tue, May 12, 2020 at 1:57 PM Robert Engels <mailto:reng...@ix.netcom.com>> wrote: > In my first response I said to use labels and referred

Re: [go-nuts] Measuring cpu usage for a tree of goroutines

2020-05-12 Thread Robert Engels
abels (https://golang.org/pkg/runtime/pprof/#WithLabels) >>> and CPU profiling. The profile should let you attribute CPU usage per >>> label. >>> >>> However, that approach would only do sampling. I do not know of an >>> approach that would let you get fa

Re: [go-nuts] Measuring cpu usage for a tree of goroutines

2020-05-12 Thread robert engels
handling. > On May 12, 2020, at 5:00 PM, robert engels wrote: > > You can do it - just create a label for each request that you want timed > individually. These will be treated as distinct events. > > The OS threads are shared across Go routines - so using OS thread CPU >

Re: [go-nuts] Measuring cpu usage for a tree of goroutines

2020-05-11 Thread robert engels
Also, you may be interested in github.com/robaho/goanalyzer <http://github.com/robaho/goanalyzer> - I feel it makes latency analysis much easier. > On May 11, 2020, at 11:46 PM, robert engels wrote: > > You don’t need to do it this way. see https://rakyll.org/profiler-la

Re: [go-nuts] Measuring cpu usage for a tree of goroutines

2020-05-11 Thread robert engels
but what about the "I can't enable profiling for every request" bit? > Assume it's actually important for me to know the cpu consumption on a per > request basis. > > On Mon, May 11, 2020 at 4:55 PM Robert Engels <mailto:reng...@ix.netcom.com>> wrote: > Look at p

Re: [go-nuts] How to work with multiple environment like dev, test, staging, prod in golang ?

2020-05-17 Thread robert engels
SpringBoot is a framework. How it handles this is different from every other framework. Easiest way in Go, just name your configuration files similar to filename.mode.yaml where mode is prod, dev, qa, etc. and set a env variable for the mode. You can then take it from there, and have base

Re: [go-nuts] Any solution to hide my source code written in Go and make it a library to develop in Go as well

2020-05-17 Thread robert engels
Just distribute it as a plugin (https://golang.org/pkg/plugin/ ) - you only need to make the calling interface public - and based on your use case - there is nothing proprietary there so it won’t be a problem. > On May 17, 2020, at 9:45 PM, Billy Cui wrote: >

Re: [go-nuts] Image Resize and Crop

2020-05-10 Thread robert engels
My bad, I didn’t read the API docs completely. Kind of a strange interface declaration in the stdlib image package - that there is only a single rectangle for Draw() and it is bounded by the two images. > On May 10, 2020, at 5:57 PM, Nigel Tao wrote: > > On Mon, May 11, 2020 at 4:28

Re: [go-nuts] Image Resize and Crop

2020-05-10 Thread robert engels
All of the code to do scaling and cropping in the ‘image’ package in the stdlib. You ‘draw’ into a new image to do scaling. You use SubImage() to perform cropping. See https://blog.golang.org/image-draw Alternatively, if you need more advanced scaling operations it would be fairly trivial to

Re: [go-nuts] Image Resize and Crop

2020-05-10 Thread robert engels
0:11 UTC+8, Nigel Tao wrote: > On Mon, May 11, 2020 at 4:28 AM robert engels > > wrote: > > All of the code to do scaling and cropping in the ‘image’ package in the > > stdlib. > > Cropping is in the stdlib but scaling is not. Various scaling > algorithms (e.g.

Re: [go-nuts] Image Resize and Crop

2020-05-11 Thread Robert Engels
I’m sorry but that makes no sense. Do you mean file size is smaller? Depends on jpeg compression options. It can be many times smaller than a png. > On May 11, 2020, at 10:22 AM, Vivi wrote: > >  > > I found a snippet, memory usage with PNG vs JPG (more than 2x memory than PNG) >

Re: [go-nuts] Memory leak in github.com/golang/freetype

2020-05-13 Thread Robert Engels
That is a very large dpi and size. The cache that is created is going to be very large. I am guessing that it allocates so much memory so fast that the process is killed by the OS Out of memory killer before the GC runs to free the “dead” faces. Thus the OOM death.. > On May 13, 2020, at 4:03

Re: [go-nuts] Any solution to hide my source code written in Go and make it a library to develop in Go as well

2020-05-18 Thread Robert Engels
>> On Monday, May 18, 2020 at 11:21:40 AM UTC+8, robert engels wrote: >> Just distribute it as a plugin (https://golang.org/pkg/plugin/) - you only >> need to make the calling interface public - and based on your use case - >> there is nothing proprietary there so it won’t b

Re: [go-nuts] Sort of misleading coverage stats

2020-03-18 Thread Robert Engels
eads to "inflated" coverage numbers, because I am explicitly not > testing any of the downstream functionality, just the top level function. > > Again, this is not really a technical issue. > >> On Wednesday, 18 March 2020 13:29:05 UTC+1, Robert Engels wrote: >>

Re: [go-nuts] Sort of misleading coverage stats

2020-03-18 Thread Robert Engels
It’s because you are not writing the top level test correctly - there is no testing at all ! > On Mar 18, 2020, at 7:02 AM, Ondrej wrote: > >  > Hey! > > There's an issue I've been grappling with for ages - say I have a custom > function, Print, that calls six other functions, FormatString,

Re: [go-nuts] Benchmarking code that mutates its input data

2020-03-18 Thread Robert Engels
The test package has methods to disable the timing - wrap the setup in these. > On Mar 18, 2020, at 10:50 AM, Sam Whited wrote: > > I'd like to quickly suggest that the "filtering without allocating" > technique from this page is probably what you want and you may want to > consider if/why

Re: [go-nuts] Sort of misleading coverage stats

2020-03-18 Thread Robert Engels
. > On Mar 18, 2020, at 10:12 AM, Ondrej wrote: > >  > But they are callable - I'm only doing this for exported functions and > methods - as I described in the initial post (and the implementation). > >> On Wednesday, 18 March 2020 15:15:30 UTC+1, Robert Engels wrote: >

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-19 Thread Robert Engels
You are only using 1.5 mb on the Go side... so if your process is consuming lots of memory it’s on the C side. > On Mar 19, 2020, at 7:55 AM, Nitish Saboo wrote: > >  > Hi Michael, > > I used something like this to generate a mem-prof for 60 minutes > > func main() { > flag.Parse() > if

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-19 Thread Robert Engels
And https://blog.golang.org/pprof > On Mar 19, 2020, at 9:27 AM, Robert Engels wrote: > >  > https://www.freecodecamp.org/news/how-i-investigated-memory-leaks-in-go-using-pprof-on-a-large-codebase-4bec4325e192/amp/ > >>> On Mar 19, 2020, at 9:24 AM, Nitish Saboo

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-19 Thread Robert Engels
each of the functions or help to analyze the memory allocation much better? > > Thanks, > Nitish > >> On Thu, Mar 19, 2020 at 7:08 PM Robert Engels wrote: >> You are only using 1.5 mb on the Go side... so if your process is consuming >> lots of memory it’s on the C

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-17 Thread Robert Engels
would add code on the Go side that does similar Go global variable handling at the call site for the C call. Then run under the race detector - I’m guessing that it will report a race on the Go global. > On Mar 16, 2020, at 2:46 PM, Robert Engels wrote: > > In the single Go rou

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-17 Thread Robert Engels
_db_reload_ruleset(patterndb, configuration, file); > pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb); > return 0; > } > > But, what made you feel that Go global variable would report a race > condition? Since it is a single goroutine what would cause a race co

Re: [go-nuts] Issue with handler

2020-03-20 Thread robert engels
You have no login.html file... > On Mar 20, 2020, at 1:50 PM, Ali Hassan wrote: > > I have two html files >index and login > Index return status = 200 > login return status = 400 > > Code snipnet > > > > > > > > > > -- > You received this message because

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-23 Thread Robert Engels
Yes. You have a leak in your Go code. It shows you the object types that are taking up all of the space. There is no root analysis available in Go. Read the paper I linked to. > On Mar 23, 2020, at 9:12 AM, Nitish Saboo wrote: > >  > Hi, > > I used something like the following to generate

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-24 Thread Robert Engels
Why is it showing memory accounting for around 17GB? 11.1 % of 8GB is .88GB > and my node is only of 8GB. I feel the way I gathered the mem profiling was > not correct ..is it ? > Please advise me what am I missing? > > Thanks, > Nitish > >> On Tue, Mar 24, 2020 at 1:28

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-24 Thread Robert Engels
gt; >> Why is it showing memory accounting for around 17GB? 11.1 % of 8GB is .88GB >> and my node is only of 8GB. I feel the way I gathered the mem profiling was >> not correct ..is it ? >> Please advise me what am I missing? >> >> Thanks, >> Nitish

Re: [go-nuts] Bencharking issue with b.StartTimer and b.StopTimer. Was: Benchmark using b.StopTimer and b.StartTimer has unexpected behaviour - to me at least

2020-03-24 Thread Robert Engels
Can you please succinctly explain the problem? > On Mar 24, 2020, at 11:24 AM, Orson Cart wrote: > >  > I posted this earlier but I realised that the code had a fundamental error in > it. I've corrected here it but the underlying problem still exists. > > I've recently started using go

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-16 Thread Robert Engels
derstand your point completely. >> I have a global variable patterndb on C side and It is getting called from a >> single goroutine every 3 mins. Why do I need to synchronize it? >> Even though the goroutine gets pinned to different threads, it can access >> the s

Re: [go-nuts] Re: Bencharking issue with b.StartTimer and b.StopTimer. Was: Benchmark using b.StopTimer and b.StartTimer has unexpected behaviour - to me at least

2020-03-24 Thread Robert Engels
One way to handle this is to generate all of the data up front in an array and then just index into the array based on the run. > On Mar 24, 2020, at 3:42 PM, Orson Cart wrote: > >  > > >> On Tuesday, 24 March 2020 20:27:39 UTC, Adrian Ratnapala wrote: >> ... >> So that sounds like the

Re: [go-nuts] Re: Mem-Leak in Go method

2020-03-24 Thread Robert Engels
>>> not called 'LoadPatternDB()' method.I have 8GB of memory on the node where >>> I am running the service. My issue is : >>> >>> Why is it showing memory accounting for around 17GB? 11.1 % of 8GB is >>> .88GB and my node is only of 8GB.

Re: [go-nuts] Re: Bencharking issue with b.StartTimer and b.StopTimer. Was: Benchmark using b.StopTimer and b.StartTimer has unexpected behaviour - to me at least

2020-03-24 Thread Robert Engels
24, 2020 at 9:56:19 PM UTC+1, Orson Cart wrote: >>>> On Tuesday, 24 March 2020 20:47:07 UTC, Robert Engels wrote: >>>> One way to handle this is to generate all of the data up front in an array >>>> and then just index into the array based on the run. >>

<    5   6   7   8   9   10   11   12   13   14   >