Re: [go-nuts] Re: Discrepancy between htop and memstats

2019-05-02 Thread Michel Levieux
Hi Vladimir, I'm gonna try that today, I'll keep you updated, thanks for the advice! Le mer. 1 mai 2019 à 11:07, Vladimir Varankin a écrit : > Hi Michel, > > Have tried collecting your program's heap profiles [1] (maybe once after > each reload cycle)? Comparing pprof results should show you

[go-nuts] using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Øyvind Teig
Hi, Louki Sumirniy This is not really a response to your problem in particular, so it may totally miss your target. It's been a while since I did anything in this group. However, it's a response to the use of buffered channels. It's a coincidence that I react to your posting (and not the

[go-nuts] What's the best practice to trace a request in log

2019-05-02 Thread Sun Frank
Hi Guys: Assume we are building a HTTP server, I think it's a pretty common requirement to give each request a requestId and add that requestId as a log field, so log generated by this request all have this unique requestId, so we can do a "grep" in central log system(Graylog, Splunk, ELK,

[go-nuts] Re: Websocket vs Template

2019-05-02 Thread amnonbc
Use a websocket. templates would give you server side rendering which will not give you live updates on the web page. Steer clear of x/net/websocket which is deprecated. Instead use the popular gorilla/websocket or the simpler nhooyr/websocket. On Wednesday, 1 May 2019 15:11:55 UTC+1,

[go-nuts] where is the repository of proxy.golang.org?

2019-05-02 Thread T L
. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Websocket vs Template

2019-05-02 Thread ThisEndUp
Thanks. You're right. After posting, I kept looking and got as far as starting to read about AJAX XMLHttpRequest, at which point I decided that templates wouldn't be a particularly pleasant way to accomplish my task. On Thursday, May 2, 2019 at 2:45:33 AM UTC-4, amn...@gmail.com wrote: > > Use

[go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
Yeah, I was able to think a bit more about it as I was falling asleep later and I realised how I meant it to run. I had to verify that indeed channels are FIFO queues, as that was the basis of this way of using them. The receiver channel is unbuffered, and lives in one goroutine. When it

[go-nuts] Re: Map inside a struct

2019-05-02 Thread Luis Furquim
Hi, Check if your method has the signature like this: func (d DP) MyMethod() { ... } If so, change to this: func (d *DP) MyMethod() { ... } I made this error so many times: a method called by value changes the value of the object and the change vanishes when the method returns, you must

Re: [go-nuts] Re: Should IP.DefaultMask() exist in today's Internet?

2019-05-02 Thread Robert Engels
The net mask is not part of the ip packet. It is a local config in the router. > On May 2, 2019, at 7:20 AM, Louki Sumirniy > wrote: > > Upon review one thing occurred to me also - Netmasks are specifically a fast > way to decide at the router which direction a packet should go. The interface

Re: [go-nuts] Re: Should IP.DefaultMask() exist in today's Internet?

2019-05-02 Thread Louki Sumirniy
The function has a very specific purpose that I have encountered in several applications, that being to automatically set the netmask based on the IP being one of the several defined ones, 192, 10, and i forget which others. Incorrect netmask can result in not recognising a LAN address that is

Re: [go-nuts] Re: Should IP.DefaultMask() exist in today's Internet?

2019-05-02 Thread Louki Sumirniy
Upon review one thing occurred to me also - Netmasks are specifically a fast way to decide at the router which direction a packet should go. The interface netmask is part of the IP part of the header and allows the router to quickly determine whether a packet should go to the external rather

[go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
It's not precisely the general functionality that I will implement for my transport, but here is a simple example of a classifier type processing queue: https://play.golang.org/p/ytdrXgCdbQH This processes a series of sequential integers and pops them into an array to find the highest factor

[go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
I have been spending my time today getting my knowledge of this subject adequate enough to use channels for a UDP transport with FEC creating sharded pieces of the packets, and I just found this and played with some of the code on it and I just wanted to mention these things:

[go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread XXX ZZZ
using go version go1.12.4 linux/amd64 El jueves, 2 de mayo de 2019, 18:50:24 (UTC-3), Anthony Martin escribió: > > What version of Go are you using? > > XXX ZZZ > once said: > > fmt.(*pp).fmtString(0xc023c17740, 0x0, 0x5, 0xc00076) > > /usr/local/go/src/fmt/print.go:448 +0x132 > >

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
Ah, so this is what they are for - the same thing implemented with channels would be a nasty big slice with empty struct quit channels to first tell the main they are done. wg.Done() and wg.Wait() eliminate the complexity that a pure channel implementation would require. With that code I also

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Ian Lance Taylor
On Thu, May 2, 2019 at 2:50 PM Anthony Martin wrote: > > What version of Go are you using? > > XXX ZZZ once said: > > fmt.(*pp).fmtString(0xc023c17740, 0x0, 0x5, 0xc00076) > > /usr/local/go/src/fmt/print.go:448 +0x132 > > fmt.(*pp).printArg(0xc023c17740, 0x9978e0, 0xc016a68a30, 0x76) > >

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Anthony Martin
Ian Lance Taylor once said: > I don't *think* the format string is changing. I think the 0 is from > the string being printed, not the format string. They both happen to > be length 5. Misled by the pair of fives. Mea culpa. Anthony -- You received this message because you are subscribed

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Steven Hartland
You can see it doesn't wait by adding a counter as seen here: https://play.golang.org/p/-eqKggUEjhQ On 02/05/2019 21:09, Louki Sumirniy wrote: I have been spending my time today getting my knowledge of this subject adequate enough to use channels for a UDP transport with FEC creating sharded

[go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Øyvind Teig
Thanks for the reference to Dave Cheney's blog note! And for this thread, quite interesting to read. I am not used to explicitly closing channels at all (occam (in the ninetees) and XC (now)), but I have sat through several presentations on conferences seen the theme being discussed, like with

Re: [go-nuts] Re: Should IP.DefaultMask() exist in today's Internet?

2019-05-02 Thread John Dreystadt
>> >> >>> On Thursday, 2 May 2019 14:09:09 UTC+2, Louki Sumirniy wrote: >>> The function has a very specific purpose that I have encountered in several >>> applications, that being to automatically set the netmask based on the IP >>> being one of the several defined ones, 192, 10, and i

[go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Anthony Martin
What version of Go are you using? XXX ZZZ once said: > fmt.(*pp).fmtString(0xc023c17740, 0x0, 0x5, 0xc00076) > /usr/local/go/src/fmt/print.go:448 +0x132 > fmt.(*pp).printArg(0xc023c17740, 0x9978e0, 0xc016a68a30, 0x76) > /usr/local/go/src/fmt/print.go:684 +0x880 >

[go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
As I mentioned earlier, I wanted to see if I could implement a waitgroup with channels instead of the stdlib's sync.Atomic counters, and using a special type of concurrent datatype called a PN Converged Replicated Datatype. Well, I'm not sure if this implementation precisely implements this

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Steven Hartland
Without the wait group it doesn't wait, so you're not guaranteed for all / any of the goroutines to complete. On 02/05/2019 21:09, Louki Sumirniy wrote: I have been spending my time today getting my knowledge of this subject adequate enough to use channels for a UDP transport with FEC

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread Ian Lance Taylor
On Thu, May 2, 2019 at 11:18 AM Marcin Romaszewicz wrote: > > If that's the actual problem, you'd just be masking it, and producing an > invalid "x". Look here: > > func (r *Subid_info) Prepare_subid_logic(){ > r.Second_subid_8=fmt.Sprintf("1%07v", r.Second_subid) > panic happens >

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread Burak Serdar
On Thu, May 2, 2019 at 11:31 AM XXX ZZZ wrote: > > Hello, > > We are having a random panic on our go application that is happening once > every million requests or so, and so far we haven't been able to reproduce it > nor to even grasp what's going on. > > Basically our code goes like: > > type

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread XXX ZZZ
I did but nothing detected. However there aren't any goroutined involved (except for the http request), other than that, this variable isn't shared among routines. El jueves, 2 de mayo de 2019, 14:54:42 (UTC-3), Ian Lance Taylor escribió: > > On Thu, May 2, 2019 at 10:31 AM XXX ZZZ > > wrote:

Re: [go-nuts] Why does this simple snip generate an infinite loop ?

2019-05-02 Thread L Godioleskky
Ok, thanks. On Thu, May 2, 2019 at 1:26 PM Robert Engels wrote: > Because when u add 1 to 0xff it goes back to 0 since it is only 8 bits > > On May 2, 2019, at 12:22 PM, lgod...@gmail.com wrote: > > func main() { > > var c8 uint8; > var S [256] uint8; > >for c8 = 0x00; c8 <= 0xff;

[go-nuts] Random panic in production with Sprintf

2019-05-02 Thread XXX ZZZ
Hello, We are having a random panic on our go application that is happening once every million requests or so, and so far we haven't been able to reproduce it nor to even grasp what's going on. Basically our code goes like: type Subid_info struct{ Affiliate_subid string

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread Ian Lance Taylor
On Thu, May 2, 2019 at 10:31 AM XXX ZZZ wrote: > > We are having a random panic on our go application that is happening once > every million requests or so, and so far we haven't been able to reproduce it > nor to even grasp what's going on. > > Basically our code goes like: > > type Subid_info

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread Marcin Romaszewicz
If that's the actual problem, you'd just be masking it, and producing an invalid "x". Look here: func (r *Subid_info) Prepare_subid_logic(){ r.Second_subid_8=fmt.Sprintf("1%07v", r.Second_subid) > panic happens here. } r.Second_subid is in an invalid state which normal Go code could not

Re: [go-nuts] Why does this simple snip generate an infinite loop ?

2019-05-02 Thread Robert Engels
Because when u add 1 to 0xff it goes back to 0 since it is only 8 bits > On May 2, 2019, at 12:22 PM, lgod...@gmail.com wrote: > > func main() { > > var c8 uint8; > var S [256] uint8; > >for c8 = 0x00; c8 <= 0xff; c8 += 0x01 { S[c8]= c8 } > } > -- > You received this message

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread XXX ZZZ
I'm testing race conditions again as we speak, however this object is created WITHIN the goroutine (the http request), there is no way, afaik, that is being used from another routine. El jueves, 2 de mayo de 2019, 15:19:02 (UTC-3), Marcin Romaszewicz escribió: > > If that's the actual problem,

[go-nuts] Why does this simple snip generate an infinite loop ?

2019-05-02 Thread lgodio2
func main() { var c8 uint8; var S [256] uint8; for c8 = 0x00; c8 <= 0xff; c8 += 0x01 { S[c8]= c8 } } -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email

[go-nuts] Re: Is it possible to simplify this snippet?

2019-05-02 Thread Sundararajan Seshadri
For your question, the answer is NO. Your version is the most simple one. If it is relating to making it meaningful or more documented, you can try something like: //checkDirection returns the direction for the key pressed: 'up', 'down', 'left', 'right'. Any other return variable (nil) will

Re: [go-nuts] Random panic in production with Sprintf

2019-05-02 Thread Burak Serdar
On Thu, May 2, 2019 at 12:12 PM Burak Serdar wrote: > > On Thu, May 2, 2019 at 11:31 AM XXX ZZZ wrote: > > > > Hello, > > > > We are having a random panic on our go application that is happening once > > every million requests or so, and so far we haven't been able to reproduce > > it nor to

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Burak Serdar
On Thu, May 2, 2019 at 3:56 PM Ian Lance Taylor wrote: > > On Thu, May 2, 2019 at 2:50 PM Anthony Martin wrote: > > > > What version of Go are you using? > > > > XXX ZZZ once said: > > > fmt.(*pp).fmtString(0xc023c17740, 0x0, 0x5, 0xc00076) > > > /usr/local/go/src/fmt/print.go:448

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Burak Serdar
On Thu, May 2, 2019 at 6:34 PM Tyler Compton wrote: > > I took a quick look and yes, it uses unsafe to convert between byte slices > and strings. I don't know enough to say that it's the problem but here's an > example: > >

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Michael Jones
Is any of this string data touched/from C via CGO? On Thu, May 2, 2019 at 3:09 PM Anthony Martin wrote: > Ian Lance Taylor once said: > > I don't *think* the format string is changing. I think the 0 is from > > the string being printed, not the format string. They both happen to > > be

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Robert Engels
Channels use sync primitives under the hood so you are not saving anything by using multiple channels instead of a single wait group. > On May 2, 2019, at 5:57 PM, Louki Sumirniy > wrote: > > As I mentioned earlier, I wanted to see if I could implement a waitgroup with > channels instead of

Re: [go-nuts] Re: Should IP.DefaultMask() exist in today's Internet?

2019-05-02 Thread Louki Sumirniy
I'm quite aware of that, it's part of the ARP, and allows the router to quickly determine which port to send to. If you put say 192.168.1.1 to a router configured with DHCP to 192.168.0.x/24 it first checks the mask by ANDing it with the list of address/port network lists' gateway to find the

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
I more or less eventually figured that out since it is impossible to query the number of workers without a race anyway, and then I started toying with atomic.Value and made that one race as well (obviously the value was copied by fmt.Println). I guess keeping track of the number of workers is

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Burak Serdar
On Thu, May 2, 2019 at 6:02 PM XXX ZZZ wrote: > > No use of C via CGO at all. > > Afaik, there isn't any unsafe use of the string, we are basically reading it > from a get parameter (fasthttp server) on an http request and then adding it > into this structure, most of the times is just a 5 char

Re: [go-nuts] Re: Should IP.DefaultMask() exist in today's Internet?

2019-05-02 Thread Robert Engels
I only mentioned it because you wrote: > The interface netmask is part of the IP part of the header and I’m also fairly certain it is not part of ARP - ARP maps MAC addresses to IP addresses on the local subnet. > On May 2, 2019, at 7:22 PM, Louki Sumirniy > wrote: > > The interface

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Tyler Compton
I took a quick look and yes, it uses unsafe to convert between byte slices and strings. I don't know enough to say that it's the problem but here's an example: https://github.com/valyala/fasthttp/blob/645361952477dfc16938fb2993065130ed7c02b9/bytesconv.go#L380 On Thu, May 2, 2019 at 5:16 PM Burak

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
oh, I did forget one thing. The race detector does not flag a race in this code: https://play.golang.org/p/M1uGq1g4vjo (play refuses to run it though) As I understand it, that's because the add/subtract operations are happening serially within the main handler goroutine. I suppose if I were to

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread Robert Engels
Whenever I see fast* I think someone took shortcuts to make something “faster” without fully implementing the spec (or external constraints, like safe data access) > On May 2, 2019, at 7:16 PM, Burak Serdar wrote: > >> On Thu, May 2, 2019 at 6:02 PM XXX ZZZ wrote: >> >> No use of C via CGO

Re: [go-nuts] Re: using channels to gather bundles from inputs for batch processing

2019-05-02 Thread Louki Sumirniy
ah yes, no, if you see the code in the play link below, it only has three channels, ops, done and ready. I just figured out that I replaced that ready by putting the close in the clause that processes incoming ops, and it's unused as well. I managed to trim it down to just one channel, the ops

Re: [go-nuts] Re: Random panic in production with Sprintf

2019-05-02 Thread XXX ZZZ
No use of C via CGO at all. Afaik, there isn't any unsafe use of the string, we are basically reading it from a get parameter (fasthttp server) on an http request and then adding it into this structure, most of the times is just a 5 char string. Out of several millions requests, this panic