[go-nuts] Re: Python cool methods in Golang

[Amnon]

Go for it!

Write a few methods. 
Put it on github
and post a link here.

Best of luck!
On Saturday, 20 August 2022 at 11:29:37 UTC+1 harald@gmx.net wrote:

> Not showing the code and attaching a (INAL) useless "confidentiality 
> notice" (good luck arguing with Google over that) looks like a big "no-go" 
> to me. What exactly is your intention?
>
> On Thursday, August 18, 2022 at 12:52:42 AM UTC+2 andres...@rappi.com 
> wrote:
>
>> Hi !
>> Currently I am working on a new library to simulate some cool and useful 
>> Python methods like (Extends, pop, contains, sets, heapify , etc) with Go 
>> 1.18 , I wanna know if you might find it useful and if someone wants to add 
>> stuff or help are more than welcome.
>> My personal email is af.ba...@uniandes.edu.co if someone would like to 
>> chat or contribute
>> Thanks
>>
>> *CONFIDENTIALITY NOTICE: *The contents of this email message and any 
>> attachments are intended solely for the addressee(s) and may contain 
>> confidential and/or privileged information and are legally protected from 
>> disclosure. If you are not the intended recipient of this message or their 
>> agent, or if this message has been addressed to you in error, please 
>> immediately alert the sender by replying to this email and then delete this 
>> message and any attachments. If you are not the intended recipient, you are 
>> hereby notified that any use, dissemination, copying, or storage of this 
>> message or its attachments without authorization is strictly prohibited.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f808efd5-56e1-471f-af22-97c0ca1c96a8n%40googlegroups.com.

Re: [go-nuts] Go multiplexers

[ramki...@hotmail.com]

Also, by chance do you know of an alternative that would provide multiple 
streams for a connection that happens different protocols or for TCP?

On Saturday, August 20, 2022 at 7:01:00 PM UTC-4 se...@liao.dev wrote:

> yamux uses its own protocol, it doesn't speak telnet
>
> - sean
>
> On Sat, Aug 20, 2022, 21:31 ramki...@hotmail.com  
> wrote:
>
>> Has anyone ever used a multiplexer ? 
>> When creating a TCP server using multiplexer, I get an error [ERR] yamux: 
>> Invalid protocol version: 102 when using Telnet to connect to the server.
>>
>> package main
>>
>> import (
>> "fmt"
>> "net"
>>
>> "github.com/hashicorp/yamux"
>> )
>>
>> func main() {
>> server()
>> }
>>
>> func server() {
>>
>> listener, _ := net.Listen("tcp", "0.0.0.0:1113")
>> for {
>>
>> // Accept a TCP connection
>> conn, err := listener.Accept()
>> if err != nil {
>> panic(err)
>> }
>>
>> // Setup server side of smux
>> session, err := yamux.Server(conn, nil)
>> if err != nil {
>> panic(err)
>> }
>>
>> // Accept a stream
>> stream, err := session.Accept()
>> if err != nil {
>> panic(err)
>> }
>> fmt.Println("DPOO")
>>
>> // Listen for a message
>> buf := make([]byte, 4)
>> stream.Read(buf)
>> stream.Close()
>> session.Close()
>> }
>> }
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/0034b133-79c3-47e7-853a-10282a5beed5n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1b75cc6b-c956-4a30-bd3e-664aca6a6af4n%40googlegroups.com.

Re: [go-nuts] Go multiplexers

[ramki...@hotmail.com]

Can  you tell me how I can figure out what it speaks?

On Saturday, August 20, 2022 at 7:01:00 PM UTC-4 se...@liao.dev wrote:

> yamux uses its own protocol, it doesn't speak telnet
>
> - sean
>
> On Sat, Aug 20, 2022, 21:31 ramki...@hotmail.com  
> wrote:
>
>> Has anyone ever used a multiplexer ? 
>> When creating a TCP server using multiplexer, I get an error [ERR] yamux: 
>> Invalid protocol version: 102 when using Telnet to connect to the server.
>>
>> package main
>>
>> import (
>> "fmt"
>> "net"
>>
>> "github.com/hashicorp/yamux"
>> )
>>
>> func main() {
>> server()
>> }
>>
>> func server() {
>>
>> listener, _ := net.Listen("tcp", "0.0.0.0:1113")
>> for {
>>
>> // Accept a TCP connection
>> conn, err := listener.Accept()
>> if err != nil {
>> panic(err)
>> }
>>
>> // Setup server side of smux
>> session, err := yamux.Server(conn, nil)
>> if err != nil {
>> panic(err)
>> }
>>
>> // Accept a stream
>> stream, err := session.Accept()
>> if err != nil {
>> panic(err)
>> }
>> fmt.Println("DPOO")
>>
>> // Listen for a message
>> buf := make([]byte, 4)
>> stream.Read(buf)
>> stream.Close()
>> session.Close()
>> }
>> }
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/0034b133-79c3-47e7-853a-10282a5beed5n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ca39944e-c85a-4e4f-9d58-b9661d535fcbn%40googlegroups.com.

Re: [go-nuts] Go multiplexers

['Sean Liao' via golang-nuts]

yamux uses its own protocol, it doesn't speak telnet

- sean

On Sat, Aug 20, 2022, 21:31 ramki...@hotmail.com 
wrote:

> Has anyone ever used a multiplexer ?
> When creating a TCP server using multiplexer, I get an error [ERR] yamux:
> Invalid protocol version: 102 when using Telnet to connect to the server.
>
> package main
>
> import (
> "fmt"
> "net"
>
> "github.com/hashicorp/yamux"
> )
>
> func main() {
> server()
> }
>
> func server() {
>
> listener, _ := net.Listen("tcp", "0.0.0.0:1113")
> for {
>
> // Accept a TCP connection
> conn, err := listener.Accept()
> if err != nil {
> panic(err)
> }
>
> // Setup server side of smux
> session, err := yamux.Server(conn, nil)
> if err != nil {
> panic(err)
> }
>
> // Accept a stream
> stream, err := session.Accept()
> if err != nil {
> panic(err)
> }
> fmt.Println("DPOO")
>
> // Listen for a message
> buf := make([]byte, 4)
> stream.Read(buf)
> stream.Close()
> session.Close()
> }
> }
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/0034b133-79c3-47e7-853a-10282a5beed5n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAGabyPrgTFhN0Meh3ohtrPkJMQjBbBdX48XknNxbRyJy2YXv7Q%40mail.gmail.com.

[go-nuts] Go multiplexers

[ramki...@hotmail.com]

Has anyone ever used a multiplexer ? 
When creating a TCP server using multiplexer, I get an error [ERR] yamux: 
Invalid protocol version: 102 when using Telnet to connect to the server.

package main

import (
"fmt"
"net"

"github.com/hashicorp/yamux"
)

func main() {
server()
}

func server() {

listener, _ := net.Listen("tcp", "0.0.0.0:1113")
for {

// Accept a TCP connection
conn, err := listener.Accept()
if err != nil {
panic(err)
}

// Setup server side of smux
session, err := yamux.Server(conn, nil)
if err != nil {
panic(err)
}

// Accept a stream
stream, err := session.Accept()
if err != nil {
panic(err)
}
fmt.Println("DPOO")

// Listen for a message
buf := make([]byte, 4)
stream.Read(buf)
stream.Close()
session.Close()
}
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0034b133-79c3-47e7-853a-10282a5beed5n%40googlegroups.com.

[go-nuts] github.com/splunk/pipelines hits v1.0.0

[K. Alex Mills]

Hi Gophers!

The API has solidified, so I've just released github.com/splunk/pipelines
at v1.0.0. This library provides a lightweight way to manage concurrency
separately from your business logic by breaking a concurrent task down into
stages in a pipeline, connected by nothing more than plain old channels.

*Features:*
* Pipeline stages based on common functional programming primitives.
* Integrated with termination signals from the context package.
* Scale out individual pipeline stages without disrupting business logic.
* Introduce queues by buffering channels mid-stream.
* Split and combine pipeline stages as pairs.
* Handle fatal and non-fatal errors from any pipeline stage.
* Sinks for draining the result of a pipeline computation.

Some future enhancements are planned. Please leave any suggestions or
issues in the issue tracker.

Thanks

K. Alex Mills

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALJzkY8nuC3v_%3DzV3xhZ_18EXqvv9wPfvDfFZWg%3DQu0krpzqQA%40mail.gmail.com.

[go-nuts] Re: Go 1.19 average goroutine stack

[tapi...@gmail.com]

BTW, the outputs of the test program:

$ GODEBUG=gctrace=1  ./main
gc 1 @0.006s 4%: 0.035+1.5+0.006 ms clock, 0.14+0/1.5/2.0+0.024 ms cpu, 
0->0->0 MB, 8 MB goal, 0 MB stacks, 8 MB globals, 4 P (forced)
2022/08/20 22:46:47 2048 1
gc 2 @2.123s 0%: 0.034+2.6+0.008 ms clock, 0.13+0/2.5/7.1+0.035 ms cpu, 
0->0->0 MB, 8 MB goal, 64 MB stacks, 8 MB globals, 4 P (forced)
2022/08/20 22:46:49 8388608 2
GC forced
gc 3 @124.390s 0%: 0.045+5.1+0.010 ms clock, 0.18+0/5.1/0+0.041 ms cpu, 
0->0->0 MB, 63 MB goal, 128 MB stacks, 8 MB globals, 4 P
GC forced
gc 4 @244.411s 0%: 0.072+5.0+0.009 ms clock, 0.28+0/5.0/0+0.038 ms cpu, 
0->0->0 MB, 118 MB goal, 128 MB stacks, 8 MB globals, 4 P
gc 5 @307.129s 0%: 0.076+2.7+0.013 ms clock, 0.30+0/2.7/7.9+0.052 ms cpu, 
0->0->0 MB, 118 MB goal, 128 MB stacks, 8 MB globals, 4 P (forced)
2022/08/20 22:51:56 16777216 3
gc 6 @313.134s 0%: 0.098+2.6+0.011 ms clock, 0.39+0/2.5/7.5+0.044 ms cpu, 
0->0->0 MB, 118 MB goal, 128 MB stacks, 8 MB globals, 4 P (forced)
2022/08/20 22:52:02 16777216 3

On Saturday, August 20, 2022 at 10:58:10 PM UTC+8 tapi...@gmail.com wrote:

> On Friday, August 19, 2022 at 12:14:55 AM UTC+8 k...@google.com wrote:
>
>> On Wednesday, August 17, 2022 at 8:18:35 PM UTC-7 tapi...@gmail.com 
>> wrote:
>>
>>> I'm a bit wondering about how the following case will be affected by the 
>>> change:
>>> 1. Initially, there is one goroutine, which stack size is large at the 
>>> time of a GC process.
>>> 2. After the GC process, a large quantity of goroutines start. They all 
>>> need small stacks.
>>>But now the runtime will allocate a very large stack for each of them.
>>>
>>> Then is much memory wasted?
>>>
>>
>> Kind of. Your description is correct, the runtime will allocate larger 
>> stacks for each of the new goroutines. But it doesn't really waste memory, 
>> it just causes the program to reach the next GC earlier. At that GC, stacks 
>> will shrink, as will the start size for subsequent goroutines. So it won't 
>> really waste memory, but waste some CPU for a possibly extra GC.
>> It isn't perfect; for instance we only shrink stacks 2x at each GC, so if 
>> the starting size is way too large it might take a couple GCs to shrink 
>> sufficiently.
>>
>
> I wrote a program to make some tests: https://go.dev/play/p/oDYrAsZz_3i
>
> It looks the new GC pacer doesn't count new stack memory as new allocated 
> heap memory.
> So large new stacks will not make the program to reach the next GC earlier.
>
> Since Go 1.18, the stack sizes (the real sizes, not the 2^n sizes) and 
> global pointers contribute in calculating target heap memory:
> https://go.dev/doc/gc-guide#GOGC, so that large stack sizes will cause 
> large GC intervals.
>
> It seems that the outputs indicate there are some hidden goroutines which 
> cause the calculated average stack sizes
> much smaller than expected.
>
> BTW, in the runtime docs, https://pkg.go.dev/runtime, "# MB stacks" and "# 
> MB globals" are listed but not shown in the format line.
>
>
>  
>
>>  
>>
>>> Will the stacks of the new goroutines shrink at the next GC process?
>>>
>>
>> Yes. The larger stack sizes are only on goroutine start. The runtime 
>> doesn't forbid shrinking below the start size.
>>  
>>
>>>
>>> On Saturday, August 13, 2022 at 10:00:58 PM UTC+8 lia...@garena.com 
>>> wrote:
>>>
 Hi masters,

 As far as I know, go 1.19 supports self-adaptive stack size when 
 spawning a goroutine, which intends to decrease the calling times of 
 morestack that aims to claim more frames from heap.

 After each GC happens, Go runtime will calculate the average stack 
 usage and the next time, goroutine would created with such a stack size.

 My question is, how do we validate the whole process or check it if 
 works well or not.
 Is there any metric or stack size checking function, or should I use 
 pprof to peek the alloc part ?

 Kindly thanks for all

 ```
 func main() {
 go func() {
 // spawn a goroutine started from a fixed stack size
 }()

 runtime.GC()

 go func() {
 // spawn a goroutine started from an avg stack size. 
 }()
 }
 ```




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d185b4fe-2de6-4ef9-94c8-bfd9f14fe1c3n%40googlegroups.com.

[go-nuts] Re: Go 1.19 average goroutine stack

[tapi...@gmail.com]

On Friday, August 19, 2022 at 12:14:55 AM UTC+8 k...@google.com wrote:

> On Wednesday, August 17, 2022 at 8:18:35 PM UTC-7 tapi...@gmail.com wrote:
>
>> I'm a bit wondering about how the following case will be affected by the 
>> change:
>> 1. Initially, there is one goroutine, which stack size is large at the 
>> time of a GC process.
>> 2. After the GC process, a large quantity of goroutines start. They all 
>> need small stacks.
>>But now the runtime will allocate a very large stack for each of them.
>>
>> Then is much memory wasted?
>>
>
> Kind of. Your description is correct, the runtime will allocate larger 
> stacks for each of the new goroutines. But it doesn't really waste memory, 
> it just causes the program to reach the next GC earlier. At that GC, stacks 
> will shrink, as will the start size for subsequent goroutines. So it won't 
> really waste memory, but waste some CPU for a possibly extra GC.
> It isn't perfect; for instance we only shrink stacks 2x at each GC, so if 
> the starting size is way too large it might take a couple GCs to shrink 
> sufficiently.
>

I wrote a program to make some tests: https://go.dev/play/p/oDYrAsZz_3i

It looks the new GC pacer doesn't count new stack memory as new allocated 
heap memory.
So large new stacks will not make the program to reach the next GC earlier.

Since Go 1.18, the stack sizes (the real sizes, not the 2^n sizes) and 
global pointers contribute in calculating target heap memory:
https://go.dev/doc/gc-guide#GOGC, so that large stack sizes will cause 
large GC intervals.

It seems that the outputs indicate there are some hidden goroutines which 
cause the calculated average stack sizes
much smaller than expected.

BTW, in the runtime docs, https://pkg.go.dev/runtime, "# MB stacks" and "# 
MB globals" are listed but not shown in the format line.


 

>  
>
>> Will the stacks of the new goroutines shrink at the next GC process?
>>
>
> Yes. The larger stack sizes are only on goroutine start. The runtime 
> doesn't forbid shrinking below the start size.
>  
>
>>
>> On Saturday, August 13, 2022 at 10:00:58 PM UTC+8 lia...@garena.com 
>> wrote:
>>
>>> Hi masters,
>>>
>>> As far as I know, go 1.19 supports self-adaptive stack size when 
>>> spawning a goroutine, which intends to decrease the calling times of 
>>> morestack that aims to claim more frames from heap.
>>>
>>> After each GC happens, Go runtime will calculate the average stack usage 
>>> and the next time, goroutine would created with such a stack size.
>>>
>>> My question is, how do we validate the whole process or check it if 
>>> works well or not.
>>> Is there any metric or stack size checking function, or should I use 
>>> pprof to peek the alloc part ?
>>>
>>> Kindly thanks for all
>>>
>>> ```
>>> func main() {
>>> go func() {
>>> // spawn a goroutine started from a fixed stack size
>>> }()
>>>
>>> runtime.GC()
>>>
>>> go func() {
>>> // spawn a goroutine started from an avg stack size. 
>>> }()
>>> }
>>> ```
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ef1f7e04-439e-4ab5-9143-527935180828n%40googlegroups.com.

[go-nuts] Re: Python cool methods in Golang

[TheDiveO]

Not showing the code and attaching a (INAL) useless "confidentiality 
notice" (good luck arguing with Google over that) looks like a big "no-go" 
to me. What exactly is your intention?

On Thursday, August 18, 2022 at 12:52:42 AM UTC+2 andres...@rappi.com wrote:

> Hi !
> Currently I am working on a new library to simulate some cool and useful 
> Python methods like (Extends, pop, contains, sets, heapify , etc) with Go 
> 1.18 , I wanna know if you might find it useful and if someone wants to add 
> stuff or help are more than welcome.
> My personal email is af.ba...@uniandes.edu.co if someone would like to 
> chat or contribute
> Thanks
>
> *CONFIDENTIALITY NOTICE: *The contents of this email message and any 
> attachments are intended solely for the addressee(s) and may contain 
> confidential and/or privileged information and are legally protected from 
> disclosure. If you are not the intended recipient of this message or their 
> agent, or if this message has been addressed to you in error, please 
> immediately alert the sender by replying to this email and then delete this 
> message and any attachments. If you are not the intended recipient, you are 
> hereby notified that any use, dissemination, copying, or storage of this 
> message or its attachments without authorization is strictly prohibited.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b5ad7670-2455-4170-b65e-3e0f8679b486n%40googlegroups.com.