Re: [go-nuts] rand seed using 'time.Now().Unix()' consequence mass common rand numbers, but using unixNano as the seed will not ,how to choose the rand's seed between 'time.Now().Unix()' and 'time.Now
It's the context:
// NewRandW creates a new RandW with a random object.
func NewRandW() *RandW {
return {r: rand.New(rand.NewSource(time.Now().Unix()))}
}
n := w.Next()
// Next returns next selected item.
func (rw *RandW) Next() (item interface{}) {
...
randomWeight := rw.r.Intn(rw.sumOfWeights)
//fmt.Printf("rand val: %d\n",randomWeight)
...
}
I am sorry about that I did't give the context of my code.
在 2019年6月27日星期四 UTC+8上午11:22:49，Kurtis Rader写道：
>
> On Wed, Jun 26, 2019 at 8:18 PM Chou Yan > wrote:
>
>> I fix it by :
>>
>> var w = weighted.NewRandW()
>>
>
> Insufficient context to understand what that does. Let alone how it
> "fixes" the problem since you haven't shown us your `NewRandW()` function.
> In general you should never initialize a RNG more than once unless you are
> doing so to create a reproducible sequence of values. In which case you
> should be initializing it with a constant value rather than a pseudo-random
> value like `time.Now().Unix()`.
>
> --
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/638a5f5b-552f-4f56-9f7f-3253fb8c2e29%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] rand seed using 'time.Now().Unix()' consequence mass common rand numbers, but using unixNano as the seed will not ,how to choose the rand's seed between 'time.Now().Unix()' and 'time.Now
thx. you are right. I make a mistake. I negleck that my func will be called
by mass goroutine, it will get the same seed and the same sequence
在 2019年6月27日星期四 UTC+8上午11:07:28，Kurtis Rader写道：
>
> Works for me: https://play.golang.org/p/zD5F7gp41re
>
> Like Burak I suspect you are initializing the RNG in a tight loop. Since
> `time.Now().Unix()` has a resolution of one second you end up generating
> the same initial value every time through the loop until the current time
> advances to the next second.
>
> On Wed, Jun 26, 2019 at 7:48 PM Chou Yan > wrote:
>
>> like this:
>> r:=rand.New(rand.NewSource(time.Now().Unix()))
>> for {
>> r.Intn(96)
>> }
>> I know the same seed will generate the same sequence. But I don't know
>> why it generate mass same number when I use seed of 'time.Now().Unix()',
>> But when I use seed of 'time.Now().UnixNano()', It will not.
>> I suspect what this is related to the seqence rand algorithm,But I have
>> no relevant evidence
>>
>> 在 2019年6月27日星期四 UTC+8上午10:32:38，Burak Serdar写道：
>>>
>>> On Wed, Jun 26, 2019 at 8:17 PM Chou Yan wrote:
>>> >
>>> > I use:
>>> > r:=rand.New(rand.NewSource(time.Now().Unix()))
>>> > r..Intn(96)
>>>
>>> How are you generating multiple random numbers? If your loop that
>>> generates these numbers include the r:=rand.New(...), then you're
>>> essentially seeding the random number generator with the same number,
>>> because the program ends before time.Now().Unix() returns a different
>>> value every second.
>>>
>>> Move the r:=rand.New(...) outsite the for loop.
>>>
>>> > I got:
>>> >
>>> > and val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 39
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > rand val: 82
>>> > ...
>>> > mass common sequential number .
>>> > But if I change that：
>>> > r:=rand.New(rand.NewSource(time.Now().UnixNano()))
>>> > It will not happen.
>>> > Why does this happened? and how do I choose it between
>>> time.Now().Unix() or UnixNano()?
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golan...@googlegroups.com.
>>> > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/ce0671c5-f2df-42d9-a7af-bdef939e6d6d%40googlegroups.com.
>>>
>>>
>>> > For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golan...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/99351298-7843-4556-b422-8635eb96c936%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/golang-nuts/99351298-7843-4556-b422-8635eb96c936%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/1e5a8486-ef43-4258-8ffc-d21dc542638f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] rand seed using 'time.Now().Unix()' consequence mass common rand numbers, but using unixNano as the seed will not ,how to choose the rand's seed between 'time.Now().Unix()' and 'time.Now
I fix it by :
var w = weighted.NewRandW()
func WeightRandom(services []*registry.Service) Next{
for _, service := range services {
for _,n := range service.Nodes {
w.Add(n,n.Weight)
}
}
return func() (*registry.Node, error) {
n := w.Next()
no, ok := n.(*registry.Node)
if !ok {
return nil,errors.New("get next err")
}
return no, nil
}
}
thx
在 2019年6月27日星期四 UTC+8上午11:16:01，Chou Yan写道：
>
> The situation I am currently experiencing is that it only occur on my
> online app. I try it at local but I can not got the result.
> haha... I make a mistake.
> the code is :
>
> func WeightRandom(services []*registry.Service) Next{
> w := weighted.NewRandW()
> for _, service := range services {
> for _,n := range service.Nodes {
> w.Add(n,n.Weight)
> }
> }
> return func() (*registry.Node, error) {
> n := w.Next()
> no, ok := n.(*registry.Node)
> if !ok {
> return nil,errors.New("get next err")
> }
> return no, nil
> }
> }
>
> The 'rand.New(...SEED)' is outside the loop of for But I neglect that this
> func 'WeightRandom' will be mass goroutine call. so each time.Now().Unix()
> ,the same seed ,the same sequence. So I got mass common rand number.But
> when I change it as time.Now().UnixNano(), the seed will not be the same
> because of the unixnano. so I got correctly. thank you very much
>
> 在 2019年6月27日星期四 UTC+8上午10:52:56，Burak Serdar写道：
>>
>> On Wed, Jun 26, 2019 at 8:48 PM Chou Yan wrote:
>> >
>> > like this:
>> > r:=rand.New(rand.NewSource(time.Now().Unix()))
>> > for {
>> > r.Intn(96)
>> > }
>> > I know the same seed will generate the same sequence. But I don't know
>> why it generate mass same number when I use seed of 'time.Now().Unix()',
>> But when I use seed of 'time.Now().UnixNano()', It will not.
>> > I suspect what this is related to the seqence rand algorithm,But I have
>> no relevant evidence
>>
>> This doesn't make sense. Can you reproduce it in the go playground?
>>
>> >
>> > 在 2019年6月27日星期四 UTC+8上午10:32:38，Burak Serdar写道：
>> >>
>> >> On Wed, Jun 26, 2019 at 8:17 PM Chou Yan wrote:
>> >> >
>> >> > I use:
>> >> > r:=rand.New(rand.NewSource(time.Now().Unix()))
>> >> > r..Intn(96)
>> >>
>> >> How are you generating multiple random numbers? If your loop that
>> >> generates these numbers include the r:=rand.New(...), then you're
>> >> essentially seeding the random number generator with the same number,
>> >> because the program ends before time.Now().Unix() returns a different
>> >> value every second.
>> >>
>> >> Move the r:=rand.New(...) outsite the for loop.
>> >>
>> >> > I got:
>> >> >
>> >> > and val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 39
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > rand val: 82
>> >> > ...
>> >> > mass common sequential number .
>> >> > But if I change that：
>> >> > r:=rand.New(rand.NewSource(time.Now().UnixNano()))
>> >> > It will not happen.
>> >> > Why does this happened? and how do
Re: [go-nuts] rand seed using 'time.Now().Unix()' consequence mass common rand numbers, but using unixNano as the seed will not ,how to choose the rand's seed between 'time.Now().Unix()' and 'time.Now
The situation I am currently experiencing is that it only occur on my
online app. I try it at local but I can not got the result.
haha... I make a mistake.
the code is :
func WeightRandom(services []*registry.Service) Next{
w := weighted.NewRandW()
for _, service := range services {
for _,n := range service.Nodes {
w.Add(n,n.Weight)
}
}
return func() (*registry.Node, error) {
n := w.Next()
no, ok := n.(*registry.Node)
if !ok {
return nil,errors.New("get next err")
}
return no, nil
}
}
The 'rand.New(...SEED)' is outside the loop of for But I neglect that this
func 'WeightRandom' will be mass goroutine call. so each time.Now().Unix()
,the same seed ,the same sequence. So I got mass common rand number.But
when I change it as time.Now().UnixNano(), the seed will not be the same
because of the unixnano. so I got correctly. thank you very much
在 2019年6月27日星期四 UTC+8上午10:52:56，Burak Serdar写道：
>
> On Wed, Jun 26, 2019 at 8:48 PM Chou Yan > wrote:
> >
> > like this:
> > r:=rand.New(rand.NewSource(time.Now().Unix()))
> > for {
> > r.Intn(96)
> > }
> > I know the same seed will generate the same sequence. But I don't know
> why it generate mass same number when I use seed of 'time.Now().Unix()',
> But when I use seed of 'time.Now().UnixNano()', It will not.
> > I suspect what this is related to the seqence rand algorithm,But I have
> no relevant evidence
>
> This doesn't make sense. Can you reproduce it in the go playground?
>
> >
> > 在 2019年6月27日星期四 UTC+8上午10:32:38，Burak Serdar写道：
> >>
> >> On Wed, Jun 26, 2019 at 8:17 PM Chou Yan wrote:
> >> >
> >> > I use:
> >> > r:=rand.New(rand.NewSource(time.Now().Unix()))
> >> > r..Intn(96)
> >>
> >> How are you generating multiple random numbers? If your loop that
> >> generates these numbers include the r:=rand.New(...), then you're
> >> essentially seeding the random number generator with the same number,
> >> because the program ends before time.Now().Unix() returns a different
> >> value every second.
> >>
> >> Move the r:=rand.New(...) outsite the for loop.
> >>
> >> > I got:
> >> >
> >> > and val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 39
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > rand val: 82
> >> > ...
> >> > mass common sequential number .
> >> > But if I change that：
> >> > r:=rand.New(rand.NewSource(time.Now().UnixNano()))
> >> > It will not happen.
> >> > Why does this happened? and how do I choose it between
> time.Now().Unix() or UnixNano()?
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> >> > To unsubscribe from this group and stop receiving emails from it,
> send an email to golan...@googlegroups.com.
> >> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ce0671c5-f2df-42d9-a7af-bdef939e6d6d%40googlegroups.com.
>
>
> >> > For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golan...@googlegroups.com .
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/99351298-7843-4556-b422-8635eb96c936%40googlegroups.com.
>
>
> > For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/1760eb39-21e7-479b-bea7-8276e03e3873%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] rand seed using 'time.Now().Unix()' consequence mass common rand numbers, but using unixNano as the seed will not ,how to choose the rand's seed between 'time.Now().Unix()' and 'time.Now
like this:
r:=rand.New(rand.NewSource(time.Now().Unix()))
for {
r.Intn(96)
}
I know the same seed will generate the same sequence. But I don't know why
it generate mass same number when I use seed of 'time.Now().Unix()', But
when I use seed of 'time.Now().UnixNano()', It will not.
I suspect what this is related to the seqence rand algorithm,But I have no
relevant evidence
在 2019年6月27日星期四 UTC+8上午10:32:38，Burak Serdar写道：
>
> On Wed, Jun 26, 2019 at 8:17 PM Chou Yan > wrote:
> >
> > I use:
> > r:=rand.New(rand.NewSource(time.Now().Unix()))
> > r..Intn(96)
>
> How are you generating multiple random numbers? If your loop that
> generates these numbers include the r:=rand.New(...), then you're
> essentially seeding the random number generator with the same number,
> because the program ends before time.Now().Unix() returns a different
> value every second.
>
> Move the r:=rand.New(...) outsite the for loop.
>
> > I got:
> >
> > and val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 39
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > rand val: 82
> > ...
> > mass common sequential number .
> > But if I change that：
> > r:=rand.New(rand.NewSource(time.Now().UnixNano()))
> > It will not happen.
> > Why does this happened? and how do I choose it between time.Now().Unix()
> or UnixNano()?
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golan...@googlegroups.com .
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/ce0671c5-f2df-42d9-a7af-bdef939e6d6d%40googlegroups.com.
>
>
> > For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/99351298-7843-4556-b422-8635eb96c936%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[go-nuts] rand seed using 'time.Now().Unix()' consequence mass common rand numbers, but using unixNano as the seed will not ,how to choose the rand's seed between 'time.Now().Unix()' and 'time.Now().U
I use: r:=rand.New(rand.NewSource(time.Now().Unix())) r..Intn(96) I got: and val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 39 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 rand val: 82 ... mass common sequential number . But if I change that： r:=rand.New(rand.NewSource(time.Now().UnixNano())) It will not happen. Why does this happened? and how do I choose it between time.Now().Unix() or UnixNano()? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ce0671c5-f2df-42d9-a7af-bdef939e6d6d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: how to decrease the HeapIdle of Memory to free memory to Linux?
I use 'debug.FreeOsMemory'. But it is not the certain. 在 2019年6月24日星期一 UTC+8上午10:59:06，Chou Yan写道： > > My server have 100G memory. > I limit my server 60G by cgroups. > But my app process use the momory more and more ! > even more than 50G. > my prof: > > # HeapIdle = 27377164288 > # HeapInuse = 21910396928 > > The Cgroup will kill the process when it use more than 60G > > But golang runtime gc will not release the memory to the os. > > It's bad. > > The Go garbage collector occasionally makes requests to the OS to release > unused memory. The OS may decide to not release the memory because the system > has plenty to spare, other some other reason . > How should this be solved? > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0bc4f86c-833c-4798-b06c-a1559edd6212%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: how to decrease the HeapIdle of Memory to free memory to Linux?
I am use 'debug.FreeOsMemory'. But it is not the certain. 在 2019年6月24日星期一 UTC+8上午10:59:06，Chou Yan写道： > > My server have 100G memory. > I limit my server 60G by cgroups. > But my app process use the momory more and more ! > even more than 50G. > my prof: > > # HeapIdle = 27377164288 > # HeapInuse = 21910396928 > > The Cgroup will kill the process when it use more than 60G > > But golang runtime gc will not release the memory to the os. > > It's bad. > > The Go garbage collector occasionally makes requests to the OS to release > unused memory. The OS may decide to not release the memory because the system > has plenty to spare, other some other reason . > How should this be solved? > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9556e37e-a59d-4249-a761-bfd026370dd5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: how to decrease the HeapIdle of Memory to free memory to Linux?
some of gc logs: gc 8639 @331692.423s 6%: 60+2541+2.3 ms clock, 1928+112357/40484/0+75 ms cpu, 22264->22299->4575 MB, 23164 MB goal, 64 P gc 8640 @331719.932s 6%: 21+2580+1.5 ms clock, 673+106151/41127/0+48 ms cpu, 21976->22064->4769 MB, 22878 MB goal, 64 P gc 8641 @331759.750s 6%: 31+2671+1.6 ms clock, 1019+115220/42609/0+54 ms cpu, 22904->22985->4785 MB, 23848 MB goal, 64 P gc 8642 @331805.499s 6%: 86+2647+2.0 ms clock, 2771+106447/42162/0+65 ms cpu, 22976->23081->4678 MB, 23926 MB goal, 64 P scvg2211: 0 MB released scvg2211: inuse: 23445, idle: 23386, sys: 46832, released: 20705, consumed: 26126 (MB) gc 8643 @331847.230s 6%: 43+2588+1.3 ms clock, 1399+112616/41295/0+44 ms cpu, 22463->22514->4605 MB, 23391 MB goal, 64 P gc 8644 @331880.496s 6%: 31+2528+4.0 ms clock, 1001+113331/40209/0+128 ms cpu, 22127->22159->4613 MB, 23025 MB goal, 64 P gc 8645 @331912.342s 6%: 35+2544+0.67 ms clock, 1145+109977/40616/0+21 ms cpu, 22151->22215->4685 MB, 23069 MB goal, 64 P scvg2212: 81 MB released scvg2212: inuse: 25495, idle: 21334, sys: 46830, released: 20773, consumed: 26056 (MB) gc 8646 @331959.029s 6%: 35+2602+9.3 ms clock, 1136+109565/40585/0+297 ms cpu, 22495->22569->4643 MB, 23425 MB goal, 64 P 在 2019年6月24日星期一 UTC+8上午10:59:06，Chou Yan写道： > > My server have 100G memory. > I limit my server 60G by cgroups. > But my app process use the momory more and more ! > even more than 50G. > my prof: > > # HeapIdle = 27377164288 > # HeapInuse = 21910396928 > > The Cgroup will kill the process when it use more than 60G > > But golang runtime gc will not release the memory to the os. > > It's bad. > > The Go garbage collector occasionally makes requests to the OS to release > unused memory. The OS may decide to not release the memory because the system > has plenty to spare, other some other reason . > How should this be solved? > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/68d6e955-5ea9-445b-99d6-ae87c569b434%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] how to decrease the HeapIdle of Memory to free memory to Linux?
My server have 100G memory. I limit my server 60G by cgroups. But my app process use the momory more and more ! even more than 50G. my prof: # HeapIdle = 27377164288 # HeapInuse = 21910396928 The Cgroup will kill the process when it use more than 60G But golang runtime gc will not release the memory to the os. It's bad. The Go garbage collector occasionally makes requests to the OS to release unused memory. The OS may decide to not release the memory because the system has plenty to spare, other some other reason . How should this be solved? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/14555dc7-2acd-48df-a206-8687a6697358%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] How many times gc stw ?
thx 在 2019年6月21日星期五 UTC+8下午12:38:50，Ian Lance Taylor写道： > > On Thu, Jun 20, 2019 at 7:22 PM Chou Yan > wrote: > > > > The describe 'release the physical memory' may not be rigorous. Because > gc remand the span,obj, or big obj to heap or central area, may not release > the physical memory. > > "When the next GC starts, it sweeps all not-yet-swept spans (if any)." > > maybe it is the sweeps termination phrase mean. > > BTW, I look the doc and look up three time of stw. > > sweep termination > > > > GC performs the mark phase. No objects may be > > > > //scanned until all Ps have enabled the write barrier, which is > > //accomplished using STW. > > > > mark termination phases. > > or "write barrier" accomplished during sweep termination? > > The write barrier is enabled as sweep termination completes. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3b7f3799-f44c-4640-908f-17e148b2fb6c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] How many times gc stw ?
The describe 'release the physical memory' may not be rigorous. Because gc remand the span,obj, or big obj to heap or central area, may not release the physical memory. "When the next GC starts, it sweeps all not-yet-swept spans (if any)." maybe it is the sweeps termination phrase mean. BTW, I look the doc and look up three time of stw. sweep termination GC performs the mark phase. No objects may be //scanned until all Ps have enabled the write barrier, which is //accomplished using STW. mark termination phases. or "write barrier" accomplished during sweep termination? 在 2019年6月21日星期五 UTC+8上午8:54:31，Ian Lance Taylor写道： > > On Thu, Jun 20, 2019 at 9:56 AM Chou Yan > wrote: > > > > I look up the doc for gctrace: > > https://godoc.org/runtime > > it show us: > > > > Currently, it is: > > gc # @#s #%: #+#+# ms clock, #+#/#/#+# ms cpu, #->#-># MB, # MB goal, # > P > > where the fields are as follows: > > gc #the GC number, incremented at each GC > > @#s time in seconds since program start > > #% percentage of time spent in GC since program start > > #+...+# wall-clock/CPU times for the phases of the GC > > #->#-># MB heap size at GC start, at GC end, and live heap > > # MB goal goal heap size > > # P number of processors used > > The phases are stop-the-world (STW) sweep termination, concurrent > > mark and scan, and STW mark termination. The CPU times > > for mark/scan are broken down in to assist time (GC performed in > > line with allocation), background GC time, and idle GC time. > > If the line ends with "(forced)", this GC was forced by a > > runtime.GC() call. > > > > > > I am very confused. Will it be swt in the cleanup phase? > > I'm sorry, I don't understand what you are asking. What is the cleanup > phase? > > > And how many times gc stw? mark start? concurrent mark? remark ? or > sweep ? > > See the long comment near the top of runtime/mgc.go. Currently each > cycle stops the world twice, very briefly, in the sweep termination > and mark termination phases. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7b0bedb9-3450-4502-bfa3-c290be1b51f8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [go-nuts] How many times gc stw ?
Sorry. Which I understand is that 'sweep termination = clean up = release the physical memory' The long comment is helpful. I got 'b. Sweep any unswept spans. There will only be unswept spans if this GC cycle was forced before the expected time.' Dose it means 'sweep = scan the root span', like java cms 'gc root scan'? not the release the physical memory? 在 2019年6月21日星期五 UTC+8上午8:54:31，Ian Lance Taylor写道： > > On Thu, Jun 20, 2019 at 9:56 AM Chou Yan > wrote: > > > > I look up the doc for gctrace: > > https://godoc.org/runtime > > it show us: > > > > Currently, it is: > > gc # @#s #%: #+#+# ms clock, #+#/#/#+# ms cpu, #->#-># MB, # MB goal, # > P > > where the fields are as follows: > > gc #the GC number, incremented at each GC > > @#s time in seconds since program start > > #% percentage of time spent in GC since program start > > #+...+# wall-clock/CPU times for the phases of the GC > > #->#-># MB heap size at GC start, at GC end, and live heap > > # MB goal goal heap size > > # P number of processors used > > The phases are stop-the-world (STW) sweep termination, concurrent > > mark and scan, and STW mark termination. The CPU times > > for mark/scan are broken down in to assist time (GC performed in > > line with allocation), background GC time, and idle GC time. > > If the line ends with "(forced)", this GC was forced by a > > runtime.GC() call. > > > > > > I am very confused. Will it be swt in the cleanup phase? > > I'm sorry, I don't understand what you are asking. What is the cleanup > phase? > > > And how many times gc stw? mark start? concurrent mark? remark ? or > sweep ? > > See the long comment near the top of runtime/mgc.go. Currently each > cycle stops the world twice, very briefly, in the sweep termination > and mark termination phases. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/58a54b06-271c-4c39-a891-2ec157c42ee9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] How many times gc stw ?
I look up the doc for gctrace: https://godoc.org/runtime it show us: Currently, it is: gc # @#s #%: #+#+# ms clock, #+#/#/#+# ms cpu, #->#-># MB, # MB goal, # P where the fields are as follows: gc #the GC number, incremented at each GC @#s time in seconds since program start #% percentage of time spent in GC since program start #+...+# wall-clock/CPU times for the phases of the GC #->#-># MB heap size at GC start, at GC end, and live heap # MB goal goal heap size # P number of processors used The phases are stop-the-world (STW) sweep termination, concurrent mark and scan, and STW mark termination. The CPU times for mark/scan are broken down in to assist time (GC performed in line with allocation), background GC time, and idle GC time. If the line ends with "(forced)", this GC was forced by a runtime.GC() call. I am very confused. Will it be swt in the cleanup phase? And how many times gc stw? mark start? concurrent mark? remark ? or sweep ? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0b509e5d-fe61-478b-9b88-f21a29d1f9cf%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
