Re: [go-nuts] performance optimization

[Jesper Louis Andersen]

On Wed, Jan 16, 2019 at 11:55 AM Wojciech S. Czarnecki 
wrote:


> You can't expect a million interrupts per second and host OS running
> simultaneously.
> (1us gives some 4k instructions inbetween on recent 3GHz cpu core)
>
>
And to add: if you do anything with memory, it is usually considerably less
than that. DRAM access in particular can be shaving many of those 4k
instructions off very easily.

If possible, it is usually better to fill out work for the future when you
are on-CPU, because you don't necessarily know when you come back to the
CPU core. This hides the latency.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[Wojciech S. Czarnecki]

On Mon, 14 Jan 2019 21:34:34 +0100
Tamás Király  wrote:

> I'm simulating the internal clock of an embedded microcontroller...

You can't expect a million interrupts per second and host OS running 
simultaneously.
(1us gives some 4k instructions inbetween on recent 3GHz cpu core) 

Usual way for emulating hw state machines maintaining RT accuracy:

loop:

1. make a batch "n" of emulated hw instructions that span observable
i/o ops or shared state commits; then

2. count expected 'hw cycles' above batch would need on real hw,
duration "d" of real time it would take then "t" point of time batch 
results are due; then

3. set "hw progress" timer P to "t" point

4. "execute" batch "n"
(at full host speed of course, hw logic/instructions are
transpiled to host instruction set)

5. sleep (give cpu to host OS) until P fires

6. commit state (results of batch) "n"
n += 1; 
goto loop

Hope this helps,

-- 
Wojciech S. Czarnecki
 << ^oo^ >> OHIR-RIPE

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[Andrei Avram]

Not sure what your need is, but maybe you need time.Since instead of saving the 
start time and substracting it from time.Now.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[Tamás Király]

thanks for the suggestions and notes, i threw out the loop, save the start
timestamp and call time.Now()-start when i need the value which is in the
second range.

- Tamás

David Anderson  ezt írta (időpont: 2019. jan. 14., H,
21:54):

> Note that even for a state of the art CPU, there's only a few thousand
> instructions in a microsecond (1000 cycles per microsecond per GHz). At
> that kind of performance, anything you do that involves the OS or
> context-switching will make it impossible to hit your target refresh rate.
> The closest you can get is to lock the goroutine to an OS thread and do
> busy waiting (and that still won't save you from OS preemption, unless
> you're doing something explicit about that too).
>
> - Dave
>
> On Mon, Jan 14, 2019 at 12:35 PM Tamás Király  wrote:
>
>> I'm simulating the internal clock of an embedded microcontroller... And
>> this sentence (and you!) gave the idea to use time.Now().
>>
>> Thanks!
>> Tamas
>>
>> 9. jan. 14., H 21:12 dátummal Matt Ho  ezt írta:
>>
>>> Can you describe what task it is that needs to be updated every
>>> microsecond?  It seems like there should be a better to resolve the
>>> underlying need.
>>>
>>> M
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[David Anderson]

Note that even for a state of the art CPU, there's only a few thousand
instructions in a microsecond (1000 cycles per microsecond per GHz). At
that kind of performance, anything you do that involves the OS or
context-switching will make it impossible to hit your target refresh rate.
The closest you can get is to lock the goroutine to an OS thread and do
busy waiting (and that still won't save you from OS preemption, unless
you're doing something explicit about that too).

- Dave

On Mon, Jan 14, 2019 at 12:35 PM Tamás Király  wrote:

> I'm simulating the internal clock of an embedded microcontroller... And
> this sentence (and you!) gave the idea to use time.Now().
>
> Thanks!
> Tamas
>
> 9. jan. 14., H 21:12 dátummal Matt Ho  ezt írta:
>
>> Can you describe what task it is that needs to be updated every
>> microsecond?  It seems like there should be a better to resolve the
>> underlying need.
>>
>> M
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[robert engels]

If you are going to use time.Now() and a ‘change loop’, you should probably 
lock the thread to the Go routine to avoid an scheduler thrashing.

> On Jan 14, 2019, at 2:34 PM, Tamás Király  wrote:
> 
> I'm simulating the internal clock of an embedded microcontroller... And this 
> sentence (and you!) gave the idea to use time.Now().
> 
> Thanks!
> Tamas
> 
> 9. jan. 14., H 21:12 dátummal Matt Ho  > ezt írta:
> Can you describe what task it is that needs to be updated every microsecond?  
> It seems like there should be a better to resolve the underlying need.
> 
> M
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[Tamás Király]

I'm simulating the internal clock of an embedded microcontroller... And
this sentence (and you!) gave the idea to use time.Now().

Thanks!
Tamas

9. jan. 14., H 21:12 dátummal Matt Ho  ezt írta:

> Can you describe what task it is that needs to be updated every
> microsecond?  It seems like there should be a better to resolve the
> underlying need.
>
> M
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[Matt Ho]

Can you describe what task it is that needs to be updated every 
microsecond?  It seems like there should be a better to resolve the 
underlying need.

M

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[robert engels]

I updated the github issue with some additional performance comparisons and 
notes.

I think there are some pretty trivial solutions for high frequency pollers, at 
least those using time.Sleep()

> On Dec 19, 2018, at 10:03 PM, Robert Engels  wrote:
> 
> Then something is broken on the implementation. 10x overhead does not seem 
> realistic. 
> 
>> On Dec 19, 2018, at 9:50 PM, Ian Lance Taylor  wrote:
>> 
>>> On Wed, Dec 19, 2018 at 6:52 PM robert engels  wrote:
>>> 
>>> I don’t think the analysis is correct in the issue cited. No one mentions 
>>> just using usleep() as the system call like a C program would, at least for 
>>> time.Sleep(), and block the calling thread.
>> 
>> That doesn't scale to large numbers of goroutines.
>> 
>> 
>>> For the timer based select, it is more difficult, and it seems the timerfd 
>>> is the best solution - but I would instead proposed a timer thread - that 
>>> manages all timers, and interacts with the scheduler as an external event - 
>>> that is it wakes up based on the earliest expiration timer.
>> 
>> That's more or less how the runtime works today, although it doesn't
>> use timerfd and there are up to 64 timer threads.
>> 
>> Ian
>> 
>> 
>>> On Dec 19, 2018, at 7:58 PM, djad...@gmail.com wrote:
>>> 
>>> see https://github.com/golang/go/issues/27707
>>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[robert engels]

Isn’t doesn’t even need to be that complex, the following uses 1.25 cores at 
100% on my machine:

package main

import "time"

func main() {
   counter := 0
   for {
  counter++
  time.Sleep(time.Microsecond)
   }
}

even if you change it to 100 usecs, which is only 10,000 times a second, it 
uses 33% of a core.

For comparison, a similar C program using usleep:

   with a 1 usec delay, uses 50% of a core,
   with 100 usecs, 5% of a core

The interesting thing is that at 1 usec delay, the idle wake-ups are only 100k 
per sec, which means even the C program requires about 10 usec to perform the 
context switch.

So if the OP is only using 33% that’s pretty good.

I am testing under OSX. So, it would appear that the Go scheduler could be 
improved - 5% vs 33% is a pretty big performance hit.


> On Dec 19, 2018, at 3:44 PM, Ian Lance Taylor  wrote:
> 
> On Wed, Dec 19, 2018 at 1:29 PM Tamás Király  wrote:
>> 
>> my task is to update a value every microsecond.
>> i did an it with an infinite loop with time.Sleep like this
>> 
>> https://play.golang.org/p/JiN3_5KiGOO
>> 
>> this causes me about 50-60% of CPU usage on my machine but i made another 
>> version with time.After:
>> 
>> https://play.golang.org/p/PQHsNq261qZ
>> 
>> this is 30%-40% of CPU usage. Can anyone optimize more this code so it does 
>> not use a hilariously lot of CPU?
> 
> Have you tried using time.Tick or time.NewTicker?
> 
> Ian
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] performance optimization

[Ian Lance Taylor]

On Wed, Dec 19, 2018 at 1:29 PM Tamás Király  wrote:
>
> my task is to update a value every microsecond.
> i did an it with an infinite loop with time.Sleep like this
>
> https://play.golang.org/p/JiN3_5KiGOO
>
> this causes me about 50-60% of CPU usage on my machine but i made another 
> version with time.After:
>
> https://play.golang.org/p/PQHsNq261qZ
>
> this is 30%-40% of CPU usage. Can anyone optimize more this code so it does 
> not use a hilariously lot of CPU?

Have you tried using time.Tick or time.NewTicker?

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.