On Sat, May 17, 2014 at 6:36 PM, Mathias Kærlev <[email protected]> wrote:
> I wouldn't want asyncio to default to calling timeBeginPeriod either, as > it would hurt battery life and potentially affect other applications, which > is not an ideal default behavior. > However, it would be great if there was an API to request a certain time > resolution from the eventloop. > This looks like it would be Windows specific functionality though. I'd prefer to do it somewhat differently. > This would be useful to e.g. my LoopingCall class, where I would be able > to find the highest call frequency and feed that to the eventloop. > Generally, if the application *knows* that it will need a high-resolution > timer, it would be able to request one. > Only on Windows, AFAIK. > I've changed the previous code to add new set_clock and > set_clock_resolution methods: > https://github.com/matpow2/cuwo/blob/py3/cuwo/win32.py > > To show what my original example would look like with the new methods: > http://bpaste.net/show/pcrIZAV3R7QaefJGMtRw/ > > If setting the clock resolution was to be added to asyncio, it would > obviously be a no-op on platforms where there are no timer resolution > issues. > In any case, I would definitely consider adding a set_clock method (so if > someone wants to use e.g. QPC, they can). > I propose to leave the resolution management to the application's Windows-specific code. You can write code that sets the resolution to what you want and then pass both it and your timer code to the event loop. One problem with using timeGetTime is that we don't know the initial > system-wide timer resolution. > We can be conservative and guess that it has not been changed from > startup, in which case we would get a resolution of 15.6ms. > However, a lot of applications (Chrome, GUI applications, Flash, games, > etc.) change the global timer resolution to 1ms, so in reality, it may > already be a lot more precise than we expect. > There is also the possibility of using a period which is higher than the > current system-wide resolution, in which case we would also be getting a > more precise timer. > From what I can see, there is no way to query the current resolution > without using NtQueryTimerResolution, which is an undocumented MS function. > All this just adds to my desire to make changing the system resolution the app's problem, and just provide set_clock(time_func, clock_resolution) as a loop method. (BTW, the resolution is only used in one place, and even that place is dubious -- it's mostly because Victor doesn't like to busy-wait if the sleep time is shorter than requested. Perhaps we can live without it after all.) > However, we can still look at timeGetTime deltas in the time() method, and > if we find a delta that is lower than the current clock resolution, we use > that as our resolution. > Again, I think that ought to be made the app's problem. > The last alternative would be to set _clock_resolution to the highest > possible resolution, i.e. 1ms, even if the actual resolution may be lower. > This way, we can ensure the scheduler never tries to fire a call which is > in the future. This is the approach I have taken in my code. > With some testing, it appears that my system in fact uses a timer > resolution of 1 ms most of the time (since a lot of applications set it by > default), so to me, this seems like the best approach. > There are very few side effects of setting _clock_resolution lower than the actual clock resolution. > At the very least, timeGetTime could be used as the default time() method > on Windows. > Unfortunately, timeBeginPeriod does not affect GetTickCount (i.e. > time.monotonic), as GetTickCount will try to emulate the 15.6ms resolution > even if the system-wide resolution has been changed. > Hm. If you want to argue over the implementation of time.monotonic(), I'd prefer it if you did that on python-dev, as it is not specific to asyncio. But it's another reason why I want the period management out of asyncio. :-) > Hopefully my input has been valuable to some extent! > I suggest that you file a tracker issue and propose a patch just to change the default time() function on Windows. And perhaps a separate one to lobby for a set_clock() API. > It would be great if asyncio becomes the go-to eventloop for all Python > things. > Yes, that's my hope too! > This is probably why I'm very keen on having a good default > implementation. For all platforms. ;-) > > Cheers, > Mathias > > Den lørdag den 17. maj 2014 23.29.30 UTC+2 skrev Guido van Rossum: >> >> Sorry, saw this after sending my reply. >> >> Looking at your code and the MS docs for timeBeginPeriod() I can see why >> we didn't choose this as our default asyncio clock or as the implementation >> of time.monotonic() -- the docs warn that the requested period can affect >> the performance of the entire system. Your implementation also selfishly >> sets exactly the period that you are interested in. >> >> But it seems that using timeBeginPeriod() is pretty independent from >> timeGetTime() -- perhaps it even affects time.monotonic()? If not, perhaps >> we can leave calling timeBeginPeriod() up to the app, but still override >> the time() method? We would then have to decide whether we want to just >> override time() and _clock_resolution in the Windows-specific Loop classes >> (_WindowsSelectorEventLoop and ProactorEventLoop) or whether we want to >> define an API to change the time implementation more generally. >> >> >> On Sat, May 17, 2014 at 6:54 AM, Mathias Kærlev <[email protected]> wrote: >> >>> I experimented with offsetting the timer using the timeout passed to the >>> selector, and that did seem to fix it for my case. >>> However, I found it difficult to see if that would be a general solution >>> to the problem, especially when the selector would actually return >>> something on each iteration (in which case we couldn't use the timeout). >>> >>> I ended up just subclassing the eventloops from windows_events.py: >>> https://github.com/matpow2/cuwo/blob/py3/cuwo/win32.py >>> >>> It uses timeGetTime and timeGetPeriod/timeEndPeriod with a timer >>> resolution of 10ms, which should be enough for the majority of cases. >>> According to the documentation, timeGetTime should work exactly like >>> GetTickCount, except for the fact that you can change the timer resolution. >>> >>> I still consider this to be a hack though. It would be nice if something >>> was done about this eventually. >>> >>> Thanks again, >>> Mathias >>> >>> Den lørdag den 17. maj 2014 11.35.08 UTC+2 skrev Mathias Kærlev: >>> >>>> That seems like a cop-out to me. >>>> >>>> I get that GetTickCount64 does not have high precision, but you could >>>> consider using a timeGetTime/GetTickCount64 hybrid (like in GLib) for >>>> asyncio. >>>> Otherwise, with these scheduling issues, the default eventloop >>>> implementation is practically unusable for games on Windows, which I think >>>> is a shame. >>>> Obviously, I agree that a precise, stable (i.e. not QPC) monotonic >>>> clock would be the only proper timer solution, but ultimately, it should >>>> not come at the expense of usability. >>>> timeGetTime/timeBeginPeriod appears to be both high-res, stable and >>>> precise, so I was wondering what the rationale is for not using these APIs? >>>> >>>> In asyncio, would an API to modify the timer implementation be >>>> applicable? >>>> Just changing loop.time will not work, as you would also have to change >>>> loop._clock_resolution, so now we're in the business of monkey-patching >>>> eventloop and implementation-specific details. >>>> Generally, having to add hacks for Windows is not very attractive when >>>> e.g. Twisted works out of the box. >>>> >>>> About the original example I posted, it seems like the stability >>>> problems only occur if the 100 FPS loop is added. >>>> Perhaps the issue could be rectified if the eventloop was more clever >>>> with its scheduling? >>>> libuv also seems to use GetQueuedCompletionStatus timeouts to adjust >>>> the timer, so I think there may be room for improvement with the timer in >>>> the Windows eventloop. >>>> I will do some more testing, and see if I can come up with a solution >>>> without changing the timer implementation. >>>> >>>> Thanks, >>>> Mathias >>>> >>>> Den lørdag den 17. maj 2014 09.52.54 UTC+2 skrev Victor Stinner: >>>>> >>>>> Hi, >>>>> >>>>> I had very very long discussion about timer granularity and choose the >>>>> "right" clock in Python and in asyncio. See for example the PEP 418 to >>>>> learn why time.monotonic() has such very low precision. >>>>> >>>>> I don't want to change the default clock of asyncio or change >>>>> time.monotonic(). As Guido wrote, it's easy to change the clock of an >>>>> event >>>>> loop. Try loop.time = myclock. >>>>> >>>>> Victor >>>>> >>>>> Le samedi 17 mai 2014, Mathias Kærlev <[email protected]> a écrit : >>>>> >>>>>> So after some quick tests, it's apparent that switching out >>>>>> time.monotonic() with either time.clock() or time.time() completely fixes >>>>>> the issue (so the framerate is always between 49~50). >>>>>> For my application, I can probably subclass the eventloop and switch >>>>>> out the timer implementation, but ideally, this should be fixed upstream >>>>>> somehow. >>>>>> I never had this issue with Twisted, even though I think Twisted uses >>>>>> time.time() with some heuristics to detect time drift and similar issues. >>>>>> Personally, I've always found a combination of timeGetTime() and >>>>>> timeBeginPeriod(1) to yield a very stable timer on Windows (especially >>>>>> for >>>>>> games), but perhaps you have greater insight into the issue. >>>>>> >>>>>> Thanks for the help! Is there any chance this will be fixed in an >>>>>> upcoming asyncio release? >>>>>> >>>>>> Mathias >>>>>> >>>>>> Den lørdag den 17. maj 2014 06.31.18 UTC+2 skrev Guido van Rossum: >>>>>>> >>>>>>> This could well be timer granularity on Windows; this has been a big >>>>>>> problem for the unittests too. Could you try with a different timer? It >>>>>>> should be easy to change the timer function. >>>>>>> >>>>>>> On Friday, May 16, 2014, Mathias Kærlev <[email protected]> wrote: >>>>>>> >>>>>>>> Hi everyone >>>>>>>> >>>>>>>> I'm currently porting some code from Python 2.7 with Twisted to >>>>>>>> Python 3.4 with asyncio. So far, it's been a pleasant experience. >>>>>>>> However, my application in question is a game server which needs to >>>>>>>> run a 50FPS update loop to simulate the game world. >>>>>>>> When running multiple update loops, I start to get some very >>>>>>>> unstable behavior, where the actual delay between a call_later call >>>>>>>> and the >>>>>>>> time it fires wobbles between the specified delay and half the delay. >>>>>>>> >>>>>>>> Please see the following example: http://bpaste.net/sho >>>>>>>> w/Cxn0jxqufnvNk7qL8MnJ/ >>>>>>>> In the example, I run two update loops: one on 50FPS, and one on >>>>>>>> 100FPS. I only monitor the update loop running at 50 FPS. >>>>>>>> Yet, on Windows 7, I get the following output: http://bpaste.net/ >>>>>>>> show/XKp4LtKpEsXWxacZjQlc/ >>>>>>>> The framerate is very wobbly, going between ~100 FPS and ~50 FPS >>>>>>>> (the 100 FPS LoopingCall is not being logged, of course). >>>>>>>> >>>>>>>> Is this a bug in my LoopingCall class, or is it a problem with the >>>>>>>> eventloop timer granularity? >>>>>>>> >>>>>>>> This is using the asyncio release found in stdlib on Python 3.4, by >>>>>>>> the way. >>>>>>>> >>>>>>>> Thanks for the help! >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> --Guido van Rossum (on iPad) >>>>>>> >>>>>> >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> > -- --Guido van Rossum (python.org/~guido)
