Re: [Async-sig] Feedback, loop.load() function
I had an idea based on the Trios implementation. Trying to move the load method implementation out of the loop scope but avoiding the subclassing. My primary concern with the instrumentation was the performance impact. I've run some tests in one experimental branch, instrumenting 4 events for the _run_once function. The idea was to see how the performance degrades, despite there were not listeners/instruments attached. My gut feelings said that only the call to emit the event and the proper check to see if there were listeners will have an expensive performance cost. The numbers were pretty clear, the default Asyncio loses a 10% of performance. Degrading from the 30K coroutines per second to 27K in my laptop. Therefore, looks like we are facing a stopper that would need to be addressed first. How might Python address this situation? Basically implementing a new object type that will provide the same interface as a Signal. Many languages have their own implementation C# [1], Java [2] or they have libraries that support Signals or Events. Indeed Python has plenty of them implemented as independent modules, the most known are Django Signals [3] or Blinker [4] >From my understanding here the problem is the language by itself, having the interpreter behind the scenes. What I would propose here is a new Python object called Signal implemented as a new type in the CPython API that will allow the developers to implement the following snippet having the chance to speed up the code or at least don't be penalized when there are no observers listening from a specific event. class Foo: def __init__(): self._cb = [] def add_listener(self, cb): self._cp.append(cb) def _signal(self, msg): for cb in self._cb: cb(msg) def run(): self._signal("start") self._signal("stop") f = Foo() f.run() def print_signal(msg): print(msg) f.add_listener(print_signal) f.run() This previous snippet might be implemented using a new object type called signal, having the next snippet: class Foo: def __init__(): self.start = signal() self.stop = signal() def run(): self.start.send() self.stop.send() f = Foo() f.run() def start(): print(msg) def stop(): print(msg) f.start.connect(start) f.stop.connect(stop) f.run() This kind of pattern is spread in many frameworks [5], and they might get benefited for free thanks to this new type of object. I'm wondering if there were other attempts in the past to implement this kind of object in the CPython API. There is some resistance to modify or add superfluous code in the code if it can be implemented as a Python module? The simple fact to become a must for an internal feature might enough to think on that? Thoughts ? [1] http://docs.oracle.com/javase/tutorial/uiswing/events/index.html [2] https://msdn.microsoft.com/en-us/library/aa645739(v=vs.71).aspx [3] https://docs.djangoproject.com/en/1.11/topics/signals/ [4] http://pythonhosted.org/blinker/ [5] http://flask.pocoo.org/docs/0.12/signals/ On Tue, Aug 22, 2017 at 2:36 PM, Pau Freixeswrote: >> One approach would be to add a generic instrumentation API. Trio has >> something like this, that I think is expressive enough to let Pau >> implement their busyness checking as a library: >> >> https://trio.readthedocs.io/en/latest/reference-core.html#instrument-api >> >> This has several advantages over subclassing: multiple libraries can >> define their own instrumentation without interfering with each other, >> you don't have to redefine the instrumentation for every loop >> implementation, and you don't have to hook up the instrumentation when >> setting up the loop, e.g. you could just do something like: >> >> import toobusy >> with toobusy.Monitor(loop) as monitor: >> if monitor.toobusy(): >> ... > > It will help also other loops to meet the same contract making them > compatibles with already implemented instruments. Maybe the major > concern here is the performance penalty, do you have some numbers > about how negligible is have all of these signals available to be > used? > -- > --pau -- --pau ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] Feedback, loop.load() function
> One approach would be to add a generic instrumentation API. Trio has > something like this, that I think is expressive enough to let Pau > implement their busyness checking as a library: > > https://trio.readthedocs.io/en/latest/reference-core.html#instrument-api > > This has several advantages over subclassing: multiple libraries can > define their own instrumentation without interfering with each other, > you don't have to redefine the instrumentation for every loop > implementation, and you don't have to hook up the instrumentation when > setting up the loop, e.g. you could just do something like: > > import toobusy > with toobusy.Monitor(loop) as monitor: > if monitor.toobusy(): > ... It will help also other loops to meet the same contract making them compatibles with already implemented instruments. Maybe the major concern here is the performance penalty, do you have some numbers about how negligible is have all of these signals available to be used? -- --pau ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] Feedback, loop.load() function
Hi, > I personally don't think we need this in asyncio. While the function has a > relatively low overhead, it's still an overhead, it's a couple more syscalls > on each loop iteration, and it's a bit of a technical debt. > > The bar for adding new functionality to asyncio is very high, and I don't > see a good rationale for adding this function. Is it for debugging > purposes? Or for profiling live applications? If it's the latter, then > there are so many other things we want to see, and some of them are > protocol-specific. Let's try to evolve the rationale and put some extra links. The load of an Asyncio loop can be at some point easily inferred using the sleeping time vs the overall time, this information brings us to understand how to saturate is the loop with a metric that informs you how many CPU resources are being used, or most important how many CPU resources left. How helpful can be this method? In our organization, e use back-pressure at the application layer of our REST microservices architecture. It allows us to prevent overloading the services. Once the back pressures kicks in we can scale horizontally our services to cope the current load. This is already implemented for other languages and we are currently working on how to implement it with the aiohttp(asyncio) stack. For more info about this technique these articles [1] [2] We are not the first ones running microservices at scale, and this pattern has been implemented by other organizations. I would like to mention the Google case [2]. From that link I would like to bold the following paragraph: """ A better solution is to measure capacity directly in available resources. For example, you may have a total of 500 CPU cores and 1 TB of memory reserved for a given service in a given datacenter. Naturally, it works much better to use those numbers directly to model a datacenter's capacity. We often speak about the cost of a request to refer to a normalized measure of how much CPU time it has consumed (over different CPU architectures, with consideration of performance differences). In a majority of cases (although certainly not in all), we've found that simply using CPU consumption as the signal for provisioning works well """ >From my understanding, the comment is pretty aligned with the implementation proposal for the Asyncio loop Having, as a result, a way to measure if there are enough resources to cope the ongoing metric. [1] https://dzone.com/articles/applying-back-pressure-when [2] http://engineering.voxer.com/2013/09/16/backpressure-in-nodejs/ [3] https://landing.google.com/sre/book/chapters/handling-overload.html > If we want to add some tracing/profiling functions there should be a way to > disable them, otherwise the performance of event loops like uvloop will > degrade, and I'm not sure that all of its users want to pay a price for > something they won't ever be using. All of this just adds to the > complexity. The goal will be, have an implementation without impact performance for real applications. I'm still not sure if this is reachable with the uvloop, I would like to start working on this as soon as possible, having the proper numbers and the possibilities to implement this in libuv will help to get the proper answer. If at last, there is no way to make it negligible, then I would agree that is needed a way to switch off or switch on. ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] Feedback, loop.load() function
> It looks like your "load average" is computing something very different than > the traditional Unix "load average". If I'm reading right, yours is a > measure of what percentage of the time the loop spent sleeping waiting for > I/O, taken over the last 60 ticks of a 1 second timer (so generally slightly > longer than 60 seconds). The traditional Unix load average is an > exponentially weighted moving average of the length of the run queue. The implementation proposed wants to expose the load of the loop. Having a direct metric that comes from the loop instead of using an external metric such as CPU, load average u others. Yes, the load average uses a decay function based on the length of the run queue for those processes that are using or waiting for a CPU, this gives us extra information about how overloaded is our system. If you compare it with the CPU load. In the case presented, the load of the loop is something equivalent with the load of the CPU and it does not have the ability to inform you about how much overloaded is your loop once reached the 100%. > > Is one of those definitions better for your goal of detecting when to shed > load? I don't know. But calling them the same thing is pretty confusing :-). > The Unix version also has the nice property that it can actually go above 1; > yours doesn't distinguish between a service whose load is at exactly 100% of > capacity and barely keeping up, versus one that's at 200% of capacity and > melting down. But for load shedding maybe you always want your tripwire to > be below that anyway. Well, I partially disagree with this. The load definition has its equivalent in computing with other metrics that have a close range, such as the CPU one. I've never had the intention to align the load of the loop with the load average, I've just used the concept as an example of the metric that might be used to check how loaded is your system. > > More broadly we might ask what's the best possible metric for this purpose – > how do we judge? A nice thing about the JavaScript library you mention is > that scheduling delay is a real thing that directly impacts the quality of > service – it's more of an "end to end" measure in a sense. Of course, if you > really want an end to end measure you can do things like instrument your > actual logic, see how fast you're replying to HTTP requests or whatever, > which is even more valid but creates complications because some requests are > supposed to take longer than others, etc. I don't know which design goals > are important for real operations. Here the key for me, something where I should have based my rationale. How good is the way presented to measure a load of your asynchronous system compared with the toobusy one? what can we achieve with this metric? I will work on that as the base of my rationale for the change proposed. Then, once if the rationale is accepted the implementation is peanuts :) -- --pau ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] Feedback, loop.load() function
It looks like your "load average" is computing something very different than the traditional Unix "load average". If I'm reading right, yours is a measure of what percentage of the time the loop spent sleeping waiting for I/O, taken over the last 60 ticks of a 1 second timer (so generally slightly longer than 60 seconds). The traditional Unix load average is an exponentially weighted moving average of the length of the run queue. Is one of those definitions better for your goal of detecting when to shed load? I don't know. But calling them the same thing is pretty confusing :-). The Unix version also has the nice property that it can actually go above 1; yours doesn't distinguish between a service whose load is at exactly 100% of capacity and barely keeping up, versus one that's at 200% of capacity and melting down. But for load shedding maybe you always want your tripwire to be below that anyway. More broadly we might ask what's the best possible metric for this purpose – how do we judge? A nice thing about the JavaScript library you mention is that scheduling delay is a real thing that directly impacts quality of service – it's more of an "end to end" measure in a sense. Of course, if you really want an end to end measure you can do things like instrument your actual logic, see how fast you're replying to http requests or whatever, which is even more valid but creates complications because some requests are supposed to take longer than others, etc. I don't know which design goals are important for real operations. On Aug 6, 2017 3:57 PM, "Pau Freixes"wrote: > Hi guys, > > I would appreciate any feedback about the idea of implementing a new > load function to ask about how saturated is your reactor. > > I have a proof of concept [1] of how the load function might be > implemented in the Asyncio python loop. > > The idea is to provide a method that can be used to ask about the load > of the reactor in a specific time, this implementation returns the > load taking into account the last 60 seconds but it can easily return > the 5m and 15minutes ones u others. > > This method can help services built on to of Asyncio to implement back > pressure mechanisms that take into account a metric coming from the > loop, instead of inferring the load using other metrics provided by > external agents such as the CPU, load average u others. > > Nowadays exists some alternatives for other languages that address > this situation using the lag of a scheduler callback, produced by > saturated reactors. The most known implementation is toobusy [2] a > nodejs implementation. > > IMHO the solution provided by tobusy has a strong dependency with the > hardware needing to tune the maximum lag allowed in terms of > milliseconds [3]. in the POF presented the user can use an exact value > meaning the percentage of the load, perhaps 0.9 > > Any comment would be appreciated. > > [1] https://github.com/pfreixes/cpython/commit/ > 5fef3cae043abd62165ce40b181286e18f5fb19c > [2] https://www.npmjs.com/package/toobusy > [3] https://www.npmjs.com/package/toobusy#tunable-parameters > -- > --pau > ___ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ > ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Re: [Async-sig] Feedback, loop.load() function
On Sun, Aug 6, 2017 at 3:57 PM, Pau Freixeswrote: > Hi guys, > > I would appreciate any feedback about the idea of implementing a new > load function to ask about how saturated is your reactor. Hi, Would it be possible for you to rephrase what you've done in terms of asyncio terminology? From what I can tell, "reactor" isn't a term used in the asyncio docs or code base. It might also improve the readability of your asyncio patch to use asyncio terminology in the code comments, doc strings, etc. --Chris > > I have a proof of concept [1] of how the load function might be > implemented in the Asyncio python loop. > > The idea is to provide a method that can be used to ask about the load > of the reactor in a specific time, this implementation returns the > load taking into account the last 60 seconds but it can easily return > the 5m and 15minutes ones u others. > > This method can help services built on to of Asyncio to implement back > pressure mechanisms that take into account a metric coming from the > loop, instead of inferring the load using other metrics provided by > external agents such as the CPU, load average u others. > > Nowadays exists some alternatives for other languages that address > this situation using the lag of a scheduler callback, produced by > saturated reactors. The most known implementation is toobusy [2] a > nodejs implementation. > > IMHO the solution provided by tobusy has a strong dependency with the > hardware needing to tune the maximum lag allowed in terms of > milliseconds [3]. in the POF presented the user can use an exact value > meaning the percentage of the load, perhaps 0.9 > > Any comment would be appreciated. > > [1] > https://github.com/pfreixes/cpython/commit/5fef3cae043abd62165ce40b181286e18f5fb19c > [2] https://www.npmjs.com/package/toobusy > [3] https://www.npmjs.com/package/toobusy#tunable-parameters > -- > --pau > ___ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
[Async-sig] Feedback, loop.load() function
Hi guys, I would appreciate any feedback about the idea of implementing a new load function to ask about how saturated is your reactor. I have a proof of concept [1] of how the load function might be implemented in the Asyncio python loop. The idea is to provide a method that can be used to ask about the load of the reactor in a specific time, this implementation returns the load taking into account the last 60 seconds but it can easily return the 5m and 15minutes ones u others. This method can help services built on to of Asyncio to implement back pressure mechanisms that take into account a metric coming from the loop, instead of inferring the load using other metrics provided by external agents such as the CPU, load average u others. Nowadays exists some alternatives for other languages that address this situation using the lag of a scheduler callback, produced by saturated reactors. The most known implementation is toobusy [2] a nodejs implementation. IMHO the solution provided by tobusy has a strong dependency with the hardware needing to tune the maximum lag allowed in terms of milliseconds [3]. in the POF presented the user can use an exact value meaning the percentage of the load, perhaps 0.9 Any comment would be appreciated. [1] https://github.com/pfreixes/cpython/commit/5fef3cae043abd62165ce40b181286e18f5fb19c [2] https://www.npmjs.com/package/toobusy [3] https://www.npmjs.com/package/toobusy#tunable-parameters -- --pau ___ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/