On 4/4/18 3:27 AM, Steven D'Aprano wrote: > So, I'm, still trying to wrap my brain around async processing, and I > started reading this tutorial: > > http://stackabuse.com/python-async-await-tutorial/ > > and the very first paragraph broke my brain. > > "Asynchronous programming has been gaining a lot of traction in the past > few years, and for good reason. Although it can be more difficult than > the traditional linear style, it is also much more efficient." > > I can agree with the first part of the first sentence (gaining a lot of > traction), and the first part of the second sentence (more difficult than > the traditional style), but the second part? Asynchronous processing is > *more efficient*? > > I'm no expert, but it seems to me that this has surely got to be crazy > talk. Whatever task you're doing, processing it asynchronously doesn't > reduce the amount of work. For example, if you want to download ten > files, you still have to download all ten files, and they're not any > smaller or the network connection any faster because you're using async. > > On the contrary, surely it is *less efficient*: there's the amount of > work you need to do, the *useful* work (downloading those files) PLUS a > bunch of book-keeping work (most, but not all, done for you in the async > framework or language) needed to swap jobs. > > I can see that the advantage of async is to *increase responsiveness*. > You aren't waiting a long time for each file to download (or the page to > render, or whatever task it is that you are doing) before being able to > start the next one. Quoting the article: > > "With asynchronous programming, you allow your code to handle other tasks > while waiting for these other resources to respond." > > Yes, this exactly. And if you're writing a web app, or certain kinds of > desktop apps, these seems sensible. I don't want my browser to come to a > complete halt just because some resource is taking a few seconds to > respond. > > But honestly, from everything I've seen, outside of browser apps (and I'm > not even sure about them), async processing looks to me like the wrong > solution to, well, everything. It combines the worst of sequential and > parallel processing with the benefits of neither. It's like cooperative > multi-tasking, only without the actual multi-tasking and even less > convenience. Its harder to use than threading or multiprocessing, but > without their ability to run in parallel. It lacks the isolation of > multiprocessing, and the preemptive task swapping of threads/processes. > > Its as hard to wrap your brain around as parallel processing in general, > but with even worse performance than sequential processing. > > Am I totally wrong? > > Asynchronous processing will use a bit more of some processing resources to handle the multi-processing, but it can be more efficient at fully using many of the resources that are available.
Take your file download example. When you are downloading a file, your processor sends out a request for a chuck of the data, then waits for the response. The computer on the other end gets that requests, sends a packet, and then waits. You get that packet, send and acknowledgement, and then wait, the other computer gets that acknowledgement and sends more data, and then waits, and so on. Even if your pipe to the Internet is the limiting factor, there is a fair amount of dead time in this operation, so starting another download to fill more of that pipe can decrease the total time to get all the data downloaded. Yes, if your single path of execution can fully use the critical resources, then adding asynchronous processing won't help, but rarely does it. Very few large machines today a single-threaded, but most have multiple cores and often even those cores have the ability to handle multiple threads at once. Thus there normally are extra resources that the asynchronous processing can better use up, so even processor usage can be improved in many cases. In most cases I am familiar with, the type of asynchronous programming you are talking about is to move I/O bound operations into a second execution path, allowing your main path to focus on keeping the CPU busy. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
