On Wed, Apr 4, 2018 at 5:27 PM, Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info> wrote: > So, I'm, still trying to wrap my brain around async processing, and I > started reading this tutorial: > > http://stackabuse.com/python-async-await-tutorial/ > > and the very first paragraph broke my brain. > > "Asynchronous programming has been gaining a lot of traction in the past > few years, and for good reason. Although it can be more difficult than > the traditional linear style, it is also much more efficient." > > I can agree with the first part of the first sentence (gaining a lot of > traction), and the first part of the second sentence (more difficult than > the traditional style), but the second part? Asynchronous processing is > *more efficient*?
The only way it could be "more efficient" is if you assume that "sequential processing" == "well, let's parallelize it by going for multiple processes, each independently started". This is an assumption, but it's not wholly unreasonable - if you write your code in a purely sequential way (say, youtube-dl) and want to do more than one of these at a time, the easiest way is to pop open another terminal and fire up another copy of the program, with an entire copy of everything. But that isn't inherent to the design of sequential processing. It's a cost of a naive form of parallelism, one that could be alleviated by threading too. IMO asyncio should be compared to threads, not to sequential processing. *First* imagine reworking your code to use threads; then consider cutting out the actual OS-managed threads and using a cooperative single-threaded system instead. You depend on application-level scheduling, which means that gethostbyname() can utterly break you if you aren't careful, and you lose about thirty or forty years of collective experience in developing operating systems with intelligent schedulers, but in return, you gain the ability to manage a hundred thousand concurrent tasks in a single process. You also lose the ability to context-swap in the middle of a logical operation, but in return, you lose the ability to context-swap in the middle of a logical operation. For people who've grown up with actual real threads, this latter is not an advantage; for people who've grown up casually mutating global state, that's huge. IN THEORY, it would be possible to create a lock-free implementation of Python that is incapable of using multiple threads, but still able to use asyncio. In practice, since CPython has a highly efficient locking system (the GIL), the advantage is unlikely to be compelling. So, no, generally async processing will NOT be more efficient than sequential processing. It has other advantages, but not that. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
