On Fri, 19 Jun 2020 20:30:03 +0100 Barry Scott <ba...@barrys-emacs.org> wrote: > > > I know very little about how this works except a vague rule of thumb > > that in the 21st century memory locality is king. If you want code to be > > fast, keep it close together, not spread out. > > Remember that the caches are large these days and can hold many cache lines. > So the idea of local really means is in the cache.
There's no such thing as "the cache". There are usually several levels of cache. L1 cache is closest to the CPU and is very fast (with latencies of access 3-4 cycle, and very high bandwidths). L2 cache is a bit further, still relatively fast, and often private per-core (typical latencies of access around 10-20 cycles). L3 cache is actually often relatively slow and shared between several or all cores (latencies vary quite a bit between CPU models, and can even be variable depending on exact data placement, but typical numbers are around 30-50 cycles). (3 levels of cache is the most used granularity these days, but you may find CPUs without a L3, or even with a L3 and a L4) So if you hit in your L3 cache but miss in your L2 cache, you're already taking a large toll during which your CPU may be stalled, waiting for data. Of course, accessing main memory is much slower yet (with latencies in the several hundreds of cycles, depending on memory speed and CPU frequency). Now, the typical L2 cache size is around 512 kiB, and the footprint of the average Python application is certainly much larger than that. This is mitigated by the fact that modern CPUs have sophisticated prefetching techniques that try to *predict* the memory ranges that will be accessed by future instructions, so as to read them from memory into (L1 or L2) cache in advance. However, this only works if those addresses are predictable at all. For example, accesses in a hash table (such as a dict) are by construction rarely predictable. Regards Antoine. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LOWPYXHZLV42IJRW33VLNXN2DB7MBRJL/ Code of Conduct: http://python.org/psf/codeofconduct/