On Fri, 19 Jun 2020 20:30:03 +0100
Barry Scott <ba...@barrys-emacs.org> wrote:
> 
> > I know very little about how this works except a vague rule of thumb 
> > that in the 21st century memory locality is king. If you want code to be 
> > fast, keep it close together, not spread out.  
> 
> Remember that the caches are large these days and can hold many cache lines.
> So the idea of local really means is in the cache.

There's no such thing as "the cache".  There are usually several levels
of cache.  L1 cache is closest to the CPU and is very fast (with
latencies of access 3-4 cycle, and very high bandwidths).  L2 cache is
a bit further, still relatively fast, and often private per-core
(typical latencies of access around 10-20 cycles).  L3 cache is
actually often relatively slow and shared between several or all cores
(latencies vary quite a bit between CPU models, and can even be
variable depending on exact data placement, but typical numbers are
around 30-50 cycles).

(3 levels of cache is the most used granularity these days, but you may
find CPUs without a L3, or even with a L3 and a L4)

So if you hit in your L3 cache but miss in your L2 cache, you're
already taking a large toll during which your CPU may be stalled,
waiting for data. Of course, accessing main memory is much slower yet
(with latencies in the several hundreds of cycles, depending on memory
speed and CPU frequency).

Now, the typical L2 cache size is around 512 kiB, and the footprint of
the average Python application is certainly much larger than that.

This is mitigated by the fact that modern CPUs have sophisticated
prefetching techniques that try to *predict* the memory ranges that
will be accessed by future instructions, so as to read them from memory
into (L1 or L2) cache in advance.  However, this only works if those
addresses are predictable at all.  For example, accesses in a hash
table (such as a dict) are by construction rarely predictable.

Regards

Antoine.

_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LOWPYXHZLV42IJRW33VLNXN2DB7MBRJL/
Code of Conduct: http://python.org/psf/codeofconduct/

Reply via email to