> On Mar 11, 2017, at 2:36 PM, Jed Brown <[email protected]> wrote: > > Barry Smith <[email protected]> writes: > >>> I think it's accurate in the sense that the performance of real >>> applications using a page migration system will be sufficiently close to >>> the best manual page mapping strategy that nobody should bother with the >>> manual system. >> >> Will such a page migration system ever exist, is Intel working hard >> on it for KNL? What if no one provides such a page migration >> system? Should we just wait around until they do (which they won't) >> and do nothing else instead? Or will we have to do a half-assed >> hacky thing to work around the lack of the mythical decent page >> migration system? > > Libnuma has move_pages. Prior to release, Intel refused to confirm that > MCDRAM would be shown to the OS as a normal numa node, such that > move_pages would work, and sometimes suggesting that it would not. Some > of the email history is me being incredulous this state before learning > that the obvious implementation that I preferred was in fact what they > did. > > Anyway, this means PETSc can track usage and call move_pages itself to > migrate hot pages into MCDRAM. > > I don't know if Intel or Linux kernel people are going to tweak the > existing automatic page migration to do this transparently, but we > probably shouldn't hold our breath. > >>> In cache mode, accessing infrequently-used memory (like TS trajectory) >>> evicts memory that you will use again soon. >> >> What if you could advise the malloc system that this chunk of >> memory should not be cached? Though this appears to be impossible >> by design? > > Malloc has nothing to do with cache, and I don't think the hardware has > an interface that would allow the kernel to set policy at this > granularity.
Intel does seem to have some load/store commands that allow? by-passing cache. How one could access these commands in any reasonable way I don't have a clue. And yes I would like to access them based on the memory locations I am loading/storing. So yes though malloc has nothing to do with cache, it is a madvise() "type" thing in that I'd like certain memory treated with different load/store.
