On 12/22/2017 04:01 PM, Kogut, Jaroslaw wrote: >> ... first thinking about redesigning the NUMA for >> heterogeneous memory may not be a good idea. Will look into this further. > I agree with comment that first a direction should be defined how to handle > heterogeneous memory system. > >> https://linuxplumbersconf.org/2017/ocw//system/presentations/4656/original/ >> Hierarchical_NUMA_Design_Plumbers_2017.pdf > I miss in the presentation a user perspective of the new approach, e.g. > - How does application developer see/understand the heterogeneous memory > system?
>From user perspective - Each memory node (with or without CPU) is a NUMA node with attributes - User should detect these NUMA nodes from sysfs (not part of proposal) - User allocates/operates/destroys VMA with new sys calls (_mattr based) > - How does app developer use the heterogeneous memory system? - Through existing and new system calls > - What are modification in API/sys interfaces? - The presentation has possible addition of new system calls with 'u64 _mattr' representation for memory attributes which can be used while requesting different kinds of memory from the kernel > > In other hand, if we assume that separate memory NUMA node has different > memory capabilities/attributes from stand point of particular CPU, it is easy > to explain for user how to describe/handle heterogeneous memory. > > Of course, current numa design is not sufficient in kernel in following areas > today: > - Exposing memory attributes that describe heterogeneous memory system > - Interfaces to use the heterogeneous memory system, e.g. more sophisticated > policies > - Internal mechanism in memory management, e.g. automigration, maybe > something else. Right, we would need - Representation of NUMA with attributes - APIs/syscalls for accessing the intended memory from user space - Memory management policies and algorithms navigating trough all these new attributes in various situations IMHO, we should not consider sysfs interfaces for heterogeneous memory (which will be an ABI going forward and hence cannot be changed easily) before we get the NUMA redesign right.
