Re: [Mesa-dev] [PATCH 3/3] radv: add support for VK_EXT_memory_budget
On Mon, 7 Jan 2019 at 17:20, Samuel Pitoiset wrote: > > On 1/7/19 6:06 PM, Alex Smith wrote: > > Hi Samuel, > > Thanks for implementing this - I've been wanting this extension for a > while so it's good it's finally available. > > This is just reporting the total heap sizes as the budget, which is the > same info we already get from the basic heap properties. The way I'd > expected budget to work (and what the spec is saying as far as I can see) > is that it's an estimate of how much is available for the calling app to > use in that heap at the time of the call, so should account for current > system-wide usage of the heap by other apps. Shouldn't this be something > like (heap size - system wide usage of the heap + current app usage of the > heap)? (+ app usage since the spec says budget includes currently allocated > device memory) > > Hi Alex, > > Yes, I was also wondering about that. We can add per-process counters for > VRAM and GTT heaps, but I don't see how we can be accurate for the visible > VRAM heap. > > As said in the commit description, that implementation is really > inacurate. Though if you need something better I can improve. > What I'm after from this extension is the ability to get an idea of when we will either start failing allocations, or start causing some allocations to be paged out from a heap (and probably cause a perf degradation). We have our own systems for relocating resources between heaps when under memory pressure, but currently we can only decide when we need to do this based on guesswork of how much we need to leave free in a heap for other apps or when we actually get to the point where allocations fail. For this extension to be useful to improve on that it needs to include some sort of reporting of app usage vs system-wide usage. Thanks, Alex > Note that I agree with you about the spec. > > > Alex > > On Mon, 7 Jan 2019 at 16:35, Samuel Pitoiset > wrote: > >> A simple Vulkan extension that allows apps to query size and >> usage of all exposed memory heaps. >> >> The different usage values are not really accurate because >> they are per drm-fd, but they should be close enough. >> >> Signed-off-by: Samuel Pitoiset >> --- >> src/amd/vulkan/radv_device.c | 44 +++ >> src/amd/vulkan/radv_extensions.py | 1 + >> 2 files changed, 45 insertions(+) >> >> diff --git a/src/amd/vulkan/radv_device.c b/src/amd/vulkan/radv_device.c >> index cef3a430555..32eaeb3b226 100644 >> --- a/src/amd/vulkan/radv_device.c >> +++ b/src/amd/vulkan/radv_device.c >> @@ -1352,12 +1352,56 @@ void radv_GetPhysicalDeviceMemoryProperties( >> *pMemoryProperties = physical_device->memory_properties; >> } >> >> +static void >> +radv_get_memory_budget_properties(VkPhysicalDevice physicalDevice, >> + >> VkPhysicalDeviceMemoryBudgetPropertiesEXT *memoryBudget) >> +{ >> + RADV_FROM_HANDLE(radv_physical_device, device, physicalDevice); >> + VkPhysicalDeviceMemoryProperties *memory_properties = >> >memory_properties; >> + uint64_t visible_vram_size = radv_get_visible_vram_size(device); >> + uint64_t vram_size = radv_get_vram_size(device); >> + uint64_t gtt_size = device->rad_info.gart_size; >> + >> + if (vram_size) { >> + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM] = vram_size; >> + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM] = >> + device->ws->query_value(device->ws, >> RADEON_VRAM_USAGE); >> + } >> + >> + if (visible_vram_size) { >> + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = >> visible_vram_size; >> + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = >> + device->ws->query_value(device->ws, >> RADEON_VRAM_VIS_USAGE); >> + } >> + >> + if (gtt_size) { >> + memoryBudget->heapBudget[RADV_MEM_HEAP_GTT] = gtt_size; >> + memoryBudget->heapUsage[RADV_MEM_HEAP_GTT] = >> + device->ws->query_value(device->ws, >> RADEON_GTT_USAGE); >> + } >> + >> + /* The heapBudget and heapUsage values must be zero for array >> elements >> +* greater than or equal to >> +* VkPhysicalDeviceMemoryProperties::memoryHeapCount. >> +*/ >> + for (uint32_t i = memory_properties->memoryHeapCount; i < >> VK_MAX_MEMORY_HEAPS; i++) { >> + memoryBudget->heapBudget[i] = 0; >> + memoryBudget->heapUsage[i] = 0; >> + } >> +} >> + >> void radv_GetPhysicalDeviceMemoryProperties2( >> VkPhysicalDevicephysicalDevice, >> VkPhysicalDeviceMemoryProperties2KHR *pMemoryProperties) >> { >> radv_GetPhysicalDeviceMemoryProperties(physicalDevice, >> >> >memoryProperties); >> + >> + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memory_budget = >> + vk_find_struct(pMemoryProperties->pNext, >> + >>
Re: [Mesa-dev] [PATCH 3/3] radv: add support for VK_EXT_memory_budget
On Mon, Jan 7, 2019 at 6:20 PM Samuel Pitoiset wrote: > > > On 1/7/19 6:06 PM, Alex Smith wrote: > > Hi Samuel, > > Thanks for implementing this - I've been wanting this extension for a while > so it's good it's finally available. > > This is just reporting the total heap sizes as the budget, which is the same > info we already get from the basic heap properties. The way I'd expected > budget to work (and what the spec is saying as far as I can see) is that it's > an estimate of how much is available for the calling app to use in that heap > at the time of the call, so should account for current system-wide usage of > the heap by other apps. Shouldn't this be something like (heap size - system > wide usage of the heap + current app usage of the heap)? (+ app usage since > the spec says budget includes currently allocated device memory) > > Hi Alex, > > Yes, I was also wondering about that. We can add per-process counters for > VRAM and GTT heaps, but I don't see how we can be accurate for the visible > VRAM heap. Actually looking at the current counters we query, those are for global usage, not the current process / drm fd usage. so we can calculate budget that way. Now a way to get at the current usage in a more accurate way than counting allocations (which does not reflect migrated data). > > As said in the commit description, that implementation is really inacurate. > Though if you need something better I can improve. > > Note that I agree with you about the spec. > > > Alex > > On Mon, 7 Jan 2019 at 16:35, Samuel Pitoiset > wrote: >> >> A simple Vulkan extension that allows apps to query size and >> usage of all exposed memory heaps. >> >> The different usage values are not really accurate because >> they are per drm-fd, but they should be close enough. >> >> Signed-off-by: Samuel Pitoiset >> --- >> src/amd/vulkan/radv_device.c | 44 +++ >> src/amd/vulkan/radv_extensions.py | 1 + >> 2 files changed, 45 insertions(+) >> >> diff --git a/src/amd/vulkan/radv_device.c b/src/amd/vulkan/radv_device.c >> index cef3a430555..32eaeb3b226 100644 >> --- a/src/amd/vulkan/radv_device.c >> +++ b/src/amd/vulkan/radv_device.c >> @@ -1352,12 +1352,56 @@ void radv_GetPhysicalDeviceMemoryProperties( >> *pMemoryProperties = physical_device->memory_properties; >> } >> >> +static void >> +radv_get_memory_budget_properties(VkPhysicalDevice physicalDevice, >> + VkPhysicalDeviceMemoryBudgetPropertiesEXT >> *memoryBudget) >> +{ >> + RADV_FROM_HANDLE(radv_physical_device, device, physicalDevice); >> + VkPhysicalDeviceMemoryProperties *memory_properties = >> >memory_properties; >> + uint64_t visible_vram_size = radv_get_visible_vram_size(device); >> + uint64_t vram_size = radv_get_vram_size(device); >> + uint64_t gtt_size = device->rad_info.gart_size; >> + >> + if (vram_size) { >> + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM] = vram_size; >> + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM] = >> + device->ws->query_value(device->ws, >> RADEON_VRAM_USAGE); >> + } >> + >> + if (visible_vram_size) { >> + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = >> visible_vram_size; >> + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = >> + device->ws->query_value(device->ws, >> RADEON_VRAM_VIS_USAGE); >> + } >> + >> + if (gtt_size) { >> + memoryBudget->heapBudget[RADV_MEM_HEAP_GTT] = gtt_size; >> + memoryBudget->heapUsage[RADV_MEM_HEAP_GTT] = >> + device->ws->query_value(device->ws, >> RADEON_GTT_USAGE); >> + } >> + >> + /* The heapBudget and heapUsage values must be zero for array >> elements >> +* greater than or equal to >> +* VkPhysicalDeviceMemoryProperties::memoryHeapCount. >> +*/ >> + for (uint32_t i = memory_properties->memoryHeapCount; i < >> VK_MAX_MEMORY_HEAPS; i++) { >> + memoryBudget->heapBudget[i] = 0; >> + memoryBudget->heapUsage[i] = 0; >> + } >> +} >> + >> void radv_GetPhysicalDeviceMemoryProperties2( >> VkPhysicalDevicephysicalDevice, >> VkPhysicalDeviceMemoryProperties2KHR *pMemoryProperties) >> { >> radv_GetPhysicalDeviceMemoryProperties(physicalDevice, >> >> >memoryProperties); >> + >> + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memory_budget = >> + vk_find_struct(pMemoryProperties->pNext, >> + PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT); >> + if (memory_budget) >> + radv_get_memory_budget_properties(physicalDevice, >> memory_budget); >> } >> >> VkResult radv_GetMemoryHostPointerPropertiesEXT( >> diff --git a/src/amd/vulkan/radv_extensions.py >>
Re: [Mesa-dev] [PATCH 3/3] radv: add support for VK_EXT_memory_budget
On 1/7/19 6:06 PM, Alex Smith wrote: Hi Samuel, Thanks for implementing this - I've been wanting this extension for a while so it's good it's finally available. This is just reporting the total heap sizes as the budget, which is the same info we already get from the basic heap properties. The way I'd expected budget to work (and what the spec is saying as far as I can see) is that it's an estimate of how much is available for the calling app to use in that heap at the time of the call, so should account for current system-wide usage of the heap by other apps. Shouldn't this be something like (heap size - system wide usage of the heap + current app usage of the heap)? (+ app usage since the spec says budget includes currently allocated device memory) Hi Alex, Yes, I was also wondering about that. We can add per-process counters for VRAM and GTT heaps, but I don't see how we can be accurate for the visible VRAM heap. As said in the commit description, that implementation is really inacurate. Though if you need something better I can improve. Note that I agree with you about the spec. Alex On Mon, 7 Jan 2019 at 16:35, Samuel Pitoiset mailto:samuel.pitoi...@gmail.com>> wrote: A simple Vulkan extension that allows apps to query size and usage of all exposed memory heaps. The different usage values are not really accurate because they are per drm-fd, but they should be close enough. Signed-off-by: Samuel Pitoiset mailto:samuel.pitoi...@gmail.com>> --- src/amd/vulkan/radv_device.c | 44 +++ src/amd/vulkan/radv_extensions.py | 1 + 2 files changed, 45 insertions(+) diff --git a/src/amd/vulkan/radv_device.c b/src/amd/vulkan/radv_device.c index cef3a430555..32eaeb3b226 100644 --- a/src/amd/vulkan/radv_device.c +++ b/src/amd/vulkan/radv_device.c @@ -1352,12 +1352,56 @@ void radv_GetPhysicalDeviceMemoryProperties( *pMemoryProperties = physical_device->memory_properties; } +static void +radv_get_memory_budget_properties(VkPhysicalDevice physicalDevice, + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memoryBudget) +{ + RADV_FROM_HANDLE(radv_physical_device, device, physicalDevice); + VkPhysicalDeviceMemoryProperties *memory_properties = >memory_properties; + uint64_t visible_vram_size = radv_get_visible_vram_size(device); + uint64_t vram_size = radv_get_vram_size(device); + uint64_t gtt_size = device->rad_info.gart_size; + + if (vram_size) { + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM] = vram_size; + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM] = + device->ws->query_value(device->ws, RADEON_VRAM_USAGE); + } + + if (visible_vram_size) { + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = visible_vram_size; + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = + device->ws->query_value(device->ws, RADEON_VRAM_VIS_USAGE); + } + + if (gtt_size) { + memoryBudget->heapBudget[RADV_MEM_HEAP_GTT] = gtt_size; + memoryBudget->heapUsage[RADV_MEM_HEAP_GTT] = + device->ws->query_value(device->ws, RADEON_GTT_USAGE); + } + + /* The heapBudget and heapUsage values must be zero for array elements + * greater than or equal to + * VkPhysicalDeviceMemoryProperties::memoryHeapCount. + */ + for (uint32_t i = memory_properties->memoryHeapCount; i < VK_MAX_MEMORY_HEAPS; i++) { + memoryBudget->heapBudget[i] = 0; + memoryBudget->heapUsage[i] = 0; + } +} + void radv_GetPhysicalDeviceMemoryProperties2( VkPhysicalDevice physicalDevice, VkPhysicalDeviceMemoryProperties2KHR *pMemoryProperties) { radv_GetPhysicalDeviceMemoryProperties(physicalDevice, >memoryProperties); + + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memory_budget = + vk_find_struct(pMemoryProperties->pNext, + PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT); + if (memory_budget) + radv_get_memory_budget_properties(physicalDevice, memory_budget); } VkResult radv_GetMemoryHostPointerPropertiesEXT( diff --git a/src/amd/vulkan/radv_extensions.py b/src/amd/vulkan/radv_extensions.py index 9952bb9c1c6..491ed9d94c3 100644 --- a/src/amd/vulkan/radv_extensions.py +++ b/src/amd/vulkan/radv_extensions.py @@ -105,6 +105,7 @@ EXTENSIONS = [ Extension('VK_EXT_external_memory_dma_buf', 1, True), Extension('VK_EXT_external_memory_host', 1, 'device->rad_info.has_userptr'), Extension('VK_EXT_global_priority', 1, 'device->rad_info.has_ctx_priority'), +
Re: [Mesa-dev] [PATCH 3/3] radv: add support for VK_EXT_memory_budget
Hi Samuel, Thanks for implementing this - I've been wanting this extension for a while so it's good it's finally available. This is just reporting the total heap sizes as the budget, which is the same info we already get from the basic heap properties. The way I'd expected budget to work (and what the spec is saying as far as I can see) is that it's an estimate of how much is available for the calling app to use in that heap at the time of the call, so should account for current system-wide usage of the heap by other apps. Shouldn't this be something like (heap size - system wide usage of the heap + current app usage of the heap)? (+ app usage since the spec says budget includes currently allocated device memory) Alex On Mon, 7 Jan 2019 at 16:35, Samuel Pitoiset wrote: > A simple Vulkan extension that allows apps to query size and > usage of all exposed memory heaps. > > The different usage values are not really accurate because > they are per drm-fd, but they should be close enough. > > Signed-off-by: Samuel Pitoiset > --- > src/amd/vulkan/radv_device.c | 44 +++ > src/amd/vulkan/radv_extensions.py | 1 + > 2 files changed, 45 insertions(+) > > diff --git a/src/amd/vulkan/radv_device.c b/src/amd/vulkan/radv_device.c > index cef3a430555..32eaeb3b226 100644 > --- a/src/amd/vulkan/radv_device.c > +++ b/src/amd/vulkan/radv_device.c > @@ -1352,12 +1352,56 @@ void radv_GetPhysicalDeviceMemoryProperties( > *pMemoryProperties = physical_device->memory_properties; > } > > +static void > +radv_get_memory_budget_properties(VkPhysicalDevice physicalDevice, > + > VkPhysicalDeviceMemoryBudgetPropertiesEXT *memoryBudget) > +{ > + RADV_FROM_HANDLE(radv_physical_device, device, physicalDevice); > + VkPhysicalDeviceMemoryProperties *memory_properties = > >memory_properties; > + uint64_t visible_vram_size = radv_get_visible_vram_size(device); > + uint64_t vram_size = radv_get_vram_size(device); > + uint64_t gtt_size = device->rad_info.gart_size; > + > + if (vram_size) { > + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM] = vram_size; > + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM] = > + device->ws->query_value(device->ws, > RADEON_VRAM_USAGE); > + } > + > + if (visible_vram_size) { > + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = > visible_vram_size; > + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = > + device->ws->query_value(device->ws, > RADEON_VRAM_VIS_USAGE); > + } > + > + if (gtt_size) { > + memoryBudget->heapBudget[RADV_MEM_HEAP_GTT] = gtt_size; > + memoryBudget->heapUsage[RADV_MEM_HEAP_GTT] = > + device->ws->query_value(device->ws, > RADEON_GTT_USAGE); > + } > + > + /* The heapBudget and heapUsage values must be zero for array > elements > +* greater than or equal to > +* VkPhysicalDeviceMemoryProperties::memoryHeapCount. > +*/ > + for (uint32_t i = memory_properties->memoryHeapCount; i < > VK_MAX_MEMORY_HEAPS; i++) { > + memoryBudget->heapBudget[i] = 0; > + memoryBudget->heapUsage[i] = 0; > + } > +} > + > void radv_GetPhysicalDeviceMemoryProperties2( > VkPhysicalDevicephysicalDevice, > VkPhysicalDeviceMemoryProperties2KHR *pMemoryProperties) > { > radv_GetPhysicalDeviceMemoryProperties(physicalDevice, > > >memoryProperties); > + > + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memory_budget = > + vk_find_struct(pMemoryProperties->pNext, > + > PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT); > + if (memory_budget) > + radv_get_memory_budget_properties(physicalDevice, > memory_budget); > } > > VkResult radv_GetMemoryHostPointerPropertiesEXT( > diff --git a/src/amd/vulkan/radv_extensions.py > b/src/amd/vulkan/radv_extensions.py > index 9952bb9c1c6..491ed9d94c3 100644 > --- a/src/amd/vulkan/radv_extensions.py > +++ b/src/amd/vulkan/radv_extensions.py > @@ -105,6 +105,7 @@ EXTENSIONS = [ > Extension('VK_EXT_external_memory_dma_buf', 1, True), > Extension('VK_EXT_external_memory_host', 1, > 'device->rad_info.has_userptr'), > Extension('VK_EXT_global_priority', 1, > 'device->rad_info.has_ctx_priority'), > +Extension('VK_EXT_memory_budget', 1, True), > Extension('VK_EXT_pci_bus_info', 2, True), > Extension('VK_EXT_sampler_filter_minmax', 1, > 'device->rad_info.chip_class >= CIK'), > Extension('VK_EXT_scalar_block_layout', 1, > 'device->rad_info.chip_class >= CIK'), > -- > 2.20.1 > > ___ > mesa-dev mailing list > mesa-dev@lists.freedesktop.org >
[Mesa-dev] [PATCH 3/3] radv: add support for VK_EXT_memory_budget
A simple Vulkan extension that allows apps to query size and usage of all exposed memory heaps. The different usage values are not really accurate because they are per drm-fd, but they should be close enough. Signed-off-by: Samuel Pitoiset --- src/amd/vulkan/radv_device.c | 44 +++ src/amd/vulkan/radv_extensions.py | 1 + 2 files changed, 45 insertions(+) diff --git a/src/amd/vulkan/radv_device.c b/src/amd/vulkan/radv_device.c index cef3a430555..32eaeb3b226 100644 --- a/src/amd/vulkan/radv_device.c +++ b/src/amd/vulkan/radv_device.c @@ -1352,12 +1352,56 @@ void radv_GetPhysicalDeviceMemoryProperties( *pMemoryProperties = physical_device->memory_properties; } +static void +radv_get_memory_budget_properties(VkPhysicalDevice physicalDevice, + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memoryBudget) +{ + RADV_FROM_HANDLE(radv_physical_device, device, physicalDevice); + VkPhysicalDeviceMemoryProperties *memory_properties = >memory_properties; + uint64_t visible_vram_size = radv_get_visible_vram_size(device); + uint64_t vram_size = radv_get_vram_size(device); + uint64_t gtt_size = device->rad_info.gart_size; + + if (vram_size) { + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM] = vram_size; + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM] = + device->ws->query_value(device->ws, RADEON_VRAM_USAGE); + } + + if (visible_vram_size) { + memoryBudget->heapBudget[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = visible_vram_size; + memoryBudget->heapUsage[RADV_MEM_HEAP_VRAM_CPU_ACCESS] = + device->ws->query_value(device->ws, RADEON_VRAM_VIS_USAGE); + } + + if (gtt_size) { + memoryBudget->heapBudget[RADV_MEM_HEAP_GTT] = gtt_size; + memoryBudget->heapUsage[RADV_MEM_HEAP_GTT] = + device->ws->query_value(device->ws, RADEON_GTT_USAGE); + } + + /* The heapBudget and heapUsage values must be zero for array elements +* greater than or equal to +* VkPhysicalDeviceMemoryProperties::memoryHeapCount. +*/ + for (uint32_t i = memory_properties->memoryHeapCount; i < VK_MAX_MEMORY_HEAPS; i++) { + memoryBudget->heapBudget[i] = 0; + memoryBudget->heapUsage[i] = 0; + } +} + void radv_GetPhysicalDeviceMemoryProperties2( VkPhysicalDevicephysicalDevice, VkPhysicalDeviceMemoryProperties2KHR *pMemoryProperties) { radv_GetPhysicalDeviceMemoryProperties(physicalDevice, >memoryProperties); + + VkPhysicalDeviceMemoryBudgetPropertiesEXT *memory_budget = + vk_find_struct(pMemoryProperties->pNext, + PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT); + if (memory_budget) + radv_get_memory_budget_properties(physicalDevice, memory_budget); } VkResult radv_GetMemoryHostPointerPropertiesEXT( diff --git a/src/amd/vulkan/radv_extensions.py b/src/amd/vulkan/radv_extensions.py index 9952bb9c1c6..491ed9d94c3 100644 --- a/src/amd/vulkan/radv_extensions.py +++ b/src/amd/vulkan/radv_extensions.py @@ -105,6 +105,7 @@ EXTENSIONS = [ Extension('VK_EXT_external_memory_dma_buf', 1, True), Extension('VK_EXT_external_memory_host', 1, 'device->rad_info.has_userptr'), Extension('VK_EXT_global_priority', 1, 'device->rad_info.has_ctx_priority'), +Extension('VK_EXT_memory_budget', 1, True), Extension('VK_EXT_pci_bus_info', 2, True), Extension('VK_EXT_sampler_filter_minmax', 1, 'device->rad_info.chip_class >= CIK'), Extension('VK_EXT_scalar_block_layout', 1, 'device->rad_info.chip_class >= CIK'), -- 2.20.1 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev